I imagine that Jack Sparrow would say something like that when he would know about web scraping and crawling. All the data about ships and cargos would definitely be useful for him and his Black Pearl crew. But how would scraping benefit me? Ordinary developer who isn’t planning to steal a gold treasure?
Tons of websites without public API When you are a web developer for a while and requesting restful APIs is your bread and butter you often come across some service and think to yourself “It would be really nice if I can just request these data through an API …”.
For a lot of websites I build I use git for version control. I always used ftp client such as FileZilla to upload files to web hosting. But it gets annoying as you got more websites to manage. You have to login to ftp, click through GUI to find your folders. It is time consuming. So one day I decided to use GitLab CI/CD to build and deploy websites to web hosting.