Web Scraping with lxml

Quick note: I recently started a web scraping project using python and the lxml project. When deploying this on a Digital Ocean droplet, naturally I chose the least expensive option with 1 cpu and 512 MB of RAM. Unfortunately, I could not pip install lxml in a virtualenv; I kept getting a gcc compile error. Turns out that 512 MB is not enough memory to compile lxml. I powered down and resized my droplet to the next best option with 1 GB of RAM and boom, it worked. Now I have a cron job that scrapes data from a webpage and parses it into a csv file daily at 3am.

Since I do not know much about web scraping, I used freelancer.com to hire someone to write the scraping script in python which I then modified to fix all of the specification mistakes I made when describing the project. I cannot recommend this service highly enough; seamless experience from start to finish. I plan to use it extensively in the future.