We are looking to extract data from several different websites using scrapy or API.
Since there will be thousands of requests to each website, you must know how to use and implement proxies for your script
There are a total of 17 fields from 7 different websites. The data will be stored in our postresql database.
The job will be considered complete if all fields from all 7 websites can be scraped for thousands of lines.
Please apply with your prior experience, and also letting me know details of the environment you will need to run the script.