Web scraping tool development
Budget
157$
per month
Posted: 5 years ago
Opened
- Description
- Require some software or script to be built to scrape product data from laptopscreen.com. The tool should not put the site under high load.
Need both brand and model product data scraped.
Index page for brands is here:
https://www.laptopscreen.com/English/brands/
For models here:
https://www.laptopscreen.com/English/section/screen-part-number/
For example, there are 2 products here that need to be scraped:
https://www.laptopscreen.com/English/screen-part-number/LTN156AT24/
Fields required:
ItemID Price DiscountedPrice Compatibility Size Resolution Surface Type Backlight Type Replacement Part Type Video Signal Connector Mountings Display Technology Part Type Image Links Product Link
The tool needs to have good error handling. E.g. some pages take a while to load, and the tool needs to wait:
https://www.laptopscreen.com/English/series/Lenovo/IDEAPAD/
Some products may not have all of the above fields. It should scrape what is available.
There are also a few different types of indexes for the products, all which have slightly different layouts. The tool needs to cater for this. E.g.
https://www.laptopscreen.com/English/brand/Gateway/
https://www.laptopscreen.com/English/brand/eMachines/
https://www.laptopscreen.com/English/brand/BenQ/
https://www.laptopscreen.com/English/brand/Lenovo/
Also, sometimes the site will stop sending requests and give a page similar to 404. In this case, the tool should pause, and try again in a minute. If the tool crashes, it should log the error and then save what it is up to into excel. Alternatively it should try and move to the next listing.
I should be able to select upfront whether to scrape brands or products. Then also be able to tell it which brands to scrape.
Skills:
data scraping,video,catering,image,software development,web scraping
- Category
Source: peopleperhour.com
Drag