Use Multiple LinkedIn Accounts to Scrape 80,000 Profiles
Budget
903$
per month
Posted: 5 years ago
Opened
- Description
- The project is quite simple in terms of the technical scraping part that is needed. You will have a LinkedIn profile URL as input and will need to visit the profile and then scrape the full employment history from the profile in order from most recent to oldest. The parameters to be captured: company name, title, years of employment, location, description
I'm not entirely certain of the output structure that I want yet - whether that be as an object array with each job as a node on the array with child elements within each node for the company name, title, years of employment, location, description, etc. - or if I would want it to be linear in typical CSV format with 5 columns for each listed job, going from most recent to oldest, 5 columns at a time, up to however many jobs are listed on the LinkedIn profile.
The difficult aspect of this project is the scale. There will initially be 80,000 LinkedIn profiles to do this full employment scrape for and then potentially an additional 120,000 profiles afterward.
If you have experience scraping LinkedIn at scale, you are likely already familiar with their commercial use limits for viewing profiles, even through the access of a LinkedIn premium account like Sales Navigator which allows at most 500 profile views per day.
I'm looking to get the first 80,000 profiles done within the span of 1 month so I'm looking for someone who will have the right strategy and approach to accomplish this and get around LinkedIn's limitations - whether that be through the use of some 3rd party API, through the use of many dummy accounts or some other method.
Ideally, you would use 6 separate LinkedIn accounts to each scrape the maximum 500 profiles per day to be able to scrape all 80,000 profiles within one month's time. Anything that will take longer than 1 month will be considered unacceptable.
Let me know your thoughts, questions, ballpark pricing, timeline, anything else you think is relevant. I'm looking for proposals that are well thought out from freelancers that seem credible and experienced in such work. Any generic proposals or boilerplate promises to somehow get the work done will be ignored. This is a serious job and I am ready to pay appropriately for the work.
Attached is a sample of the kind of profiles that will need to be scraped.
They are all lawyers/attorneys, primarly located in the United States.
Please write the name of your favorite food at the top of your proposal to help weed out robo-proposals.
Many thanks and I look forward to discussing the work with you.
Skills:
legal consulting,application programming interface (API),comma separated values(csv),LinkedIn,scraping,software development
- Category
Source: peopleperhour.com
Drag