We are looking for an exceptional Web Scraping Expert to architect and build advanced web scraping with our cross-functional data sources team, a core function in MP mission.
As a Web Scraping focused Data Engineer, you will be responsible for extracting and ingesting data from websites using web crawling tools. In this role you will own the creation process of these tools, services, and workflows to improve crawl/ scrape analysis, reports and data management. We will rely on you to test the data and the scrape to insure accuracy and quality. You will own the process to identify and rectify any issues with breaks as well as scale scrapes as needed
- 2+ years of web scraping experience
- Experience with one or more of the following web scraping frameworks and tools: Scrapy , Puppeteer, Selenium, Import.io, Webhose.io, ScrapingHub etc.
- Basic knowledge of data engineering (database ingestion, ETL, etc.)
- Solution orientation and “can do” attitude – with a desire to tackle complex problems.