As a Web Scraping focused Data Engineer, you will be responsible for extracting and ingesting data from websites using web crawling tools. In this role you will own the creation process of these tools, services, and workflows to improve crawl/ scrape analysis, reports and data management. We will rely on you to test the data and the scrape to insure accuracy and quality. You will own the process to identify and rectify any issues with breaks as well as scale scrapes as needed.

Responsibilities:

Develop and maintain web crawlers in Python.

Crawl large websites and extract data.

Store data in a database.

Analyze and report on data.

Work with other engineers to develop and improve our web crawling infrastructure.

Stay up to date on the latest crawling tools and techniques.