Job Responsibilities : -
As a Python / Javascipt Developer, your role is to apply your knowledge set to fetch data from multiple online sources, cleanse it and apply ETL-
Develop a deep understanding of our vast data sources on the web and know exactly how, when, and which data to scrap, parse and store this data-
Work closely with Data Scientists / Data Engineers to support Machine Learning Algorithm Development-
Develop frameworks for automating and maintaining the constant flow of data from multiple sources-
Work independently with little supervision to research and test innovative solutions Skills and Qualifications : -
Strong coding experience in Python (knowledge of Java, Javascript is a plus)-
Experience with ETL, Big Data Concepts, SQL and NoSQL databases-
Experience with multi-processing, multi-threading and AWS/Azure-
Strong knowledge of scraping frameworks such as Python (Request, BeautifulSoup),WebHarvest, Scrapy and others-
Previous experience with web crawling is a must