Job purpose:
To transform data into a format that can be easily analyzed by developing, maintaining, and testing infrastructures for data generation You will be working closely with data scientists and are largely in charge of architecting solutions for data scientists that enable them to do their jobs
Accountabilities
· Data Pipeline - Develop and maintain scalable data pipelines and builds out new API integrations to support continuing increases in data volume and complexity
· Data Integration - Connect offline and online data to continuously improve overall understanding of customer behavior and journeys for personalization Data pre-processing including collecting, parsing, managing, analyzing and visualizing large sets of data
· Data Quality Management - Cleanse the data and improve data quality and readiness for analysis Drive standards, define and implement/improve data governance strategies and enforce best practices to scale data analysis across platforms
· Data Transformation - Processes data by cleansing data and transforming them to proper storage structure for the purpose of querying and analysis using ETL and ELT process
· Data Enablement - Ensure data is accessible and useable to wider enterprise to enable a deeper and more timely understanding of operation
Qualifications & Specifications
· Bachelor’s degree in Engineering /Computer Science/ Math/ Statistics or equivalent Master's degree in relevant specification will be first preference
· Strong programming skills in Python/R/SAS
· Proven experience with large data sets and related technologies – SQL, NoSQL, Google / AWS Cloud, Hadoop, Hive, Spark
· Excellent understanding of computer science fundamentals, data structures, and algorithms
· Data pipeline software - Airflow, RJ Metrics, Segment, Amazon Data Pipeline, Apache Pig
· ETL software’s - Amazon RedShift, CA Erwin Data Modeler, Oracle Warehouse Builder, SAS Data Integration Server, Pentaho Kettle, Apatar
· Hands-on experience and knowledge of the Data Lake technology
· For Analyst – 3 to 5 years of experience and for Sr Analyst – 5 to 8 years of experience
