Home>Jobs>India >Hyderabad >Big Data Engineer

Big Data Engineer

InvestingChannel

Experience : 0 to 3 Years

Education : ME/M.Tech, Other Course, BCA, BE/B.Tech

Location : Hyderabad

Job Role : IT Software-Engineer

Posted on 21 Oct 20

Apply now

Job DescriptionLast Date 19 Dec 20

InvestingChannel, an innovative digital media company featuring a valuable repertoire of financial websites, award winning content and robust advertising and subscription solutions

We are looking for an experienced and passionate Data Engineer with 6+ years of experience in building scalable, high-performance distributed systems that deal with large data volumes You will be responsible for development work on all aspects of Big Data, data provisioning, modeling, performance tuning and optimization

Responsibilities

Work closely with business and dev teams to translate the business/functional requirements into technical specifications that drive Big Data solutions to meet functional requirements
Participate in software design meetings and write technical design documents
Design, develop, implement and tune large-scale distributed systems and pipelines that process large volumes of data; focusing on scalability, low -latency, and fault-tolerance in every system built
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS ‘big data’ technologies
Maintain application stability and data integrity by monitoring key metrics and improving code base accordingly
Understand & maintain existing codebase by regular re-factoring and applying requested fixes and features
Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader
Should be flexible to learn new technology / required frameworks

Requirements

6+ years of experience in core Java and Spring Batch
Highly Proficient in SQL Expertise in writing complex, highly-optimized queries across large data sets
4+ years of experience in ingestion pipeline and data processing
Experience with data governance (Data Quality, Metadata Management, Security, etc)
Hands-on with Parquet formats, AWS Redshift, Spectrum and Athena
Experience with multi-data sources and getting it all into a Data Lake
2+ years of experience in Hadoop technologies
Experience in a Linux/UNIX environment is mandatory
Ability to grapple with log files and Unix processes
Strong analytic skills related to working with structured and unstructured datasets
Excellent understanding of Data Structures and Algorithms
Prior exposure to building real time data pipelines would be an added advantage
Experience on data visualization tools is good to have
Experience of working within a fast-paced Agile development process
Excellent problem solving skills
Technology 'Must Haves': Core Java, Spring Batch, Data Structures, Web Services, RDBMS (MySQL/SQL Server),NoSQL, HDFS, MapReduce, Hive, Kafka, Airflow, Spark, Hibernate Framework, Shell Scripting
Technology 'Good to Haves': Design Patterns, Jenkins, Python, R

Big Data Engineer
Experience : 0 to 3
Job Type : Full Time
Interview Type : Face to Face Interview

Job Type : Full Time

Interview Type : Face to Face Interview

Company Description

InvestingChannel

Apply now

Similar Jobs

Big Data Engineer

BRISA TECHNOLOGIES PRIVATE LIMITED

Experience : 0 to 3 Years

Education : BE/B.Tech

Location : Bangalore

Last Date : 04 Feb 21