Big Data Engineer jobs in Bangalore at Intel Corporation

Big Data Engineer

Job ID: JR0126123
Job Category: Information Technology
Primary Location: Bangalore, KA IN

Qualifications

Navigate the Hadoop Ecosystem and know how to leverage or optimize the use of what Hadoop has to offer
Hadoop development, debugging, and implementation of workflows and common algorithms
Apache Hadoop and data ETL (extract, transform, load),ingestion, and processing with Hadoop tools
Knowledge of building a scalable and integrated Data Lake for an Enterprise
Use the HDFS architecture, including how HDFS implements file sizes, block sizes, and block abstraction. Understand default replication values and storage requirements for replication. Determine how HDFS stores, reads, and writes files.
Analyze the order of operations in a MapReduce job, how data moves from place to place, how partition and combiners function, and the sort and shuffle process
Analyze and determine which of Hadoop's data types for keys and values are appropriate for a job. Understand common key and value types in the MapReduce framework and the interfaces they implement
Organizing data into tables, performing transformations, performance tuning and simplifying complex queries with Hive and Impala
How to pick the best tool for a given task in Hadoop, achieve interoperability, and manage recurring workflows
Strong programing skills in Java or Python
Working Knowledge of data ingestion using spark for supporting various file types like Json, Xml, Csv and databases
Hands on development experience to extract the data from various sources like SFTP, Amazon S3 and other cloud data sources
Designing optimal HBase schemas for efficient data storage and ingestion to HBase using the native API
Knowledge of Kafka, Spark Streaming and stream data loads types and techniques
Strong Sql and Data Analysis Skills
Strong Shell Script or any other scripting language

Intel AI, leveraging Intel's world leading position in silicon innovation and proven history in creating the compute standards that power our world, is transforming Artificial Intelligence (AI) with the Intel AI products portfolio. Harnessing silicon designed specifically for AI, end to end solutions that broadly span from the data center to the edge, and tools that enable customers to quickly deploy and scale up, Intel AI is inside AI and leading the next evolution of compute.