Please Look AWS), Databricks, Snowflake, and ETL tools with Generative AI initiatives.
Senior Developer - Unstructured Data Processing
Unstructured data processing is a key component for any Generative AI solution, and we are seeking a strong engineer in this area.
This capability will support enterprise-wide projects and enable us to deliver high-value Generative AI initiatives.
Responsibilities:
Understand various document types and extraction methods of information from these documents
Experience in tools such as Databricks, Snowflake, and ETL tools
Experience in building Generative AI capabilities
Experience in building end-to-end pipelines for data sourcing, extraction, vectorization and storing information in multiple target databases
Develop business processes for operationalization of selected vendor tool
Build evaluation framework of extraction accuracy, and optimization
Setup and train developer team on new tool features, and functionality
Constantly review, evaluate and deploy new and emerging capabilities (ex. Agentic AI patterns, prompt templates, vector databases, application patterns, evaluation and optimization frameworks such as LLM as a Judge etc.)
Have continuous improvement mindset to constantly learn, improve and advance AI capabilities.
Applicable to both positions:
General Responsibilities:
Infrastructure: Design, deploy, and manage AWS infrastructure to support data science and machine learning pipelines.
Data & Platforms: Set up and optimize infrastructure for vector databases (e.g., Pinecone, Weaviate etc.), unstructured data processing (documents, audio, video), and large-scale data warehousing (Snowflake).
ML/LLM Operations: Implement CI/CD pipelines and automation for model training, serving, and deployment.
Containerization: Architect and manage containerized environments for AI applications using Amazon EKS and ECS, ensuring high availability and scalability.
Tooling: Support data science teams by integrating and managing platforms such as Databricks, Jupyter, and VS Code.
Monitoring & Optimization: Proactively monitor and scale AI/ML infrastructure to ensure optimal performance, resource utilization, and cost efficiency.
Required Skills:
AWS: Expert-level proficiency in core AWS services for AI/ML and data (S3, EC2, EKS, ECS, SageMaker, Lambda, etc.).
Data Platforms: Strong experience with Snowflake and Databricks.
ML/LLM Operations: Hands-on experience with vector databases and processing unstructured data and GenAI/ML Observability.
DevOps: Expertise in DevOps principles, including automation, IaC (e.g., Terraform), and CI/CD pipelines.
Tools: Familiarity with Jupyter notebooks and VS Code environments.
Experience:
7-8 years of architecting hybrid cloud (preferably AWS) solutions using industry leading and cloud native capabilities and tools.
1-2 years of delivering Generative AI solutions.
Works well in a matrix organization with onshore and offshore teams.
Good written and verbal communication skills.
Strong interpersonal skills and continuous learning mindset.
1.The more the Jobs you apply, the higher your chances of getting a job.
2. Keep your profile updated Update
Recruiters prefer candidates with complete profile information.
3. Keep visiting the Teamlease.com daily
Daily visit will ensure you won’t miss out on any Job opportunity.
4. Watch videos to improve Watch videos
Be a better candidate than others by watching these Job-related videos.