Infosys BPM - Technology Lead - Databricks AI Services

Description:

Key Responsibilities:

- Lead the design and development of scalable data pipelines and ETL/ELT workflows using Databricks on Azure.

- Architect and implement data lakehouse solutions, optimizing storage, compute, and query performance for large datasets.

- Develop, optimize, and maintain Databricks notebooks, jobs, and workflows for batch and near real-time data processing.

- Define and enforce coding standards, best practices, and reusable frameworks for Databricks-based data engineering.

- Collaborate with data architects, analysts, and business teams to understand requirements and translate them into technical designs.

- Implement robust data quality checks, validation rules, and monitoring to ensure reliability and trust in data products.

- Optimize Spark jobs for performance and cost efficiency, including cluster sizing, caching strategies, and partitioning.

- Ensure security, governance, and compliance of data solutions, including access control and data lineage within Databricks and Azure.

- Troubleshoot complex data issues, perform root cause analysis, and drive long-term fixes and improvements.

- Mentor junior engineers, provide technical guidance, conduct code reviews, and support knowledge sharing within the team.

- Contribute to roadmap planning, effort estimation, and delivery management for data engineering initiatives.- Knowledge of more than one technology

- Basics of Architecture and Design fundamentals

- Knowledge of Testing tools

- Knowledge of agile methodologies

- Understanding of Project life cycle activities on development and maintenance projects

- Understanding of one or more Estimation methodologies, Knowledge of Quality processes

- Basics of business domain to understand the business requirements

- Analytical abilities, Strong Technical Skills, Good communication skills

- Good understanding of the technology and domain

- Ability to demonstrate a sound understanding of software quality assurance principles, SOLID design principles and modelling methods

- Awareness of latest technologies and trends

- Excellent problem solving, analytical and debugging skills

Note : For your candidature to be considered on this job, you need to apply necessarily on the company's redirected page of this job. Please make sure you apply on the redirected page as well.