Role Overview:

We are looking for a Director Data Engineering to lead the design, development, and scaling of modern data platforms powering our analytics and AI solutions.

Key Responsibilities:

Platform Architecture & Engineering:

- Design and build scalable data platforms leveraging Databricks Lakehouse Architecture and Snowflake.

- Implement Medallion Architecture (Bronze, Silver, Gold layers) to standardize enterprise data pipelines.

- Optimize Delta Lake tables for performance, scalability, and cost efficiency.

- Establish governance frameworks using Unity Catalog and enterprise data modeling practices.

- Drive modern cloud-native data architecture and engineering best practices.

Data Pipeline Development:

- Architect and manage end-to-end batch and near real-time data pipelines.

- Enable ingestion, transformation, and serving of large-scale datasets, including healthcare claims, benefits, and workforce datasets.

- Ensure pipelines are scalable, maintainable, and aligned to downstream analytics and AI requirements.

AI-Ready Data Platforms:

- Build data platforms that support AI and Machine Learning workloads.

- Enable feature engineering, model-ready datasets, and AI data pipelines.

- Leverage capabilities such as Databricks ML, MLflow, and Feature Engineering.

- Collaborate closely with Data Science teams to operationalize ML solutions.

DataOps & Reliability:

- Implement CI/CD practices for data engineering workflows.

- Establish monitoring, observability, logging, and data quality frameworks.

- Drive high standards of reliability, governance, security, and operational excellence.

Collaboration & Solutioning:

- Partner with Product, Analytics, Consulting, and Client teams to deliver scalable data solutions.

- Participate in architecture discussions and solution design for client engagements.

Practice Development & Go-to-Market:

- Contribute to building and scaling company's Data Engineering / Databricks Services practice.

- Support pre-sales activities including technical presentations, solution architecture discussions, and proposal development.

- Help define reusable accelerators, frameworks, and best practices for enterprise data engineering engagements.

Team Leadership:

- Lead and mentor a team of Data Engineers, fostering technical excellence and accountability.

- Conduct architecture reviews, code reviews, and establish engineering best practices.

Key Requirements:

- 11-15 years of experience in Data Engineering, Data Platform Development, or Cloud Data Engineering.

- Proven experience leading engineering teams and delivering scalable enterprise data platforms.

- Strong hands-on expertise in Databricks, Apache Spark, PySpark, Delta Lake, and Unity Catalog.

- Cloud platforms experience (AWS, Azure, or GCP).

- Snowflake experience is strongly preferred.

- Experience with AI/ML workflows (Databricks ML, MLflow, Vector Search, RAG) is highly desirable.

- Strong stakeholder management and executive communication skills.

- Ability to balance technical depth with business impact.