Role Summary:
The ML Lead is responsible for defining and owning the intelligence strategy of an agentic AI platform built on on prem LLMs. This role leads model selection, multi model orchestration, quantization and inference optimization, domain knowledge ingestion, RAG grounding, fine tuning approaches, and evaluation frameworks. The ML Lead ensures that AI agents meet defined quality, performance, cost, and reliability metrics and continuously improve in production.
Key Responsibilities:
- Identify and evaluate appropriate LLMs and auxiliary models; define and implement a multi model strategy aligned to task complexity, latency, and cost constraints.
- Design and own quantization strategies, inference parameters, and routing logic to optimize on prem performance without degrading agent behavior.
- Lead domain knowledge integration, including RAG grounding strategies and decisions on what knowledge is retrieved vs fine tuned.
- Drive LLM fine tuning and PEFT/LoRA based adaptations, ensuring stable, task specific behavior across agent workflows.
- Define, implement, and track AI quality metrics (accuracy, grounding, tool use, drift, cost, latency) and ensure systems consistently meet agreed targets.
- Provide technical leadership to ML Engineers and collaborate with Agent Engineers, Data Engineering Leads, MLOps, Security, and Product teams.
- Set standards for model experimentation, evaluation, and rollout; review and approve behavior impacting changes.
- Act as the primary owner for AI intelligence decisions in architecture and production governance discussions.
Required Experience & Skills:
- Strong background in applied ML / LLM systems with leadership experience in production environments.
- Hands on experience with LLM inference, quantization (FP16, INT8/INT4, GPTQ/AWQ), RAG architectures, and PEFT techniques (LoRA/QLoRA).
- Proven ability to design evaluation metrics and benchmarking frameworks for complex, multi step AI systems.
- Experience with on prem or restricted environments (resource constraints, compliance, cost control).
Didn’t find the job appropriate? Report this Job
