Lead Data Engineer - Auto/Auto Ancillary

We are looking for someone who can join within 30 days with skills like Azure,Python and Spark

Responsibilities:

- Responsible to Ingest data from files, streams and databases. Process the data with Python and Pyspark and its storage into time series database.

- Develop programs in Python as part of data extraction, data cleaning, transformation and processing.

- Develop and maintain scalable data pipelines.

- Rest API's development. Responsibilities:

Mandatory Skills:

- Overall 6 -13 years of experience as Data Engineer.

- Advanced working SQL knowledge to create complex queries

. - Experience in working with Time-series Database, relational databases as well as working familiarity with a variety of databases (structured & Unstructured).

- Hands On experience on visualization tools like Grafana & Power BI.

- Experience in designing and implementing scalable architecture.

- Good experience in doing object-oriented programming in python

- Very strong in Object Oriented Analysis and Design (OOAD)

- Strong knowledge on REST APIs.

- Experience working on Azure Cloud services IaaS, PaaS

- Hands on Experience in working on Microsoft Azure Services like ADLS/Blob Storage solutions, Event Hubs, Service Bus, scale sets, Load Balancers, Azure Functions, Databricks.

- Hands on Experience in working on Kafka.

- Knowledge on continuous integration/continuous deployment.

- Experience of data migration and deployment from On-Premise to Cloud environment and vice-versa.

- Individual Contributor.

Requirements

- Candidates must possess atleast a Engineering/equivalent.

- Atleast 8.00 year(s) of working experience in the related field is required for this position.

- Preferably Middle Level/Management. Job role in Lead Data Engineer