Lead Data Engineer - Auto/Auto Ancillary (6-13 yrs)
We are looking for someone who can join within 30 days with skills like Azure,Python and Spark
- Responsible to Ingest data from files, streams and databases. Process the data with Python and Pyspark and its storage into time series database.
- Develop programs in Python as part of data extraction, data cleaning, transformation and processing.
- Develop and maintain scalable data pipelines.
- Rest API's development. Responsibilities:
- Overall 6 -13 years of experience as Data Engineer.
- Advanced working SQL knowledge to create complex queries
. - Experience in working with Time-series Database, relational databases as well as working familiarity with a variety of databases (structured & Unstructured).
- Hands On experience on visualization tools like Grafana & Power BI.
- Experience in designing and implementing scalable architecture.
- Good experience in doing object-oriented programming in python
- Very strong in Object Oriented Analysis and Design (OOAD)
- Strong knowledge on REST APIs.
- Experience working on Azure Cloud services IaaS, PaaS
- Hands on Experience in working on Microsoft Azure Services like ADLS/Blob Storage solutions, Event Hubs, Service Bus, scale sets, Load Balancers, Azure Functions, Databricks.
- Hands on Experience in working on Kafka.
- Knowledge on continuous integration/continuous deployment.
- Experience of data migration and deployment from On-Premise to Cloud environment and vice-versa.
- Individual Contributor.
- Candidates must possess atleast a Engineering/equivalent.
- Atleast 8.00 year(s) of working experience in the related field is required for this position.
- Preferably Middle Level/Management. Job role in Lead Data Engineer