Data Engineer - Consulting Firm (3-5 yrs)
Experience: 3.0 - 5.0 Years
- Bachelor's degree in a quantitative field (e.g., Engineering, Mathematics, Operations Research, Statistics, etc.) from Tier 1/2 institute.
- Experience in Big data Technologies like Hands-on with Scala, Apache Spark, Hadoop, HDFS, MLlib, Airflow
- Experience in AWS
- Bachelor degree in a quantitative field (e.g., Engineering, Mathematics, Operations Research, Statistics, etc.) from Tier 1/2 colleges.
- 4 or more years of professional experience in areas like development of data pipelines, implementation of data lakes, setting up of cloud ecosystem, development of ETL tools etc.
- Previous exposure of working with clickstream data coming from Adobe analytics and Google analytics will be given extra consideration.
- Previous experience of working directly with clients in US or Europe is a must
- Strong AWS Experience. Experience of working on different components of the ecosystem.
- A certification would be helpful.
- Experience with the following aspect of data engineering
- Building and optimizing - big data- data pipelines, architectures and data sets.
- Manipulating, processing, and extracting value from large disconnected datasets
- Development of the data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
- Knowledge and working experience of Spark/Scala
- Good experience with at least one ETL tool like Talend, Informatica etc.
- Build processes supporting data transformation, data structures, metadata, dependency and workload management
- Expertise in SQL and Relational Databases.
- Expertise in at least one of the NoSQL databases.