Gladwin Analytics - Big Data Engineer

Responsibilities:

- We are looking for an experienced data engineer to help manage our ever growing information needs and support analysis of increasingly complex business questions in Airline industry. As a data engineer, you would be responsible to help design and build out the data engineering platform that will form the core of the Analytics, BI and Machine Learning initiatives for our business.

- The role will involve building and maintaining database solutions that are used for daily reporting and drive business decision making while dealing efficiently with the massive scale of data available through our Data Warehouse as well as our software systems.

- You will be responsible for designing and implementing solutions that involve data interfaces with both internal and external systems, and administering the platform software. You are expected to build efficient, flexible, extensible, and scalable ETL design and mappings.

- You must be a self - starter and be able to learn on the go.

Candidate Profile:

- Proficient understanding of distributed computing principles

- Management of Hadoop cluster, with all included services

- Ability to solve any ongoing issues with operating the cluster

- Proficiency with Hadoop v2, MapReduce, HDFS

- Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming

- Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala

- Experience with Spark

- Experience with integration of data from multiple data sources

- Experience with NoSQL databases, such as HBase, Cassandra, MongoDB

- Knowledge of various ETL techniques and frameworks, such as Flume

- Experience with various messaging systems, such as Kafka or RabbitMQ

- Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2O

- Good understanding of Lambda Architecture, along with its advantages and drawbacks

- Experience with Cloudera/MapR/Hortonworks