ZS Associates - Big Data Architect

ZS is one of the world's largest business services firms specializing in transforming sales and marketing from an art to a science. We help clients gain market share at lower cost by creating data-driven strategies that they can implement rapidly, by taking on sales and marketing operations to make them more competitive, and by helping them deliver impact where it matters. With 22 offices around the world, we have worked with more than 1,200 companies in 70 countries across consumer products, energy, high-tech, insurance, medical products and services, pharmaceuticals and other industries.

As part of ZS's Capability and Expertise Center (CEC), you will work with team members based around the world. As a firm, ZS partners with clients during all parts of the process, from analyzing issues to implementing solutions. To address the increasingly diverse nature of our business, work is delivered in three broad capability areas. Within each of these areas, ZSers collaborate with one another on client solutions.

Business Technology Group Overview:

ZS Associates' Business Technology group focuses on strategizing, designing and building marketing and sales technology solutions to improve commercial effectiveness. The types of solutions include enterprise information management, reporting, analytics, CRM/SFA and business process automation enablement. We provide a complete spectrum of business technology solutions, from strategy and roadmap definition to full implementation and on-going operations. Our team possesses deep expertise in leading analytic, business intelligence, CRM and system integration technologies, which we apply in a pragmatic fashion to create and deliver optimal architecture and implementation solutions.

Role and Responsibilities:

- Design, Develop and Maintain a big data platform using open source technologies such as Spark, Impala, Hive and Scala

- Translate business requirements, data models and algorithms into code (Scala/ Spark SQL/ Python) running on Spark clusters

- Recommend/ suggest inputs for setup and design based on the business requirements

- Provide design recommendations based on specific use cases

- Develop automation for the setup and maintenance of the Big Data Platform

- Lead performance tuning activities of the system

- Help the team with set up of the environment (especially with CDH cluster setup), and install additional software if required

- Knowledge of AWS cloud infrastructure will be an added benefit

Required Qualifications:

- Bachelor's/master's degree with specialization in Computer or IT or other computer related disciplines

- Strong Background in Big Data Infrastructure, Engineering and Development and working with Big Data and Hadoop File System

- 10-12 years of relevant consulting-industry experience working on medium-large scale technology solution delivery engagements.

- Experience in programming (Scala/ Python) on a Distributed Spark platform in a Linux environment is a must

- Very good knowledge and hands on experience of Big Data querying tools, such as Pig, Hive, and Impala

- Strong working experience on Spark, Impala and Hive (with development as well as optimization)

- Good experience working on Cloudera distribution and on EMR

- Experience with development on AWS (Amazon web services)

- Good hands on experience of automation using Oozie

- Deep understanding and ability to use SQL, XML, JSON and LINUX

- Experience in deploying a distributed Spark cluster from scratch

- Experience in Management and Maintenance of Large Data Sets over an extended period of time

- Experience in Linux/Unix scripting is preferable

- Experience working in a development team responsible for delivering an enterprise product