Hey Guys, I have an opening to share with you.

Required Skills: Cloud platforms (AWS, Azure, GCP),Python and Pyspark,Apache Spark, Spark tuning and creating RDDs,Various file formats like Parquet, Avro, etc. for large volumes of data.

Description

About The Role

A successful candidate will be a self-starter, who drives excellence, is ready to jump into a variety of big data technologies & frameworks, and is able to coordinate and collaborate with other engineers, as well as mentor other engineers in the tea.

What You’ll Be Doing

1. Build highly scalable, available, fault-tolerant distributed data processing systems (batch and streaming systems) processing over 100s of terabytes of data ingested every day and petabyte-sized data warehouse and elastic search cluster.
2. Build quality data solutions and refine existing diverse datasets to simplified models encouraging self-service
3. Build data pipelines that optimize on data quality and are resilient to poor quality data sources
4. Own the data mapping, business logic, transformations and data quality
5. Low level systems debugging, performance measurement & optimization on large production clusters
6. Participate in architecture discussions, influence product roadmap, and take ownership and responsibility over new projects
7. Maintain and support existing platforms and evolve to newer technology stacks and architectures

We’re excited if you have

1. Proficiency in Python and pyspark
2. Deep understanding of Apache Spark, Spark tuning, creating RDDs, and building data frames. Create Java/ Scala Spark jobs for data transformation and aggregation.
3. Experience in big data technologies like HDFS, YARN, Map-Reduce, Hive, Kafka, Spark, Airflow, Presto, etc.
4. Experience in building distributed environments using any of Kafka, Spark, Hive, Hadoop, etc.
5. Good understanding of the architecture and functioning of Distributed database systems
6. Experience working with various file formats like Parquet, Avro, etc for large volumes of data
7. Experience with one or more NoSQL databases
8. Experience with AWS, GCP
9. 5+ years of professional experience as a data or software engineer.

If this is interesting for you then please share your resume while applying. Also, please feel free to reach out in case of any queries.

Thanks,

S Naithani