Working Hours: Full overlap with PST

About the Role

We are seeking a Senior Data Engineer who excels in SQL, large-scale data processing, and MPP databases. This is a lead role, requiring someone with deep technical expertise, strategic thinking, and the ability to design and optimize data pipelines at scale. You will be responsible for architecting high-performance data solutions, working with Hadoop, MPP databases, and on-prem environments, and mentoring junior engineers.

Key Responsibilities

• Lead the development and optimization of data pipelines for processing massive datasets.
• Design scalable ETL processes, focusing on performance and efficiency.
• Write, optimize, and debug complex SQL queries, including window functions, partitioning, and query tuning.
• Work extensively with MPP databases (Snowflake, Redshift, BigQuery, Teradata, Vertica) and understand their architecture and performance optimizations.
• Deep understanding of Hadoop and its ecosystem (MapReduce, Hive, Spark, HDFS).
• Strong programming background to develop custom data processing solutions.
• Work with Linux environments, including system monitoring and optimization.
• Architect solutions for on-prem environments, ensuring seamless data movement and storage.
• Collaborate with business teams, understanding their needs and translating them into technical solutions.
• Mentor junior engineers, fostering best practices in SQL optimization, MPP architecture, and data engineering.

Must-Have Qualifications

• Hadoop expertise – Strong understanding of distributed computing and performance optimizations.
• MPP Databases – Extensive experience with Teradata, Snowflake, Redshift, BigQuery, or Vertica.
• Programming background – Proficiency in Python, Java, or Scala for data processing.
• Linux expertise – Comfortable with command-line tools, system monitoring, and performance tuning.
• On-Prem Data Engineering – Experience working with large-scale on-prem infrastructure.
• SQL Mastery – Deep understanding of query optimization, performance tuning, and ETL logic.
• Leadership experience – Ability to design solutions, mentor teams, and drive technical decisions.

Nice-to-Have Qualifications

• ETL Tools (Pentaho, Informatica, Talend, DBT, Apache NiFi) – Helpful but not mandatory.
• Cloud experience (AWS, GCP, Azure) – Cloud migration knowledge is beneficial.