Role: Senior Data Engineer
Role Overview :
Experienced Data Engineer to analyze, design and develop enterprise-scale data pipelines across legacy and strategic cloud data platforms.
A key component of the position is analyzing existing data pipelines, particularly Omnia-based Spark pipelines, understanding the underlying business and transformation logic, and translating that logic into scalable data solutions on modern cloud platforms. The role requires strong hands-on experience with AWS, Snowflake, dbt, Airflow, PySpark, Python and advanced SQL.
Technical Expertise :
- Strong hands-on experience designing, developing and supporting scalable enterprise data pipelines.
- Strong Spark-etl experience, including the ability to analyze existing Omnia pipelines, understand their processing logic and derive new transformation and business rules.
- Strong working knowledge of AWS services, including S3, Glue, EMR, Lambda, Athena, Step Functions, SQS, IAM, Secrets Manager and CloudWatch, combined with hands-on experience in orchestrating data pipelines using Apache Airflow.
- Practical experience with Snowflake, covering data ingestion, data modelling, transformation and performance optimization.
- Experience using dbt to develop data models, incremental processing, reusable macros, automated tests, documentation and dependency management.
- Excellent SQL skills, with experience in complex data transformations, query optimization, reconciliation and understanding existing business logic.
- Familiarity with Git, automated testing, CI/CD, Jenkins and DevSecOps practices within a data engineering environment.
- Good Python programming skills for data processing, automation, and reusable framework components.
- Previous CBA experience, particularly with Omnia or CBA data platforms and engineering standards, is highly desirable.
Key Responsibilities
- Conduct detailed analysis of existing end-to-end data pipelines across multiple platforms, with a particular focus on Omnia - PySpark-based processing.
- Review existing source code, SQL, transformation rules, mappings and data flows to understand the current business and technical logic.
- Derive, document, and implement new transformation logic based on the analysis of existing pipelines and business requirements.
- Design, develop and maintain reliable, scalable, and reusable data pipelines using AWS, Snowflake and dbt.
- Develop batch and, where required, event-driven data processing solutions using cloud-native and distributed data technologies.
- Translate existing Omnia data processing logic into strategic cloud-based data engineering solutions.
- Build ingestion, transformation, enrichment and data integration pipelines across data lakes, data warehouses and cloud lakehouse platforms.
- Develop complex and performance-optimized SQL for data transformation, reconciliation, validation and analytical processing.
- Analyze end-to-end data flows from source capture through transformation, enrichment and downstream consumption.
- Work with data designers, product owners, and subject matter experts to translate business requirements into technical data rules and durable engineering solutions.
- Implement data quality, reconciliation and balancing controls across data pipelines.
- Troubleshoot data pipeline failures, transformation issues, data discrepancies and performance bottlenecks across multiple environments.
SYNECHRON’S DIVERSITY & INCLUSION STATEMENT
Diversity & Inclusion are fundamental to our culture, and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity, Equity, and Inclusion (DEI) initiative ‘Same Difference’ is committed to fostering an inclusive culture – promoting equality, diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger, successful businesses as a global company. We encourage applicants from across diverse backgrounds, race, ethnicities, religion, age, marital status, gender, sexual orientations, or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements, mentoring, internal mobility, learning and development programs, and more.
All employment decisions at Synechron are based on business needs, job requirements and individual qualifications, without regard to the applicant’s gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law.
Candidate Application Notice
