Overview
We are looking for a Senior Data Engineer with strong 7-10 year hands-on experience in Azure and Microsoft Fabric to help modernize and migrate data products from Azure Synapse to Microsoft Fabric. The ideal candidate will have experience with Azure Data Factory, notebooks, Spark/PySpark, and building scalable data pipelines that support analytics and business reporting needs.
Responsibilities
- Migrate existing data products, code, and workloads from Azure Synapse to Microsoft Fabric
- Design, build, and support scalable data pipelines and data processing workflows
- Develop and enhance data products that support business and analytics use cases
- Build solutions using Azure Data Factory, notebooks, Spark/PySpark, SQL, and Python
- Refactor pandas-based transformation logic into scalable PySpark and Python implementations
- Work with structured and semi-structured data to enable downstream consumption
- Implement full, delta, and CDC-based ingestion patterns from multiple data sources
- Apply data warehousing concepts and medallion architecture principles in solution design
- Validate migrated code and data outputs to ensure accuracy, consistency, and completeness
- Perform data quality checks, performance tuning, monitoring, and issue resolution
- Troubleshoot pipeline, code, and environment issues across Azure-based platforms
- Collaborate with data analysts, engineers, and business stakeholders to gather requirements and deliver solutions
- Document technical designs, migration approaches, and data product logic
- Support code reviews, testing, release, and deployment activities
Required Technical Skills
- 7-10 years of strong hands-on experience with Microsoft Fabric and Azure fundamentals
- Strong hands-on experience with Azure Synapse and Azure Data Factory
- Strong hands-on experience with Spark and PySpark
- Proficiency in Python and SQL
- Experience with pandas and refactoring to PySpark/Spark-based implementations
- Strong experience with Git and Azure DevOps, including pull requests, CI/CD, and deployment automation
- Experience building reusable engineering patterns and shared libraries
- Solid understanding of Azure storage, compute, security, and identity
- Strong knowledge of data modeling, data transformation, and warehousing concepts
- Experience with performance optimization and tuning
Technologies, Platforms, and Tools
- Microsoft Fabric
- Microsoft Azure Fundamentals
- Azure Data Factory
- Preferred knowledge on source systems such as SAP ECC, S/4 HANA, Oracle ERP, Salesforce
- Worked with Delta, Parquet, JSON, and CSV file formats
- Azure DevOps, Terraform
