IN-Senior Associate_Azure Data Engineer_Data & Analytics_Advisory_Pune

Responsibilities: 

• Analyses current business practices, processes, and procedures as well as identifying future business opportunities for leveraging Microsoft Azure Data & Analytics Services. 

• Provide technical leadership and thought leadership as a senior member of the Analytics Practice in areas such as data access & ingestion, data processing, data integration, data modeling, database design & implementation, data visualization, and advanced analytics. 

• Engage and collaborate with customers to understand business requirements/use cases and translate them into detailed technical specifications. 

• Develop best practices including reusable code, libraries, patterns, and consumable frameworks for cloud-based data warehousing and ETL. 

• Maintain best practice standards for the development or cloud-based data warehouse solutioning including naming standards. 

• Designing and implementing highly performant data pipelines from multiple sources using Apache Spark and/or Azure Databricks 

• Integrating the end-to-end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is always maintained 

• Working with other members of the project team to support delivery of additional project components (API interfaces) 

• Evaluating the performance and applicability of multiple tools against customer requirements 

• Working within an Agile delivery / DevOps methodology to deliver proof of concept and production implementation in iterative sprints. 

• Integrate Databricks with other technologies (Ingestion tools, Visualization tools). 

• Proven experience working as a data engineer 

• Highly proficient in using the spark framework (python and/or Scala) 

• Extensive knowledge of Data Warehousing concepts, strategies, methodologies. 

• Direct experience of building data pipelines using Azure Data Factory and Apache Spark (preferably in Databricks). 

• Hands on experience designing and delivering solutions using Azure including Azure Storage, Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics 

• Experience in designing and hands-on development in cloud-based analytics solutions. 

• Expert level understanding on Azure Data Factory, Azure Synapse, Azure SQL, Azure Data Lake, and Azure App Service is required. 

• Designing and building of data pipelines using API ingestion and Streaming ingestion methods. 

• Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code is essential. 

• Thorough understanding of Azure Cloud Infrastructure offerings. 

• Strong experience in common data warehouse modeling principles including Kimball. 

• Working knowledge of Python is desirable 

• Experience developing security models. 

• Databricks & Azure Big Data Architecture Certification would be plus 

Mandatory skill sets: 

ADE, ADB, ADF 

Preferred skill sets: 

ADE, ADB, ADF 

Years of experience required: 

3-8 Years 

Education qualification: 

BE, B.Tech, MCA, M.Tech