IN-Sr Associate_Databricks Engineer _D&A _Advisory _Mumbai

Responsibilities: 

Minimum 5 years of professional experience with working knowledge in a Data and Analytics role with a Global organization   

-5 to 8 years of experience in working with Databricks tech stacks   

-Experience in leading development of Data and Analytics products, from Requirement Gathering State to Driving User Adoption   

-Develop and optimize ETL processes using Databricks and related tools like Apache Spark   

-Design efficient data processing systems and pipelines using Databricks, APIs, and other cloud services   

-Candidate with strong data transformation experience on Unity Catalog, Delta Tables, DLT   

-Strong proficiency in writing and optimizing SQL queries and working with databases   

-Ability to acquire specialized domain knowledge required to be more effective in all work activities   

-BI & Data-warehousing concepts are a must.   

-Design, develop, and maintain scalable ETL/ELT pipelines using PySpark on Databricks.   

-Ingest and transform data from multiple structured and unstructured sources including cloud storage (Azure Data Lake, AWS S3, etc.).   

-Optimize Spark jobs for performance and cost-efficiency on the Databricks platform.   

-Collaborate with data scientists, analysts, and stakeholders to understand data requirements and deliver high-quality solutions.   

-Implement best practices in data engineering, including modular coding, unit testing, and version control (e.g., Git).   

-Automate data workflows and schedule jobs using Databricks Workflows or external orchestration tools (e.g., Airflow, Azure Data Factory).   

-Ensure data quality, integrity, and governance in all data pipelines.   

-Participate in code reviews, performance tuning, and system monitoring.   

-Document solutions, processes, and configurations.   

Mandatory skill sets: 

Pyspark, Databricks 

Preferred skill sets: 

Databricks, Apache Spark 

Years of experience required: 

7 – 10 yrs 

Education qualification: 

Btech/MBA/MCA