SEMANTIC & GENIE ENGINEER (DATABRICKS)

Accountabilities

As a Semantic & Genie Engineer, you will design, implement, and optimize data platform capabilities while integrating intelligent automation into enterprise workflows. Your responsibilities will include:

• Design and implement Unity Catalog strategies to manage data access, lineage, permissions, and governance effectively.

• Architect and maintain Delta Lake structures to support reliable, scalable, and high-performance data storage and processing.

• Conduct comprehensive data profiling and establish validation frameworks to maintain data accuracy, consistency, and integrity.

• Optimize complex SQL queries and database performance to support efficient analytics and reporting requirements.

• Develop custom applications using Python, Streamlit, and Databricks Apps to visualize insights and automate business workflows.

• Engineer advanced multi-agent systems and Databricks Genie Agents to enable intelligent data operations and automated decision-making.

• Configure and monitor Databricks Metric Views to track key performance indicators and evaluate data and system performance.

• Manage version control and collaborative development workflows using Git across codebases and configurations.

• Integrate Machine Learning Pipelines (MCP) into data workflows to enhance predictive capabilities and analytical solutions.

• Ensure seamless integration between data engineering components, semantic layers, and generative AI capabilities.

• Continuously improve platform reliability, query efficiency, data governance, and automation processes.

Requirements

The ideal candidate combines strong data engineering expertise with practical experience in Databricks, intelligent automation, and application development. You should be comfortable managing complex data environments, optimizing performance, and implementing emerging AI capabilities. Key qualifications include:

• At least 5 years of professional experience in data engineering or a related technical field.

• Strong proficiency in Python programming and advanced SQL for data manipulation, analysis, and optimization.

• Deep expertise in the Databricks ecosystem, particularly Unity Catalog, Delta Lake, and Databricks Genie.

• Strong understanding of data governance, access management, and data lineage.

• Proven experience with data quality assurance, profiling techniques, and validation methodologies.

• Demonstrated ability to optimize SQL queries and improve data pipeline performance.

• Experience building interactive dashboards and applications using Streamlit and Databricks Apps.

• Knowledge of multi-agent orchestration patterns and practical experience implementing Genie Agents and Genie Spaces.

• Familiarity with Git version control and collaborative software development practices.

• Understanding of Machine Learning Pipelines (MCP) and their integration into data workflows.

• Strong analytical and problem-solving skills, with the ability to translate complex technical requirements into reliable data solutions.

Benefits

• Competitive annual compensation of ₹15 LPA.

• Full-time employment.

• Fully remote working arrangement within India.

• Opportunity to work with advanced Databricks technologies and generative AI solutions.

• Exposure to intelligent automation, multi-agent orchestration, and modern data architecture.

• Opportunity to contribute to data governance, analytics, and enterprise-scale engineering initiatives.