Databricks Technical Lead

What You’ll Do

• Own the AI data product roadmap and translate business priorities into a sequenced backlog of reusable data products with defined ownership, service levels, and lifecycle controls

• Partner with dashboard, analytics, and AI delivery teams to identify reporting assets that should evolve into governed AI-ready data products

• Establish data product charters defining purpose, consumers, source ownership, data contracts, access models, and retention requirements

• Implement table-, column-, and row-level access standards, classification, audit trails, and data handling controls using Databricks Unity Catalog and AWS services

• Lead the design and operation of incremental, observable, and cost-efficient data ingestion and curation pipelines across enterprise data sources

• Create and maintain versioned, reproducible training datasets, evaluation datasets, inference inputs, feature tables, and monitoring datasets with clear metadata and documentation

• Define and implement AI-specific data quality standards including completeness, consistency, timeliness, uniqueness, referential integrity, distribution stability, and drift sensitivity

• Establish automated quality controls and enforce readiness gates that prevent model training, deployment, or production promotion when critical data quality requirements are not met

• Develop operational runbooks for recovery, backfills, reprocessing, incident response, and escalation management

• Participate in operational reviews and drive root-cause analysis and corrective actions to improve platform reliability

• Optimize storage, compute utilization, partitioning, scheduling, and overall platform cost efficiency

• Approve data readiness gates for model training, evaluation, deployment, and significant production inference input changes

• Escalate data ownership, quality, access, and cost issues that introduce risk or impede business outcomes

• Drive dashboard-to-data-product transformations and establish governed, reusable data products that support critical AI use cases

• Deliver quality and cost scorecards that provide transparency into data health, reliability, usage, and platform cost drivers

What You’ll Need

Required:

• Bachelor’s degree or higher in Computer Science, Data Science, AI/ML, Applied Mathematics, Engineering, or a related field

• 8+ years of experience in data engineering, data management, AI data foundations, or related technology leadership roles

• Demonstrated ownership of data products or mission-critical data pipelines delivering measurable business impact

• Hands-on experience with AWS, Databricks, Delta Lake, Unity Catalog, data lineage, observability, and production support

• Experience defining and managing data contracts, quality gates, service levels, and data access models

• Strong stakeholder management skills with experience partnering across business leaders, product owners, data and AI engineering teams, security, and platform organizations

• Ability to operate at a senior level, challenge incomplete requirements, evaluate trade-offs, and drive decisions in ambiguous environments

• Strong communication skills with the ability to simplify complex technical concepts for business stakeholders

• Proven ownership mindset with accountability for business and technical outcomes

• Strong judgment balancing speed, quality, security, cost, and maintainability

• Excellent problem-solving skills and ability to adapt in fast-paced, evolving environments

• Commitment to data quality, operational excellence, documentation, and continuous improvement

Preferred:

• Databricks Data Engineer Professional certification

• Databricks Machine Learning Professional certification

• AWS Certified Data Analytics certification

• AWS Solutions Architect Associate certification

• AWS Solutions Architect Professional certification

• Data governance, data management, product ownership, or agile delivery certifications

Physical Demands

• Ability to safely and successfully perform the essential job functions

• Sedentary work that involves sitting or remaining stationary most of the time with occasional need to move around the office to attend meetings, etc.

• Ability to conduct repetitive tasks on a computer, utilizing a mouse, keyboard, and monitor