Forward Deployed Data Engineer IV, Databricks

Accountabilities:

• Embed directly with utility clients to design and build production data platforms covering ingestion, transformation, orchestration, quality, governance, and serving using Databricks, Spark, Python, SQL, and AWS.
• Own the technical architecture of client engagements from discovery through deployment, clearly communicating and defending architectural decisions with client architects, security teams, and platform owners.
• Lead technical discovery independently, challenge assumptions when appropriate, identify underlying business needs, and recommend solutions that address the actual problem rather than simply the requested implementation.
• Deliver useful working systems early in engagements, typically within the first few weeks, and progressively harden them for production based on real-world usage and operational requirements.
• Lead migrations from legacy data warehouses and on-premises environments to modern cloud platforms, including identifying and addressing undocumented dependencies.
• Develop robust data models using dimensional, lakehouse, medallion, graph, or semantic approaches according to the requirements of each domain and use case.
• Integrate utility systems such as asset and work management platforms, historians and SCADA systems, customer information systems, advanced metering infrastructure, GIS, and ERP platforms through APIs, change data capture, file-based integrations, and other available mechanisms.
• Build observable and operable pipelines with lineage, data quality controls, alerting, monitoring, and runbooks that client engineering teams can effectively maintain.
• Develop operator-facing applications such as data quality dashboards, reconciliation interfaces, review and correction tools, and self-service data applications, using Databricks Apps or comparable technologies where appropriate.
• Manage engagement scope, timelines, expectations, and delivery risks against defined statements of work, escalating issues early and maintaining alignment with stakeholders.
• Contribute reusable ingestion frameworks, engineering patterns, and reference architectures that can be applied across client engagements while sharing field insights with internal product and engineering teams.
• Apply appropriate data governance, security, privacy, lineage, and retention controls within the regulatory and operational context of the utility industry.

Requirements:

• Hold a Bachelor’s degree in Computer Science, Information Technology, or a related field, with a Master’s degree in a relevant STEM discipline considered an advantage.
• Bring 5+ years of experience in data engineering, data platforms, or analytics engineering, including ownership of at least one production data platform.
• Demonstrate expert proficiency in Python, SQL, Databricks, and Spark, including an understanding of Spark runtime behavior and the ability to diagnose performance issues without simply increasing compute resources.
• Have experience building and operating cloud-based data pipelines in AWS, with working knowledge of an additional cloud ecosystem considered a plus.
• Demonstrate strong knowledge of core data engineering challenges, including incremental processing, full rebuilds, idempotency, late-arriving data, deduplication, backfill strategies, and schema evolution.
• Bring experience designing both batch and streaming pipelines, with the judgment to determine which architecture best addresses the underlying business and technical requirements.
• Have practical knowledge of data governance, security, privacy, lineage, and data quality practices, together with proficiency in Git, Docker, CI/CD tooling, and at least one modern orchestration framework.
• Demonstrate experience using AI coding assistants and agentic development tools to rapidly prototype, test, and iterate on production-oriented software; this is a particularly important capability for the role.
• Be capable of owning the full system lifecycle, from technical discovery and architecture through development, deployment, and post-go-live support.
• Have experience working directly with enterprise clients and navigating competing priorities among technical contributors, business stakeholders, and senior executives.
• Demonstrate strong project delivery skills, including managing technical scope, timelines, risks, and measurable outcomes in fast-moving engagements.
• Communicate clearly through written documentation and technical whiteboarding, and remain effective when working with ambiguity, incomplete data, and changing objectives.
• Experience in the energy, utility, or another asset-intensive industry is valuable, particularly familiarity with the operational systems, data environments, and constraints common to these sectors.
• Experience with graph databases and knowledge graph modeling, Kafka or equivalent streaming platforms, change data capture tooling, ML/AI feature pipelines, Databricks Apps, React, Streamlit, Dash, REST or GraphQL APIs, or Databricks performance optimization is advantageous.
• A Databricks certification is considered a plus.
• Must be authorized to work for any employer in the United States; employment visa sponsorship or transfer of sponsorship is not available for this role.

Benefits:

• Budgeted salary of $175,000–$200,000 USD plus an annual bonus, with actual compensation adjusted based on experience.
• Medical, dental, and vision insurance options.
• Company-paid life insurance.
• Company-paid short-term and long-term disability insurance.
• Medical and dependent-care flexible spending accounts.
• Paid parental leave.
• Flexible Time Off (FTO), subject to manager approval and business coverage requirements.
• 401(k) plan with a 3% employer match.
• Remote-based work within the United States.
• Significant client travel, generally 30–50% depending on the engagement, including potential extended on-site periods during discovery, deployment, and go-live.
• Opportunities to develop technical expertise while working on meaningful data and technology challenges within the utility and energy sector.