Senior Data Engineer - Data Platform

Accountabilities

• Architect and build scalable transformation, warehouse, and data-processing systems capable of handling billions of records across multiple markets.
• Design production patterns for LLM-powered extraction, enrichment, entity resolution, and semantic validation, including structured outputs, retries, fallbacks, and reusable abstractions.
• Develop evaluation frameworks with labelled datasets, scoring mechanisms, judge calibration, regression suites, and measurable precision/recall targets for AI-powered data workflows.
• Establish clear standards for when deterministic rules, SQL, data contracts, and dbt tests should be used versus when LLM-based validation is appropriate.
• Own warehouse architecture and optimization, including dbt transformation layers, Snowflake performance, clustering, materialization strategies, access controls, and cost management.
• Build reliable orchestration and infrastructure patterns using Airflow and AWS, with appropriate recovery, dependency management, scheduling, and infrastructure-as-code practices.
• Implement observability across AI and data pipelines by tracking model versions, prompts, costs, latency, decisions, and operational alerts.
• Develop matching, deduplication, embedding, and retrieval solutions to support entity resolution across company and people data.
• Provide technical leadership in architecture discussions with data quality, sourcing, analytics, product, and other cross-functional teams.
• Balance technical excellence with product impact, making informed decisions about when to optimize, refactor, redesign, or ship.

Requirements

• 5+ years of experience building and owning production-grade data platforms in business-critical environments, preferably with end-to-end architectural responsibility.
• Proven experience working with billions of rows and designing systems, pipelines, and quality controls that remain reliable and performant at significant scale.
• Demonstrated experience deploying LLMs within production data pipelines for extraction, enrichment, or validation, including structured outputs, versioned prompts, and labelled evaluation datasets.
• Experience designing evaluation or test harnesses used by other engineers, including scorers, regression suites, labelled datasets, and judge-calibration approaches.
• Strong architectural judgment when deciding between deterministic data-quality approaches and LLM-based semantic evaluation.
• Expert-level Python and SQL skills, with strong knowledge of performance optimization, concurrency, and large-scale data transformations.
• Extensive experience with dbt and Snowflake, including modular transformation architecture, business logic, query optimization, clustering, RBAC, and cost management.
• Extensive production experience with Airflow, including orchestration, dependencies, recovery strategies, and cost-aware scheduling.
• Solid AWS experience across services such as S3, Lambda, Glue, ECS, and RDS.
• Practical experience using AI coding tools such as Claude Code, Cursor, or equivalent, with evidence of real production work delivered through these tools.
• Strong product mindset and engineering judgment, with an ability to connect platform decisions to data quality and customer outcomes.
• Experience with evaluation frameworks, LLM tracing, embeddings, vector search, fuzzy matching, Spark/PySpark, streaming technologies, B2B data, or data privacy and compliance would be highly valued.
• Experience in startup or scale-up environments where you have established technical standards and architectures rather than simply inherited them is a plus.

Benefits

• Fully remote position with the flexibility to work from India.
• Opportunity to work on greenfield data and AI infrastructure at significant scale.
• High technical ownership, with the ability to shape architecture, engineering standards, and production practices.
• Exposure to frontier challenges involving LLM evaluation, AI-powered data pipelines, entity resolution, observability, and large-scale data quality.
• Lean, senior engineering environment with minimal hierarchy and strong individual ownership.
• Fast-paced delivery culture with frequent releases and end-to-end responsibility for the technology stack.
• Competitive base salary with meaningful equity participation.
• Opportunity to build systems that directly influence the quality and reliability of customer-facing data.
• Inclusive environment that values diverse perspectives, collaboration, professional growth, and technical excellence.