Want to know if this job is worth applying to?
Brasil
Remote only
This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Tech Lead – Data Engineering based in Brazil.
In this senior, hands-on role, you will lead the development and evolution of the data layer powering an enterprise artificial intelligence product within the mobility infrastructure sector. You will design and maintain scalable data pipelines, prepare reliable datasets for generative AI services, and connect data platforms with APIs and analytical applications. Working primarily with Databricks, PySpark, and SQL, you will help shape modern data architecture while ensuring performance, quality, security, and governance. You will collaborate closely with product teams, guide technical decisions, and promote engineering best practices through reusable frameworks and incremental delivery. This is an opportunity to combine technical leadership with practical data engineering expertise in a remote, AI-focused environment.
Understand the product, its users, and underlying data sources, taking technical ownership of requirements and delivery.
Build, maintain, and optimize data pipelines using PySpark and SQL within Databricks.
Implement and evolve medallion architecture across landing, bronze, silver, and gold layers.
Work with Delta Lake, Unity Catalog, and Databricks Workflows to support reliable and scalable data operations.
Prepare and manage datasets for generative AI services, including embeddings, vector indexes, data quality, and versioning.
Integrate data with AI services and Databricks Genie.
Consume and expose data through APIs, ensuring appropriate security, monitoring, and observability.
Design and maintain the gold data layer, making trusted datasets available to the product, Power BI, and analytical applications.
Develop reusable engineering frameworks, automated validations, and data quality controls.
Apply software delivery best practices, including Git versioning, CI/CD, automated testing, code reviews, and technical documentation.
Collaborate with product teams through short development cycles, prioritizing incremental and valuable deliveries.
Support technical estimates, guide squad members, and share engineering standards and best practices.
Work with Data Governance teams to implement data classification, anonymization, lineage, and access controls.
Solid professional experience in data engineering, including acting as a technical reference or lead within an engineering squad.
Strong hands-on experience with Databricks, PySpark, and Spark SQL in production environments.
Practical knowledge of medallion architecture, including landing, bronze, silver, and gold layers.
Experience with Delta Lake, Unity Catalog, and Databricks Workflows.
Experience preparing and managing data for generative AI applications, including embeddings, vector indexes, data quality, and version control.
Ability to integrate data with APIs and external services while maintaining security and observability.
Strong data modeling skills, particularly across silver and gold analytical layers.
Familiarity with Git, CI/CD pipelines, code reviews, automated testing, and data quality validation.
Experience working in agile product teams and delivering improvements incrementally.
Strong communication skills, with the ability to collaborate effectively with both technical and business stakeholders.
Degree or formal education in Information Technology or a related field.
Nice to have: Experience with Databricks Genie and enterprise generative AI implementation standards.
Nice to have: Knowledge of data governance for AI applications and Brazilian data protection requirements (LGPD), including classification, anonymization, lineage, and access management.
Nice to have: Experience with Oracle and GoldenGate/CDC replication, SAP data extraction through Open Hub, SQL Server, or Microsoft Graph (SharePoint/Microsoft 365).
Nice to have: Familiarity with Power BI, service identities, secrets management solutions, and technical documentation using Confluence.
Meal and/or food allowance.
Medical and dental insurance.
Health coverage for pets.
Home office allowance for remote employees.
Welcome kit during onboarding.
Life insurance.
Access to psychological and nutritional support through Zenklub.
TotalPass wellness and fitness benefits.
Certification and continuous learning programs with partners including Adobe, Anthropic, AWS, Google, Microsoft, and Salesforce.
Professional development partnerships and discounts.
Access to UOL digital content.
Maternity welcome kit.
Extended maternity and paternity leave.
Employee referral program.
Fully remote, full-time employment.
Opportunity to work with advanced AI, generative AI, and modern data engineering technologies.