This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Data Engineer (Databricks + Fabric) based in Brazil.
In this role, you will design, build, and operate production-ready data platforms for clients, transforming complex data requirements into scalable and reliable solutions. You will work hands-on with Databricks and Microsoft Fabric to develop advanced ETL/ELT pipelines, implement modern lakehouse architectures, and optimize data processing workflows. Using technologies such as PySpark, Delta Lake, Python, and SQL, you will help organizations improve data accessibility, quality, governance, and operational efficiency. You will also implement Medallion architecture principles and establish robust data governance practices through Unity Catalog. Working closely with clients and cross-functional teams, you will identify technical challenges, recommend improvements, and contribute to new data engineering initiatives. This is an opportunity to take ownership of impactful production environments while working remotely in an international technology consulting environment.
Accountabilities:
- Design, develop, deploy, and maintain production-grade data platforms for clients, taking hands-on responsibility for implementation and ongoing operations.
- Build and optimize scalable ETL/ELT pipelines using Databricks and Microsoft Fabric to support complex data integration and transformation requirements.
- Design and implement modern lakehouse architectures based on the Medallion framework, organizing data into Bronze, Silver, and Gold layers using Delta Lake.
- Develop and maintain data processing solutions using PySpark, Python, SQL, Databricks Workflows, and Microsoft Fabric components.
- Implement and manage data governance through Unity Catalog, including permissions, access controls, data lineage, and data quality practices.
- Work across the Microsoft Fabric ecosystem, leveraging Lakehouse, OneLake, Data Pipelines, Notebooks, Dataflows Gen2, and Warehouse capabilities.
- Establish and automate deployment processes using CI/CD pipelines and Infrastructure as Code (IaC) practices.
- Ensure data platforms are scalable, secure, reliable, and optimized for performance and maintainability.
- Gather, analyze, and translate business and technical requirements in collaboration with clients and internal teams.
- Identify opportunities to improve existing data architectures, infrastructure, and engineering workflows.
- Proactively recognize potential client needs and recommend technical enhancements that may lead to additional projects and engagements.
- Collaborate with cross-functional teams to ensure successful project delivery and alignment with business objectives.
Requirements:
- Proven hands-on professional experience implementing and managing Databricks solutions in production environments. Proofs of concept or theoretical knowledge alone are not sufficient.
- Strong experience with PySpark, Delta Lake, Databricks Workflows, and Unity Catalog, including catalog management, permissions, and data lineage.
- Demonstrated experience designing and implementing Medallion architectures (Bronze, Silver, and Gold) in production data environments.
- Proven hands-on experience with Microsoft Fabric across its full platform, including Lakehouse, OneLake, Data Pipelines, Notebooks, Dataflows Gen2, and Warehouse. Experience limited to proofs of concept is not sufficient.
- Advanced proficiency in SQL and Python, with the ability to develop efficient data transformation and processing solutions.
- Strong understanding of modern data engineering principles, lakehouse architectures, data governance, and production data platform management.
- Experience building and maintaining automated deployment workflows using CI/CD and Infrastructure as Code.
- Current Microsoft DP-900 or DP-700 certification, with a commitment to obtain the remaining certification within the first 60 days of employment. Certification status should be verifiable through Microsoft Learn.
- Professional English proficiency at B2 level or above, with the ability to communicate effectively with international clients and technical teams.
- Fluent Spanish, with the ability to participate confidently in professional discussions and client interactions.
- Strong analytical and problem-solving skills, combined with a practical, hands-on approach to technical implementation.
- Excellent communication and collaboration skills, with the ability to gather requirements, understand client needs, and translate them into effective technical solutions.
- Proactive mindset, attention to detail, and the ability to identify opportunities for continuous improvement.
Benefits:
- Fully remote work: Work remotely from Brazil, with a location-independent working environment.
- International exposure: Collaborate with clients and technical teams across Latin America and potentially other international markets.
- Advanced technology stack: Gain hands-on experience with Databricks, Microsoft Fabric, Delta Lake, PySpark, and modern cloud data engineering technologies.
- Technical ownership: Take direct responsibility for building, deploying, and operating production data platforms.
- Professional development: Strengthen your expertise in modern lakehouse architectures, data governance, automation, and enterprise data solutions.
- Client-facing experience: Work closely with clients, understand their business challenges, and contribute to impactful technology initiatives.
- Continuous learning: Develop your skills through practical exposure to evolving data engineering tools and methodologies.
- Employment type: Remote engagement; specific contractual arrangements and compensation details to be confirmed during the recruitment process.
- Location: Latin America, with Brazil as the target location.
How Jobgether works:
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
#LI-CL1
