Want to know if this job is worth applying to?
Brasil
Remote only
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Data Engineer – Azure Databricks based in Brazil.
We are seeking a highly experienced Senior Data Engineer with over eight years of experience in data engineering to design, develop, optimize, and maintain large-scale enterprise data solutions. In this remote role, you will work within a global environment, leveraging the Microsoft Azure ecosystem to deliver reliable, scalable, and high-performance data platforms. You will build and optimize ETL/ELT pipelines, integrate data from multiple sources, and ensure data quality for analytics and business needs. The role combines hands-on engineering, performance optimization, and production support, including troubleshooting critical data workflows. Your technical expertise and independent approach will help improve operational efficiency and ensure dependable data processing across complex environments. Advanced English proficiency is essential for effective collaboration with international teams.
Design, develop, maintain, and optimize data pipelines using Azure Databricks, PySpark, Azure Data Factory (ADF), Python, and Azure Data Lake.
Build and support ETL/ELT processes to integrate enterprise data and meet analytical and business requirements.
Perform data ingestion, transformation, validation, and reconciliation across multiple source systems.
Conduct data profiling and quality assessments to identify inconsistencies, resolve issues, and ensure data accuracy and reliability.
Develop, write, and optimize complex SQL queries to improve data processing performance.
Perform performance tuning and troubleshoot large-scale data workloads to ensure efficiency, scalability, and reliability.
Provide Level 2 production support for incidents involving Azure Data Factory pipelines, Databricks jobs, and data processing workflows.
Monitor and maintain production pipelines, ensuring availability, operational stability, and efficient resource utilization.
Support deployment and release processes across development, testing, and production environments.
Manage competing priorities across multiple projects, working independently while meeting deadlines and maintaining high delivery standards.
More than eight years of professional experience in data engineering, including the development and maintenance of ETL processes.
Strong hands-on experience with Azure Databricks and Azure Data Factory (ADF).
Extensive knowledge of ETL/ELT development, data transformation, and enterprise data integration.
Advanced SQL skills, including complex query development, optimization, and performance tuning.
Practical experience with Python and PySpark for data processing and engineering.
Experience working with Azure Data Lake and integrating data from multiple enterprise systems.
Proven ability to perform data profiling, validation, reconciliation, and quality assurance.
Experience monitoring, maintaining, and troubleshooting data pipelines in production environments.
Advanced English proficiency, both written and spoken, to collaborate effectively in a global working environment.
Strong analytical and investigative skills, with a structured, solution-oriented approach to complex technical challenges.
Ability to work autonomously, organize activities, manage competing demands, and prioritize effectively.
Excellent verbal and written communication skills, with the ability to collaborate across multidisciplinary teams.
Commitment to code quality, performance, reliability, and timely delivery.
Nice to have:
Experience implementing CI/CD pipelines and DevOps practices.
Familiarity with Git, Azure DevOps, or similar version control and deployment tools.
Knowledge of cloud monitoring, logging, and observability solutions.
Fully remote work arrangement.
CLT employment contract in Brazil.
Opportunity to contribute to large-scale enterprise data engineering initiatives in a global environment.
Salary, healthcare coverage, paid time off, and additional benefits are not specified in the provided job description.