Senior Cloud Data Engineer

Senior Cloud Data Engineer

Req number:

R7366

Employment type:

Full time

Worksite flexibility:

Remote

Who we are

CAI is a global services firm with over 9,000 associates worldwide and a yearly revenue of $1.3 billion+. We have over 40 years of excellence in uniting talent and technology to power the possible for our clients, colleagues, and communities. As a privately held company, we have the freedom and focus to do what is right—whatever it takes. Our tailor-made solutions create lasting results across the public and commercial sectors, and we are trailblazers in bringing neurodiversity to the enterprise.

Job Summary

We are looking for a motivated Senior Cloud Data Engineer to join our dynamic team who will be responsible for managing the existing cloud data platform to make it more scalable, reliable, and cost efficient. You will also contribute to additional projects, leveraging the existing architecture where applicable and adopting newer technologies as needed and looking for your next career move, apply now.

Job Description

We are seeking a Senior Cloud Data Engineer who will manage the existing cloud data platform to make it more scalable, reliable, and cost efficient. This position will be Full-time and Remote position.

What You’ll Do

• Analyze and understand existing data warehouse implementations to support migration and consolidation efforts.

• Reverse-engineer legacy stored procedures (PL/SQL, SQL) and translate business logic into scalable Spark SQL code within Databricks notebooks.

• Design and develop data lake solutions on AWS using S3 and Delta Lake architecture, leveraging Databricks for processing and transformation.

• Build and maintain robust data pipelines using ETL tools with ingestion into S3 and processing in Databricks.

• Collaborate with data architects to implement ingestion and transformation frameworks aligned with enterprise standards.

• Evaluate and optimize data models (Star, Snowflake, Flattened) for performance and scalability in the new platform.

• Document ETL processes, data flows, and transformation logic to ensure transparency and maintainability.

• Perform foundational data administration tasks including job scheduling, error troubleshooting, performance tuning, and backup coordination.

• Work closely with cross-functional teams to ensure smooth transition and integration of data sources into the unified platform.

• Participate in Agile ceremonies and contribute to sprint planning, retrospectives, and backlog grooming

• Triage, debug and fix technical issues related to Data Lakes

• Maintain and Manage Code repositories like Git

What You'll Need

Required:

• 8–12 years of experience in data engineering, data warehousing, or big data platform development.

• 5+ years of hands-on experience with Databricks including Spark, Delta Lake, and performance optimization.

• 4+ years of experience designing and implementing cloud-based data lake or lakehouse architectures on Databricks.

• Strong expertise in Spark technologies including Spark SQL, PySpark, or Scala.

• Advanced SQL and PL/SQL skills with the ability to interpret and refactor legacy stored procedures.

• Strong understanding of data modeling techniques including Star Schema, Snowflake, and modern Lakehouse modeling approaches.

• Proficiency in at least one programming language (Python, Scala, Java).

• Hands-on experience with data modeling and warehouse design principles.

• Experience designing enterprise-scale ETL/ELT pipelines and ingestion frameworks.

• Strong understanding of performance tuning, partitioning strategies, and cost optimization for large-scale data platforms.

• Experience implementing CI/CD pipelines and DevOps practices for data engineering workflows.

• Bachelor’s degree in computer science, Information Technology, Data Engineering, or related field.

• Experience working in Agile environments and contributing to iterative development cycles. Experience working on Agile projects and Agile methodology in general

Preferred:

• Databricks Professional or Associate Cloud Certification.

• Experience with enterprise data governance, metadata management, and data catalog platforms.

Reasonable accommodation statement

If you require a reasonable accommodation in completing this application, interviewing, completing any pre-employment testing, or otherwise participating in the employment selection process, please direct your inquiries to application.accommodations@cai.io or (888) 824 – 8111.