Want to know if this job is worth applying to?
Township of Langley, British Columbia, Canada
Onsite
Come join Pattison Food Group Ltd., a Jim Pattison business, and Canada’s largest Western-based provider of food and health products.
Pattison Food Group Ltd. consists of Save-On-Foods and other well-known grocery banners. With nearly 300 retail locations, we proudly employ more than 30,000 team members.
We have an exciting opportunity for a full-time QA Data Engineer to join our IT team in Langley, BC.
Reporting to the Manager, Data & Analytics you will work closely with the Data Scientists, Data Engineers, and Machine Learning Engineers to help extract value from our data by supporting validation of data processes across ingestion, transformation, and delivery, ensuring robustness for machine learning and analytics use cases.
In this role you will ensure the quality, reliability, and accuracy of data pipelines powering analytics, reporting, and machine learning applications. You will design and automate data quality tests, build validation frameworks, and collaborate closely with Data Engineers and Data Scientists.
You will be responsible for:
Building test and validation cases on data engineering and occasionally data science code changes to ensure standards and quality is maintained.
Designing, monitoring, and maintaining QA reports, KPIs & quality trends for the internal data systems.
Creating test automation frameworks that are used to validate data engineering pipelines throughout the end-to-end lifecycles of large-scale, distributed machine learning systems using the latest open source and cloud technologies.
Scripting and automating tests and validating the data pipelines used for machine learning models, dashboards, and reports.
Defining data quality and unit testing best practices, and ensuring all work meets those requirements.
Developing scalable tools and services for conducting peer reviews and unit testing.
Collaborating with data scientists and data engineers to test and deploy data pipelines and data solutions at scale.
Identifying project requirements, tracking, and communicating project progress with peers, and driving projects to completion.
Sharing knowledge through the creation of development, support documentation and presentations to peers and senior leaders.
Identifying and addressing unexpected pipeline production issues related to data tables and pipelines.
Bridging the gap between the project team and production team to ensure smooth roll-out changes and transition of ownership.
You have:
A bachelor’s degree in computer science, applied mathematics, or engineering.
3+ years of experience in conducting Quality Assurance.
Experience in QA for data pipelines within large-scale data solutions.
Experience in conducting thorough peer reviews for code changes and defining and implementing unit tests.
Experience building systems with scalable data processing technologies using Spark, Python, SQL.
Experience with data modeling concepts and best practices.
Experience in CI/CD and be able to follow standard workspace branching practices.
Exposure to Databricks and associated tools like ML Flow and Delta Lake.
Familiarity with data-oriented workflow orchestration frameworks (Databricks Jobs, Azure Data Factory, Airflow etc.)
Experience developing in cloud computing environments (Azure, AWS, GCloud, etc.)
Commitment to maintaining high code quality, automated testing, and other best practices in data engineering.
Experience working with cloud computing and database systems.
Fluency in python and potentially other programming languages (Scala, etc..).
Experience with version control tools such as Git and Git workflows.
You are:
Collaborative and innovative.
Motivated to identify and address potential issues.
Equipped with strong analytic skills for uncovering hidden bugs and issues in code.
A critical and analytical thinker.
What we offer:
In addition to a highly competitive salary, we offer a great range of benefits, a company pension plan, free parking, exclusive team member offers, and opportunities for development and career progression.
If you’re looking to join a team that puts people first, you’re in the right place. Apply online today!
IND6
Compensation Details:
$94,552.00 - $118,190.00The compensation offered for this position will take into consideration location, education, skills, experience and other factors.