This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Software Engineer, AI Training based in Canada.
This is a unique opportunity for an experienced software engineer to help evaluate the quality of interactions with modern AI coding agents.
Rather than writing production code, you will assess whether AI systems demonstrate the judgment, reasoning, and communication expected from strong engineers.
You will evaluate coding interactions end to end, from technical outputs to explanations, reasoning, and overall developer experience.
Your feedback will help define what high-quality AI-assisted development should look like in real-world scenarios.
The role requires strong engineering taste, rigorous judgment, and the ability to clearly explain what works, what does not, and why.
You will work remotely on flexible, project-based engagements with opportunities for ongoing assignments based on performance.
Accountabilities:
- Evaluate AI-generated coding interactions from end to end, assessing whether responses are useful, accurate at a high level, and aligned with strong engineering practices.
- Assess the quality of AI reasoning, explanations, preambles, and guidance rather than focusing solely on whether generated code is technically correct.
- Distinguish between different levels of response quality and articulate what separates an acceptable interaction from an exceptional one.
- Provide clear, direct, and well-reasoned written feedback on what worked, what failed, and what felt misleading, ineffective, or inconsistent with experienced engineering judgment.
- Help establish standards for what excellent AI-assisted development interactions should look like across tools such as Cursor, Codex, and Claude Code.
- Evaluate whether AI interactions demonstrate sound engineering judgment, build user trust, and guide developers effectively rather than simply producing technically valid output.
- Record concise video explanations when needed to communicate evaluation findings and reasoning clearly.
Requirements
- Senior, Staff, Principal-level software engineering experience, or equivalent depth of professional engineering experience.
- Strong background in TypeScript/JavaScript or Python, with a deep understanding of modern software engineering practices.
- Hands-on experience with at least one AI-assisted coding tool such as OpenAI Codex, Claude Code, or Cursor.
- Strong familiarity with modern AI-assisted development workflows and the ways developers interact with coding agents.
- Ability to evaluate code and engineering decisions at a high level without necessarily executing the code or reviewing every individual line.
- Strong written and spoken English at B2 level or above, with the ability to provide detailed written feedback and record short video explanations.
- Comfortable making subjective but rigorous judgments and communicating direct, well-supported opinions.
- High standards for engineering quality, developer experience, technical reasoning, and effective communication.
Benefits
- Contract engagement with a fully remote work model, available worldwide.
- Compensation of $100–$200/hour, depending on experience and location.
- Flexible workload of 10–20 hours per week.
- Ongoing project opportunities, with individual projects typically lasting from approximately two weeks to a few months.
- Potential for additional projects based on successful performance and available opportunities.
- Opportunity to contribute to the evaluation and development of modern AI-assisted software engineering workflows.
- Start as soon as the take-home evaluation is completed and a project opening becomes available.
- Selection process consists of one take-home evaluation exercise with a recorded Loom walkthrough; no interview is required.
How Jobgether works:
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.
#LI-CL1
