AI Test Engineer II

Accountabilities:

Test Automation and Engineering: Design, develop, and maintain automated test suites covering API, user interface, security, and performance testing. Build reliable, scalable, and maintainable automation frameworks that support continuous product validation.
End-to-End and Integration Testing: Develop end-to-end and integration tests using Playwright or comparable modern automation frameworks. Validate application workflows, system interactions, and integrations across distributed environments.
API Testing: Create and maintain automated tests for REST and GraphQL services, validating API contracts, error handling, edge cases, and expected system behavior.
AI-Assisted Testing: Leverage AI-powered coding and testing tools, such as Claude Code, GitHub Copilot, and similar solutions, to generate, refactor, and scale automated tests. Establish best practices for using AI assistants safely and effectively throughout the testing lifecycle.
Testing Innovation: Evaluate emerging AI-based testing technologies and identify opportunities to integrate useful tools into existing engineering workflows to improve productivity and test coverage.
Security and Performance Validation: Conduct security-focused testing covering authentication, authorization, input validation, and common OWASP vulnerabilities. Design and execute load and performance tests to assess scalability, response times, and system resilience under different workloads.
Defect Identification and Resolution: Identify performance bottlenecks, vulnerabilities, system failures, and other quality issues. Collaborate with engineering teams to investigate root causes and implement appropriate corrective actions.
Data and Integration Validation: Use SQL and data-validation techniques to verify data integrity across APIs, pipelines, and integrations. Test interactions between services and downstream systems while identifying anomalies, regressions, and gaps in end-to-end workflows.
CI/CD Integration: Integrate automated test suites into continuous integration and continuous delivery pipelines to enable consistent, automated quality checks throughout the software development lifecycle.
Cross-functional Collaboration: Work closely with Engineering, Quality Assurance, Product, and Operations teams to support sprint planning, backlog refinement, release validation, and ongoing product improvements.
Quality Standards and Documentation: Contribute to testing standards, automation best practices, technical documentation, and quality assurance processes. Participate in root cause analysis for defects and production incidents.
Requirements:
Bachelor's degree in Computer Science, Engineering, or a related technical discipline, or equivalent professional experience.
Previous professional experience in quality assurance automation, test engineering, software engineering, or a closely related role.
Hands-on experience with modern test automation frameworks, particularly Playwright. Experience with Cypress, Selenium, or comparable technologies is also valuable.
Demonstrated experience testing APIs, distributed systems, software integrations, and data pipelines.
Strong understanding of software testing principles, test automation architecture, and quality engineering methodologies.
Solid SQL knowledge and practical experience validating data integrity and consistency across complex systems.
Experience integrating automated testing solutions into CI/CD pipelines and supporting continuous delivery processes.
Familiarity with Agile software development methodologies, including sprint planning, backlog refinement, and release management.
Experience working with cloud-based applications and infrastructure, preferably within AWS environments.
Strong analytical and troubleshooting skills, with the ability to investigate system behavior, identify defects, and determine root causes.
Excellent communication and collaboration skills, with the ability to work effectively across technical and non-technical teams.
Ability to manage multiple testing priorities, demonstrate attention to detail, and take ownership of quality outcomes.
Preferred Qualifications:
Experience testing AI/ML models, intelligent applications, or data-driven software systems.
Familiarity with validating AI-generated outputs, prompt-based applications, and LLM-powered systems.
Understanding of large language models (LLMs), AI agent workflows, and AI-assisted software development.
Practical experience using AI-powered coding and testing tools such as Claude Code, GitHub Copilot, or similar platforms.
Experience with security testing and performance/load testing tools such as k6, JMeter, or OWASP ZAP.
Knowledge of DevOps practices, CI/CD automation, and modern software delivery pipelines.
Previous experience in healthcare technology or regulated-data environments, including familiarity with HIPAA and related compliance requirements.
General understanding of AI/ML concepts and their application in modern software environments.
Benefits:
Competitive annual salary: $125,000–$150,000, depending on experience, skills, and qualifications.
Comprehensive healthcare coverage: Access to highly valued health insurance benefits designed to support employees' wellbeing.
Retirement savings: 401(k) retirement plan with employer matching contributions.
Paid parental leave: Benefits designed to support employees and their families.
Remote working flexibility: Fully remote working arrangement offering flexibility and work-life balance.
Collaborative environment: Opportunity to join a growing, innovative organization that values teamwork, creativity, and continuous improvement.
Professional development: Opportunities to expand expertise in AI, software testing, automation, and healthcare technology.
Meaningful impact: Contribute to technology solutions that improve collaboration between healthcare payers and providers, reduce administrative waste, and support better access to healthcare.
Additional Information:
This is primarily a computer-based role performed in an indoor working environment, involving extended periods of sitting, typing, and using computer equipment.
The position requires sustained concentration, analytical thinking, and regular written and verbal communication.
Additional responsibilities or compliance-related requirements may be assigned according to business needs, potentially including background checks, drug screenings, and applicable information disclosures.
Reasonable accommodations may be provided to qualified individuals with disabilities to support the performance of essential job functions.