QA Engineer: AI-Assisted Testing

Accountabilities:

• Use AI-assisted workflows to analyze tickets, merge requests, code changes, and acceptance criteria; identify risks and edge cases; and develop focused test charters and checklists.
• Perform risk-based exploratory, functional, regression, and integration testing across web dashboards, administrative tools, email notifications, document workflows, and external integrations.
• Validate end-to-end document retrieval, parsing, categorization, storage, and delivery across external portals, authentication flows, and third-party platforms.
• Investigate backend and platform functionality using API calls, SQL queries, logs, dashboards, and monitoring tools such as Grafana/Loki and Sentry.
• Generate realistic test data, sample documents, email scenarios, and edge cases with AI tools and use simulators and test environments to reproduce difficult production scenarios.
• Evaluate AI-powered document categorization and LLM-facing functionality for accuracy, consistency, robustness, and unexpected behavior, including testing unusual or adversarial inputs.
• Test authentication, SSO, OAuth, user roles, permissions, sessions, and data isolation to help ensure secure access and protect information between customers.
• Investigate customer-reported and production issues, reproduce defects consistently, identify likely root causes, and create clear, actionable bug reports.
• Build or maintain targeted automated checks when appropriate, using tools such as Playwright or API testing frameworks, with AI assisting development while carefully reviewing generated code.
• Collaborate with engineers and product managers during sprint planning, backlog refinement, and retrospectives, helping establish testable acceptance criteria and identifying quality risks early.
• Continuously improve testing strategies, documentation, test coverage, and quality processes within an AI-assisted development environment.
• Apply careful data-handling practices when using AI tools, ensuring customer and sensitive data is never submitted to unauthorized services.

Requirements

• 3–7 years of professional software QA experience, particularly testing web or SaaS applications and complex, data-intensive systems.
• Strong hands-on experience with exploratory and manual testing, including functional, regression, integration, and risk-based testing approaches.
• Demonstrated ability to use AI assistants such as Claude, Claude Code, Cursor, ChatGPT, or comparable tools to generate test ideas, create test data, write queries, analyze logs and code changes, investigate defects, and produce test documentation.
• Strong AI judgment, including the ability to provide effective context and prompts, verify AI-generated results, recognize incorrect or unreliable outputs, and retain independent ownership of testing decisions.
• Experience testing AI or LLM-based features, including classification accuracy, hallucinations, prompt injection, consistency, and evaluation using representative sample sets and repeated testing.
• Experience writing or maintaining automated tests using tools such as Playwright, Selenium, API test frameworks, or similar technologies, including reviewing and correcting AI-generated test code.
• Strong SQL skills, preferably with PostgreSQL, for data validation, troubleshooting, and investigation.
• Experience testing APIs directly using tools such as Postman, curl, Bruno, or equivalent, with a solid understanding of HTTP, REST, and JSON.
• Experience with web security testing, including authentication, sessions, roles and permissions, OAuth2/SAML, CSRF, and XSS.
• Experience with browser automation or AI browser tools, such as Playwright MCP or browser agents, to support exploratory and functional testing.
• Experience testing third-party integrations, email-driven workflows, and browser automation involving external systems.
• Familiarity with command-line tools, Git, and application logs; experience with Docker or Kubernetes is a plus.
• Experience with Jira or comparable defect tracking and test management platforms.
• Experience with accessibility testing and knowledge of WCAG 2.2 AA standards.
• Knowledge of legal technology, particularly court e-filing, docketing, or legal document management, is desirable.
• ISTQB or comparable QA certification is a plus.
• Familiarity with Lean Startup principles is beneficial.
• Excellent written and verbal communication skills, with the ability to produce precise bug reports, test notes, and status updates.
• Strong analytical thinking, curiosity, attention to detail, ownership, and persistence when investigating difficult or unclear problems.

Benefits

• Fully remote position open to candidates across all U.S. time zones, including Hawaii.
• Flexible working hours designed to support a healthy work-life balance.
• Health, dental, vision, and life insurance options.
• Discretionary Time Off (DTO), allowing employees to take additional unplanned time away from work when needed, subject to supervisor approval.
• Scheduled vacation and holiday time.
• Employee stock option plan participation.
• Company-provided AI tools with guidelines designed to support safe and responsible use.
• Opportunity to shape QA practices within an AI-assisted engineering organization.
• Hands-on exposure to AI features operating in production and used by real customers.
• Opportunity to influence testing strategy rather than simply follow predefined test scripts.
• Collaborative environment with close interaction across QA, engineering, and product teams.