QA Automation Engineer with AI experience
.svg)
You will be cooperating with a leading provider of AI evaluation and optimization solutions, trusted by multinational companies to optimize AI agents and detect performance issues in large language models.
In this role, you will contribute to a product that addresses one of the most important and challenging problems facing society’s adoption of AI today.
Due to the client’s time zone, we would appreciate a candidate who can work until 5:00 p.m., or occasionally until 6:00 p.m. If you prefer working slightly later hours, that’s perfectly okay with the client - but it’s not a requirement.
This is a 100% remote position, but if you enjoy working from an office, you’re warmly welcome to join us there too 😊
Join us and make a real impact!
If you’re ready to broaden your horizons and work with an innovative company at the forefront of AI, we’d love to hear from you. You’ll help build the environments that shape how future AI systems are trained, evaluated, and aligned - and collaborate with world-class engineers and researchers on one of the most important technical challenges of our time.
What will be your responsibilities?
- Design and implement automated tests.
- Build and maintain automated regression pipelines to ensure continuous platform stability after deployments and LLM-model updates.
- Design and execute tests for RAG evaluation and the relevance of AI-generated content.
- Work closely with ML, AI, and FE Engineers to identify automation opportunities and improve overall software quality
Requirements
- Min. 5 years of experience in automation, including API testing.
- Practice in testing LLM and RAG applications, including prompt validation.
- Knowledge of tools supporting automation using AI (e.g., GitHub Copilot, Cursor, Claude Code).
- Skills in validating RESTful APIs and microservices.
- Proficiency in testing applications hosted on various cloud infrastructures.
100 – 175 PLN
Our benefits
.png)
.png)
Recruitment process
See similar jobs
Technical Project Manager (AI project)
Senior Python Engineer
.svg)
.svg)
Since working with Acaisoft, our delivery velocity has increased at least 20 times. In the past months, we’ve done more developments than what had been done in the last four years in our company.
