OpenRecently verified

Freelance AI Test Scenario Designer - CS & AI Students Eligible

  • Data & Machine Learning
  • Platform: Mindrift
  • Location: 7 countries

About this job

This freelance role involves designing structured evaluation scenarios that test how LLM-based agents carry out tasks a person would normally perform. The work includes defining the ideal path, acceptable agent behavior and edge cases, then refining them with developers. It suits candidates with a computer science, AI or data background who are comfortable with complex guidelines.

What you'll do

  • Design structured test scenarios based on real-world tasks.
  • Define the golden path and acceptable agent behavior.
  • Annotate task steps, expected outputs and edge cases.
  • Review agent outputs and adapt tests accordingly.

Requirements

Machine LearningNLPLLM
  • Bachelor's or Master's degree in Computer Science, Software Engineering, Data Science, Artificial Intelligence, Machine Learning or a related field

Pay

Pay not disclosed

The platform does not publish the pay for this job.

Location

Open to residents of: Canada, United Kingdom, Greece, Ireland, Italy, Singapore, United States.

How to apply

You apply on Mindrift, the AI training platform run by Toloka: submit your application for the role, pass a qualification step, then work on projects that match your expertise.

All Mindrift jobs and how the platform works

Source

Official posting: https://apply.workable.com/j/390BCEEF9A

Last checked on October 16, 2026.

Similar jobs

  • Data & Machine Learning
  • Platform: Mercor
  • Location: United States
  • Language: Danish

You join a red teaming team specialized in adversarial evaluation of conversational AI models. Your role consists of testing AI systems by exploring their vulnerabilities (jailbreaks, prompt injections, biases), generating high-quality data documenting these vulnerabilities, and producing reproducible reports to strengthen model safety. This position is for bilingual English-Danish experts with prior experience in red teaming or related fields (cybersecurity, adversarial ML, socio-technical analysis).

Posted July 30, 2026

OpenRecently verified$48-62/hr

Referral link: we may earn a fee. Apply without it

  • Data & Machine Learning
  • Platform: Mercor
  • Location: United States
  • Language: Dutch

This role consists of testing the robustness and security of conversational AI models by subjecting them to sophisticated adversarial attacks. You will generate high-quality training data by identifying vulnerabilities, biases, and systemic risks that automated tests do not detect. The ideal profile has prior experience in red teaming, structured adversarial thinking, and the ability to clearly communicate security risks.

Posted July 30, 2026

OpenRecently verified$48-62/hr

Referral link: we may earn a fee. Apply without it

  • Software Engineering
  • Platform: Alignerr
  • Level: Intermediate

This remote contract role asks analysts to bring offensive security knowledge to AI development without writing exploits or taking part in live intrusions. The work centers on analyzing attack paths and adversary behavior in structured scenarios, then producing labeled reasoning data and written assessments that help AI systems reason about cybersecurity. It suits experienced security professionals who can explain how real attacks unfold in production environments.

  • Data & Machine Learning
  • Platform: Alignerr

This role involves testing and evaluating AI chatbot responses across diverse topics by engaging in conversations, identifying issues, and providing structured feedback. The position is designed for individuals with strong critical thinking and communication skills who want to contribute to AI safety and improvement without requiring prior technical or AI experience.

  • Data & Machine Learning
  • Platform: DataAnnotation
  • Level: Intermediate

This role involves evaluating how well AI models can perform computational biology tasks. Computational biologists and bioinformaticians will design realistic analysis scenarios from their own work, run them through AI systems, and grade the results against professional standards to help benchmark model capabilities.

Talent poolRecently verified$40-125/hr
  • Data & Machine Learning
  • Platform: micro1
  • Location: 58 countries

This contract role asks QA specialists to test AI system outputs against defined requirements, label and review datasets, and report model errors and patterns. It is aimed at people with software quality assurance and domain knowledge, and no prior AI experience is needed.

Posted October 16, 2026

OpenRecently verified$63-120/hr

Referral link to micro1's job list: search for this role there. Open this exact job

Keep exploring