Skip to content

AI Evaluation

Machine Learning Expert

Mercor logoMercor$150–$200/hrRemote, most countriesContractPosted today

Some links on AITraining.jobs are referral links. If you sign up through one, AITraining.jobs may earn a commission at no extra cost to you. It never changes what a platform pays you.

About the role

What you'll do

  • Come up with ideas for hard machine learning engineering tasks ("plans") for a long-horizon AI research project with a leading AI lab
  • Design tasks hard enough to stump frontier models: a strong task is solved in no more than 2 out of 8 attempts
  • Lean toward inference, evaluation and systems work rather than heavy model training. About half of the tasks run on GPU and half are CPU-only
  • Explain clearly why each problem is hard for a frontier model
  • You mainly contribute ideas. You do not need to build or review the tasks yourself

Must-haves

  • Current or recent role as an ML research engineer, research scientist or applied scientist
  • Hands-on experience personally training, fine-tuning or optimizing models in PyTorch or JAX
  • Deep knowledge of at least one of these areas:
  • LLMs: pretraining, alignment, evals
  • GPU inference and performance: vLLM, Triton, CUDA, profiling
  • Reinforcement learning (RL)
  • Scientific ML: weather and climate, physics, biology
  • Able to explain why a problem is hard for a frontier model

Nice-to-haves

  • Experience writing benchmark or evaluation questions (a big plus)
  • PhD in ML, CS or physics, or an MSc plus strong research output
  • Publications
  • Prior Mercor contracts with strong performance

Not a fit

  • Software or full-stack engineers whose AI experience is mainly building LangChain or RAG applications
  • Backgrounds limited to AI trainer, rater or annotator work

Where you can work

This role is 100% remote: you can work from any country except Andorra, Afghanistan, Armenia, Azerbaijan, Barbados, Burkina Faso, Burundi, Bahamas, Bhutan, Belarus, Belize, DR Congo, Central African Republic, Congo (Republic), Cameroon, China, Cuba, Cape Verde, Djibouti, Dominica, Eritrea, Ethiopia, Fiji, Micronesia, Grenada, Georgia, Guinea, Equatorial Guinea, Guinea-Bissau, Honduras, Haiti, Iraq, Iran, Kenya, Kyrgyzstan, Cambodia, Kiribati, Comoros, St. Kitts and Nevis, North Korea, Kazakhstan, Lebanon, Liechtenstein, Liberia, Lesotho, Libya, Moldova, Montenegro, Marshall Islands, Mali, Myanmar, Mauritania, Maldives, Malawi, Niger, Nicaragua, Nauru, Papua New Guinea, Pakistan, Palestine, Palau, Russia, Solomon Islands, Seychelles, Sudan, Sierra Leone, Somalia, Suriname, South Sudan, São Tomé and Príncipe, Syria, Eswatini, Chad, Togo, Tajikistan, Timor-Leste, Turkmenistan, Tonga, Tuvalu, Ukraine, Uganda, Vatican City, St. Vincent and Grenadines, Venezuela, Vanuatu, Samoa, Yemen, Zambia and Zimbabwe.

From Mercor's job posting, checked Oct 20, 2026.

About Mercor

Premium expert network: physicians, lawyers, engineers, PhDs.

How it pays: Weekly via Stripe (1099 contractors).

Similar open roles

Some links on AITraining.jobs are referral links. If you sign up through one, AITraining.jobs may earn a commission at no extra cost to you. It never changes what a platform pays you.

Want companies to ask for you?

One profile for every AI training platform. Companies ask us for people with a specific language, country or credential: add yours once, and when a role fits, we introduce you, only with your yes. Free.

Create my profile

Get new Mercor roles by email

We check the Mercor job board every hour and email you its newest roles each week, plus similar paid roles elsewhere. Free, unsubscribe anytime.