Join us in helping advance the next generation of AI models!

We are looking for experienced Software Engineers to contribute to a cutting-edge AI evaluation project. This project plays a key role in improving how frontier language models understand, reason about, and generate production-quality software. You'll evaluate AI-generated solutions through the lens of an experienced software engineer, helping shape the future of AI-assisted software development.

Project Timeline

• Project Duration: Approximately 1 week
• Expected Timeline: July 8–14
• Expected commitment: at least 10 hours
• Candidates must be located in US/ Canada due to the need for real-time collaboration with the project team.

Background & Qualifications

YOE

• 3+ years of professional software engineering experience (or equivalent open-source experience)

Technical Expertise

• JavaScript / TypeScript: Node.js, React, TypeScript, modern web application development
• Go: Backend services, distributed systems, REST/gRPC APIs, cloud-native development (Kubernetes experience is a plus)
• Python: Backend development, open-source libraries, DevOps/infrastructure automation, testing frameworks

Software Engineering

• Experience working with large production codebases
• Ability to quickly understand unfamiliar repositories and software architectures
• Experience reviewing code for readability, maintainability, and software design
• Strong engineering judgment and a solid understanding of software engineering best practices

GitHub & Open Source

• Strong understanding of Git and GitHub workflows (e.g., branching, Issues, Pull Requests, and Code Reviews)
• Experience navigating open-source repositories and collaborating through GitHub
• Familiarity with open-source development workflows is highly preferred
• Familiarity with SWE-Bench or SWE-Bench Pro is a plus

Communication

• Strong English communication skills

Responsibilities

• Become familiar with assigned open-source codebases and their engineering practices
• Review code changes, technical discussions, and related software artifacts
• Evaluate AI-generated code for correctness, quality, maintainability, and engineering best practices
• Provide structured evaluations with clear technical reasoning
• Collaborate with fellow technical reviewers to ensure high-quality and consistent evaluations

Note: This is not a software development role. Rather than writing code, you'll leverage your software engineering expertise to evaluate AI-generated solutions and help improve the capabilities of state-of-the-art coding models.

We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.

Contract and Payment Terms

• You will be engaged as an independent contractor.
• This is a fully remote role that can be completed on your own schedule.
• Projects can be extended, shortened, or concluded early depending on needs and performance.
• Your work at Mercor will not involve access to confidential or proprietary information from any employer, client, or institution.
• Payments are weekly on Stripe or Wise based on services rendered.
• Please note: We are unable to support H1-B or STEM OPT candidates at this time.