Data Science Evaluator (contract)
Overview
MarketerHire is seeking Data Science Evaluators for a 12-week, remote contract engagement supporting leading AI research organizations. The role involves designing grading criteria, evaluating data science work produced by AI systems or people, and writing evidence-based evaluations across exploratory analysis, statistical modeling, machine learning pipelines, experimentation, and technical notebooks. Candidates must have 2–3 years of professional data science experience, strong Python and SQL skills, excellent written English, sound evaluation judgment, and availability for 30–40+ hours per week from the US, Canada, or UK.
Responsibilities
<h2>Key Responsibilities</h2><ul><li>Design task-specific grading criteria for exploratory analyses, statistical models, ML pipelines, A/B test write-ups, and technical notebooks.</li><li>Evaluate AI-generated and human-created data science work against defined criteria.</li><li>Assess the quality, correctness, completeness, and reasoning demonstrated in technical deliverables.</li><li>Write clear, evidence-based justifications for every evaluation score.</li><li>Apply consistent judgment across a broad range of data science tasks and outputs.</li><li>Identify subtle errors and distinguish well-supported conclusions from unsupported inferences.</li><li>Follow complex evaluation instructions and reason through ambiguous cases.</li><li>Incorporate feedback from senior reviewers and iterate on evaluation decisions and written justifications.</li><li>Work independently while learning new tools and evaluation workflows quickly.</li></ul>
Requirements
<h2>Required Qualifications</h2><ul><li>2–3 years of professional data science experience.</li><li>Strong skills in <strong>Python, SQL, statistics, machine learning, experimentation, and causal inference</strong>.</li><li>Excellent written English and the ability to produce clear, evidence-based explanations.</li><li>Ability to follow complex instructions, reason through ambiguity, and spot subtle errors in polished work.</li><li>Clear separation of what you know from what you are inferring.</li><li>Ability to work independently, learn new tools quickly, and take feedback well.</li><li>Based in and working from the US, Canada, or UK.</li><li>Availability for 30–40+ hours per week for the full 12-week engagement.</li></ul>
Benefits
<h2>Benefits and Perks</h2><ul><li>Remote contract engagement for professionals in the US, Canada, and UK.</li><li>Competitive compensation of <strong>$100–$150 per hour</strong>, paid weekly via Stripe or Wise.</li><li>Flexible scheduling within the required 30–40+ hours per week.</li><li>Defined 12-week engagement with the possibility that projects may be extended based on needs and performance.</li><li>Opportunity to contribute to the evaluation of real-world data science capabilities in AI systems.</li></ul>