LLM Training - Biomedical Informatics - PhD or PhD candidates

Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.

Role Overview

We are seeking highly skilled Biomedical Informatics Subject Matter Experts (SMEs) to join our team in developing rigorous, high-quality evaluation questions designed to assess and challenge advanced AI models. The primary objective is to create benchmark questions that test the boundaries of AI capabilities across diverse bioinformatics domains.

This role requires deep technical expertise combined with the ability to craft precise, unambiguous questions with well-defined evaluations. The ideal candidate will possess not only strong theoretical knowledge but also practical experience in designing assessment materials that have deterministic, verifiable outcomes.

Responsibilities :

• Design and develop challenging bioinformatics coding questions that push the limits of AI model capabilities.
• Create automated, coding-based evaluations with unit tests that objectively verify correctness.
• Build comprehensive evaluation rubrics with 7–10 distinct criteria for each bioinformatics question.
• Ensure every question is self-contained, well-defined, and completely unambiguous, including all domain rules and edge cases.
• Develop problems where there is exactly one correct answer or outcome that can be deterministically validated.
• Translate theoretical bioinformatics concepts into concrete, measurable outputs.
• Enforce performance requirements with explicit constraints and tests to prevent inefficient or brute-force solutions.

Commitments Required : 30-40h / week

Employment type : Contractor assignment (no medical / paid leave)

Duration of contract : ~8 weeks

Location : India, Pakistan, Nigeria, Kenya, Egypt, Ghana, Bangladesh, Turkey, Mexico, Europe

Selection Process : Home Assignment (takes ~4h to complete)

Requirements