Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[eval] Add IMO problems with exact answers #1528

Merged
merged 9 commits into from
Jul 13, 2024
Merged

[eval] Add IMO problems with exact answers #1528

merged 9 commits into from
Jul 13, 2024

Commits on May 7, 2024

  1. evals: Add imo_solutions_only.yaml to registry

    @justinlinw
    justinlinw committed May 7, 2024
    Configuration menu
    Copy the full SHA
    148c091 View commit details
    Browse the repository at this point in the history

  2. evals: Add samples.jsonl for imo_solutions_only containing 19 sol… 

    …utions-only IMO questions
    @justinlinw
    justinlinw committed May 7, 2024
    Configuration menu
    Copy the full SHA
    1bff78d View commit details
    Browse the repository at this point in the history

Commits on May 15, 2024

  1. fix: Add description

    @justinlinw
    justinlinw committed May 15, 2024
    Configuration menu
    Copy the full SHA
    52ca817 View commit details
    Browse the repository at this point in the history

  2. fix: Rename imo_solutions_only → imo_exact_answers

    @justinlinw
    justinlinw committed May 15, 2024
    Configuration menu
    Copy the full SHA
    839c267 View commit details
    Browse the repository at this point in the history

Commits on May 17, 2024

  1. fix: Replace <eval_name> in .yaml file

    @justinlinw
    justinlinw committed May 17, 2024
    Configuration menu
    Copy the full SHA
    67542df View commit details
    Browse the repository at this point in the history

  2. fix: Escape backslashes in LaTeX content for JSON compatibility

    @justinlinw
    justinlinw committed May 17, 2024
    Configuration menu
    Copy the full SHA
    6fcf137 View commit details
    Browse the repository at this point in the history

  3. fix: Remove instructions in system prompt to answer with []

    @justinlinw
    justinlinw committed May 17, 2024
    Configuration menu
    Copy the full SHA
    be71181 View commit details
    Browse the repository at this point in the history

  4. fix: Update evaluation method to Includes instead of Match

    @justinlinw
    justinlinw committed May 17, 2024
    Configuration menu
    Copy the full SHA
    bd7ab27 View commit details
    Browse the repository at this point in the history

  5. fix: Update system prompt to include reasoning steps and update answe… 

    …r/response format
    @justinlinw
    justinlinw committed May 17, 2024
    Configuration menu
    Copy the full SHA
    66a00b6 View commit details
    Browse the repository at this point in the history