Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve error reporting #12

Open
RyanGlScott opened this issue Nov 15, 2021 · 0 comments
Open

Improve error reporting #12

RyanGlScott opened this issue Nov 15, 2021 · 0 comments
Labels
enhancement New feature or request

Comments

@RyanGlScott
Copy link
Collaborator

The error messages that copilot-verifier gives when unable to verify a program can be rather difficult to understand by non-experts. Here is one example observed in #5:

$ cabal run verify-examples -- WCV
Up to date
=====
== Running the WCV example...
=====

Generated "results/wcv/wcv.c"
Compiled wcv into results/wcv/crux~wcv.bc
Translated bitcode into Crucible
Generating proof state data
Computing initial state verification conditions
Proving initial state verification conditions
All obligations proved by concrete simplification
Computing step bisimulation verification conditions
Proving step bisimulation verification conditions
verify-examples: user error (user error (evalGroundExpr: could not evaluate expression: let uninterpreted_float_lt = ??
    uninterpreted_sbv_to_float = ??
    uninterpreted_float_to_sbv = ??
    -- results/wcv/wcv.c:21:208
    v366 = apply uninterpreted_float_to_sbv 0 crelative_velocity_x_r0@58:bv
    -- results/wcv/wcv.c:21:202
    v373 = ite (bvSlt v366 0x0:[32]) (bvSum (bvMul 0xffffffff:[32] v366)) v366
    -- results/wcv/wcv.c:21:201
    v374 = apply uninterpreted_sbv_to_float 0 v373
 in apply uninterpreted_float_lt v374 0x3f50624dd2f1a9fc:[64])
)

This was fixed by picking a different translation of abs, but it seems quite likely that there are other similarly perplexing error-message scenarios awaiting us. We should:

  • Identify situations in which copilot-verifier produces dense error messages, and
  • Improve the reporting of these errors so that they are more comprehensible by users who aren't experts in What4/Crucible.
@RyanGlScott RyanGlScott added the enhancement New feature or request label Nov 15, 2021
@RyanGlScott RyanGlScott mentioned this issue Nov 15, 2021
RyanGlScott added a commit that referenced this issue Nov 18, 2021
Previously, `copilot-verifier` would simply print out
`Simulation aborted for multiple reasons` when there were multiple failures,
which wasn't terribly helpful. After this patch, we get something more
informative:

```
verify-examples: user error (Simulation aborted for multiple reasons.
Simulation aborted!
Abort due to assertion failure:
  results/wcv/wcv.c:21:202: error: in well_clear_violation_guard
  Failed to load function handle
  Details:
    No implementation or override found for pointer: "llvm.fabs.f64"
Simulation aborted!
Abort due to assertion failure:
  results/wcv/wcv.c:21:4576: error: in well_clear_violation_guard
  Failed to load function handle
  Details:
    No implementation or override found for pointer: "llvm.fabs.f64")
```

It's a tad verbose, but we can improve the error-reporting story later.
See #12.
RyanGlScott added a commit that referenced this issue Nov 23, 2021
Previously, `copilot-verifier` would simply print out
`Simulation aborted for multiple reasons` when there were multiple failures,
which wasn't terribly helpful. After this patch, we get something more
informative:

```
verify-examples: user error (Simulation aborted for multiple reasons.
Simulation aborted!
Abort due to assertion failure:
  results/wcv/wcv.c:21:202: error: in well_clear_violation_guard
  Failed to load function handle
  Details:
    No implementation or override found for pointer: "llvm.fabs.f64"
Simulation aborted!
Abort due to assertion failure:
  results/wcv/wcv.c:21:4576: error: in well_clear_violation_guard
  Failed to load function handle
  Details:
    No implementation or override found for pointer: "llvm.fabs.f64")
```

It's a tad verbose, but we can improve the error-reporting story later.
See #12.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Development

No branches or pull requests

1 participant