Generate annotated source listings with inline visualization of code metrics

[devdanzin]

Wily should be able to generate color coded source listings with inline visualization of code metrics from detailed metric results. The inspiration for this comes from #88 (which has an early preview of the results), #60 and #177. An example bulk report containing annotated source files can be found at https://devdanzin.github.io/wily/.

Describe the solution you'd like
Adding a new annotate command which can either generate HTML files of source code annotated with metric values or print to console. There's working code for this in the annotated_source branch and PRs to integrate it will be proposed.

Describe alternatives you've considered
It would be possible, with some added code to wily, to keep annotate as a separate tool instead.

Additional context

wily annotate design and implementation

wily annotate is a new command being proposed for addition to wily. It allows
inline visualization of code metrics with color-coded lines for a given metric.
It works on an already built wily cache and uses detailed metric information to
add metric values to code blocks in source file listings. From its docs:

The annotate command will generate annotated Python source and output it to
console or save as HTML, with metric values for a revision or single file.
It currently supports Cyclomatic Complexity and Halstead metrics.

Here we document the design and current implementation choices of the
annotate command for wily.

General design

For source code annotation with metrics, we work in three phases:

• Build: record start and end lines for detailed metric results.
• Map: generate a mapping of line numbers to metric values.
• Annotate: render the source code with added metric information.

There's a fourth phase represented by the bulk-annotate command, which creates
annotated HTML files in batch mode.

Current implementation

Build

During build, we record the line numbers (lineno and endline) that radon
provides for Cyclomatic Complexity. We also customize radon to record line
numbers for Halstead metrics.

Map

We currently fetch detailed information from the cache, then use
lineno/endline for each entry to map its metric value to source lines. Only
one revision of a file is currently annotated.

Annotation

We use customized pygments formatters to render highlighted source code
with attached metric values. There is support for console printing and
HTML file generation. For HTML, the metric values are used to choose
background colors for code blocks (functions, methods and classes), and
rely on JS and CSS to allow changing what metric and colors get displayed.
There's a very simple HTML template used to display pygments output.

We can restrict annotation by revision and/or by path.

Planned improvements

Build

It's possible to also generate detailed Raw results with some modifications
3to radon. This can work either as a contribution to radon or local customization
of its visitor.

Annotation

Allowing users to provide simple templates for the HTML files is planned.

Creating an index page is planned, and might reuse the code from the proposed
wily bulk command.

Alternative design choices

Build

Instead of customizing radon, we might prefer to wait for it to add support
for the line numbers we need. Part of this has already been submitted as a PR
to radon, another PR will be issued when the first one lands. And there's an
old open PR for Raw line numbers.

Instead of generating annotated sources after building, it could be a step
of the build process. This would make it easier to annotate multiple revisions.

Map

Changing the mapping representation of metric values to lines may make annotation
simpler.

It'd also be possible to map and annotate multiple versions of a given file, to
allow comparisons between revisions.

Annotation

It's possible to ditch pygments and work on a simpler, non-highlighting renderer
for terminal and HTML.

Command or tool?

Given the size and complexity of the current code, it might make sense to keep
annotate as a separate tool. Supporting code would still be needed in wily
(mainly build changes, maybe information retrieval for mapping). The obvious
advantages of keeping it as a command is that it will always be in sync with
wily and available to all of its users.