revise

tmpl/wiki/package-quality.md

@@ -1,10 +1,12 @@
 ## Package Quality
 
+NOTE: this is WIP
+
 The MirageOS ecosystem of libraries is booming:
 
 ```
 $ opam list -a | grep mirage | wc -l
-      90
+      95
 ```
 
 We now have various libraries covering all aspects of the OS: from network to
@@ -17,40 +19,116 @@ developing your own application instead of spending time re-implementing
 something that the OS would usually do for you. There are of course still
 gaps in the ecosystem, but things are getting better!
 
-The next problem is knowing the level of confidence to attribute to each of
-these libraries. Would be great to be able to answer questions such as:
+A MirageOS unikernel contains an unmodified OCaml runtime, but the `Unix` module
+is intentionally! not implemented on MirageOS, neither is the `Str` or `Threads`
+module. Any OCaml library depending directly or indirectly on these is not
+usable with MirageOS.
+
+Only a very small set of C functions are implemented in the MirageOS lower
+layers (such as
+[ocaml-freestanding](https://github.com/mirage/ocaml-freestanding)). If an
+OCaml library uses external libraries written in C via OCaml's C foreign
+function interface, this library will likely not work on MirageOS since the
+C functions are missing. It is possible to use C functions on MirageOS, but
+the build system of such a library needs to "cross-compile" the C functions for
+the different virtualization technologies.
+Examples include
+[mirage-entropy](https://github.com/mirage/mirage-entropy),
+[checkseum](https://github.com/mirage/checkseum),
+[digestif](https://github.com/mirage/digestif),
+[bigstringaf](https://github.com/inhabitedtype/bigstringaf)
+and [nocrypto](https://github.com/mirleft/ocaml-nocrypto) (which
+uses [gmp](https://gmplib.org) via [zarith](https://github.com/ocaml/zarith) and
+the gmp-freestanding and gmp-xen package in opam-repository),
+
+A problem is how to know the level of confidence to attribute to a MirageOS
+library. Would be great to be able to answer questions such as:
+Does this library work with MirageOS?
 Does the library has a good test coverage? Was it formally verified?
-Was it used in production? In order to do this, I would like to introduce
-the "MirageOS quality levels". Ideally this will live in a tool (e.g.
+Was it used in production? In order to do this, we are introducing
+the "MirageOS quality tags". Ideally this will live in a tool (e.g.
 `mirage lint`) but to start we list the criteria that package aiming
 for increased quality should follow.
 
-### ![Level 1:](https://img.shields.io/badge/level-1-blue.svg) Follow Packaging Guidelines
-
-See the [packaging guidelines](https://mirage.io/wiki/packaging) for more
-details. In summary:
-
-- The package should work well with [odig](http://erratique.ch/software/odig)
-  e.g. `CHANGES.md`, `LICENSE.md` and `README.md` should exist.
+### ![ready:](https://img.shields.io/badge/quality-ready-orange.svg) usable with MirageOS
+
+The OCaml library is packaged with [opam](https://opam.ocaml.org), and released
+to the [opam repository](https://github.com/ocaml/opam-repository).
+
+In addition, the library follows our [packaging guidelines](https://mirage.io/wiki/packaging).
+
+Checks to be in this level:
+- The commands `opam lint` and `dune-release lint` or `topkg lint` return no
+  error.
+- The metadata files `CHANGES.md`, `LICENSE.md` and `README.md` exist
+- The use of [dune](http://dune.readthedocs.io/en/latest/) is
+  strongly encouraged but not yet mandatory.
+- Immutable strings, since 4.06.0 default (`-safe-string`).
+- The not available modules `Unix`, `Str` and `Threads` are not referenced.
+- Either there are no C dependencies or the build system cross-compiles to
+  MirageOS targets.
+- If a side-effecting computation, such as network access, random, etc. is
+  required, an implementation using the [mirage-types](https://github.com/mirage/mirage-types)
+  interfaces is provided in a sublibrary (e.g. [tls-mirage](https://github.com/mirleft/ocaml-tls/tree/master/mirage).
+
+## ![integrated:](https://img.shields.io/badge/quality-tested-yellow.svg) follows our best practises
+
+MirageOS follows software development best practises. The library has to
+follow the [OCaml programming guidelines](https://ocaml.org/learn/tutorials/guidelines.html).
+
+For some functionality in MirageOS we agreed on OCaml libraries:
+- A MirageOS unikernel should have a unified logging configuration of log level
+  and subsystems. We use the [logs](http://erratique.ch/software/logs/doc/Logs.html)
+  library for logging, which nicely separates the log source from the log reporter.
+  The [mirage-logs](https://github.com/mirage/mirage-logs) contains command-line
+  logging source and level support, a
+  [syslog reporter](https://github.com/hannesm/logs-syslog) is integrated.
+- If a library reports metrics, e.g. it uses a cache or wants to measure
+  durations between request and response. The
+  [metrics](https://github.com/mirage/metrics) is used.
+
+Recommended OCaml libraries:
+- Use [lwt](http://ocsigen.org/lwt/) for asynchronous tasks.
+- Use [ipaddr](https://github.com/mirage/ocaml-ipaddr) for Internet Protocol
+  addresses.
+- Use [astring](http://erratique.ch/software/astring/doc/Astring.html) for
+  string processing.
+- Provide pretty printers by using
+  [fmt](http://erratique.ch/software/fmt/doc/Fmt.html).
+- Have unit tests (using [alcotest](https://github.com/mirage/alcotest)),
+  ideally with automatedcoverage report output.
+- [Ptime](http://erratique.ch/software/ptime/doc/Ptime.html) for POSIX time
+  computations.
+- [Duration](https://github.com/hannesm/duration) converts time units
+  (milliseconds, seconds, ..)
+- [Randomconv](https://github.com/hannesm/randomconv) converting bytes to
+  number ranges.
+
+Code best pracises:
+- Avoid polymorphic equality and comparison -- when possible define your own
+  specialized functions.
+- Avoid global mutable state.
+- Use tail recursion.
+- Do not expose any exceptions in the public interface, but use the
+  [result](http://caml.inria.fr/pub/docs/manual-ocaml/libref/Pervasives.html#TYPEresult)
+  type and [rresult](http://erratique.ch/software/rresult/doc/Rresult.html)
+  combinators.
 
-- The package should work well with [topkg](http://erratique.ch/software/topkg),
-  e.g. `pkg/pkg.ml` should exist.
+Style best practises
+- Have proper indentation (using ocp-indent + checked-in ocp-indent file).
+- Use at most 80 columns (`ocp-indent` unfortunately doesn't check this).
 
-- The use of [jbuilder](http://jbuilder.readthedocs.io/en/latest/) is
-  strongly encouraged but not yet mandatory as 1.0.0 has not been
-  officially released. However, we plan to enforce its use as soon as
-  it is the case -- assuming that it has all the features needed for
-  building MirageOS applications.
+The library is documented with a well-defined scope and its API is documented,
+accessible online (e.g. [docs.mirage.io](https://docs.mirage.io))
+and can be generated locally with [odig](http://erratique.ch/software/odig).
 
-- Use proper names: same name for opam and ocamlfind package, and also ideally
-  for the top-level module.
+## ![automated:](https://img.shields.io/badge/quality-automated-yellow-green.svg) automated with continuous integration
 
-- All direct dependencies are specified in the opam file (e.g. sometimes some
-  direct dependencies are included by transitivity -- this is not very stable
-  and could break the package in the future if one of a transitive dependency
-  is updated) (NOTE: should probably be added in the packaging guideline)
+The library uses continuous integration systems to catch issues early:
+- The [ocaml-ci-scripts](https://github.com/ocaml/ocaml-ci-scripts) contain
+  Travis and AppVeyor integration.
 
-### ![Level 2](https://img.shields.io/badge/level-2-blue.svg) Define Package Scope
+## ![approved:](https://img.shields.io/badge/quality-approved-green.svg) approved member of the MirageOS ecosystem
 
 - Have well-identified maintainers. GitHub recently
   [introduced code owners](https://github.com/blog/2392-introducing-code-owners),
@@ -59,80 +137,30 @@ details. In summary:
 - Have a design document specifying the scope of the library (and esp. what is
   out of scope)
 
-- Use the correct set of tags in opam metadata
-
-- Have documentation
-
-### ![Level 3:](https://img.shields.io/badge/level-3-blue.svg) Use Good Coding Style
-
-- Have proper indentation (using ocp-indent + checked-in ocp-indent file).
-
-- Use at most 80 columns (`ocp-indent` unfortunately doesn't check this).
-
-- Follow [OCaml programming guidelines](https://ocaml.org/learn/tutorials/guidelines.html).
-
-### ![Level 4:](https://img.shields.io/badge/level-4-blue.svg) Keep your Style Functional
-
-- Avoid global mutable state.
-
-- Use tail recursion.
-
-- Use `-safe-string` (looks like this will be the default in 4.06).
-
-- Avoid polymorphic equality and comparison -- when possible define your own
-  specialized functions.
-
-### ![Level 5:](https://img.shields.io/badge/level-5-blue.svg) Test
-
-- Have unit tests (using alcotest) (ideally with coverage report).
-
-### ![Level 6:](https://img.shields.io/badge/level-6-blue.svg) Keep Sane Dependencies
-
-- Depends only on libraries released in opam.
-
-- Provide pretty printers by using [fmt](http://erratique.ch/software/fmt/doc/Fmt.html).
-
-- Do not expose any exceptions in the public interface, but use the
-  [result](http://caml.inria.fr/pub/docs/manual-ocaml/libref/Pervasives.html#TYPEresult)
-  type and [rresult](http://erratique.ch/software/rresult/doc/Rresult.html)
-  combinators.
-
-- Use [logs](http://erratique.ch/software/logs/doc/Logs.html) for logging.
-
-- Use [astring](http://erratique.ch/software/astring/doc/Astring.html) for
-  string processing.
-
-- Use [bos](http://erratique.ch/software/bos/doc/Bos.html) for operating system
-  interaction.
-
-- Use [fpath](http://erratique.ch/software/fpath/doc/Fpath.html) for file paths.
-
 - Avoid the use of C bindings for no good reasons. A good reason would be to
   improve performance by an order of magnitude, or re-use an existing C library
   that has not been rewritten yet.
 
-### ![Level 7:](https://img.shields.io/badge/level-7-blue.svg) Randomized Test
-
 - Have randomized property-based testing. Using QuickCheck-like libraries or
-  even better using fuzz testing (and crowbar) when the tooling will be ready.
-
-### ![Level 8:](https://img.shields.io/badge/level-8-blue.svg) Count with Care
+  fuzz testing (e.g. [crowbar](https://github.com/stedolan/crowbar/), automated
+  fuzz testing with [bun](https://github.com/yomimono/ocaml-bun) and
+  [ppx_deriving_crowbar](https://github.com/yomimono/ppx_deriving_crowbar/)).
 
-- Avoid integer overflows (basically every addition and subtraction, as well
+- Avoid integer overflows: basically every addition and subtraction, as well
   as multiplication needs to be guarded unless you know that an overflow can
-  never happen (in this case, a comment should be suggested))
+  never happen (in this case, a comment should be suggested).
 
-- Work on 32bit (esp. in regards to the last point)
+- Work on 32bit (esp. in regards to the last point), tested by CI.
 
-### ![Level 9:](https://img.shields.io/badge/level-9-blue.svg) Used in Production
+- Is reproducible by having a way to reproduce the exact binary result. See
+  [reproducible builds](https://reproducible-builds.org/) for further information.
+  OCaml produces reproducible binaries. A reproducible MirageOS unikernel needs
+  to record the exact version of its opam package dependencies, and each opam
+  package needs to be reproducible.
 
 - Have a clear indication if the library is used in production (and if yes by
   which project).
 
-- For distributed binaries: have a way to reproduce the exact set of packages
-  needed to build the released version of the binary, for instance by vendoring
-  opam metadata.
-
-### ![Level 10:](https://img.shields.io/badge/level-10-blue.svg) Verify Formally
+## ![verified:](https://img.shields.io/badge/quality-verified-purple.svg) Formally verified
 
-- Has been formally verified.
+- Has been formally verified with a proof assistant.

tmpl/wiki/packages.md

@@ -66,6 +66,10 @@ A Mirage library should have
   and [mirage-block-lwt.1.1.0](https://github.com/mirage/mirage-block/blob/1.1.0/mirage-block-lwt.opam).
   These should have a github pages `doc:` link in order that `topkg` can detect
   the upstream repo.
+- The opam file specifies all direct dependencies are specified. Sometimes
+  direct dependencies are included by transitivity, which is not very stable
+  and could break the package in the future if one of the transitive dependencies
+  is updated
 - `Makefile`: contains `jbuilder` invocations including the `--dev` argument
   to enable warnings as errors for local builds.
   For example [mirage-block.3.1.2](https://github.com/mirage/mirage-block/blob/1.1.0/Makefile)