This document is a work-in-progress attempt to provide useful information for people willing to inspect or modify the compiler distribution’s codebase. Feel free to improve it by sending change proposals for it.
If you already have a patch that you would like to contribute to the official distribution, please see CONTRIBUTING.md.
-
Create a new git branch to store your changes.
git checkout -b my-modification
-
Consult INSTALL.adoc for build instructions. Here is the gist of it:
./configure make
-
Try the newly built compiler binaries
ocamlc,ocamloptor their.optversion. To try the toplevel, use:make runtop
-
Hack frenetically and keep rebuilding.
-
Run the testsuite from time to time.
make tests
-
Install in a new opam switch to try things out:
opam compiler-conf install
With opam 2, create a local opam switch with the compiler installed from the current source directory:
opam switch create . --empty opam install .
-
You did it, Well done! Consult CONTRIBUTING.md to send your contribution upstream.
See our Development tips and tricks for various helpful details, for example on how to automatically create an opam switch from a compiler branch.
There is always a lot of potential tasks, both for old and newcomers. Here are various potential projects:
-
The OCaml bugtracker contains reported bugs and feature requests. Some changes that should be accessible to newcomers are marked with the tag junior_job.
-
The OCaml Labs compiler-hacking wiki contains various ideas of changes to propose, some easy, some requiring a fair amount of work.
-
Documentation improvements are always much appreciated, either in the various
.mlifiles or in the official manual (See manual/README.md). If you invest effort in understanding a part of the codebase, submitting a pull request that adds clarifying comments can be an excellent contribution to help you, next time, and other code readers. -
The github project contains a lot of pull requests, many of them being in dire need of a review — we have more people willing to contribute changes than to review someone else’s change. Picking one of them, trying to understand the code (looking at the code around it) and asking questions about what you don’t understand or what feels odd is super-useful. It helps the contribution process, and it is also an excellent way to get to know various parts of the compiler from the angle of a specific aspect or feature.
Again, reviewing small or medium-sized pull requests is accessible to anyone with OCaml programming experience, and helps maintainers and other contributors. If you also submit pull requests yourself, a good discipline is to review at least as many pull requests as you submit.
The compiler codebase can be intimidating at first sight. Here are a few pointers to get started.
The driver — driver/
The driver contains the "main" function of the compilers that drive compilation. It parses the command-line arguments and composes the required compiler passes by calling functions from the various parts of the compiler described below.
Parsing — parsing/
Parses source files and produces an Abstract Syntax Tree (AST) (parsing/parsetree.mli has lot of helpful comments). See parsing/HACKING.adoc.
The logic for Camlp4 and Ppx preprocessing is not in parsing/, but in driver/, see driver/pparse.mli and driver/pparse.ml.
Typing — typing/
Type-checks the AST and produces a typed representation of the program (typing/typedtree.mli has some helpful comments). See typing/HACKING.adoc.
The bytecode compiler — bytecomp/
The native compiler — middle_end/ and asmcomp/
- stdlib/
-
The standard library. Each file is largely independent and should not need further knowledge.
- otherlibs/
-
External libraries such as
unix,threads,dynlink,strandbigarray.
Instructions for building the full reference manual are provided in
manual/README.md. However, if you only modify the documentation
comments in .mli files in the compiler codebase, you can observe the
result by running
make html_doc
and then opening ./ocamldoc/stdlib_html/index.html in a web browser.
- lex/
-
The
ocamllexlexer generator. - yacc/
-
The
ocamlyaccparser generator. We do not recommend using it for user projects in need of a parser generator. Please consider using and contributing to menhir instead, which has tons of extra features, lets you write more readable grammars, and has excellent documentation.
- BOOTSTRAP.adoc
-
instructions for bootstrapping
- Changes
-
what’s new with each release
- CONTRIBUTING.md
-
how to contribute to OCaml
- HACKING.adoc
-
this file
- INSTALL.adoc
-
instructions for installation
- LICENSE
-
license and copyright notice
- Makefile
-
main Makefile
- Makefile.common
-
common Makefile definitions
- Makefile.tools
-
used by manual/ and testsuite/ Makefiles
- README.adoc
-
general information on the compiler distribution
- README.win32.adoc
-
general information on the Windows ports of OCaml
- VERSION
-
version string
- asmcomp/
-
native-code compiler and linker
- boot/
-
bootstrap compiler build-aux/: autotools support scripts
- bytecomp/
-
bytecode compiler and linker
- compilerlibs/
-
the OCaml compiler as a library
- configure
-
configure script configure.ac: autoconf input file
- debugger/
-
source-level replay debugger
- driver/
-
driver code for the compilers
- flexdll/
-
git submodule — see README.win32.adoc
- lex/
-
lexer generator
- man/
-
man pages
- manual/
-
system to generate the manual
- middle_end/
-
the flambda optimisation phase
- ocamldoc/
-
documentation generator
- ocamltest/
-
test driver
- otherlibs/
-
several additional libraries
- parsing/
-
syntax analysis — see parsing/HACKING.adoc
- runtime/
-
bytecode interpreter and runtime systems
- stdlib/
-
standard library
- testsuite/
-
tests — see testsuite/HACKING.adoc
- tools/
-
various utilities
- toplevel/
-
interactive system
- typing/
-
typechecking — see typing/HACKING.adoc
- utils/
-
utility libraries
- yacc/
-
parser generator
The separately-distributed script
opam-compiler-conf can
be used to easily build opam switches out of a git branch of the
compiler distribution. This lets you easily install and test opam
packages from an under-modification compiler version.
Besides the targets listed in INSTALL.adoc for build and installation, the following targets may be of use:
make runtop-
builds and runs the ocaml toplevel of the distribution (optionally uses
rlwrapfor readline+history support) make natruntop-
builds and runs the native ocaml toplevel (experimental)
make partialclean-
Clean the OCaml files but keep the compiled C files.
make depend-
Regenerate the
.dependfile. Should be used each time new dependencies are added between files. make -C testsuite parallel
Additionally, there are some developer specific targets in Makefile.dev. These targets are automatically available when working in a Git clone of the repository, but are not available from a tarball.
If you have options to configure which you always (or at least frequently)
use, it’s possible to store them in Git, and configure will automatically add
them. For example, you may wish to avoid building the debug runtime by default
while developing, in which case you can issue
git config --global ocaml.configure '--disable-debug-runtime'. The configure
script will alert you that it has picked up this option and added it before
any options you specified for configure.
Options are added before those passed on the command line, so it’s possible to
override them, for example ./configure --enable-debug-runtime will build the
debug runtime, since the enable flag appears after the disable flag. You can
also use the full power of Git’s config command and have options specific to
particular clone or worktree.
configure includes the standard -C option which caches various test results
in the file config.cache and can use those results to avoid running tests in
subsequent invocations. This mechanism works fine, except that it is easy to
clean the cache by mistake (e.g. with git clean -dfX). The cache is also
host-specific which means the file has to be deleted if you run configure with
a new --host value (this is quite common on Windows, where configure is
also quite slow to run).
You can elect to have host-specific cache files by issuing
git config --global ocaml.configure-cache .. The configure script will now
automatically create ocaml-host.cache (e.g. ocaml-x86_64-pc-windows.cache,
or ocaml-default.cache). If you work with multiple worktrees, you can share
these cache files by issuing git config --global ocaml.configure-cache ... The
directory is interpreted relative to the configure script.
The OCaml compiler is bootstrapped. This means that previously-compiled bytecode versions of the compiler and lexer are included in the repository under the boot/ directory. These bytecode images are used once the bytecode runtime (which is written in C) has been built to compile the standard library and then to build a fresh compiler. Details can be found in BOOTSTRAP.adoc.
Once you’ve built a natively-compiled ocamlc.opt, you can use it to
speed up future builds by copying it to boot:
cp ocamlc.opt boot/
If boot/ocamlc changes (e.g. because you ran make bootstrap), then
the build will revert to the slower bytecode-compiled ocamlc until
you do the above step again.
The script that is run on Travis continuous integration servers is tools/ci/travis/travis-ci.sh; its configuration can be found as a Travis configuration file in .travis.yml.
For example, if you want to reproduce the default build on your machine, you can use the configuration values and run command taken from .travis.yml:
CI_KIND=build XARCH=x64 bash -ex tools/ci/travis/travis-ci.sh
The scripts support two other kinds of tests (values of the
CI_KIND variable) which both inspect the patch submitted as part of
a pull request. tests checks that the testsuite has been modified
(hopefully, improved) by the patch, and changes checks that the
Changes file has been modified (hopefully to add a new entry).
These tests rely on the $TRAVIS_COMMIT_RANGE variable which you can
set explicitly to reproduce them locally.
The changes check can be disabled by including "(no change
entry needed)" in one of your commit messages — but in general all
patches submitted should come with a Changes entry; see the guidelines
in CONTRIBUTING.md.
INRIA provides a Jenkins continuous integration service that OCaml uses, see https://ci.inria.fr/ocaml/. It provides a wider architecture support (MSVC and MinGW, a zsystems s390x machine, and various MacOS versions) than the Travis/AppVeyor testing on github, but only runs on commits to the trunk or release branches, not on every PR.
You do not need to be an INRIA employee to open an account on this jenkins service; anyone can create an account there to access build logs and manually restart builds. If you would like to do this but have trouble doing it, please email [email protected].
To be notified by email of build failures, you can subscribe to the [email protected] mailing list by visiting its web page.
If you have suspicions that your changes may fail on exotic architectures (they touch the build system or the backend code generator, for example) and would like to get wider testing than github’s CI provides, it is possible to manually start INRIA’s CI on arbitrary git branches even before opening a pull request as follows:
-
Make sure you have an account on Inria’s CI as described before.
-
Make sure you have been added to the ocaml project.
-
Prepare a branch with the code you’d like to test, say "mybranch". It is probably a good idea to make sure your branch is based on the latest trunk.
-
Make your branch publicly available. For instance, you can fork OCaml’s GitHub repository and then push "mybranch" to your fork.
-
Visit https://ci.inria.fr/ocaml/job/precheck and log in. Click on "Build with parameters".
-
Fill in the REPO_URL and BRANCH fields as appropriate and run the build.
-
You should receive a bunch of e-mails with the build logs for each slave and each tested configuration (with and without flambda) attached.
INRIA’s CI "main" and "precheck" jobs run the script tools/ci-build. In particular, when running the CI on a publicly available branch via the "precheck" job as explained in the previous section, you can edit this script to change what the CI will test.
For instance, parallel builds are only tested for the "trunk" branch. In order to use "precheck" to test parallel build on a custom branch, add this at the beginning of tools/ci-build:
OCAML_JOBS=10
If you would like to receive email notifications of all commits made to the main git repository, you can subscribe to the [email protected] mailing list by visiting its web page.
Happy Hacking!