I'm using erlang for a personal research project, and wanted easy support for literate programming. This is what I hacked together. I'm making this available to the public in case it's useful to anyone else.
compyl makes it easy to apply arbitrary text transformations to a file before passing it to the erlang compiler.
The compyl module is a simple wrapper around the standard erlang compiler, which accepts two additional options:
{text_transform, Module, Function}
{scan_transform, Module, Function}
If the option {text_transform, mymod, do_stuff} is specified, the text of the file will be transformed by mymod:do_stuff/1 before it's passed on to the erlang compiler. The scan_transform option allows you to specify a function to transform the list of tokens generated by erl_scan.
These are, of course, roughly analogous to the built-in {parse_transform, Module} option.
One obvious application of this is (haskell-style) literate programming. For example, consider the following Markdown file, "my_math.md":
My Great Math Module
====================
This module defines some commonly used math functions.
(Note: This is an erlang module defined in a markdown file. Any line prefixed
with 4 spaces or a tab character will be interpreted as erlang code.)
-module(my_math).
-export([add/2]).
To treat this file as erlang code and compile it, we must pass it through the
appropriate `text_transform` function.
-compile({text_transform, literate, markdown}).
Now, onto the good stuff.
add/2: returns the sum of the two arguments
add(X, Y) -> X + Y.
If we compile this with compyl, we can use it as a normal erlang module:
1> compyl:c("my_math.md").
{ok, my_math}
2> my_math:add(2, 2).
4
Here, the line
-compile({text_transform, literate, markdown}).
tells compyl to pass the text of the file through literate:markdown/1 before compiling it. literate:markdown/1 simply uses a regular expression to delete the contents of any line that doesn't start with 4 spaces or a tab character.
git clone [email protected]:sbillig/compyl.git
Compile the .erl files and put them on your erlang path.
erlc compyl.erl literate.erl multiline_comments.erl
1> code:add_pathz("/path/to/compyl").
2> compyl:c("/path/to/compyl/test/latex_test2.tex").
{ok, latex_test2}.
Within literate.erl there are currently text_transform functions for literate source code files written in latex, markdown, and Bird-style. Additional transforms can be added easily, see below.
The appropriate compiler options are, respectively:
{text_transform, literate, latex} % see latex_test2.tex
{text_transform, literate, markdown} % see markdown_test.md
{text_transform, literate, bird} % see bird_test.lerl
There's also support for (ugly) multiline comments:
-module(amod).
-compile([export_all, {scan_transform, multiline_comments, strip_comments}]).
::"
this is a
comment spanning
several lines
"::
f() -> ::"this is also a comment"::
X = ::"and so is this":: 2,
X*2.
The choice of characters is arbitrary. Other multiline comment syntax could be supported by writing a different scan_transform or text_transform function.
A text_transform can be any unary function that accepts a string and returns a string, usually one containing valid erlang syntax. The returned string will be passed onto the erlang compiler (or to the next text_transform function). Input and output are binary strings, like those returned by file:read_file/1 (eg. <<"-module(amod).\n-compile(...">>).
For example, the literate:markdown/1 is defined as:
markdown(Bin) -> B = re:replace(Bin, "^(?! |\\t)[^\\n]+","", [global,multiline]), iolist_to_binary(B).
which simply finds all lines that don't start with four spaces or a tab, and replaces them with nothingness. Ideally, transforms should maintain line numbers. That is, if a piece of erlang code is on line 14 of the input string, it should be on line 14 of the output string as well.
If you want to apply the text_transform function do_some_stuff of the module mytransforms to a file, just add the compiler option to the file:
-compile({text_transform, mytransforms, do_some_stuff}).
A scan_tranform function should accept a list of tokens (as returned by erl_scan:string/1), and return a list of tokens.
compyl:file will compile the module defined in somefile.whatever, and save it as <modulename>.beam in the current directory. Note that filename and extension can be anything, and don't have to match the name of the module defined in the file, or the content type of the file. Analogous to compile:file.
compyl:file("path/to/somefile.whatever")
compyl:file(Path, Options)
compyl:c compiles, then purges and loads the code for a file, as in c:c.
compyl:c("path/to/somefile.whatever").
compyl:c(Path, Options).
If you're interested in literate programming in erlang, see also Joe Armstrong's EWEB.