'doctest' in Factor
Wednesday, December 13, 2006
The doctest module for Python scans your source file for code examples in docstrings, and evaluates them to check against the correct result. The documentation for this module seems to suggest it is a rather non-trivial affair (and looking at the code confirms this). In comparison, implementing this feature in Factor takes a few minutes:
: all-word-help ( -- seq )
all-words [ word-help ] map [ ] subset ;
: all-examples ( words -- elements )
[ \ $example swap elements ] map [ empty? not ] subset ;
: example-valid? ( elements -- ? )
1 tail
[ 1 head* "\n" join eval>string "\n" ?tail drop ] keep
peek = ;
: incorrect-examples ( words -- elements )
all-examples
[ [ example-valid? ] all? not ] subset ;
all-word-help incorrect-examples [ . ] each
Factor’s help is written in a structured markup language where examples
are explicitly denoted as such (and in the UI, you can click them to
evaluate them). So all you have to do is iterate over all words in the
dictionary, extract their documentation, collect $example
markup
elements, and ensure they do the right thing. Note that $example
elements look like so:
{ $example "2 2 + ." "4" }
All but the last element of the array are strings which are joined together with newlines in between to form the expression to be tested, and the last string is the expected output.
Using this tool I actually found some typos in the Factor documentation. I think the idea is great, and while Python makes it easier than something like Java (where you’d have to write a Doclet and use BeanShell, or something), Factor makes it even easier because it has an help system with a markup language, not just docstrings.
I have some other bits of code sitting around for checking consistency of parameter names in documentation, and checking for broken links and markup. One day I’m going to put them together into a single ’lint’-style tool for documentation.