Factor Language Blog

New suite of benchmarks

Friday, August 10, 2007

I spent a bit of time cleaning up various benchmarks and putting them over a common framework. Now you can do "benchmark" run to run a suite of benchmarks. The results are collected and printed in a table:

Benchmark                    Run time (ms) GC time (ms)
benchmark.continuations      473           3
benchmark.empty-loop         480           0
benchmark.fib1               194           0
benchmark.fib2               855           0
benchmark.fib3               1418          0
benchmark.fib4               3334          60
benchmark.fib5               2126          11
benchmark.iteration          8829          23
benchmark.mandel             4421          77
benchmark.nsieve             2221          3
benchmark.nsieve-bits        65679         239
benchmark.partial-sums       26293         183
benchmark.raytracer          26031         242
benchmark.recursive          36740         233
benchmark.reverse-complement 15637         167
benchmark.ring               9124          79
benchmark.sort               1934          38
benchmark.spectral-norm      32142         991
benchmark.sum-file           287           0

I want to set up a continuous testing cluster with a number of machines. The machines would perform the following tasks:

  • Build Factor
  • Run unit tests
  • Run benchmarks
  • Create deployment images for various modules and upload them to a “Factor application central” web site