The shootin

NEWS! the first specification (not complete yet) is here.

teaser

On the face of it, shootin is a system like the 'Great Computer Language Shootout' . The shootin's focus is on comparing programs that produce the same results (aka 'the same thing') so the process can be automated. The shootout focuses on comparing 'the same way', ie. comparing the same algorithm written in different languages on different implementations. (the shootout is a little subjective due to the algorithm restriction so the process cannot be automated).

Two measures of software goodness are execution time (shorter is usually better) and source length which is often measured as lines-of-code. As many languages have programs that do not change after stripping newlines, the shootin measures the size of source in bytes after stripping comments.

The shootout will also measure how long it takes to produce a working program: a snapshot will be taken at each commit and see who got there first.

Mechanism

see sourceforge CVS instructions

anonymous CVS:

$ export CVSROOT=:pserver:anonymous@cvs.sourceforge.net:/cvsroot/shootin

CVS for shootin developers:

$ export CVSROOT=:ext:SFNAME@cvs.sf.net:/cvsroot/shootin

First steps

$ mkdir shootin && cd shootin
$ cvs checkout s
$ s/c ls       # list cvs modules which your system may support
python lisp c  # it appears that your system has python, lisp and c
$ cvs checkout `s/c ls`
and checkout the shootout source. the shootin uses tests from the shootout while there is not a decent set of shootin tests. 
$ cvs -z3 -d:pserver:anonymous@cvs.alioth.debian.org:/cvsroot/shootout co shootout

file structure after all that

shootin/
       s/
              c       # the controller: calls drivers and times tests
              found   # a table of commands needed for each testsuite
              soe.py  # two files supporting c and driver programs
              drv.py  #
       data/
              words
       c/
              gcc-3/
                           driver  # runs configure and make for s/c
                           cmds    # contributes to s/found
       python/
              python-2.3.5/
                           cmds 
                           driver 
       shootout/      # source for the shootout
just a one-liner to build, make and report 
$ s/c report
you'll find that you need python 2.3 or greater as well as matplotlib to draw the graph (if you want to draw one)

not just languages

the shootin is designed to accommodate other types of tests. for example, there may be a test that compares windows to linux for reading and writing hard disk files or sockets or spawning threads.

contacts

SourceForge.net Logo