Sci-logger

Introduction

Suppose you are doing a lots of machine learning experiments (or other experiments which process data) with many different parameter and algorithm settings and you want to keep track of what you have done. Sci-logger keeps track of what you changed in your code and records outcome of your experiments so you can compare them later.

Requirements

• You use command line to run your experiments
• Git
• Python 2.7

Basic demo

Let's create a new directory for our demo:

:~$ mkdir sci-logger-demo
:~$ cd sci-logger-demo/
:~/sci-logger-demo$

Now download the zipball and extract logic.py and sci-loger.py into this directory:

~/sci-logger-demo$ ls
logic.py  sci-loger.py

Now we need to init and empty git repository (note that we don't need any github or other git repository provider):

~/sci-logger-demo$ git init

We can create a simple "experiment" now, which outcome would be "Hello world!":

~/sci-logger-demo$ cat >test.py
print "Hello world!"
~/sci-logger-demo$ python test.py 
Hello world!

Now we need to tell our logger, that we want to track our code:

~/sci-logger-demo$ python sci-loger.py -a test.py
creating folder structure (first run)
test.py added succesfully

We can run our experiment with tracking (note that you need to put in comment for experiment):

~/sci-logger-demo$ python sci-loger.py -x "python test.py"
Write comment to this experiment                                                                                                                                                                                                                                               
First test                                                                                                                                                                                                                                                                     
Committed                                                                                                                                                                                                                                                                      
Code is running                                                                                                                                                                                                                                                                
Hello world!                                                                                                                                                                                                                                                                   
2014-09-18_12:02:26.859468 c8be73

You can list all experiments:

~/sci-logger-demo$ python sci-loger.py -s                                                                                                                                                                                                                 
1 2014-09-18_12:02:26  c8be73  First test 

Also all the outputs:

python sci-loger.py -n code_out
/home/usamec/sci-logger-demo/.snaps/2014-09-18_12:02:26.859468/code_out

You can check the output:

cat /home/usamec/sci-logger-demo/.snaps/2014-09-18_12:02:26.859468/code_out
Hello world!

Or you can write a script which processes each output:

~/sci-logger-demo$ python sci-loger.py -e code_out "cat"
/home/usamec/sci-logger-demo/.snaps/2014-09-18_12:02:26.859468/code_out
Hello world!

Also if you want to get back to some place, just checkout the appropriate git commit.

Support or Contact

If you have trouble using scilogger or have any idea for improvement, you can contact us at vlado.boza@blackswanrational.com and we’ll help you sort it out.