Manage benchmark results#
Collect benchmark results#
Once the benchmark is run, the results are stored in a directory
./results
in the benchmark directory, with a .parquet
file.
By default, the name of the file include the date and time of the run,
but a custom name can be given using the --output
option of
benchopt run
.
This result file is produced only once the full benchmark has been run.
When the benchmark is run in parallel, the results that have already been
computed can be collected using the --collect
option with
benchopt run
. Adding this option with the same command line will
produce a parquet file with all the results that have been computed so far.
Clean benchmark results#
The results and cache of previously run benchmark can be cleaned using the
benchopt clean
command. This command will remove the ./results
and
./__cache__
directories in the benchmark directory.
Publish benchmark results#
Benchopt allows you to publish your benchmark results to
the Benchopt Benchmarks website
with one command benchopt publish
.
The publish
command will send your results to GitHub by opening
a pull-request on the Benchopt results repository.
Once the pull-request is merged it will appear automatically online.
Workflow example:
$ git clone https://github.com/benchopt/benchmark_logreg_l2
$ benchopt run ./benchmark_logreg_l2
$ benchopt publish ./benchmark_logreg_l2 -t <GITHUB_TOKEN>
You see that to publish you need to specify the value of <GITHUB_TOKEN>
.
This GitHub access token contains 40 alphanumeric characters that allow GitHub
to identify you and use your account.
After getting your personal token as explained below the last
line will read something like:
$ benchopt publish ./benchmark_logreg_l2 -t 1gdgfej73i72if0852a685ejbhb1930ch496cda4
Warning
Your GitHub access token is a sensitive information. Keep it secret as it is as powerful as your GitHub password!
Let’s now see how to create your personal GitHub token.
Obtaining a GitHub token#
Visit settings/tokens
and click on generate new token
.
Then create a token named benchopt, ticking the repo box as shown below:
![../_images/github_benchopt_token.png](../_images/github_benchopt_token.png)
Then click on generate token
and copy this token of 40 characters in a
secure location. Note that the token can be stored in a config file for benchopt
using benchopt config set github_token <TOKEN>
. More info on config files can
be found in config_doc.