Using different programming languages with benchopt#
Benchopt supports different types of solvers:
Python solver#
The simplest solvers to use are solvers using Python code. Here is an example:
from benchopt import BaseSolver
import numpy as np
class Solver(BaseSolver):
# Name of the Solver, used to select it in the CLI
name = 'gd'
# By default, benchopt will evaluate the result of a method after various
# number of iterations. Setting the sampling_strategy controls how this is
# done. Here, we use a callback function that is called at each iteration.
sampling_strategy = 'callback'
# Parameters of the method, that will be tested by the benchmark.
# Each parameter ``param_name`` will be accessible as ``self.param_name``.
parameters = {'lr': [1e-3, 1e-2]}
# The three methods below define the necessary methods for the Solver, to
# get the info from the Objective, to run the method and to return a
# result that can be evaluated by the Objective.
def set_objective(self, X):
"""Set the info from a Objective, to run the method.
This method is also typically used to adapt the solver's parameters to
the data (e.g. scaling) or to initialize the algorithm.
The kwargs are the keys of the dictionary returned by
``Objective.get_objective``.
"""
self.X = X
self.X_hat = np.zeros_like(X)
def run(self, cb):
"""Run the actual method to benchmark.
Here, as we use a "callback", we need to call it at each iteration to
evaluate the result as the procedure progresses.
The callback implements a stopping mechanism, based on the number of
iterations, the time and the evoluation of the performances.
"""
while cb():
self.X_hat = self.X_hat - self.lr * (self.X_hat - self.X)
def get_result(self):
"""Format the output of the method to be evaluated in the Objective.
Returns a dict which is passed to ``Objective.evaluate_result`` method.
"""
return {'X_hat': self.X_hat}
If your Python solver requires some packages, benchopt allows you to list its requirements. See Specifying requirements for more details on how to specify the requirements for benchopt classes.
R solver#
A solver written in R needs two files.
A .R file that contains the solver and a .py file that knows how to
call the R solver using Rpy2. Only the
extensions should differ between the two files. Here is the Python file:
from pathlib import Path
from benchopt import BaseSolver
import numpy as np
# Import helpers from rpy2 and benchopt.helpers.r_lang
from benchopt.helpers.r_lang import import_func_from_r_file, converter_ctx
# Import R function defined in r_pgd.R so they can be retrieved as python
# functions using `func = robjects.r['FUNC_NAME']`
R_FILE = str(Path(__file__).with_suffix('.R'))
class Solver(BaseSolver):
name = "R-PGD"
install_cmd = 'conda'
requirements = ['r-base', 'rpy2']
sampling_strategy = 'iteration'
parameters = {'lr': [1e-3, 1e-2]}
def set_objective(self, X):
self.X = X
robjects = import_func_from_r_file(R_FILE)
self.r_gd = robjects.r['gradient_descent']
def run(self, n_iter):
with converter_ctx():
coefs = self.r_gd(
self.X, self.lr, n_iter=n_iter
)
self.X_hat = np.asarray(coefs)
def get_result(self):
return {'X_hat': self.X_hat}
It uses the R code in:
##' Functions used in GD algorithm
##'
##' @title Functions used in GD algorithm
##' @author Thomas Moreau
##' @export
# Main algorithm
gradient_descent <- function(X, lr, n_iter) {
# --------- Initialize parameter ---------
p <- ncol(X)
parameters <- X * 0
# --------- Run GD for n_iter iterations ---------
for (i in 1:n_iter) {
# Compute the gradient
grad <- (parameters - X)
# # Update the parameters
parameters <- parameters - lr * grad
}
return(parameters)
}
Julia solver#
A solver written in Julia needs two files.
A .jl file that contains the solver and a .py file that knows how to
call the Julia solver using PyJulia.
Only the extensions should differ between the two files.
Here is the Python file:
from pathlib import Path
from benchopt.helpers.julia import JuliaSolver
from benchopt.helpers.julia import get_jl_interpreter
JULIA_SOLVER_FILE = str(Path(__file__).with_suffix('.jl'))
class Solver(JuliaSolver):
name = "Julia-GD"
sampling_strategy = "iteration"
parameters = {"lr": [1e-3, 1e-2]}
requirements = [
"https://repo.prefix.dev/julia-forge::julia",
"pip::julia",
]
def set_objective(self, X):
self.X = X
jl = get_jl_interpreter()
self.julia_gd = jl.include(JULIA_SOLVER_FILE)
def warm_up(self):
# Make sure we don't account for the Julia loading time in the
# first iteration of the benchmark.
self.julia_gd(self.X, self.lr, 20)
def run(self, n_iter):
# Here we cannot call a python callback, so we call iteratively
# the solver with a growing number of iterations.
self.X_hat = self.julia_gd(self.X, self.lr, n_iter)
def get_result(self):
return dict(X_hat=self.X_hat)
It uses the Julia code in:
using Core
function gradient_descent(X, lr, n_iter)
X_hat = zeros(size(X))
for i ∈ 1:n_iter
grad = X_hat - X
X_hat -= lr * grad
end
return X_hat
end
Installing Julia dependencies
Note that it is also possible to install julia dependencies using benchopt install with the class attribute julia_requirements. This attribute should be a list of package names, whose string are directly passed to Pkg.add.
In case it is necessary to install dependencies from a GitHub repository, one can use the following format: PkgName::https://github.com/org/Pkg.jl#branch_name. This will be processed to recover both the url and the package name. Note that the branch_name is optional. Using the :: to specify the PkgName is necessary to allow benchopt to check if it is installed in the targeted environment.
Solver from source#
You can install a package from source in case it is not available as binaries from the package managers from either Python, R or Julia.
Note
A package available from source may require a C++ or Fortran compiler.
Note
See for example on the L1 logistic regression benchmark for
an example
that uses a 'shell' as install_cmd.