Performance Evaluation

In this notebook we explore the performance evaluation tools of cbxpy. These are provided in the module cbx.utils.success. For a test problem, we consider the Rastrigin function in dimension d=20. We employ N=40 particles and perform M=100 runs.

%load_ext autoreload
%autoreload 2
import cbx
import cbx.dynamics as dyns
import cbx.utils.success as success
import numpy as np

f = cbx.objectives.Rastrigin()
M = 100
N = 40
d = 20

x = np.random.uniform(-3,3,(M,N,d))
kwargs = {'x':x, 'sigma':10.1, 'verbosity':0, 'max_it':5000, 'noise':'anisotropic', 'alpha':30, 'dt':0.01, 'f_dim':'3D'}
x_true = np.zeros((x.shape[-1],))

Perform optimization

Using the keywords from above, we define a dynamic and run the optimization.

dyn = dyns.CBO(f,**kwargs)
dyn.optimize();

Performance evaluation

To evaluate the performance, we use the evaluation class, which accepts a list of performance criteria. Each criterion should have a call function that accepts a dynamic as the input and outputs a dictionary containing the following key-value pairs:

• rate: the success rate,

• num: the absolute number of successful runs,

• idx: the indices of the successful runs.

We can directly apply this to our dynamic to get a result. Here we use the criterion dist_to_min, which needs the true minimum as the input. This criterion will be satisfied if

\[\|x_{\text{true}} - x\|_p < \texttt{tol},\]

where the tolerance and \(p\) of the norm can also be specified.

seval = success.evaluation(criteria = [success.dist_to_min(x_true, tol=0.25, p=float('inf'))])
seval(dyn);

------------------------------
Results of success evaluation:
Success Rate: 1.0
Succesful runs: 100
Succesful idx: [ 0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19 20 21 22 23
 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47
 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71
 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95
 96 97 98 99]