Evaluators module

`Prequential`([metrics])	Prequential data stream evaluator.
`TestThenTrain`([metrics, verbose])	Test Than Train data stream evaluator.

class strlearn.evaluators.Prequential(metrics=(<function accuracy_score>, <function balanced_accuracy_score>))

Bases: object

Prequential data stream evaluator.

Implementation of prequential evaluation procedure, based on sliding windows instead of separate data chunks. Window moves by a fixed number of instances in order to preserve some of the already processed ones. After each step, samples that are currently in the window are used to test the classifier and then for training.

Parameters:

metrics (tuple or function) – Tuple of metric functions or single metric function.

Variables:

classes (array-like, shape (n_classes, )) – The class labels.
scores (array-like, shape (stream.n_chunks, len(metrics))) – Values of metrics for each processed data chunk.

Example:

>>> import strlearn as sl
>>> stream = sl.streams.StreamGenerator()
>>> clf = sl.classifiers.AccumulatedSamplesClassifier()
>>> evaluator = sl.evaluators.PrequentialEvaluator()
>>> evaluator.process(clf, stream, interval=50)
>>> print(evaluator.scores_)
...
[[0.95       0.9483469  0.94805282 0.9483469  0.95412844]
[0.96       0.95728313 0.95696445 0.95728313 0.96460177]
[0.96       0.95858586 0.95848154 0.95858586 0.96396396]
...
[0.92       0.91987179 0.91986621 0.91987179 0.91666667]
[0.91       0.91065705 0.91050889 0.91065705 0.90816327]
[0.925      0.92567027 0.9250634  0.92567027 0.92610837]]

process(stream, clfs, interval=100)

Perform learning procedure on data stream.

Parameters:

stream (object) – Data stream as an object
clfs (tuple or function) – scikit-learn estimator of list of scikit-learn estimators.
interval (integer, optional (default=100)) – The number of instances by which the sliding window moves before the next evaluation and training steps.

class strlearn.evaluators.TestThenTrain(metrics=(<function accuracy_score>, <function balanced_accuracy_score>), verbose=False)

Bases: object

Test Than Train data stream evaluator.

Implementation of test-then-train evaluation procedure, where each individual data chunk is first used to test the classifier and then it is used for training.

Parameters:

metrics (tuple or function) – Tuple of metric functions or single metric function.
verbose (boolean) – Flag to turn on verbose mode.

Variables:

classes (array-like, shape (n_classes, )) – The class labels.
scores (array-like, shape (stream.n_chunks, len(metrics))) – Values of metrics for each processed data chunk.

Example:

>>> import strlearn as sl
>>> stream = sl.streams.StreamGenerator()
>>> clf = sl.classifiers.AccumulatedSamplesClassifier()
>>> evaluator = sl.evaluators.TestThenTrainEvaluator()
>>> evaluator.process(clf, stream)
>>> print(evaluator.scores_)
...
[[0.92       0.91879699 0.91848191 0.91879699 0.92523364]
[0.945      0.94648779 0.94624912 0.94648779 0.94240838]
[0.92       0.91936979 0.91936231 0.91936979 0.9047619 ]
...
[0.92       0.91907051 0.91877671 0.91907051 0.9245283 ]
[0.885      0.8854889  0.88546135 0.8854889  0.87830688]
[0.935      0.93569212 0.93540766 0.93569212 0.93467337]]

process(stream, clfs)

Perform learning procedure on data stream.

Parameters:

stream (object) – Data stream as an object
clfs (tuple or function) – scikit-learn estimator of list of scikit-learn estimators.