cv_stability

Builtin index · Statistics & ML

result = cv_stability(X, y, [n_repeats=10], [cv_folds=5])

Input: X, y, n_repeats, cv_folds · Output: result

What it is

Repeats cross-validation n_repeats times (integer, default 10) with cv_folds folds each (integer, default 5) on an N-row X and length-N y, reshuffling between repeats.

Overloads

2 forms. Each carries its own inputs, outputs and worked example — pick the tab for the one you are calling.

classification (the default) regression

Code	Input	Output	Description
`result = cv_stability(X, y, [n_repeats=10], [cv_folds=5])`	`X` —not documented yet `y` —not documented yet `n_repeats` optional (default `10`)not documented yet `cv_folds` optional (default `5`)not documented yet	`result` vector or number	Same default as `ablation_study`: with no `kind=`, `cv_stability` repeats classifier cross-validation and reports accuracy scores.

Examples

Written for this form and run by the build -- the output below is what the engine actually produced.

Xcs = randn(60, 2, seed = 91)
ycs = zeros(60)
for i in 0 to 59
    if Xcs[i, 0] + Xcs[i, 1] > 0
        ycs[i] = 1
    end if
end for
print("cv_stability, default kind (classification): {round(cv_stability(Xcs, ycs, n_repeats = 3, cv_folds = 3), 4)}")

output

cv_stability, default kind (classification): [0.8333, 0.7667, 0.85]

Code	Input	Output	Description
`result = cv_stability(X, y, [n_repeats=10], [cv_folds=5])`	`X` —Repeats cross-validation `n_repeats` times (integer, default 10) with `cv_folds` folds each (integer, default 5) on an `N`-row `X` and length-`N` `y`, reshuffling between repeats. `y` —On a continuous `y`, the same default silently returns all zeros. `n_repeats` optional (default `10`)Repeats cross-validation `n_repeats` times (integer, default 10) with `cv_folds` folds each (integer, default 5) on an `N`-row `X` and length-`N` `y`, reshuffling between repeats. `cv_folds` optional (default `5`)Repeats cross-validation `n_repeats` times (integer, default 10) with `cv_folds` folds each (integer, default 5) on an `N`-row `X` and length-`N` `y`, reshuffling between repeats.	`result` vector or number	—

Examples

Written for this form and run by the build -- the output below is what the engine actually produced.

Xrs = randn(60, 3, seed = 94)
betas = [2.0, -1.0, 0.5]
yrs = Xrs * betas + 0.05 * randn(60, seed = 95)
rk2 = "regression"
print("default kind on a continuous target: {round(cv_stability(Xrs, yrs, n_repeats = 3, cv_folds = 3), 4)}")   # wrong! all zero

print("kind=regression on the same target:  {round(cv_stability(Xrs, yrs, n_repeats = 3, cv_folds = 3, kind = rk2), 4)}")

output

default kind on a continuous target: [0, 0, 0]
kind=regression on the same target:  [0.293, 0.5255, 0.3916]

How it works

cv_stability(X, y, [n_repeats=10], [cv_folds=5], [max_depth=3], [seed=], [kind="classification"]) — see cv_stability_scores's own doc comment. Returns the raw per-repeat CV scores; mean(...)/std(...) on the result gives the usual "performance +- variability" summary.

From the note above this builtin in the interpreter, where the reasoning was written down as it was built.