permutation_importance

Builtin index · Statistics & ML

result = permutation_importance(model, X, y, [n_repeats=])

Input: model, X, y, n_repeats · Output: result

What it is

Shuffles one column of an N-row, D-column X at a time and re-scores an already-fitted model Record model against the matching length-N target y, over n_repeats shuffles per column (integer).

Overloads

One call shape — this function is not overloaded.

Code	Input	Output	Description
`result = permutation_importance(model, X, y, [n_repeats=])`	`model` Recordnot documented yet `X` Recordnot documented yet `y` Recordnot documented yet `n_repeats` Record, optionalnot documented yet	`result` vector	Returns a length-`D` vector of importance scores (drop in score caused by shuffling that column); works on any fitted model because it perturbs the input rather than reaching inside the model itself.

Examples

From the documentation, and run by the build -- the output and the figure below are what the engine actually produced. Simplest first.

print("corrcoef matrix:\n{round(corrcoef(X), 3)}")

m = ols_model(X, y)
yhat = m.predict(X)
print("mae: {round(mae(y, yhat), 4)}, mse: {round(mse(y, yhat), 4)}")

cm = knn_model(Xc, yc, 5, kind = "classification")
predc = cm.predict(Xc)
print("precision: {round(precision(yc, predc), 3)}")
print("recall:    {round(recall(yc, predc), 3)}")
print("f1:        {round(f1(yc, predc), 3)}")

folds = k_fold(X, y, 4)
print("k_fold: {len(folds)} folds, first fold trains on {rows(folds[0].X_train)} rows")

stab = cv_stability(Xc, yc, n_repeats = 5, cv_folds = 3)
print("cv_stability: {round(mean(stab), 3)} +/- {round(std(stab), 3)}")

ab = ablation_study(Xc, yc, cv_folds = 3)
print("ablation_study scores: {round(ab, 4)}")
print("rfe keeps:             {rfe(Xc, yc, 1)}")
print("mutual_info_classif:   {round(mutual_info_classif(Xc, yc, k = 3), 4)}")
print("astar_mrmr keeps:      {astar_mrmr(Xc, yc, 1)}")
print("permutation_importance: {round(permutation_importance(m, X, y, n_repeats = 5), 4)}")

output

corrcoef matrix:
[1, -0.119, 0.007; -0.119, 1, -0.036; 0.007, -0.036, 1]
mae: 0.0847, mse: 0.011
precision: 0.995
recall:    0.995
f1:        0.995
k_fold: 4 folds, first fold trains on 90 rows
cv_stability: 0.908 +/- 0.012
ablation_study scores: [0.2401, 0.1301]
rfe keeps:             [0]
mutual_info_classif:   [0.2398, 0.1939]
astar_mrmr keeps:      [0]
permutation_importance: [1.5394, 0.3327, 0.0947]

from statistics-ml

How it works

permutation_importance(model, X, y, [n_repeats=10], [seed=]) — the standard model-agnostic explainability method (any fitted _model handle with .predict/.score — not just the tree-based kinds ablation_study/rfe/ astar_mrmr are restricted to, since this perturbs the *input*, never touches the model's own internals): shuffle one feature column at a time (breaking its relationship with y while leaving every other column's joint distribution untouched), re-score the already-fitted model, and average the score drop over n_repeats independent shuffles. A large drop means the model actually depends on that feature; ~0 or negative means it doesn't (or the shuffle got lucky). Genuinely single-threaded (each model_predict/model_score call needs &mut Interp, unlike ablation_study/cv_stability/rfe/astar_mrmr's direct-Rust TreeBuilder calls) — but each call is one cheap prediction pass, not a fit, so this doesn't need the same acceleration those do.

From the note above this builtin in the interpreter, where the reasoning was written down as it was built.