mutual_info_classif

Builtin index · Statistics & ML

result = mutual_info_classif(X, y, [k=3])

Input: X, y, k · Output: result

What it is

k-nearest-neighbour mutual information between each column of an N-row, D-column X and a length-N discrete label vector y, using k neighbours (integer, default 3).

Overloads

One call shape — this function is not overloaded.

Code	Input	Output	Description
`result = mutual_info_classif(X, y, [k=3])`	`X` vectornot documented yet `y` vectornot documented yet `k` vector, optional (default `3`)not documented yet	`result` vector	Returns a length-`D` vector of non-negative scores, one per feature; catches a relationship that is real but not linear, which `corr`/`corrcoef` misses.

Examples

From the documentation, and run by the build -- the output and the figure below are what the engine actually produced. Simplest first.

print("corrcoef matrix:\n{round(corrcoef(X), 3)}")

m = ols_model(X, y)
yhat = m.predict(X)
print("mae: {round(mae(y, yhat), 4)}, mse: {round(mse(y, yhat), 4)}")

cm = knn_model(Xc, yc, 5, kind = "classification")
predc = cm.predict(Xc)
print("precision: {round(precision(yc, predc), 3)}")
print("recall:    {round(recall(yc, predc), 3)}")
print("f1:        {round(f1(yc, predc), 3)}")

folds = k_fold(X, y, 4)
print("k_fold: {len(folds)} folds, first fold trains on {rows(folds[0].X_train)} rows")

stab = cv_stability(Xc, yc, n_repeats = 5, cv_folds = 3)
print("cv_stability: {round(mean(stab), 3)} +/- {round(std(stab), 3)}")

ab = ablation_study(Xc, yc, cv_folds = 3)
print("ablation_study scores: {round(ab, 4)}")
print("rfe keeps:             {rfe(Xc, yc, 1)}")
print("mutual_info_classif:   {round(mutual_info_classif(Xc, yc, k = 3), 4)}")
print("astar_mrmr keeps:      {astar_mrmr(Xc, yc, 1)}")
print("permutation_importance: {round(permutation_importance(m, X, y, n_repeats = 5), 4)}")

output

corrcoef matrix:
[1, -0.119, 0.007; -0.119, 1, -0.036; 0.007, -0.036, 1]
mae: 0.0847, mse: 0.011
precision: 0.995
recall:    0.995
f1:        0.995
k_fold: 4 folds, first fold trains on 90 rows
cv_stability: 0.908 +/- 0.012
ablation_study scores: [0.2401, 0.1301]
rfe keeps:             [0]
mutual_info_classif:   [0.2398, 0.1939]
astar_mrmr keeps:      [0]
permutation_importance: [1.5394, 0.3327, 0.0947]

from statistics-ml

How it works

mutual_info_classif(X, y, [k=3]) — Ross (2014) k-NN mutual information between each column of X and discrete labels y; see mutual_info_classif_column's own doc comment for the algorithm. Returns one score per feature column (same shape as sklearn's mutual_info_classif) — A*-mRMR's own relevance criterion, but also useful standalone as a quick per-feature relevance ranking.

From the note above this builtin in the interpreter, where the reasoning was written down as it was built.