gradient_boosting_model

Builtin index · Statistics & ML

result = gradient_boosting_model(X, y, n_trees, [learning_rate=0.1], [max_depth=3])

Input: X, y, n_trees, learning_rate, max_depth · Output: result

What it is

Regression-only boosted ensemble on an N-row, D-column X and length-N y: n_trees (positive integer) shallow trees fitted in sequence, each to the running residual, scaled by learning_rate (a number, default 0.1) and capped at max_depth (default 3, shallower than tree_model's 20 because a boosting weak learner is supposed to be weak).

Overloads

One call shape — this function is not overloaded.

Examples

From the documentation, and run by the build -- the output and the figure below are what the engine actually produced. Simplest first.

km = kmedians_model(G, 3, seed = 5)
kd = kmedoids_model(G, 3, seed = 5)
gm = gmm_model(G, 3, seed = 5)
print("kmedians label for row 0: {km.predict(G)[0]}")
print("kmedoids label for row 0: {kd.predict(G)[0]}")
print("gmm log-likelihood: {round(gm.log_likelihood, 1)}")

gb = gradient_boosting_model(X, y, 20, learning_rate = 0.1, max_depth = 3)
print("gradient boosting r2: {round(gb.score(X, y), 3)}")

sv = svm_model(Xc, yc, kernel = "linear", C = 1.0)
lg = logistic_model(Xc, yc)
print("svm accuracy:      {round(sv.score(Xc, yc), 3)}")
print("logistic accuracy: {round(lg.score(Xc, yc), 3)}")

svr = svr_model(X, y)
print("svr r2: {round(svr.score(X, y), 3)}")

kmedians label for row 0: 1
kmedoids label for row 0: 1
gmm log-likelihood: [-448.7, -448.7, -448.7, -448.7]
gradient boosting r2: 0.953
svm accuracy:      0.995
logistic accuracy: 0.995
svr r2: 0.596

from statistics-ml

How it works

gradient_boosting_model(X, y, n_trees, [learning_rate=], [max_depth=], [kind=]) — sequential boosting, kind= "regression" (default) or kind="classification" (binary, added 2026-08-25 — the "stretch goal" earlier passes deferred). Regression: starts from the constant baseline mean(y), then repeatedly fits a shallow regression tree (default max_depth=3, deliberately shallower than tree_model's own default — a boosting "weak learner" is supposed to be weak) to the *current* residual and adds learning_rate * tree.predict(X) to the running prediction, which becomes the next round's residual target. Classification: the same algorithm in log-odds space (the standard Friedman gradient-boosting-for-logistic-deviance construction) — y mapped to canonical 0/1 (svm_fit_model's own smaller-value-> lower-canonical-label convention), running score starts at the log-odds of the base rate (ln(p/(1-p))), each tree is fit to the *negative gradient of log-loss* (y01 - sigmoid(running) — algebraically identical in shape to the regression residual, just in probability space instead of the target's own units), and .predict thresholds sigmoid(running) at 0.5 the same way logistic_model does. Both kinds are fully deterministic (every tree sees the whole training set — no bootstrap, no feature subsampling), so there's no seed= parameter to thread through, unlike random_forest_model above.

From the note above this builtin in the interpreter, where the reasoning was written down as it was built.