Qu

lr_adaptive

Builtin index · Statistics & ML

result = lr_plateau([lr=], [patience=], [factor=], [base=])

Input: lr, patience, factor, base · Output: result

What it is

Learning-rate controllers that wrap a base optimizer rather than replacing it: lr= the starting learning rate (number), patience an integer number of epochs to wait for improvement before shrinking the rate, factor the multiplicative shrink (a number in (0, 1)), and base a string naming the wrapped optimizer (e.g. "adam"). Passed as optimizer= wherever a plain optimizer recipe would go.

Overloads

One call shape — this function is not overloaded.

CodeInputOutputDescription
result = lr_plateau([lr=], [patience=], [factor=], [base=])
lr number or string, optionalnot documented yet
patience number or string, optionalnot documented yet
factor number or string, optionalnot documented yet
base number or string, optionalnot documented yet
result Record
Returns an optimizer-recipe Record.

Examples

From the documentation, and run by the build -- the output and the figure below are what the engine actually produced. Simplest first.

x = [1.0, 2.0, 3.0, 4.0]
y_true = [2.0, 4.0, 6.0, 8.0]
w = param(0.0)
function loss_fn(w)
    pred = w * x
    diff = pred - y_true
    return mean(diff .^ 2)
end function
opt = sgd(w, 0.01)
losses = train_loop("loss_fn", w, 100, optimizer = "opt")
print("train_loop (sgd): loss {round(losses[0], 3)} -> {round(losses[99], 5)}")
tape_reset()

w2v = 0.0
opt2 = adam(param(w2v), lr = 0.1)
for t = 0 to 19
    tape_reset()
    w2 = param(w2v)
    pred = w2 * x
    loss2 = mean((pred - y_true) .^ 2)
    g = grad(loss2, wrt = w2)
    opt2 = optimizer_step(opt2, g)
    w2v = opt2._params[0]
end for
print("optimizer_step (adam): w -> {round(w2v, 4)} (target 2.0)")
tape_reset()

# One step each from w=1.0, grad=3.0 -- same optimizer_step(opt, grad) call.

w3 = param(1.0)
grad3 = [3.0]
r1 = optimizer_step(nesterov_sgd(w3, lr = 0.05, momentum = 0.9), grad3)
r2 = optimizer_step(adamw(w3, lr = 0.05), grad3)
r3 = optimizer_step(nadam(w3, lr = 0.05), grad3)
r4 = optimizer_step(adamax(w3, lr = 0.05), grad3)
r5 = optimizer_step(rmsprop(w3, lr = 0.05), grad3)
r6 = optimizer_step(adagrad(w3, lr = 0.05), grad3)
r7 = optimizer_step(adadelta(w3), grad3)
print("nesterov_sgd -> {round(r1._params[0], 4)}, adamw -> {round(r2._params[0], 4)}, nadam -> {round(r3._params[0], 4)}")
print("adamax -> {round(r4._params[0], 4)}, rmsprop -> {round(r5._params[0], 4)}, adagrad -> {round(r6._params[0], 4)}, adadelta -> {round(r7._params[0], 4)}")

Xxor = [0.0, 0.0; 0.0, 1.0; 1.0, 0.0; 1.0, 1.0]
Yxor = [0.0; 1.0; 1.0; 2.0]
net = input(2) |> dense(4, activation = "relu") |> dense(1, activation = "none") |>
      compile(optimizer = lr_plateau(lr = 0.1, patience = 5, base = "adam"), loss = "mse", seed = 7)
net = net.fit(Xxor, Yxor, epochs = 60)
n = length(net.loss_history)
print("compile + fit(lr_plateau): loss {round(net.loss_history[0], 4)} -> {round(net.loss_history[n - 1], 4)}")

net2 = input(2) |> dense(4, activation = "relu") |> dense(1, activation = "none") |>
       compile(optimizer = lr_adaptive(lr = 0.05), loss = "mse", seed = 7)
net2 = net2.fit(Xxor, Yxor, epochs = 60)
n2 = length(net2.loss_history)
print("compile + fit(lr_adaptive): loss {round(net2.loss_history[0], 4)} -> {round(net2.loss_history[n2 - 1], 4)}")

quick = quick_mlp(Xxor, Yxor, seed = 3)
nq = length(quick.loss_history)
print("quick_mlp: loss {round(quick.loss_history[0], 4)} -> {round(quick.loss_history[nq - 1], 4)}")
output
train_loop (sgd): loss 30 -> 0
optimizer_step (adam): w -> 1.7775 (target 2.0)
nesterov_sgd -> 0.715, adamw -> 0.9495, nadam -> 0.9263
adamax -> 0.95, rmsprop -> 0.8419, adagrad -> 0.95, adadelta -> 0.9996
compile + fit(lr_plateau): loss 3.3387 -> 0.0341
compile + fit(lr_adaptive): loss 0.3897 -> 0.0002
quick_mlp: loss 3.1877 -> 0.0922

from statistics-ml

How it works

lr_adaptive(lr=0.01, increase_factor=1.1, decrease_factor=0.5) lr_plateau(lr=0.01, patience=10, factor=0.5, min_lr=0.0, base="sgd"|"adam") — 2026-08-26, two learning-rate CONTROLLERS usable as optimizer= on .fit(...)/compile(...) exactly like sgd(...)/adam(...), both spec-only "recipe" Models (same zero-arg carve-out those two use) — neither is a base update rule on its own, see sequential_fit's own doc comment for how each wraps a base rule and why lr_adaptive is restricted to plain SGD while lr_plateau composes with either.

From the note above this builtin in the interpreter, where the reasoning was written down as it was built.

See also

lr_plateau — documented in the same entry.