sequential

Builtin index · Statistics & ML

result = sequential(layers, [seed=])

Input: layers, seed · Output: result

What it is

Initializes a stack of layer specs (a list of dense_layer/dropout_layer Records, or a single one) into a trainable model: weights use He initialization (sqrt(2/in_dim)) on a layer whose activation is "relu", Xavier-style (sqrt(1/in_dim)) otherwise, biases start at zero, and an optional integer seed= makes the draw reproducible.

Overloads

One call shape — this function is not overloaded.

Examples

From the documentation, and run by the build -- the output and the figure below are what the engine actually produced. Simplest first.

X = [0, 0, 1, 1, 0, 1, 0, 1] as matrix(4, 2)
Y = [0, 1, 1, 0] as matrix(4, 1)

layers = (dense_layer(2, 8, activation = "relu"), dense_layer(8, 1))
net = sequential(layers, seed = 42)
trained = net.fit(X, Y, epochs = 2000, lr = 0.5, loss = "mse")

print("loss {trained.loss_history[0]:.4f} -> {trained.loss_history[1999]:.6f}")
print("XOR predictions (want 0, 1, 1, 0): {trained.predict(X)}")

loss 0.9916 -> 0.000000
XOR predictions (want 0, 1, 1, 0): [4.134172e-84; 1; 1; -6.676147e-84]

from specialized

dl1 = dense_layer(2, 8, activation = "relu")
dl2 = dropout_layer(0.3)
dl3 = dense_layer(8, 1, activation = "none")
net = sequential((dl1, dl2, dl3), seed = 3)
print("layer count: {length(net.layers)}")

Xxor = [0.0, 0.0; 0.0, 1.0; 1.0, 0.0; 1.0, 1.0]
Yxor = [0.0; 1.0; 1.0; 2.0]
net = net.fit(Xxor, Yxor, epochs = 60, optimizer = adam(lr = 0.05), loss = "mse")
n = length(net.loss_history)
print("sequential + dropout_layer: loss {round(net.loss_history[0], 4)} -> {round(net.loss_history[n - 1], 4)}")

mc = mlp_classifier(2, [8], 2, seed = 3)
Yc = [0, 1, 1, 0]
mc = mc.fit(Xxor, Yc, epochs = 60, loss = "cross_entropy")
nc = length(mc.loss_history)
print("mlp_classifier: loss {round(mc.loss_history[0], 4)} -> {round(mc.loss_history[nc - 1], 4)}")
preds = mc.predict(Xxor)
print("mlp_classifier raw logits shape: {rows(preds)} x {cols(preds)}")

layer count: 3
sequential + dropout_layer: loss 3.0224 -> 0.5
mlp_classifier: loss 0.7146 -> 0.6938
mlp_classifier raw logits shape: 4 x 2

from statistics-ml

How it works

sequential(layers, [seed=]) — takes a List of layer specs (from dense_layer/dropout_layer) and returns an initialized Value::Model (kind="sequential"): a "layers" field (the spec list, unchanged) and a "params" field (a parallel List of per-layer weight Records — {w, b} for "dense", an empty Record for "dropout"). Weights are scaled-random-normal (random_matrix, gru_init's own initializer): He (sqrt(2/in_dim)) when the layer's activation is "relu", Xavier-ish (sqrt(1/in_dim)) otherwise — biases start at zero, the standard convention. Like randn, an explicit seed= reseeds the persistent RNG first so init is reproducible.

From the note above this builtin in the interpreter, where the reasoning was written down as it was built.