dense_layer

Builtin index · Statistics & ML

result = dense_layer(in_dim, out_dim, [activation=])

Input: in_dim, out_dim, activation · Output: result

What it is

Builds one unfitted dense-layer spec: in_dim/out_dim (positive integers) the layer's input and output widths, activation a string, "relu" or "none" (default "none", a bare affine layer).

Overloads

One call shape — this function is not overloaded.

Examples

From the documentation, and run by the build -- the output and the figure below are what the engine actually produced. Simplest first.

X = [0, 0, 1, 1, 0, 1, 0, 1] as matrix(4, 2)
Y = [0, 1, 1, 0] as matrix(4, 1)

layers = (dense_layer(2, 8, activation = "relu"), dense_layer(8, 1))
net = sequential(layers, seed = 42)
trained = net.fit(X, Y, epochs = 2000, lr = 0.5, loss = "mse")

print("loss {trained.loss_history[0]:.4f} -> {trained.loss_history[1999]:.6f}")
print("XOR predictions (want 0, 1, 1, 0): {trained.predict(X)}")

loss 0.9916 -> 0.000000
XOR predictions (want 0, 1, 1, 0): [4.134172e-84; 1; 1; -6.676147e-84]

from specialized

dl1 = dense_layer(2, 8, activation = "relu")
dl2 = dropout_layer(0.3)
dl3 = dense_layer(8, 1, activation = "none")
net = sequential((dl1, dl2, dl3), seed = 3)
print("layer count: {length(net.layers)}")

Xxor = [0.0, 0.0; 0.0, 1.0; 1.0, 0.0; 1.0, 1.0]
Yxor = [0.0; 1.0; 1.0; 2.0]
net = net.fit(Xxor, Yxor, epochs = 60, optimizer = adam(lr = 0.05), loss = "mse")
n = length(net.loss_history)
print("sequential + dropout_layer: loss {round(net.loss_history[0], 4)} -> {round(net.loss_history[n - 1], 4)}")

mc = mlp_classifier(2, [8], 2, seed = 3)
Yc = [0, 1, 1, 0]
mc = mc.fit(Xxor, Yc, epochs = 60, loss = "cross_entropy")
nc = length(mc.loss_history)
print("mlp_classifier: loss {round(mc.loss_history[0], 4)} -> {round(mc.loss_history[nc - 1], 4)}")
preds = mc.predict(Xxor)
print("mlp_classifier raw logits shape: {rows(preds)} x {cols(preds)}")

layer count: 3
sequential + dropout_layer: loss 3.0224 -> 0.5
mlp_classifier: loss 0.7146 -> 0.6938
mlp_classifier raw logits shape: 4 x 2

from statistics-ml

How it works

dense_layer(in_dim, out_dim, [activation="relu"|"none"]) — an UNINITIALIZED layer spec (no weights yet; sequential(...) does the actual init). activation defaults to "none" (a bare affine layer, e.g. a regression readout). Pipe-chain accumulation (2026-08-26): dense_layer(2,8, activation="relu") |> dense_layer(8,1) |> sequential(seed=1) needs zero new grammar because eval_pipe already inserts the piped-in value as arg0 (see its own doc comment) — so a SECOND dense_layer in a chain sees its predecessor's spec (or the growing chain so far) sitting in arg_all(&args)[0], with its own real (in_dim, out_dim) shifted right to arg_all(&args)[1]/ arg_all(&args)[2]. Detected by shape, not position: arg_all(&args)[0] is either a bare layer-spec Value::Record (kind="dense"| "dropout", from a single prior dense_layer/dropout_layer call) or a Value::List of them (a chain already 2+ long) — a shape a real in_dim (always numeric) can never take, so this can't misfire on the ordinary dense_layer(2, 8) direct-call form. Accumulation rule: always fold into and return a flat Value::List (never nest lists-of-lists) — a lone prior Record becomes a 2-element list, an existing list gets the new spec appended. sequential(...) separately accepts EITHER a List (the >=2-layer case) or a lone Record (the never-piped sequential(dense_layer(...)) one-layer case this rule never needs to produce) — see that arm's own doc comment.

From the note above this builtin in the interpreter, where the reasoning was written down as it was built.