gru_init

Builtin index · Statistics & ML

result = gru_init(input_size, hidden_size, [seed=])

Input: input_size, hidden_size, seed · Output: result

What it is

Builds fresh GRU weights for an input of size input_size (integer) and a hidden state of size hidden_size (integer): Xavier-scaled weight matrices and zero biases for the update, reset and candidate gates, plus an optional integer seed=.

Overloads

One call shape — this function is not overloaded.

Examples

From the documentation, and run by the build -- the output and the figure below are what the engine actually produced. Simplest first.

raw = gru_init(2, 3, seed = 1)
w = {wz=param(raw.wz), uz=param(raw.uz), bz=param(raw.bz),
     wr=param(raw.wr), ur=param(raw.ur), br=param(raw.br),
     wh=param(raw.wh), uh=param(raw.uh), bh=param(raw.bh)}
Xseq = [0.5, -0.3, 0.8; 0.2, 0.4, -0.1]   # (input_size, seq_len)

hs = gru_forward(Xseq, [0.0, 0.0, 0.0], w)
h_last = hs[length(hs) - 1]
print("gru_forward last hidden state: {round(h_last, 4)}")

loss = sum(h_last .^ 2)
g = grad(loss, wrt = w.wz)
print("dL/dwz: {round(g, 4)}")
tape_reset()

h1 = gru_cell([1.0], [1.0], {wz=0.5, uz=0.5, bz=0.0, wr=0.3, ur=0.3, br=0.0, wh=0.2, uh=0.2, bh=0.0})
print("gru_cell one step: {round(h1, 4)}")

gru_forward last hidden state: [-0.542; -0.0036; -0.0815]
dL/dwz: [0.2321, -0.0486; -0.0006, 0.0001; -0.0079, 0.0034]
gru_cell one step: [0.5012]

from statistics-ml

How it works

gru_init(input_size, hidden_size, [seed=]) — weight initializer matching particle_filter_init's own seed= reproducibility convention (an explicit seed reseeds self.rng before drawing, so the SAME seed always produces the SAME weights; no seed just advances the persistent stream). Returns a Value::Record (fields wz/uz/bz/ wr/ur/br/wh/uh/bh) of PLAIN (untracked) values — matching particle_filter_init's own choice to return plain data rather than pre-wrapped param() tensors, since not every caller trains every field (e.g. a fixed, non-trained h0). Wrap whichever fields should be trainable with param(...) before building the weights record gru_cell/gru_forward expect. Wz/Wr/Wh are (hidden_size, input_size); Uz/Ur/ Uh are (hidden_size, hidden_size); each gate's bias is a zero-initialized Value::Vec of length hidden_size (the standard zero-bias-init choice — no per-gate bias tricks like LSTM's common "init the forget gate bias to 1" shortcut, since the task asked for the standard equations, not a variant). Each weight matrix is Xavier/Glorot-scaled (std = 1/sqrt(fan_in), fan-in input_size for the Ws and hidden_size for the Us) — the same reasoning any dense-layer init uses, not a GRU-specific trick.

From the note above this builtin in the interpreter, where the reasoning was written down as it was built.