gru_forward

Builtin index · Statistics & ML

result = gru_forward(x_sequence, h0, weights)

Input: x_sequence, h0, weights · Output: result

What it is

Unrolls gru_cell over a whole sequence: x_sequence an (input_size, seq_len) matrix, h0 a length-hidden_size initial hidden state, weights the same Record gru_cell takes.

Overloads

One call shape — this function is not overloaded.

Examples

From the documentation, and run by the build -- the output and the figure below are what the engine actually produced. Simplest first.

raw = gru_init(2, 3, seed = 1)
w = {wz=param(raw.wz), uz=param(raw.uz), bz=param(raw.bz),
     wr=param(raw.wr), ur=param(raw.ur), br=param(raw.br),
     wh=param(raw.wh), uh=param(raw.uh), bh=param(raw.bh)}
Xseq = [0.5, -0.3, 0.8; 0.2, 0.4, -0.1]   # (input_size, seq_len)

hs = gru_forward(Xseq, [0.0, 0.0, 0.0], w)
h_last = hs[length(hs) - 1]
print("gru_forward last hidden state: {round(h_last, 4)}")

loss = sum(h_last .^ 2)
g = grad(loss, wrt = w.wz)
print("dL/dwz: {round(g, 4)}")
tape_reset()

h1 = gru_cell([1.0], [1.0], {wz=0.5, uz=0.5, bz=0.0, wr=0.3, ur=0.3, br=0.0, wh=0.2, uh=0.2, bh=0.0})
print("gru_cell one step: {round(h1, 4)}")

gru_forward last hidden state: [-0.542; -0.0036; -0.0815]
dL/dwz: [0.2321, -0.0486; -0.0006, 0.0001; -0.0079, 0.0034]
gru_cell one step: [0.5012]

from statistics-ml

How it works

gru_forward(x_sequence, h0, weights) — unrolls gru_cell across a whole sequence, returning a Value::List of every timestep's hidden state (so a caller can read the final one via hs[length(hs)-1], or feed all of them into a further layer — the same "return everything, let the caller slice" choice train_loop makes for its own per-epoch losses). Backprop-through-time needs no special handling here at all: each gru_cell_compute call below records its own ordinary tape entries (matmuls/adds/sigmoids/tanh), and a later timestep's entries reference an EARLIER timestep's h by tape node id exactly the way any other tracked value gets reused — eval_grad's existing reverse walk over self.tape already chains gradients through that reuse correctly, the same mechanism that makes w * x + w * y correctly sum w's gradient from both uses. BPTT is just that mechanism applied across a longer chain, not a separate algorithm — verified independently against finite differences in the acceptance tests rather than just assumed from this argument. x_sequence is either: - a Value::Mat of shape (input_size, seq_len) — one COLUMN per timestep, single un-batched sequence (a plain Value::Vec column, to_matrix's existing convention, read out via col_vec per step); or - a Value::List of length seq_len, each element a Value::Mat of shape (input_size, batch) for that timestep — a batch of independent sequences, time-major. See this function's own module-level doc comment (in Matrix::matmul's/Matrix::broadcast's OWN existing rayon parallelism (both already parallelize once a matrix crosses their size threshold) rather than a hand-rolled rayon loop here: every op inside gru_cell_compute goes through self.tape (&mut self, strictly sequential), so there is no safe way to run two timesteps' — or two batch sequences' — worth of tape-recording concurrently without a per-thread-tape redesign this task did not ask for.

From the note above this builtin in the interpreter, where the reasoning was written down as it was built.