lstm_cell

Builtin index · Statistics & ML

result = lstm_cell(x_t, h_prev, c_prev, weights)

Input: x_t, h_prev, c_prev, weights · Output: result

What it is

One LSTM timestep: x_t (length-input_size input vector), h_prev/c_prev (length-hidden_size previous hidden/cell state vectors), and weights (a Record with the twelve gate fields lstm_init produces, param-wrapped where you want gradients).

Overloads

One call shape — this function is not overloaded.

Examples

From the documentation, and run by the build -- the output and the figure below are what the engine actually produced. Simplest first.

w1 = {wf=0.5, uf=0.5, bf=0.0, wi=0.3, ui=0.2, bi=0.0,
      wo=0.4, uo=0.1, bo=0.0, wg=0.6, ug=0.3, bg=0.0}
hc = lstm_cell(1.0, 0.0, 0.0, w1)
print("h = {round(hc.h, 4)}, c = {round(hc.c, 4)}")

h = 0.1791, c = 0.3085

from statistics-ml

How it works

lstm_cell(x_t, h_prev, c_prev, weights) — one LSTM timestep (Hochreiter & Schmidhuber 1997, the now-standard forget/input/ output-gate formulation with a per-gate shared bias, the same "paper's own equations, not a framework variant" choice gru_cell_compute's own doc comment makes for GRU): text f_t = sigmoid(Wf @ x_t + Uf @ h_{t-1} + bf) -- forget gate i_t = sigmoid(Wi @ x_t + Ui @ h_{t-1} + bi) -- input gate o_t = sigmoid(Wo @ x_t + Uo @ h_{t-1} + bo) -- output gate g_t = tanh (Wg @ x_t + Ug @ h_{t-1} + bg) -- candidate cell state c_t = f_t .* c_{t-1} + i_t .* g_t h_t = o_t .* tanh(c_t) weights is a Value::Record with fields wf/uf/bf/ wi/ui/bi/wo/uo/bo/wg/ug/bg (see lstm_init). Deliberately zero-initialized biases for every gate (no "initialize the forget-gate bias to 1" folklore trick some frameworks apply) — same reasoning gru_init's own doc comment gives for skipping GRU-side init tricks: the task asked for the standard equations, and keeping every Qu recurrent cell's init convention identical is worth more than a training-speed folklore trick neither cell currently has. Two states thread through a sequence (h AND c, unlike GRU's single h), and Qu has no tuple/multi-return value, so this returns a Value::Record {h = h_t, c = c_t} — plain field access (eval_field's existing Value::Record arm) reads either one back out with no new mechanism, exactly the same "record of named values, some of which happen to be tracked Tensors" shape the weights record argument already relies on. See lstm_forward for how a whole sequence carries c across timesteps using this record. Composed entirely from already-tracked primitives (tb's "*"/"+"/".*", tm1's "sigmoid"/"tanh"), exactly like gru_cell_compute — no new tensor_vjp rule needed; BPTT falls out of the existing tape mechanism the same way GRU's own doc comment explains. See the module-level gradient-check test (mirroring gru_gradient_matches_finite_differences_for_ weights_and_initial_hidden_state) for the finite-difference verification, and a separate hand-computed-forward-pass test for the gate-assignment/activation correctness gradient- checking alone can't catch.

From the note above this builtin in the interpreter, where the reasoning was written down as it was built.