softmax

Builtin index · Core maths

result = softmax(x)

Input: x · Output: result

What it is

Elementwise exp(x_i) / sum(exp(x)) — the gradient of logsumexp, and a probability distribution that sums to 1. x is a scalar, Vec, or Mat/Signal, flattened.

Overloads

One call shape — this function is not overloaded.

Code	Input	Output	Description
`result = softmax(x)`	`x` scalar, Vec, Mat, or Signalnot documented yet	`result` Vec or Mat	Returns a `Vec` the same length as `x` whatever `x`'s original shape was — a `Mat` in still gives a flat `Vec` out, and an empty `x` an empty `Vec`.

Examples

From the documentation, and run by the build -- the output and the figure below are what the engine actually produced. Simplest first.

crest(x) := max(abs(x)) / rms(x)

xs = [0, 1, 2, 3]
print("hard max = {max(xs)}")
print("logsumexp = {logsumexp(xs):.4f}")       # >= hard max, a proven upper bound

print("smoothmax b=50 = {smoothmax(xs, 50):.4f}")  # -> hard max as beta grows

print("softmax = {softmax(xs)}")                # the gradient of logsumexp

output

hard max = 3
logsumexp = 3.4402
smoothmax b=50 = 3.0000
softmax = [0.032059, 0.087144, 0.236883, 0.643914]

from numerics-dsp