Qu

step

Builtin index · Statistics & ML

result = env.step(state, action)

Input: state, action · Output: result

What it is

Applies one action (an integer 0-3) from the current integer state index in environment env.

Overloads

One call shape — this function is not overloaded.

CodeInputOutputDescription
result = env.step(state, action)
state —not documented yet
action —not documented yet
next_state —integer
reward numbera number, step_reward or goal_reward
done booleana boolean, true once the goal is reached
Returns a Record with next_state (integer), reward (a number, step_reward or goal_reward), and done (a boolean, true once the goal is reached).

Examples

From the documentation, and run by the build -- the output and the figure below are what the engine actually produced. Simplest first.

env = gridworld_env(4)
start = env.reset()
print("start state: {start}")

q = q_learning(env, 300, alpha = 0.3, gamma = 0.9, epsilon = 0.1, seed = 7)
s = sarsa(env, 300, alpha = 0.3, gamma = 0.9, epsilon = 0.1, seed = 7)
print("q_learning reward: early {round(mean(q.rewards[0:19]), 2)} -> late {round(mean(q.rewards[280:299]), 2)}")
print("sarsa reward:      early {round(mean(s.rewards[0:19]), 2)} -> late {round(mean(s.rewards[280:299]), 2)}")

state = env.reset()
steps = 0
done = false
while done == false and steps < 20
    action = q.predict(state)
    result = env.step(state, action)
    state = result.next_state
    steps = steps + 1
    done = result.done
end while
print("greedy policy reached the goal in {steps} steps")
output
start state: 0
q_learning reward: early -4 -> late 4.45
sarsa reward:      early -1.5 -> late 4.6
greedy policy reached the goal in 6 steps

from statistics-ml

How it works

step(env, state, action) — callable as env.step(state, action). Returns a Value::Record (.next_state/.reward/.done) — Qu has no tuples, so this follows the same "named-field return value" precedent as train_test_split's Model, except a Record here since this is a one-shot data bag, not a handle with further methods.

From the note above this builtin in the interpreter, where the reasoning was written down as it was built.