stormvogel.simulator

Classes

Path

Represent a path created by a simulator on a certain model.

Functions

get_action_at_state(→ stormvogel.model.Action)

Obtain the chosen action in a state by a scheduler.

step(→ tuple[stormvogel.model.State, ...)

Simulate one step from the given state.

simulate_path(→ Path)

Simulate the model and return the path created by the process.

simulate(→ stormvogel.model.Model)

Simulate the model over multiple runs.

Module Contents

class stormvogel.simulator.Path(path: list[tuple[stormvogel.model.Action, stormvogel.model.State]] | list[stormvogel.model.State], model: stormvogel.model.Model)

Represent a path created by a simulator on a certain model.

Parameters:

  • path – The path itself is a list where we either store for each step a state or a state-action pair, depending on whether we are working with a DTMC or an MDP.

  • model – Model that the path traverses through.

path: list[tuple[stormvogel.model.Action, stormvogel.model.State]] | list[stormvogel.model.State]
model: stormvogel.model.Model
get_state_in_step(step: int) → stormvogel.model.State | None

Return the state discovered in the given step in the path.

Parameters:

step – The step index to look up.

Returns:

The state at the given step, or None.

get_action_in_step(step: int) → stormvogel.model.Action | None

Return the action discovered in the given step in the path.

Parameters:

step – The step index to look up.

Returns:

The action at the given step, or None.

get_step(step: int) → tuple[stormvogel.model.Action, stormvogel.model.State] | stormvogel.model.State

Return the state or state-action pair discovered in the given step.

Parameters:

step – The step index to look up.

Returns:

The state or state-action pair at the given step.

to_state_action_sequence() → list[stormvogel.model.Action | stormvogel.model.State]

Convert a Path to a list containing actions and states.

Returns:

A flat list of actions and states from this path.

__str__() → str
__len__()
stormvogel.simulator.get_action_at_state(state: stormvogel.model.State, scheduler: stormvogel.result.Scheduler | Callable[[stormvogel.model.State], stormvogel.model.Action]) → stormvogel.model.Action

Obtain the chosen action in a state by a scheduler.

Parameters:

  • state – The state to get the action for.

  • scheduler – A scheduler or callable that maps states to actions.

Returns:

The action chosen by the scheduler at the given state.

Raises:

TypeError – If scheduler is not a Scheduler or callable.

stormvogel.simulator.step(state: stormvogel.model.State, action: stormvogel.model.Action | None = None, seed: int | None = None) → tuple[stormvogel.model.State, list[stormvogel.model.Number], list[str]]

Simulate one step from the given state.

Rewards are always the state-exit rewards of the current state. Missing rewards default to 0.

Parameters:

  • state – The current state to step from.

  • action – The action to take, or None for models without actions.

  • seed – Seed for the random state selection. Random if not provided.

Returns:

A tuple of (next_state, state-exit rewards, next_state labels).

stormvogel.simulator.simulate_path(model: stormvogel.model.Model, steps: int = 1, scheduler: stormvogel.result.Scheduler | Callable[[stormvogel.model.State], stormvogel.model.Action] | None = None, seed: int | None = None) → Path

Simulate the model and return the path created by the process.

Parameters:

  • model – The stormvogel model that the simulator should run on.

  • steps – The number of steps the simulator walks through the model.

  • scheduler – A stormvogel scheduler to determine what actions should be taken. Random if not provided. A callable from states to actions can also be provided.

  • seed – The seed for the random state selection. Random if not provided.

Returns:

A path object representing the simulated trajectory.

stormvogel.simulator.simulate(model: stormvogel.model.Model, steps: int = 1, runs: int = 1, scheduler: stormvogel.result.Scheduler | Callable[[stormvogel.model.State], stormvogel.model.Action] | None = None, seed: int | None = None) → stormvogel.model.Model

Simulate the model over multiple runs.

Parameters:

  • model – The stormvogel model that the simulator should run on.

  • steps – The number of steps the simulator walks through the model.

  • runs – The number of times the model gets simulated.

  • scheduler – A stormvogel scheduler to determine what actions should be taken. Random if not provided. A callable from states to actions can also be provided.

  • seed – The seed for the random state selection. Random if not provided.

Returns:

The partial model discovered by all the runs of the simulator together.