stormvogel.gym_env

Gymnasium-compliant environment wrapping a stormvogel MDP, DTMC, POMDP, or HMM.

Exceptions

ActionUnavailableError

Raised when the chosen action is not available in the current state.

Classes

ModelEnv

Gymnasium environment wrapping a stormvogel MDP, DTMC, POMDP, or HMM.

Functions

_domain_size(→ int)
_encode_value(→ int)	Encode a single domain value as a non-negative integer for gymnasium.

Module Contents

exception stormvogel.gym_env.ActionUnavailableError

Bases: Exception

Raised when the chosen action is not available in the current state.

stormvogel.gym_env._domain_size(domain: stormvogel.model.variable.VariableDomain) → int

stormvogel.gym_env._encode_value(domain: stormvogel.model.variable.VariableDomain, value: Any) → int

Encode a single domain value as a non-negative integer for gymnasium.

class stormvogel.gym_env.ModelEnv(model: stormvogel.model.Model, reward_model_name: str | None = None, obs_type: Literal['index', 'valuations'] = 'index')

Bases: gymnasium.Env

Gymnasium environment wrapping a stormvogel MDP, DTMC, POMDP, or HMM.

Two observation modes are supported, selected by obs_type:

• "index" (default): - MDP/DTMC: Discrete(n_states) — integer index of the current state. - POMDP/HMM: Discrete(n_observations) — integer index of the

  observation associated with the current state.

• "valuations": - MDP/DTMC: Dict space built from state variables that carry a

  declared domain (Variable.domain is not None).

  • POMDP/HMM: Dict space built from observation variables that carry a declared domain (Variable.domain is not None).

  Each variable becomes a Discrete component; values are encoded as non-negative integers. IntDomain(lo, hi): value v → v - lo; BoolDomain: False→0, True→1; CategoricalDomain: index into domain.values. None maps to the last index when allow_none is set. Missing variable values are raised as ValueError at step time.

For MDP/POMDP models the action space is Discrete(n_actions) where actions are indexed in the order returned by model.actions.

For DTMC/HMM models the action space is Discrete(1); the single legal action index is 0 (mapped internally to EmptyAction).

Stochastic observations (Distribution over observations) are not supported and raise NotImplementedError at step time.

Parameters:

• model – A stormvogel model of type MDP, DTMC, POMDP, or HMM.

• reward_model_name – Name of the reward model to use for the step reward. If None and exactly one reward model exists it is used automatically. If None and multiple reward models exist a ValueError is raised. No reward models → reward is always 0.

• obs_type – "index" for a flat observation space, "valuations" for a Dict space built from variable domains.

Raises:

ValueError – If obs_type="valuations" but no domain-bearing variables are found.

model

_obs_type = 'index'

_is_obs_model

_index_to_state: list[stormvogel.model.State]

_state_to_index: dict[stormvogel.model.State, int]

_current_state: stormvogel.model.State

_collect_obs_vars() → list[stormvogel.model.variable.Variable]

Return domain-bearing variables in sorted label order.

_encode_obs(state: stormvogel.model.State) → int | dict[str, int]

reset(*, seed: int | None = None, options: dict[str, Any] | None = None) → tuple[int | dict[str, int], dict[str, Any]]

Reset the environment to the model’s initial state.

Returns:

(observation, info) where info["state"] is the initial stormvogel State.

step(action: int) → tuple[int | dict[str, int], float, bool, bool, dict[str, Any]]

Take one step in the environment.

Parameters:

action – Integer action index. For MDPs this selects from _index_to_action; for DTMCs only 0 is valid.

Returns:

(observation, reward, terminated, truncated, info) where info["state"] is the next stormvogel State and info["labels"] are its labels.

Raises:

ActionUnavailableError – If the chosen action is not available in the current state, or if action != 0 for a DTMC.