stormvogel.extensions.gym_sampling

Functions

sample_gym(→ Tuple)

Sample the gym environment.

sample_to_stormvogel(→ stormvogel.model.Model)

Create a stormvogel MDP from a sampling.

sample_gym_to_stormvogel(env[, no_samples, ...])

Sample the gym environment and convert it to a stormvogel MDP.

Module Contents

stormvogel.extensions.gym_sampling.sample_gym(env: gymnasium.Env, no_samples: int = 10, sample_length: int = 20, gymnasium_scheduler: Callable[[Any], int] | None = None, convert_obs: Callable[[Any], Any] = lambda x: ...) → Tuple

Sample the gym environment.

Gym environments are POMDPs; gymnasium only exposes observations. States that differ in gym but share the same observation and termination are considered the same state in the result.

Parameters:

  • env – Gymnasium environment.

  • no_samples – Total number of samples (starting at an initial state). To resolve multiple initial states, a new single initial state is added if necessary.

  • sample_length – Maximum length of a single sample.

  • gymnasium_scheduler – Function mapping states to action numbers.

  • convert_obs – Convert observations to a hashable type. Rounding can also be applied here.

Returns:

A 6-tuple (initial_states, states, transition_counts, transition_samples, reward_sums, no_actions).

stormvogel.extensions.gym_sampling.sample_to_stormvogel(initial_states: collections.defaultdict[Any, int], transition_counts: collections.defaultdict[Tuple[Any, Any], collections.defaultdict[Any, int]], transition_samples: collections.defaultdict[Tuple[Any, Any], int], reward_sums: collections.defaultdict[Tuple[Any, Any], float], no_actions: int, no_samples: int, max_size: int = 10000) → stormvogel.model.Model

Create a stormvogel MDP from a sampling.

Use sample_gym() to obtain a sample from gym. Probabilities are frequentist estimates whose accuracy depends on how often each state is visited.

Parameters:

  • initial_states – Mapping from initial states to observation counts.

  • transition_counts – Counts how many times each (state, action) to state transition was observed.

  • transition_samples – Counts how many times each (state, action) pair was observed.

  • reward_sums – Sum of rewards for each (state, action) pair.

  • no_actions – Number of different actions observed.

  • no_samples – Number of samples used to obtain this sampling.

  • max_size – Maximum number of states in the resulting model.

Returns:

Stormvogel MDP model.

stormvogel.extensions.gym_sampling.sample_gym_to_stormvogel(env: gymnasium.Env, no_samples: int = 10, sample_length: int = 20, gymnasium_scheduler: Callable[[Any], int] | None = None, convert_obs: Callable[[Any], Any] = lambda x: ..., max_size: int = 10000)

Sample the gym environment and convert it to a stormvogel MDP.

Gym environments are POMDPs; gymnasium only exposes observations. The result is an MDP where states with the same observations (and termination) are lumped together. Probabilities are frequentist estimates whose accuracy depends on how often each state is visited.

Parameters:

  • env – Gymnasium environment.

  • no_samples – Total number of samples (starting at an initial state). To resolve multiple initial states, a new single initial state is added if necessary.

  • sample_length – Maximum length of a single sample.

  • gymnasium_scheduler – Function mapping states to action numbers.

  • convert_obs – Convert observations to a hashable type. Rounding can also be applied here.

  • max_size – Maximum number of states in the resulting model.

Returns:

Stormvogel MDP model.