stormvogel.teaching.belief

Teaching module: POMDP belief type and belief computations.

Provides the canonical Belief type and exact Bayesian belief tracking for POMDPs with deterministic state observations. All arithmetic uses Fraction.

Typical usage:

from stormvogel.teaching.belief import Belief, initial_belief, belief_trace

b0 = initial_belief(pomdp, "z")
beliefs = belief_trace(pomdp, b0, [("b", "z"), ("a", "z_target")])

Classes

Belief

Exact probability distribution over POMDP states.

Functions

initial_belief(→ Belief)

Derive the initial belief from the POMDP's initial-state distribution.

belief_update(→ Belief)

Compute the updated belief after taking an action and receiving an observation.

belief_table(→ None)

Render a belief trace as an HTML table in a Jupyter notebook.

belief_trace(→ list[Belief])

Compute the sequence of beliefs induced by an observation trace.

Module Contents

class stormvogel.teaching.belief.Belief(dist: dict[State, Fraction])

Bases: collections.abc.Mapping[stormvogel.model.state.State, fractions.Fraction]

Exact probability distribution over POMDP states.

Implements the Mapping interface over State → Fraction, so b[s], b.get(s, 0), b.items(), etc. all work directly. Zero-probability states are silently dropped.

Parameters:

dist – Mapping from POMDP states to their belief probabilities.

dist: dict[stormvogel.model.state.State, fractions.Fraction]
_key: tuple[tuple[uuid.UUID, fractions.Fraction], Ellipsis]
__getitem__(key: stormvogel.model.state.State) → fractions.Fraction
__iter__() → collections.abc.Iterator[stormvogel.model.state.State]
__len__() → int
__hash__() → int
__eq__(other: object) → bool
__repr__() → str
static _state_name(state: stormvogel.model.state.State) → str

Best short name for state: friendly_name > first label > index.

static _fraction_latex(f: fractions.Fraction) → str
_repr_latex_() → str
classmethod normalize(unnorm: dict[State, Fraction]) → Belief

Normalize unnorm to a probability distribution and return a Belief.

Parameters:

unnorm – Unnormalized weights (non-negative, at least one > 0).

Raises:

ValueError – If all weights are zero.

stormvogel.teaching.belief.initial_belief(pomdp: stormvogel.model.model.Model, obs_alias: str) → Belief

Derive the initial belief from the POMDP’s initial-state distribution.

The initial state is assumed to have a single EmptyAction transition that encodes the prior distribution over states. The resulting distribution is filtered to states whose observation matches obs_alias and then normalised.

Parameters:

  • pomdp – The POMDP model.

  • obs_alias – Observation alias that filters which successor states belong to the initial belief support.

Returns:

Normalised belief over states with observation obs_alias.

Raises:

ValueError – If no states with obs_alias are reachable from the initial state, or if the initial state has no EmptyAction transition.

stormvogel.teaching.belief.belief_update(pomdp: stormvogel.model.model.Model, belief: Belief, action_label: str, obs_alias: str) → Belief

Compute the updated belief after taking an action and receiving an observation.

Applies the standard Bayesian filter for POMDPs with deterministic observations:

b'(s') ∝  Σ_s  P(s' | s, a) · b(s)   if obs(s') = o
          0                             otherwise
Parameters:

  • pomdp – The POMDP model.

  • belief – Current belief distribution over states.

  • action_label – Label of the action taken.

  • obs_alias – Alias of the observation received after the action.

Returns:

Updated, normalised belief.

Raises:

ValueError – If the observation is unreachable from the current belief under the given action.

stormvogel.teaching.belief.belief_table(beliefs: list[Belief], trace: list[tuple[str, str]]) → None

Render a belief trace as an HTML table in a Jupyter notebook.

Displays a table with columns Step / Action / Observation / Belief, where the belief column uses the LaTeX representation of each Belief. Row 0 shows the initial belief (no action/observation).

Parameters:

  • beliefs – List of beliefs as returned by belief_trace() (length len(trace) + 1).

  • trace – Sequence of (action_label, obs_alias) pairs passed to belief_trace().

stormvogel.teaching.belief.belief_trace(pomdp: stormvogel.model.model.Model, b0: Belief, trace: list[tuple[str, str]]) → list[Belief]

Compute the sequence of beliefs induced by an observation trace.

Parameters:

  • pomdp – The POMDP model.

  • b0 – Initial belief distribution.

  • trace – Sequence of (action_label, obs_alias) pairs.

Returns:

List of beliefs of length len(trace) + 1: the initial belief followed by one updated belief per step.