stormvogel.examples.sup_not_attained_pomdp¶

POMDP where the supremum reachability probability is not attained.

Hidden states are L (left) and R (right). Action listen does not change the hidden state but produces a noisy observation: from L the agent hears l with probability 9/10 and r with probability 1/10; from R the probabilities are reversed. Actions guessL and guessR commit: guessL transitions to goal from L and to failure from R; guessR is symmetric.

Supremum = 1. By listening n times and guessing the majority observation, the agent is correct with probability approaching 1 (law of large numbers), so the supremum over all observation-based policies is 1.

Not attained. Every finite observation history has positive probability under both hidden states, so whenever the agent guesses there is positive probability of failure. No policy achieves probability 1.

Because stormvogel uses deterministic state observations, the stochastic observation function is encoded by splitting each hidden state into observation-suffixed variants: L_l/R_l (last obs was l) and L_r/R_r (last obs was r). States sharing an observation suffix are indistinguishable to the agent. The start state distributes uniformly over all four variants, giving initial belief {L: 1/2, R: 1/2} regardless of the first observed observation.

Functions¶

create_sup_not_attained_pomdp(→ stormvogel.model.Model)

Return the "supremum not attained" POMDP.

Module Contents¶

stormvogel.examples.sup_not_attained_pomdp.create_sup_not_attained_pomdp() → stormvogel.model.Model¶

Return the “supremum not attained” POMDP.

The start state distributes uniformly over L_l, R_l, L_r, R_r (probability 1/4 each), so the initial belief is {L: 1/2, R: 1/2} under either first observation.

Transitions:

L_l / L_r  --listen-->  L_l (9/10),  L_r (1/10)
R_l / R_r  --listen-->  R_l (1/10),  R_r (9/10)

L_l / L_r  --guessL-->  g (1)
L_l / L_r  --guessR-->  f (1)
R_l / R_r  --guessL-->  f (1)
R_l / R_r  --guessR-->  g (1)

Returns:: A stormvogel POMDP model.