stormvogel.examples.six_state_reachability¶

6-State POMDP reachability example.

A two-level extension of the 4-state reachability POMDP.

States s1 and s2 share observation z and behave exactly as in the 4-state model: action a commits towards the target, action b mixes the belief over {s1, s2}.

States s3 and s4 share observation z2 and mirror the structure of s1/s2, but action a sends them to s1/s2 rather than to target/sink, and action b mixes within {s3, s4}.

The agent therefore faces two indistinguishable pairs of states at different levels of the problem: it must first decide (at obs z2) when to transition to the inner level, and then (at obs z) when to commit to the target.

Functions¶

`create_6state_reachability_pomdp`(...)	Return the 6-state reachability POMDP.
`create_6state_reachability_pomdp_variantb`(...)	Return the 6-state reachability POMDP, variant b (three actions).

Module Contents¶

stormvogel.examples.six_state_reachability.create_6state_reachability_pomdp(p: Fraction | float = Fraction(1, 2)) → stormvogel.model.Model¶

Return the 6-state reachability POMDP.

The initial distribution puts weight p on s3 and 1 − p on s4.

Transitions at the outer level (obs z2):

s3  --a-->  s1 (7/10),  s2 (3/10)
s3  --b-->  s3 (8/10),  s4 (2/10)
s4  --a-->  s1 (2/10),  s2 (8/10)
s4  --b-->  s3 (3/10),  s4 (7/10)

Transitions at the inner level (obs z) are identical to the 4-state model:

s1  --a-->  target (7/10),  sink (3/10)
s1  --b-->  s1 (8/10),     s2 (2/10)
s2  --a-->  target (2/10),  sink (8/10)
s2  --b-->  s1 (3/10),     s2 (7/10)

Parameters:: p – Initial probability of being in s3 (must be in (0, 1)).
Returns:: A stormvogel POMDP model.
Raises:: ValueError – If p is not in (0, 1).

stormvogel.examples.six_state_reachability.create_6state_reachability_pomdp_variantb(p: Fraction | float = Fraction(1, 2)) → stormvogel.model.Model¶

Return the 6-state reachability POMDP, variant b (three actions).

Mirrors create_4state_reachability_pomdp_variantb(): the inner level uses three actions (a, b, c), and the outer level (s3/s4) mirrors the same structure with s1/s2 in place of target/sink.

Transitions at the outer level (obs z2):

s3  --a-->  s1 (1)
s3  --b-->  s3 (3/4),  s4 (1/4)
s3  --c-->  s1 (1/4),  s2 (3/4)
s4  --a-->  s2 (1)
s4  --b-->  s3 (2/3),  s4 (1/3)
s4  --c-->  s1 (4/5),  s2 (1/5)

Transitions at the inner level (obs z) are identical to the 4-state variant b:

s1  --a-->  target (1)
s1  --b-->  s1 (3/4),     s2 (1/4)
s1  --c-->  target (1/4), sink (3/4)
s2  --a-->  sink (1)
s2  --b-->  s1 (2/3),     s2 (1/3)
s2  --c-->  target (4/5), sink (1/5)

Parameters:: p – Initial probability of being in s3 (must be in (0, 1)).
Returns:: A stormvogel POMDP model.
Raises:: ValueError – If p is not in (0, 1).