{ "cells": [ { "cell_type": "markdown", "id": "7e796657", "metadata": {}, "source": [ "# Step Two: Model Checking\n", "After creating and inspecting the model in the previous *Step One*, we will now apply model checking to the Orchard MDP. While, strictly speaking, model checking asks whether a property holds on an MDP, we will see that modern probabilistic model checking tools, including Storm, can actually go beyond such queries.\n", "The section is structured based on the type of properties that we consider." ] }, { "cell_type": "markdown", "id": "9968a722", "metadata": {}, "source": [ "## Preparation\n", "We start be loading the models from the previous notebook.\n", "The complete Stormvogel model for the Orchard game is given in `orchard/orchard_game_stormvogel.py`.\n", "The Prism specification is given in `orchard/orchard_stormvogel.pm`.\n", "These models are built using functions in `orchard/orchard_builder.py`." ] }, { "cell_type": "code", "execution_count": 1, "id": "342e90e0", "metadata": { "execution": { "iopub.execute_input": "2026-03-26T10:39:36.136219Z", "iopub.status.busy": "2026-03-26T10:39:36.135967Z", "iopub.status.idle": "2026-03-26T10:39:36.366512Z", "shell.execute_reply": "2026-03-26T10:39:36.365963Z" } }, "outputs": [], "source": [ "# Import stormvogel and prism models from the previous step\n", "import stormvogel\n", "import stormpy\n", "\n", "from orchard.orchard_builder import build_simple, build_full, build_prism" ] }, { "cell_type": "code", "execution_count": 2, "id": "127595c6", "metadata": { "execution": { "iopub.execute_input": "2026-03-26T10:39:36.368614Z", "iopub.status.busy": "2026-03-26T10:39:36.368330Z", "iopub.status.idle": "2026-03-26T10:39:38.715217Z", "shell.execute_reply": "2026-03-26T10:39:38.714623Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/__w/stormvogel/stormvogel/stormvogel/bird.py:126: UserWarning: State-action rewards are not supported in this version of stormvogel. Will assign None to the second parameter of your reward function.\n", " warnings.warn(\n" ] } ], "source": [ "orchard_simple = build_simple()\n", "orchard, orchard_storm = build_full()\n", "orchard_prism, _ = build_prism()" ] }, { "cell_type": "markdown", "id": "7d30416d", "metadata": {}, "source": [ "We have the following four models:\n", "- `orchard_simple` is the simple Orchard game (with only two fruits) modeled in Stormvogel.\n", "- `orchard` is the full Orchard game model in Stormvogel.\n", "- `orchard_storm` is the `orchard` model in Storm data structures.\n", "- `orchard_prism` is the full Orchard game model specified in the Prism modeling language.\n", "\n", "The latter three models are semantically equivalent." ] }, { "cell_type": "markdown", "id": "b59398a7", "metadata": {}, "source": [ "## Reachability\n", "One of the simplest properties for MDPs is a reachability query *\"what is the\n", "maximal probability to reach a specific set of states, described by $\\varphi$?\"*.\n", "In Storm and in Stormvogel, we specify formulas in the defacto standard [Prism representation](https://www.prismmodelchecker.org/manual/PropertySpecification/Introduction), for example `Pmax=? [F \"Label\"]`.\n", "\n", "In the Orchard game, we are mostly interested in the *\"maximal probability of winning the game\"*. Using the label `\"PlayersWon\"`, we express this property as `Pmax=?(F \"PlayersWon\")`, i.e. maximizing the probability of reaching a state where the players have won.\n", "\n", "Note that, for many queries, we get a result for every state and we therefore clarify in the code that we are interested in the probability from the initial state (`initial_state`)." ] }, { "cell_type": "code", "execution_count": 3, "id": "db29c044", "metadata": { "execution": { "iopub.execute_input": "2026-03-26T10:39:38.717424Z", "iopub.status.busy": "2026-03-26T10:39:38.717184Z", "iopub.status.idle": "2026-03-26T10:39:38.729387Z", "shell.execute_reply": "2026-03-26T10:39:38.728906Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0.5711805425946498\n" ] } ], "source": [ "# Probability of winning the simple game\n", "prob_players_won = 'Pmax=? [F \"PlayersWon\"]'\n", "result = stormvogel.model_checking(orchard_simple, prob_players_won)\n", "print(result.get_result_of_state(orchard_simple.initial_state))" ] }, { "cell_type": "markdown", "id": "3912cbf6", "metadata": {}, "source": [ "The resulting winning probability is $0.5712$." ] }, { "cell_type": "markdown", "id": "7c58793a", "metadata": {}, "source": [ "We can also visualize the result of this query on the MDP state space.\n", "\n", "Note again that a potential warning of the form `Test request failed` can be safely ignored." ] }, { "cell_type": "code", "execution_count": 4, "id": "b2799547", "metadata": { "editable": true, "execution": { "iopub.execute_input": "2026-03-26T10:39:38.731263Z", "iopub.status.busy": "2026-03-26T10:39:38.731086Z", "iopub.status.idle": "2026-03-26T10:39:38.939909Z", "shell.execute_reply": "2026-03-26T10:39:38.939262Z" }, "slideshow": { "slide_type": "" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "
\n", "