{ "cells": [ { "cell_type": "markdown", "id": "0d93a53e", "metadata": {}, "source": [ "# Building MDPs\n", "In Stormvogel, a **Markov Decision Process (MDP)** consists of:\n", "* states $S$,\n", "* actions $A$,\n", "* an initial state $s_0$,\n", "* a mapping from states to sets of *enabled actions*,\n", "* a successor distribution $P(s,a)$ for every state $s$ and every enabled action $a$, i.e., sets of transitions between states $s$ and $s'$, each annotated with an action and a probability.\n", "* state labels $L(s)$.\n", "\n", "\n", "Here we show how to construct a simple example mdp using the bird API and the model builder API.\n", "The idea is that you can choose to study (you will likely pass the exam but you have less free time) or not to study (you will have more free time but risk failing the exam)." ] }, { "cell_type": "markdown", "id": "bece1df6", "metadata": {}, "source": [ "## The study dilemma\n", "This little MDP is supposed to help you decide whether you should stuy or not." ] }, { "cell_type": "markdown", "id": "ef9dec3d", "metadata": {}, "source": [ "### Bird API\n", "For MDPs, you specify the availaible actions in `available_actions`. An action here is simply a string. You specify the transition of a state-action pair in `delta`." ] }, { "cell_type": "code", "execution_count": 1, "id": "5c5ff61c", "metadata": { "execution": { "iopub.execute_input": "2026-03-26T10:41:27.435671Z", "iopub.status.busy": "2026-03-26T10:41:27.435489Z", "iopub.status.idle": "2026-03-26T10:41:27.903705Z", "shell.execute_reply": "2026-03-26T10:41:27.903062Z" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "
\n", "