Department of Industrial and Systems Engineering, University of Wisconsin-Madison, Madison, WI 53706, USA.
Med Decis Making. 2010 Jul-Aug;30(4):474-83. doi: 10.1177/0272989X09353194. Epub 2009 Dec 31.
We provide a tutorial on the construction and evaluation of Markov decision processes (MDPs), which are powerful analytical tools used for sequential decision making under uncertainty that have been widely used in many industrial and manufacturing applications but are underutilized in medical decision making (MDM). We demonstrate the use of an MDP to solve a sequential clinical treatment problem under uncertainty. Markov decision processes generalize standard Markov models in that a decision process is embedded in the model and multiple decisions are made over time. Furthermore, they have significant advantages over standard decision analysis. We compare MDPs to standard Markov-based simulation models by solving the problem of the optimal timing of living-donor liver transplantation using both methods. Both models result in the same optimal transplantation policy and the same total life expectancies for the same patient and living donor. The computation time for solving the MDP model is significantly smaller than that for solving the Markov model. We briefly describe the growing literature of MDPs applied to medical decisions.
我们提供了一个关于构建和评估马尔可夫决策过程（MDP）的教程，MDP 是一种在不确定性下进行序贯决策的强大分析工具，已广泛应用于许多工业和制造业应用中，但在医疗决策（MDM）中未得到充分利用。我们演示了如何使用 MDP 来解决不确定情况下的序贯临床治疗问题。MDP 相对于标准的基于马尔可夫的模拟模型具有显著优势。MDP 在模型中嵌入了决策过程，并且可以随着时间的推移进行多次决策。我们通过使用这两种方法解决活体供肝移植的最佳时机问题，将 MDP 与标准基于马尔可夫的模拟模型进行了比较。两种模型都得到了相同的最优移植策略和相同的患者和活体供者的总预期寿命。解决 MDP 模型的计算时间明显小于解决马尔可夫模型的计算时间。我们简要描述了应用于医疗决策的 MDP 不断增长的文献。
