Simulation-based approximate policy iteration for dynamic patient scheduling for radiation therapy.

We study radiation therapy scheduling problem where dynamically and stochastically arriving patients of different types are scheduled to future days. Unlike similar models in the literature, we consider cancellation of treatments. We formulate this dynamic multi-appointment patient scheduling problem as a Markov Decision Process (MDP). Since the MDP is intractable due to large state and action spaces, we employ a simulation-based approximate dynamic programming (ADP) approach to approximately solve our model. In particular, we develop Least-square based approximate policy iteration for solving our model. The performance of the ADP approach is compared with that of a myopic heuristic decision rule.

Dynamic scheduling with due dates and time windows: an application to chemotherapy patient appointment booking.

We study a scheduling problem in which arriving patients require appointments at specific future days within a treatment specific time window. This research is motivated by a study of chemotherapy scheduling practices at the British Columbia Cancer Agency (Canada). We formulate this problem as a Markov Decision Process (MDP). Since the resulting MDPs are intractable to exact methods, we employ linear-programming-based Approximate Dynamic Programming (ADP) to obtain approximate solutions. Using simulation, we compare the performance of the resulting ADP policies to practical and easy-to-use heuristic decision rules under diverse scenarios. The results indicate that ADP is promising in several scenarios, and that a specific easy-to-use heuristic performs well in the idealized chemotherapy scheduling setting we study.

A Markov decision process approach to multi-category patient scheduling in a diagnostic facility.

OBJECTIVES: To develop a mathematical model for multi-category patient scheduling decisions in computed tomography (CT), and to investigate associated tradeoffs from economic and operational perspectives. METHODS: We modeled this decision-problem as a finite-horizon Markov decision process (MDP) with expected net CT revenue as the performance metric. The performance of optimal policies was compared with five heuristics using data from an urban hospital. In addition to net revenue, other patient-throughput and service-quality metrics were also used in this comparative analysis. RESULTS: The optimal policy had a threshold structure in the two-scanner case - it prioritized one type of patient when the queue-length for that type exceeded a threshold. The net revenue gap between the optimal policy and the heuristics ranged from 5% to 12%. This gap was 4% higher in the more congested, single-scanner system than in the two-scanner system. The performance of the net revenue maximizing policy was similar to the heuristics, when compared with respect to the alternative performance metrics in the two-scanner case. Under the optimal policy, the average number of patients that were not scanned by the end of the day, and the average patient waiting-time, were both nearly 80% smaller in the two-scanner case than in the single-scanner case. The net revenue gap between the optimal policy and the priority-based heuristics was nearly 2% smaller as compared to the first-come-first-served and random selection schemes. Net revenue was most sensitive to inpatient (IP) penalty costs in the single-scanner system, whereas to IP and outpatient revenues in the two-scanner case. CONCLUSIONS: The performance of the optimal policy is competitive with the operational and economic metrics considered in this paper. Such a policy can be implemented relatively easily and could be tested in practice in the future. The priority-based heuristics are next-best to the optimal policy and are much easier to implement.