Handbook of Markov Decision Processes

Chapter 1 Introduction

Chapter 2 Finite State and Action MDPS

Chapter 3 Bias Optimality

Chapter 4 Singular Perturbations of Markov Chains and Decision Processes

Chapter 5 Average Reward Optimization Theory for Denumerable State Spaces

Chapter 6 Total Reward Criteria

Chapter 8 Blackwell Optimality

Chapter 9 The Poisson Equation for Countable Markov Chains: Probabilistic Methods and Interpretations

Chapter 10 Stability, Performance Evaluation, and Optimization

Chapter 11 Convex Analytic Methods in Markov Decision Processes

Chapter 12 The Linear Programming Approach

Chapter 13 Invariant Gambling Problems and Markov Decision Processes

Chapter 14 Neuro-Dynamic Programming: Overview and Recent Trends

Chapter 15 Markov Decision Processes in Finance and Dynamic Options

Chapter 16 Applications of Markov Decision Processes in Communication Networks

Chapter 17 Water Reservoir Applications of Markov Decision Processes