Stochastic Control Problems with Probability and Risk sensitive Criteria

Abstract

In this thesis we consider stochastic control problems with probability and risk-sensitive criterion. We consider both single and multi controller problems. Under probability criterion we first consider a zero-sum game with semi-Markov state process. We consider a general state and finite action spaces. Under suitable assumptions, we establish the existence of value of the game and also characterize it through an optimality equation. In the process we also prescribe a saddle point equilibrium. Next we newlineconsider a zero-sum game with probability criterion for continuous time Markov chains. We consider denumerable state space and unbounded transition rates. Again under suitable assumptions, we show the existence of value of the game and also characterize it as the unique solution of a pair of Shapley equations. We also establish the existence of a randomized stationary saddle point equilibrium. newline newlineIn the risk-sensitive setup we consider a single controller problem with semi-Markov state process. The state space is assumed to be discrete. In place of the classical risk-sensitive utility function, which is the exponential function, we consider general utility functions. The optimization criteria also contains a discount factor. We investigate random finite horizon and infinite horizon problems. Using a state augmentation technique we characterize the value functions and also prescribe optimal controls. We then consider risk-sensitive game problems. We study zero and non-zero sum risk-sensitive average criterion games for semi-Markov processes with a finite state space. For the zero-sum case, under suitable assumptions we show that the game has a value. We also establish the existence of a stationary saddle point equilibrium. For the non-zero sum case, under suitable assumptions we establish the existence of a stationary Nash equilibrium. newline newlineFinally, we also consider a partially observable model. More specifically, we investigate partially observable zero sum games where the state process is a discrete time Markov chain.