Multi-state models are frequently used when data come from subjects observed over time and where focus is on the occurrence of events that the subjects may experience. A convenient modeling assumption is that the multi-state stochastic process is Markovian, in which case a number of methods are available when doing inference for both transition intensities and transition probabilities. The Markov assumption, however, is quite strict and may not fit actual data in a satisfactory way. Therefore, inference methods for non-Markov models are needed. In this paper, we review methods for estimating transition probabilities in such models and suggest ways of doing regression analysis based on pseudo observations. In particular, we will compare methods using land-marking with methods using plug-in. The methods are illustrated using simulations and practical examples from medical research.