First-step analysis

First-step analysis conditions on the first transition of a stochastic process, turning hitting probabilities, hitting times, and accumulated rewards into linear recurrence equations.