In hybrid control problems, the dynamics can be modeled as periods of continuous control, punctuated by discrete switches between different dynamic modes. We additionally consider situations where the continuous state variables can stochastically evolve with time, and as a result also when performing switching. Efficient methods are developed to handle these stochastic hybrid optimal control problems using dynamic programming algorithms. Extending the ideas of Ferreti and Festa, we apply these methods to a model situation of optimizing the expected time-to-target of a sailboat navigating in a headwind whose direction stochastically evolves with time. Here, we must determine the optimal navigation policy conditioned on the current wind direction, as well as the optimal times to “tack” and switch course by swinging the bow through the headwind. The problem is first mapped to reduced coordinates, and a two-stage solution process is implemented. In the first stage we solve for the optimal control policy over the entire state space, which is utilized in the second stage to perform live navigation under stochastic wind trajectories.