Modern control theory has provided a large number of tools for dealing with probabilistic systems. However, most of these tools solve for local policies; there are relatively few tools for solving for complex plans that, for instance, gather information. In contrast, the planning community has provided ways to compute plans that handle complex probabilistic uncertainty, but these often don't work for large or continuous problems. Recently, our group has developed techniques for planners that can efficiently search for complex plans in probabilistic domains by taking advantage of local solutions provided by feedback and open-loop controllers, and predicting a distribution over the posteriors. This approach of planning over distributions of posteriors can incorporate a surprisingly wide variety of sensor models and objective functions. I will show some results in a couple of domains including helicopter flight in GPS-denied environments.