Symmetric partition functions for efficient MaxEnt in high dimensions

Maximum Entropy (MaxEnt) methods such as Conditional Random Fields (CRF) have enjoyed great successes in modeling sequences of low-dimensional, discrete data. High-dimensional, continuous data, on the other hand, pose computational challenges for existing methods, which scale exponentially in the dimensionality of the data. We have identified a natural class of MaxEnt models for continuous sequences in which inference is tractable; in particular, we have shown that if the MaxEnt features are functions of low-dimensional projections, then the MaxEnt partition function can be computed directly in a compressed form of tractable size.
This method allows us to apply MaxEnt modeling to high-dimensional sequences such as human motion capture data without having to perform dimensionality reduction as a preprocessing step. Furthermore, our method enables the efficient computation of a variety of probabilistic inferences, such as the probability that an observed path is generated by a learned model, or the probability that a path generated by the model ever visits some region of space. We are currently researching further applications of the method to fields such as robotic manipulation, where it may be used to build context-based models of robot motion from demonstration.