Many problems in machine learning involve collecting high-dimensional multivariate observations or sequences of observations, and then fitting a compact model which explains these observations. Recently, linear algebra techniques have given a fundamentally different perspective on how to fit and perform inference in these models. Exploiting the underlying spectral properties of the model parameters has led to fast, provably consistent methods for parameter learning that stand in contrast to previous approaches, such as Expectation Maximization, which suffer from bad local optima and slow convergence. In the past several years, these Spectral Learning algorithms have become increasingly popular. They have been applied to learn the structure and parameters of many models including predictive state representations, finite state transducers, hidden Markov models, latent trees, latent junction trees, probabilistic context free grammars, and mixture/admixture models. Spectral learning algorithms have also been applied to a wide range of application domains including system identification, video modeling, speech modeling, robotics, and natural language processing. The focus of this workshop will be on spectral learning algorithms, broadly construed as any method that fits a model by way of a spectral decomposition of moments of (features of) observations. We would like the workshop to be as inclusive as possible and encourage paper submissions and participation from a wide range of research related to this focus. We will encourage submissions on the following themes: - How can spectral techniques help us develop fast and local minima free solutions to real world problems where existing methods such as Expectation Maximization are unsatisfactory? - How do spectral/moment methods compare to maximum-likelihood estimators and Bayesian methods, especially in terms of robustness, statistical efficiency, and computational efficiency? - What notions of spectral decompositions are appropriate for latent variable models and structured prediction problems? - How can spectral methods take advantage of multi-core/multi-node computing environments? - What computational problems, besides parameter estimation, can benefit from spectral decompositions and operator parameterizations? (For example, applications to parsing.)
https://sites.google.com/site/spectrallearningworkshop/
