Many machine learning problems exhibit a rich geometric structure either in the data space or in the model parameter space. Applications abound involving data that lies on a low-dimensional manifold (for example, the orthogonal group, or the manifold of fixed-rank matrices). In some cases, models may also be invariant along some directions in the space of parameters, resulting in some geometric structure in the training problem.

In this talk, we consider two machine learning problems. First, motivated by hyperparameter optimization in deep learning, we describe an algorithmic framework for optimizing functions with low effective dimension, that are only varying over a low-dimensional subspace. Our framework relies on successive low-dimensional random embeddings of the objective. In the second part of the talk, we explain how electroencephalogram signals can be classified by representing them by covariance matrices. Since the latters are positive semidefinite, we present recent results on the geometry of positive-semidefinite matrix manifolds that help to address this second problem.