Research

Statistical machine learning provides tools for understanding complex data collections, using Bayesian inference to cope with uncertainty on model parameters originating from learning from finite data. We develop computationally efficient and maximally automatic approximate algorithms for Bayesian inference, in context of probabilistic programming and machine learning. Our goal is to allow the user to focus on model specification, not needing to worry about the specifics of inference.

Highlights:

1. Efficient model-independent variational approximation by re-using gradient computations during optimization. (Sakaya and Klami. Importance sampled stochastic optimization for variational inference, UAI 2017)
2. Probabilistic programs for complicated matrix factorization tasks. (Klami, Bouchard and Tripathi. Group-sparse embeddings in collective matrix factorization, ICLR 2014)
3. Bayesian inference under combinatorial constraints. (Klami and Jitta. Probabilistic size-constrained microclustering, UAI 2016)

Projects:

1. Scalable probabilistic analytics (SPA, 2016-2018). Development of computationally efficient variational inference algorihms for probabilistic programs. Funded by Tekes, Reaktor, Ekahau and M-Brain. Collaboration with Petri Myllymäki and Teemu Roos.
2. Reliable Automatic Bayesian Machine Learning (RAB-ML, 2018-2019). Development of reliable and efficient solutions for Bayesian machine learning and probabilistic programming. Funded by Academy of Finland ICT-2023 program. Collaboration with Aki Vehtari and Antti Honkela.
3. Finnish Center for Artificial Intelligence (FCAI), Agile probabilistic AI research theme.

Machine learning research is often carried out in the context of elegant but simplified setups: It is assumed that all relevant data is provided in form a simple matrix or tensor. In most practical applications this is not the case, but instead we need to combine information scattered in multiple data sources of heterogeneous nature. We provide fundamental modeling solutions for combining such data sources.

Highlights:

1. Bayesian canonical correlation analysis and inter-battery factor analysis (IBFA) and their extension Group factor analysis (GFA) for discovering relationships between more than two parallel data sources. (Klami, Virtanen and Kaski. Bayesian canonical correlation analysis, JMLR, 2013; Klami et al. Group factor analysis, IEEE Transactions on Neural Networks and Learning Systems, 2015).
2. Matrix factorization tools for discovering relationships between more complex setups and heterogeneous data. (Klami, Bouchard and Tripathi. Group-sparse embeddings in collective matrix factorization, ICLR 2014; Klami. Polya-gamma augmentations for factor models, ACML 2015)
3. Cross-domain object matching for discovering relationships between data sources with no known pairing between objects. (Jitta and Klami. Few-to-few cross-domain object matching, Advanced methodologies for Bayesian networks, 2017; Klami. Bayesian object matching, Machine Learning, 2013).

Projects:

1. Traces of Information: Intelligence from Fragmented Sources (ToI, 2013-2019). Academy Research Fellow project of Arto Klami.
2. Improved Learning by Combinin Information Sources (ILCIS, 2013-2015). Funded by Xerox Research Foundation, collaboration with Abhishek Tripathi (Xerox Research Center India) and Petri Myllymäki.

Many modern machine learning models are complex and require large training data sets, which makes learning difficult in applications where labeling examples is costly or difficult. We study data-efficient techniques for learning complex models from limited supervision, by utilizing related learning tasks (multi-task learning, transfer learning) and unlabeled observations or external constraints (semi-supervised learning). We also develop solutions for changing enrivonments based on domain adaptation.

Highlights:

1. Structured pseudo-labels for semi-supervised learning based on output-space constraints or smoothness assumptions. (Longi, Pulkkinen and Klami. Semi-supervised convolutional neural networks for identifying Wi-Fi interference sources, ACML'17).
2. Transfer learning in computer science education. (Lagus et al. Transfer learning methods in programming course outcome prediction. To appear in ACM TOCE).

Today it is easy to collect information about individuals by monitoring their activities, either based on explicit sensors or by log data collected by computers they interact with. We develop models required for inferring interesting and useful information based on such data, to describe, understand and enchance our daily life.

Highlights:

1. Modeling keyboard usage in programming education and other educational contexts. (Leinonen et al. Automatic inference of programming performance and experience from typing patterns, ACM TSCSE'16)
2. Models for complex movement trajectories in indoor spaces. (Jitta and Klami. Partially hidden Markov models for privacy-preserving modeling of indoor trajectories, Neurocomputing 2017)
3. Human computer interaction in touch-based information retrieval. (Andolina et al. Intentstreams: smart parallel search streams for branching exploratory search, IUI'15)
4. Understanding intentions based on brain signal analysis, in particular MEG recordings during natural tasks. (Kauppi et al. Towards brain-activity-controlled information retrieval: Decoding image relevance from MEG signals, NeuroImage 2015)

Projects:

1. Traces of Information: Intelligence from Fragmented Sources (ToI, 2013-2019). Academy Research Fellow project of Arto Klami.

Several of the applications we are interested in build on analysis of high-dimensional time series and spectral data. We develop practical machine learning solutions for such data.

Highlights:

1. Artificial intelligence solutions for advanced signal processing in ultrasonic cleaning, in collaboration with Edward Haegsström and Altum Technologies. (Rajani et al. Detecting industrial fouling by monotonicity during ultrasonic cleaning, under review).
2. Detection of radio-frequency devices from spectral data. (Longi, Pulkkinen and Klami. Semi-supervised convolutional neural networks for identifying Wi-Fi interference sources, ACML'17)
3. Mobile hyperspectral imaging. We are currently working on analysis methods for hyperspectral images of natural environment.

Projects:

1. Scalable probabilistic analytics (SPA, 2016-2018). Development of computationally efficient variational inference algorihms for probabilistic programs. Funded by Tekes, Reaktor, Ekahau and M-Brain. Collaboration with Petri Myllymäki and Teemu Roos.