GPstuff - Gaussian process models for Bayesian analysis 4.1
Multinomial probit with EP, marginal posterior corrections (cm2 and fact), and other improvments.
Download GPstuff-4.1.zip or code only for Matlab GPstuff_matlab-4.1.zip or code only for Octave GPstuff_octave-4.1.zip.
Multi-latent models, survival models, Octave compatibility and many other improvements.
For Matlab download GPstuff_matlab-4.0.zip and for Octave download GPstuff_octave-4.0.zip.
- 2012-10-29 Version 3.4.1 Bug fixes. Release notes. User guide.
- 2012-10-08 Version 3.4 Some improvements and bug fixes. Release notes. Updated user guide.
If you use GPstuff, please use the reference (available online):
- Jarno Vanhatalo, Jaakko Riihimäki, Jouni Hartikainen, Pasi Jylänki, Ville Tolvanen, and Aki Vehtari (2013). GPstuff: Bayesian Modeling with Gaussian Processes. In Journal of Machine Learning Research, 14(Apr):1175 1179.
- To get release announcements, you can subscribe to the GPstuff Announcement Mailing List.
- Or subscribe to announcements at mloss.org by clicking the tiny letter symbol on the second line showing the last update date and time.
The GPstuff toolbox is a versatile collection of Gaussian process models and computational tools required for inference. The tools include, among others, various inference methods, sparse approximations and model assessment methods.
The GPstuff toolbox works (at least) with Matlab versions r2009b (7.9) or newer (older versions down to 7.7 should work also, but the code is not tested with them). Most of the functionality works also with Octave (3.6.4 or newer, see release notes for detials). Most of the code is written in m-files but some of the most computationally critical parts have been coded in C.
The GPstuff-toolbox has been developed by BECS Bayes group, Aalto University. The coding of the GPstuff-toolbox started in 2006 based on the MCMCStuff-toolbox (1998-2006), which was based on Netlab-toolbox (1996-2001). The main authors of the GPstuff have been Jarno Vanhatalo, Jaakko Riihimäki, Jouni Hartikainen, Pasi Jylänki, Ville Tolvanen and Aki Vehtari, but the package contains code written by many more people. In the Department of Biomedical Engineering and Computational Science at Aalto University these persons are (in alphabetical order): Toni Auranen, Pasi Jylänki, Tuomas Nikoskinen, Tomi Peltola, Eero Pennala, Heikki Peura, Ville Pietiläinen, Markus Siivola, Simo Särkkä and Ernesto Ulloa. People outside Aalto University are (in alphabetical order): Christopher M. Bishop, Timothy A. Davis, Matthew D. Hoffman, Kurt Hornik, Dirk-Jan Kroon, Iain Murray, Ian T. Nabney, Radford M. Neal and Carl E. Rasmussen. We want to thank them all for sharing their code under a free software license.
This software is distributed under the GNU General Public License (version 3 or later); please refer to the file License.txt, included with the software, for details.
Download and Installation
Using GPstuff from R
Features of the toolbox
Covariance and mean functions
- Several covariance functions (e.g. squared exponential, exponential, Matérn, periodic and a compactly supported piece wise polynomial function)
- Sums, products and scaling of covariance functions
- Euclidean and delta distance
- Several mean functions with marginalized parameters
- Continuous observations: Gaussian, Gaussian scale mixture (MCMC only), Student's-t, quantile regression
- Classification: Logit, Probit, multinomial logit (softmax), multinomial probit
- Count data: Binomial, Poisson, (Zero truncated) Negative-Binomial, Hurdle model, Zero-inflated Negative-Binomial, Multinomial
- Survival: Cox-PH, Weibull, log-Gaussian, log-logistic
- Point process: Log-Gaussian Cox process
- Density estimation and regression: logistic GP
- Other: derivative observations (for sexp covariance function only)
Priors for parameters (theta)
- Several priors, Hierarchical priors
- Sparse matrix routines for compactly supported covariance functions
- Fully and partially independent conditional (FIC, PIC)
- Compactly supported plus FIC (CS+FIC)
- Variational sparse (VAR), Deterministic training conditional (DTC), Subset of regressors (SOR) (Gaussian/EP only)
- Exact (Gaussian only)
- Laplace, Expectation propagation (EP), Parallel EP, Robust-EP
- Marginal posterior corrections (cm2 and fact)
- Scaled Metropolis, Hamiltonian Monte Carlo (HMC), Scaled HMC, Elliptical slice sampling
- Type II ML/MAP
- Leave-one-out cross-validation (LOO-CV), Laplace/EP LOO-CV
- Metropolis, HMC, No-U-Turn-Sampler (NUTS), Slice Sampling (SLS), Surrogate SLS, Shrinking-rank SLS, Covariance-matching SLS
- Grid, CCD, Importance sampling
- LOO-CV, Laplace/EP LOO-CV, IS-LOO-CV, k-fold-CV
- WAIC, DIC
- Average predictive comparison
Contents of the toolbox
The contents of the toolbox can be examined here.
There are many demos in the toolbox. Here are few of them:
- demo_regression1: A regression demo for full GP, compact support GP, FIC and PIC.
- demo_classific: A classification problem.
- demo_spatial1: A disease mapping problem with FIC sparse GP approximation.
- demo_births: Demonstration of analysis of birthday frequencies in USA 1969-1988 using Gaussian process with several components.
- MCMCstuff toolbox for Matlab
- MCMC Diagnostics for Matlab
- FBM tools for Matlab
- Gaussian processes web site
If you use GPstuff, please use the reference:
- Jarno Vanhatalo, Jaakko Riihimäki, Jouni Hartikainen, Pasi Jylänki, Ville Tolvanen, Aki Vehtari (2013). GPstuff: Bayesian Modeling with Gaussian Processes. In Journal of Machine Learning Research, in press.
GPstuff has also been used, for example, in the following publications:
- Jaakko Riihimäki, Pasi Jylänki and Aki Vehtari (2013). Nested Expectation Propagation for Gaussian Process Classification with a Multinomial Probit Likelihood. In Journal of Machine Learning Research, 14(Jan):75-109. Available online. See also a classification demo using modified GPstuff code (will be implemented as part of GPstuff later).
- Lari Veneranta, Richard Hudd and Jarno Vanhatalo (2013). Reproduction areas of sea-spawning Coregonids reflect the environment in shallow coastal waters. Marine Ecology Progress Series, 477:231-250.
- Jarno Vanhatalo, Laura Tuomi, Arto Inkala, Inari Helle, and Heikki Pitkänen (2013). Probabilistic Ecosystem Model for Predicting the Nutrient Concentrations in the Gulf of Finland under Diverse Management Actions. Environmental Science & Technology, 47(1):334-341.
- Jarno Vanhatalo, Lari Veneranta and Richard Hudd (2012). Species Distribution Modelling with Gaussian Processes: a Case Study with the Youngest Stages of Sea Spawning Whitefish (Coregonus lavaretus L. s.l.) Larvae. Ecological Modelling, 228:49-58.
- Teppo Juntunen, Jarno Vanhatalo, Heikki Peltonen and Samu Mäntyniemi (2012). Bayesian spatial multispecies modelling to assess pelagic fish stocks from acoustic- and trawl-survey data. ICES Journal of Marine Science, 69: 95-104.
- Heikki Joensuu, Aki Vehtari, Jaakko Riihimäki, Toshirou Nishida, Sonja E Steigen, Peter Brabec, Lukas Plank, Bengt Nilsson, Claudia Cirilli, Chiara Braconi, Andrea Bordoni, Magnus K Magnusson, Zdenek Linke, Jozef Sufliarsky, Federico Massimo, Jon G Jonasson, Angelo Paolo Dei Tos and Piotr Rutkowski (2011). Risk of gastrointestinal stromal tumour recurrence after surgery: an analysis of pooled population-based cohorts. In The Lancet Oncology, 13(3):265-274. Published Online: 07 December 2011.
- Pasi Jylänki, Jarno Vanhatalo and Aki Vehtari (2011). Robust Gaussian Process Regression with a Student-t Likelihood. Journal of Machine Learning Research, 12:3227-3257 (available online). The EP implementation described in the paper is included in the GPstuff toolbox. See also a short demo on the regression examples described in the paper.
- Jorma Rantonen, Satu Luoto, Aki Vehtari, Markku Hupli, Jaro Karppinen, Antti Malmivaara and Simo Taimela (2011). The effectiveness of two active interventions compared to self-care advice in employees with non-acute low back symptoms. A randomised, controlled trial with a 4-year follow-up in the occupational health setting. Occupational and Environmental Medicine, oem.2009.054312 (Available online 20 May 2011)
- Jarno Vanhatalo, Ville Pietiläinen and Aki Vehtari (2010). Approximate inference for disease mapping with sparse Gaussian processes. Statistics in Medicine, 29(15):1580-1607. online
- Jarno Vanhatalo, Pia Mäkelä ja Aki Vehtari (2010). Alkoholikuolleisuuden alueelliset erot Suomessa 2000-luvun alussa. Yhteiskuntapolitiikka, 75(3):265-273 (Available online in Finnish) (English translation) (Online maps in Finnish)
- Jaakko Riihimäki and Aki Vehtari (2010). Gaussian processes with monotonicity information. In Journal of Machine Learning Research: Workshop and Conference Proceedings, 9:645-652, AISTATS2010 special issue. (abstract, PDF)
- Jarno Vanhatalo and Aki Vehtari (2010). Speeding up the binary Gaussian process classification. In Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence (UAI 2010), AUAI Press. (Available online).
- Jarno Vanhatalo, Pasi Jylänki and Aki Vehtari (2009). Gaussian process regression with Student-t likelihood. In Bengio et al, editors, Advances in Neural Information Processing Systems 22, pp. 1910-1918, NIPS Foundation (Available online)
- Jarno Vanhatalo and Aki Vehtari (2009). Discussion to 'Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations' by Håvard Rue, Sara Martino and Nicolas Chopin. Journal of the Royal Statistical Society, Series B (Statistical Methodology)., 71(2):383 (Available online 6 April 2009)
- Jarno Vanhatalo and Aki Vehtari (2008). Modelling local and global phenomena with sparse Gaussian processes. Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence. (PDF)
- Jarno Vanhatalo and Aki Vehtari (2007). Sparse Log Gaussian Processes via MCMC for Spatial Epidemiology. JMLR Workshop and Conference Proceedings, 1:73-89. (Gaussian Processes in Practice) (PDF) (Slides related to the paper in PDF)