Improving on Expectation Propagation (2009)
A series of corrections is developed for the fixed points of Expectation Propagation (EP), which is one of the most popular methods for approximate probabilistic inference. These corrections can lead...
Genome-wide detection and analysis of hippocampus core promoters using DeepCAGE (2009)
Valen, Eivind, Pascarella, Giovanni, Chalk, Alistair, Maeda, Norihiro, Kojima, Miki, Kawazu, Chika, ...
Finding and characterizing mRNAs, their transcription start sites (TSS), and their associated promoters is a major focus in post-genome biology. Mammalian cells have at least 5–10 magnitudes more...
BIOINFORMATICS ORIGINAL PAPER (2008)
Gene Expression, Thomas Grotkjær, Ole Winther, Birgitte Regenberg, Jens Nielsen, Lars Kai Hansen
Robust multi-scale clustering of large DNA microarray datasets with the consensus algorithm Vol. 22 no. 1 2006, pages 58–67 doi:10.1093/bioinformatics/bti746
Exact Inference in Tree Graphs (2008)
• Contemporary machine learning uses complex flexible probabilistic models. • Bayesian inference is typically intractable. • Approximate polynomial complexity methods needed. • VB, Bethe, EP...
Computing with Finite and Infinite Networks (2008)
Using statistical mechanics results, I calculate learning curves (average generalization error) for Gaussian processes (GPs) and Bayesian neural networks (NNs) used for regression. Applying the...
Approximate inference in probabilistic models (2008)
Abstract. We present a framework for approximate inference in probabilistic data models which is based on free energies. The free energy is constructed from two approximating distributions which...
Joaquin Quiñonero-candela, Mathematical Modelling, Ole Winther
In this paper, we consider Tipping’s relevance vector machine (RVM) [1] and formalize an incremental training strategy as a variant of the expectation-maximization (EM) algorithm that we call...
We propose a novel a framework for deriving approximations for intractable probabilistic models. This framework is based on a free energy (negative log marginal likelihood) and can be seen as a...
Joaquin Quiñonero-candela, Mathematical Modelling, Ole Winther
In this paper, we consider Tipping’s relevance vector machine (RVM) [1] and formalize an incremental training strategy as a variant of the expectation-maximization (EM) algorithm that we call...
Improving on Expectation Propagation. (2008)
Opper, Manfred, Paquet, Ulrich, Winther, Ole
A series of corrections is developed for the fixed points of Expectation Propagation (EP), which is one of the most popular methods for approximate probabilistic inference. These corrections can lead...
Genome-wide detection and analysis of hippocampus core promoters using DeepCAGE (2008)
Valen, Eivind, Pascarella, Giovanni, Chalk, Alistair Morgan, Maeda, Norihiro, Kojima, Miki, Kawazu, Chika, ...
Finding and characterizing mRNAs, their transcription start sites (TSS) and their associated promoters is a major focus in post-genome biology. Mammalian cells have at least 5-10 magnitudes more TSS...
Genome-wide detection and analysis of hippocampus core promoters using DeepCAGE (2008)
Valen, Eivind, Pascarella, Giovanni, Chalk, Alistair Morgan, Maeda, Norihiro, Kojima, Miki, Kawazu, Chika, ...
Finding and characterizing mRNAs, their transcription start sites (TSS) and their associated promoters is a major focus in post-genome biology. Mammalian cells have at least 5-10 magnitudes more TSS...
Genome-wide detection and analysis of hippocampus core promoters using DeepCAGE (2008)
Valen, Eivind, Pascarella, Giovanni, Chalk, Alistair Morgan, Maeda, Norihiro, Kojima, Miki, Kawazu, Chika, ...
Finding and characterizing mRNAs, their transcription start sites (TSS) and their associated promoters is a major focus in post-genome biology. Mammalian cells have at least 5-10 magnitudes more TSS...
Bryne, Jan Christian, Valen, Eivind, Tang, Man-Hung Eric, Marstrand, Troels, Winther, Ole, Da Piedade, Isabelle, ...
JASPAR is a popular open-access database for matrix models describing DNA-binding preferences for transcription factors and other DNA patterns. With its third major release, JASPAR has been expanded...
Incremental Gaussian Processes (2007)
In this paper, we consider Tipping's relevance vector machine (RVM) [1] and formalize an incremental training strategy as a variant of the expectation-maximization (EM) algorithm that we call...
Computing with Finite and Infinite Networks (2007)
Using statistical mechanics results, I calculate learning curves (average generalization error) for Gaussian processes (GPs) and Bayesian neural networks (NNs) used for regression. Applying the...
A mean field algorithm for Bayes learning in large feed-forward neural networks
We develop mean eld approaches for probabilistic independent component analysis (ICA). The sources are estimated from the mean of their posterior distribution and the mixing matrix (and noise level)...
Bayesian online learning in the perceptron (2007)
Abstract. In a Bayesian approach to online learning a simple approximate parametric form for posterior is updated in each online learning step. Usually in online learning only an estimate of the...
Optimal online learning: a Bayesian approach (2007)
A recently proposed Bayesian approach to online learning is applied to learning a rule defined as a noisy single layer perceptron. In the Bayesian online approach, the exact posterior distribution is...
Regenberg, Birgitte, Grotkjær, Thomas, Winther, Ole, Fausbøll, Anders, Åkesson, Mats, Bro, Christoffer, ...
Abstract Background Growth rate is central to the development of cells in all organisms. However, little is known about the impact of changing growth rates. We used continuous cultures to control...
Kaare Brandt Petersen, Michael Syskind Pedersen, Suggestions Bill Baxter, Brian Templeton, Christian Rishøj, L. Theobald, ...
What is this? These pages are a collection of facts (identities, approximations, inequalities, relations,...) about matrices and matters relating to them. It is collected in this form for the...
Robust multi-scale clustering of large DNA microarray datasets with the consensus algorithm (2006)
Grotkjær, Thomas, Winther, Ole, Regenberg, Birgitte, Nielsen, Jens, Hansen, Lars Kai
Motivation: Hierarchical and relocation clustering (e.g. K-means and self-organizing maps) have been successful tools in the display and analysis of whole genome DNA microarray expression data....
Expectation Consistent Approximate Inference (2005)
We propose a novel framework for approximations to intractable probabilistic models which is based on a free energy formulation. The approximation can be understood from replacing an average over the...
Approximate inference techniques with expectation constraints (2005)
Heskes, Tom, Opper, Manfred, Wiegerinck, Wim, Winther, Ole, Zoeter, Onno
This paper discusses inference problems in probabilistic graphical models that often occur in a machine learning setting. In particular it presents a unified view of several recently proposed...
Approximate inference techniques with expectation constraints (2005)
Heskes, Tom, Opper, Manfred, Wiegerinck, Wim, Winther, Ole, Zoeter, Onno
This article discusses inference problems in probabilistic graphical models that often occur in a machine learning setting. In particular it presents a unified view of several recently proposed...
Expectation Consistent Free Energies for Approximate Inference (2005)
We propose a novel a framework for deriving approximations for intractable probabilistic models. This framework is based on a free energy (negative log marginal likelihood) and can be seen as a...
Expectation consistent approximate inference (2005)
Ole Winther, Mathematical Modelling
We propose a novel framework for approximations to intractable probabilistic models which is based on a free energy formulation. The approximation can be understood from replacing an average over the...
Approximate inference techniques with expectation constraints (2005)
Tom Heskes, Wim Wiegerinck, Ole Winther, Onno Zoeter
constraints
Ole Winther, Kaare Br, T Petersen
In this paper we present an empirical Bayes method for flexible and efficient Independent Component Analysis (ICA). The method is flexible with respect to choice of source prior, dimensionality and...
Expectation consistent approximate inference (2005)
Ole Winther, Mathematical Modelling
We propose a novel framework for approximations to intractable probabilistic models which is based on a free energy formulation. The approximation can be understood as replacing an average over the...
Kaare Br, T Petersen, Ole Winther, Kaare Brandt Petersen, Kaare Brandt Petersen, Kaare Brandt Petersen, ...
Linear mixing and gaussian noise
Flexible and Efficient Implementations of Bayesian Independent Component Analysis (2005)
Variational Methods, Ole Winther, Phd Kaare Br, T Petersen, Ole Winther, Kaare Br, ...
Abstract: In this paper we present an empirical Bayes method for flexible and efficient Independent Component Analysis (ICA). The method is flexible with respect to choice of source prior,...
Robust multi-scale clustering of large DNA microarray datasets with the consensus algorithm (2005)
Grotkjær, Thomas, Winther, Ole, Regenberg, Birgitte, Nielsen, Jens, Hansen, Lars Kai
Motivation: Hierarchical and relocation clustering (e.g. K-means and self-organising maps) have been successful tools in the display and analysis of whole genome DNA microarray expression data....
Approximate Inference in Probabilistic Models (2004)
We present a framework for approximate inference in probabilistic data models which is based on free energies. The free energy is constructed from two approximating distributions which encode...
Expectation Consistent Approximate Inference (2004)
We propose a novel framework for approximations to intractable probabilistic models. The method is based on a free energy formulation of inference and allows for a simultaneous computation of...
Variational linear response (2004)
A general linear response method for deriving improved estimates of correlations in the variational Bayes framework is presented. Three applications are given and it is discussed how to use linear...
Variational linear response (2004)
A general linear response method for deriving improved estimates of correlations in the variational Bayes framework is presented. Three applications are given and it is discussed how to use linear...
Teaching computers to fold proteins (2003)
A new general algorithm for optimization of potential functions for protein folding is introduced. It is based upon gradient optimization of the thermodynamic stability of native folds of a training...
Tractable Inference for Probabilistic Data Models (2003)
Based on ideas from statistical physics, we present an approximation technique for probabilistic data models with a large number of hidden variables. We give examples for two non–trivial...
Variational Linear Response (2003)
A general linear response method for deriving improved estimates of correlations in the variational Bayes framework is presented. Three applications are given and it is discussed how to use linear...
Analysis of Mean Field Annealing in Subtractive Interference Cancellation (2002)
Thomas Fabricius, Student Member Ieee, Ole Winther, Mathematical Modelling
In this contribution we derive the cost function corresponding to the linear complexity Subtractive Interference Cancellation with tangent hyperbolic tentative decisions. We use the cost function to...
TAP Gibbs free energy, belief propagation and sparsity (2002)
The adaptive TAP Gibbs free energy for a general densely connected probabilistic model with quadratic interactions and arbritary single site constraints is derived. We show how a specific sequential...
TAP Gibbs free energy, belief propagation and sparsity (2002)
The adaptive TAP Gibbs free energy for a general densely connected probabilistic model with qaudratic interactions and arbritary single site constraints is derived. We show how a specific sequential...
TAP Gibbs free energy, belief propagation and sparsity (2002)
The adaptive TAP Gibbs free energy for a general densely connected probabilistic model with quadratic interactions and arbritary single site constraints is derived. We show how a specific sequential...
TAP Gibbs free energy, belief propagation and sparsity (2002)
The adaptive TAP Gibbs free energy for a general densely connected probabilistic model with qaudratic interactions and arbritary single site constraints is derived. We show how a specific sequential...
Tractable Inference for Probabilistic Data Models (2002)
Lehel Csato, Manfred Opper, Ole Winther
We present an approximation technique for probabilistic data models with a large number of hidden variables, based on ideas from statistical physics. We give examples for two nontrivial applications....
Tractable approximations for probabilistic models: The adaptive TAP mean field approach (2001)
We develop an advanced mean field method for approximating averages in probabilistic data models that is based on the TAP approach of disorder physics. In contrast to conventional TAP, where the...
Tractable approximations for probabilistic models: The adaptive TAP mean field approach (2001)
We develop an advanced mean field method for approximating averages in probabilistic data models that is based on the TAP approach of disorder physics. In contrast to conventional TAP, where the...
Tractable approximations for probabilistic models: The adaptive TAP mean field approach (2001)
We develop an advanced mean field method for approximating averages in probabilistic data models that is based on the TAP approach of disorder physics. In contrast to conventional TAP, where the...
Tractable approximations for probabilistic models: The adaptive TAP mean field approach (2001)
We develop an advanced mean field method for approximating averages in probabilistic data models that is based on the TAP approach of disorder physics. In contrast to conventional TAP, where the...
Tractable approximations for probabilistic models: The adaptive TAP mean field approach (2001)
We develop an advanced mean field method for approximating averages in probabilistic data models that is based on the TAP approach of disorder physics. In contrast to conventional TAP, where the...
Tractable approximations for probabilistic models: The adaptive TAP mean field approach (2001)
We develop an advanced mean field method for approximating averages in probabilistic data models that is based on the TAP approach of disorder physics. In contrast to conventional TAP, where the...
Tractable approximations for probabilistic models: The adaptive TAP mean field approach (2001)
We develop an advanced mean field method for approximating averages in probabilistic data models that is based on the TAP approach of disorder physics. In contrast to conventional TAP, where the...
Tractable approximations for probabilistic models: The adaptive TAP mean field approach (2001)
We develop an advanced mean field method for approximating averages in probabilistic data models that is based on the TAP approach of disorder physics. In contrast to conventional TAP, where the...
Tractable approximations for probabilistic models: The adaptive TAP mean field approach (2001)
We develop an advanced mean field method for approximating averages in probabilistic data models that is based on the TAP approach of disorder physics. In contrast to conventional TAP, where the...
Tractable approximations for probabilistic models: The adaptive TAP mean field approach (2001)
We develop an advanced mean field method for approximating averages in probabilistic data models that is based on the TAP approach of disorder physics. In contrast to conventional TAP, where the...
Ole Winther, Mathematical Modelling
We develop a generalization of the TAP mean eld approach of disorder physics which makes the method applicable to the computation of approximate averages in probabilistic models for real data. In...
From Naive Mean Field Theory to the TAP Equations (2001)
Manfred Opper, Ole Winther, London England
Introduction Mean field (MF) methods provide tractable approximations for the computation of high dimensional sums and integrals in probabilistic models. By neglecting certain dependencies between...
Gaussian processes for classification: Mean field algorithms (2000)
We derive a mean field algorithm for binary classification with Gaussian processes which is based on the TAP approach originally proposed in Statistical Physics of disordered systems. The theory also...
Ensemble Learning and Linear Response Theory for (2000)
We propose a general Bayesian framework for performing independent component analysis (ICA) which relies on ensemble learning and linear response theory known from statistical physics. We apply it to...
Efficient Approaches to Gaussian Process Classification (2000)
Lehel Csato, Ernest Fokoué, Manfred Opper, Bernhard Schottky, Ole Winther
We present three simple approximations for the calculation of the posterior mean in Gaussian Process classification. The first two methods are related to mean field ideas known in Statistical...
Efficient Approaches to Gaussian Process Classification (2000)
Lehel Csato, Ernest Fokoué, Manfred Opper, Bernhard Schottky, Ole Winther
We present three simple approximations for the calculation of the posterior mean in Gaussian Process classification. The first two methods are related to mean field ideas known in Statistical...
Ensemble Learning and Linear Response Theory for ICA (2000)
Pedro Højen-Sørensen, Ole Winther, Lars Kai Hansen
We propose a general framework for performing independent component analysis (ICA) which relies on ensemble learning and linear response theory known from statistical physics. We apply it to both...
Computing with Finite and Infinite Networks (2000)
Using statistical mechanics results, I calculate learning curves (average generalization error) for Gaussian processes (GPs) and Bayesian neural networks (NNs) used for regression. Applying the...
Introduction Mean field (MF) methods provide efficient approximations which are able to cope with the increasing complexity of modern probabilistic data models. They replace the intractable task of...
Efficient Approaches to Gaussian Process Classification (2000)
Lehel Csat'o Ernest, Bernhard Schottky, Ole Winther
We present three simple approximations for the calculation of the posterior mean in Gaussian Process classification. The first two methods are related to mean field ideas known in Statistical...
Efficient Approaches to Gaussian Process Classification (2000)
Lehel Csato, Ernest Fokoue, Manfred Opper, Bernhard Schottky, Ole Winther
.71> y = sign (ha(x)i) where ha(x)i is the posterior mean: ha(x)i = E a(x) Q t i=1 P (y i ja(x i )) E Q t i=1 P (y i ja(x i )) E is the expectation over the GP prior and t is the number of...
Ensemble Learning and Linear Response Theory for (2000)
We propose a general framework for performing independent component analysis (ICA) which relies on ensemble learning and linear response theory known from statistical physics. We apply it to both...
Mean field methods for classification with Gaussian processes (1999)
We discuss the application of TAP mean field methods known from the Statistical Mechanics of disordered systems to Bayesian classification models with Gaussian processes. In contrast to previous...
Gaussian Processes for Classification: Mean Field Algorithms (1999)
We derive a mean field algorithm for binary classification with Gaussian processes which are based on the TAP approach originally proposed in statistical physics of disordered systems. The theory...
Mean field methods for classification with Gaussian processes (1999)
We discuss the application of TAP mean field methods known from the Statistical Mechanics of disordered systems to Bayesian classification models with Gaussian processes. In contrast to previous...
Bayesian Mean Field Algorithms for Neural Networks and Gaussian Processes (1998)
The front page figure shows an example of a mapping of input space by a committee machine with three hidden units. The subject of this thesis is the derivation and study of Bayesian mean field...
Optimal Bayesian Online Learning (1998)
Abstract. In a Bayesian approach to online learning a simple parametric approximate posterior over rules is updated in each online learning step. Predictions on new data are derived from averages...
Optimal Perceptron Learning: an Online Bayesian Approach (1998)
The recently proposed Bayesian approach to online learning is applied to learning a rule defined as a noisy single layer perceptron with either continuous or binary weights. In the Bayesian online...
Optimal Bayesian Online Learning (1998)
. In a Bayesian approach to online learning a simple parametric approximate posterior over rules is updated in each online learning step. Predictions on new data are derived from averages over this...
Regenberg, Birgitte, Grotkjær, Thomas, Winther, Ole, Fausbøll, Anders, Åkesson, Mats, Bro, Christoffer, ...
Analysis of S. cerevisiae cultures with generation times varying between 2 and 35 hours shows that the expression of half of all yeast genes is affected by the specific growth rate.
Bryne, Jan Christian, Valen, Eivind, Tang, Man-Hung Eric, Marstrand, Troels, Winther, Ole, Da Piedade, Isabelle, ...
JASPAR is a popular open-access database for matrix models describing DNA-binding preferences for transcription factors and other DNA patterns. With its third major release, JASPAR has been expanded...
Gaussian Process Classification and SVM: Mean Field Results and Leave-One-Out Estimator
In this chapter, we elaborate on the well-known relationship between Gaussian Processes (GP) and Support Vector Machines (SVM). Secondly, we present approximate solutions for two computational...
Genome-wide detection and analysis of hippocampus core promoters using DeepCAGE
Valen, Eivind, Pascarella, Giovanni, Chalk, Alistair, Maeda, Norihiro, Kojima, Miki, Kawazu, Chika, ...
Finding and characterizing mRNAs, their transcription start sites (TSS), and their associated promoters is a major focus in post-genome biology. Mammalian cells have at least 5–10 magnitudes more...
Discovery of Regulatory Elements is Improved by a Discriminatory Approach
Valen, Eivind, Sandelin, Albin, Winther, Ole, Krogh, Anders
A major goal in post-genome biology is the complete mapping of the gene regulatory networks for every organism. Identification of regulatory elements is a prerequisite for realizing this ambitious...