Information Cut for Clustering using a Gradient Descent Approach (2009)
Robert Jenssen, Deniz Erdogmus, Jose C. Principe, Torbjørn Eltoft
We introduce a new graph cut for clustering which we call the Information Cut. It is derived using Parzen windowing to estimate an information theoretic distance measure between probability density...
The Laplacian Classifier (2009)
Robert Jenssen, Deniz Erdogmus, Jose C. Principe, Torbjørn Eltoft
Abstract—We develop a novel classifier in a kernel feature space related to the eigenspectrum of the Laplacian data matrix. The classification cost function measures the angle between class mean...
Cognitive State Estimation Based on EEG for Augmented Cognition (2009)
Deniz Erdogmus, Andre Adami, Michael Pavel, Tian Lan, Santosh Mathan, Stephen Whitlow, ...
Abstract ⎯ Augmented cognition is an emerging concept that aims to enhance user performance and cognitive capabilities on the basis of adaptive assistance. An integral part of such systems is the...
Recursive Least Squares for an Entropy Regularized MSE Cost Function (2008)
Deniz Erdogmus, Ana N. Rao, Jose C. Principe
Abstract. Minimum MSE plays an indispensable role in learning and adaptation of neural systems. Nevertheless, the instantaneous value of the modeling error alone does not convey sufficient...
ESTIMATING COGNITIVE STATE USING EEG SIGNALS (2008)
Tian Lan, Andre Adami, Deniz Erdogmus, Misha Pavel
Using EEG signals to estimate cognitive state has drawn increasing attention in recently years, especially in the context of brain-computer interface (BCI) design. However, this goal is extremely...
Robert Jenssen, Deniz Erdogmus, Josec. Principefellow, Torbjørn Eltoft
Abstract — Recent work has revealed a close connection between certain information theoretic divergence measures and properties of Mercer kernel feature spaces. Specifically, it has been proposed...
Stochastic Blind Equalization Based on PDF Fitting using Parzen Estimator (2008)
Marcelino Lázaro, Ignacio Santamaría, Deniz Erdogmus, Carlos Pantaleón, Jose C. Principe
Abstract — This paper presents a new blind equalization approach that aims to force the probability density function (pdf) at the equalizer output to match the known constellation pdf. Quadratic...
Accelerating the convergence speed of neural networks (2008)
Oscar Fontenla-romero, Deniz Erdogmus, Jose C. Principe, Amparo Alonso-betanzos, Enrique Castillo
learning methods using least squares
Vector-Quantization using Information Theoretic Concepts (2008)
Anant Hegde, Deniz Erdogmus, Jose C. Principe
Abstract. The process of representing a large data set with a smaller number of vectors in the best possible way, also known as vector quantization, has been intensively studied in the recent years....
Deniz Erdogmus, Oscar Fontenla-romero, Jose C. Principe, Amparo Alonso-betanzos, Enrique Castillo
Abstract—Training multilayer neural networks is typically carried out using descent techniques such as the gradient-based backpropagation (BP) of error or the quasi-Newton approaches including the...
Minimax Mutual Information Approach for ICA of Complex-Valued Linear Mixtures (2008)
Jian-wu Xu, Deniz Erdogmus, Ana N. Rao, José Carlos Príncipe
Abstract. Recently, the authors developed the Minimax Mutual Information algorithm for linear ICA of real-valued mixtures, which is based on a density estimate stemming from Jaynes ’ maximum...
RECURSIVE PRINCIPAL COMPONENTS ANALYSIS USING EIGENVECTOR MATRIX PERTURBATION (2008)
Deniz Erdogmus, Ana N. Rao, Hemanth Peddaneni, Anant Hegde, Jose C. Principe
Abstract. Principal components analysis is an important and well-studied subject in statistics and signal processing. The literature has an abundance of algorithms for solving this problem, where...
Sequential Feature Extraction Using Information-Theoretic Learning (2008)
Deniz Erdogmus, Kari Torkkola, Jose C. Principe
Abstract-- A classification system typically includes both a feature extractor and a classifier. The two components can be trained either sequentially or simultaneously. The former option has an...
A MUTUAL INFORMATION EXTENSION TO THE MATCHED FILTER (2008)
Deniz Erdogmus, Rati Agrawal, Jose C. Principe
Abstract. Matched filters are the optimal linear filters for signal detection under linear channel and white noise conditions. Their optimality is guaranteed in the additive white Gaussian noise...
Deniz Erdogmus, Jose C. Principe
Fano’s inequality has proven to be one important result in Shannon’s information theory having found applications in innumerous proofs of convergence. It also provides us with a lower bound on...
Robert Jenssen, Deniz Erdogmus, Josec. Principefellow, Torbjørn Eltoft
Abstract — Recent work has revealed a close connection between certain information theoretic divergence measures and properties of Mercer kernel feature spaces. Specifically, it has been proposed...
Kernel Maximum Entropy Data Transformation and an Enhanced Spectral Clustering Algorithm (2008)
Robert Jenssen, Torbjørn Eltoft, Mark Girolami, Deniz Erdogmus
We propose a new kernel-based data transformation technique. It is founded on the principle of maximum entropy (MaxEnt) preservation, hence named kernel MaxEnt. The key measure is Renyi’s entropy...
Zero-entropy minimization for blind extraction of bounded sources (BEBS (2008)
Frédéric Vrins, Deniz Erdogmus, Christian Jutten, Michel Verleysen
Abstract. Renyi’s entropy can be used as a cost function for blind source separation (BSS). Previous works have emphasized the advantage of setting Renyi’s exponent to a value different from one...
Robert Jenssen, Deniz Erdogmus, Jose C. Principe, Torbjørn Eltoft
Abstract. This paper addresses the problem of efficient information theoretic, non-parametric data clustering. We develop a procedure for adapting the cluster memberships of the data patterns, in...
Channel Selection and Feature Projection for Cognitive Load Estimation Using Ambulatory EEG (2007)
Tian Lan, Deniz Erdogmus, Andre Adami, Santosh Mathan, Misha Pavel
We present an ambulatory cognitive state classification system to assess the subject's mental load based on EEG measurements. The ambulatory cognitive state estimator is utilized in the context of a...
Anant Hegde, Deniz Erdogmus, Deng S. Shiau, Jose C. Principe, Chris J. Sackellares
Abnormal dynamical coupling between brain structures is believed to be primarily responsible for the generation of epileptic seizures and their propagation. In this study, we attempt to identify the...
Channel Selection and Feature Projection for Cognitive Load Estimation Using Ambulatory EEG (2007)
Tian Lan, Deniz Erdogmus, Andre Adami, Santosh Mathan, Misha Pavel
We present an ambulatory cognitive state classification system to assess the subject's mental load based on EEG measurements. The ambulatory cognitive state estimator is utilized in the context of a...
Anant Hegde, Deniz Erdogmus, Deng S. Shiau, Jose C. Principe, Chris J. Sackellares
Abnormal dynamical coupling between brain structures is believed to be primarily responsible for the generation of epileptic seizures and their propagation. In this study, we attempt to identify the...
Some Equivalences between Kernel Methods and Information Theoretic Methods (2006)
Robert Jenssen, Deniz Erdogmus, Jose C. Principe, Torbjørn Eltoft
Abstract. In this paper, we discuss some equivalences between two recently introduced statistical learning schemes, namely Mercer kernel methods and information theoretic methods. We show that Parzen...
Sung-Phil Kim, Yadunandana N. Rao, Deniz Erdogmus, Justin C. Sanchez, Jose C. Principe
We propose the use of nonnegative matrix factorization (NMF) as a model-independent methodology to analyze neural activity. We demonstrate that, using this technique, it is possible to identify local...
The Laplacian PDF Distance: A Cost Function for Clustering in a Kernel Feature Space (2005)
Robert Jenssen, Deniz Erdogmus, Jose Principe, Torbjørn Eltoft
A new distance measure between probability density functions (pdfs) is introduced, which we refer to as the Laplacian pdf distance. The Laplacian pdf distance exhibits a remarkable connection to...
The Laplacian Spectral Classifier (2005)
Robert Jenssen, Deniz Erdogmus, Jose C. Principe, Torbjørn Eltoft
We develop a novel classifier in a kernel feature space defined by the eigenspectrum of the Laplacian data matrix. The classification cost function is derived from a distance measure between...
Sung-Phil Kim, Yadunandana N. Rao, Deniz Erdogmus, Justin C. Sanchez, Jose C. Principe
We propose the use of nonnegative matrix factorization (NMF) as a model-independent methodology to analyze neural activity. We demonstrate that, using this technique, it is possible to identify local...
Recursive Principal Components Analysis Using Eigenvector Matrix Perturbation (2004)
Deniz Erdogmus, Yadunandana N. Rao, Hemanth Peddaneni, Anant Hegde, Jose C. Principe
Principal components analysis is an important and well-studied subject in statistics and signal processing. The literature has an abundance of algorithms for solving this problem, where most of these...
Feature selection in MLPs and SVMs based on maximum output information (2004)
Vikas Sindhwani, Subrata Rakshit, Dipti Deodhare, Deniz Erdogmus, Jose Principe, Partha Niyogi
Abstract — We present feature selection algorithms for multilayer
Vector- quantization by density matching in the minimum Kullback–Leibler divergence sense (2004)
Anant Hegde, Deniz Erdogmus, Tue Lehn-schioler, Ana N. Rao, Jose C. Principe
Abstract – Representation of a large set of highdimensional data is a fundamental problem in many applications such as communications and biomedical systems. The problem has been tackled by...
Minimax Mutual Information Approach for Independent Component Analysis (2004)
Deniz Erdogmus, Yadunandana N. Rao, Jose C. Principe
Minimum output mutual information is regarded as a natural criterion for independent component analysis (ICA) and is used as the performance measure in many ICA algorithms. Two common approaches in...
Recursive Principal Components Analysis Using Eigenvector Matrix Perturbation (2004)
Jose C. Principe, Anant Hegde, Hemanth Peddaneni, Yadunandana N. Rao, Deniz Erdogmus
Principal components analysis is an important and well-studied subject in statistics and signal processing. The literature has an abundance of algorithms for solving this problem, where most of these...
Recursive Principal Components Analysis Using Eigenvector Matrix Perturbation (2004)
Deniz Erdogmus, Yadunandana N. Rao, Hemanth Peddaneni, Anant Hegde, Jose C. Principe
Principal components analysis is an important and well-studied subject in statistics and signal processing. The literature has an abundance of algorithms for solving this problem, where most of these...
Deniz Erdogmus, Yadunandana N. Rao, Jose C. Principe
SIPEX-G is a fast-converging, robust, gradient-based PCA algorithm that has been recently proposed by the authors. Its superior performance in synthetic and real data compared with its benchmark...
Clustering using Renyi's Entropy (2003)
Robert Jenssen Kenneth, Deniz Erdogmus, Jose C. Principe, Torbjørn Eltoft
We propose a new clustering algorithm using Renyi's entropy as our similarity metric. The main idea is to assign a data pattern to the cluster, which among all possible clusters, increases its...
Jose C. Principe, Yadunandana N. Rao, Deniz Erdogmus
SIPEX-G is a fast-converging, robust, gradient-based PCA algorithm that has been recently proposed by the authors. Its superior performance in synthetic and real data compared with its benchmark...
Thesis (Ph. D.)--University of Florida, 2002.
A neural network perspective to extended Luenberger observers (2002)
Deniz Erdogmus, A. Umut Genç, José C. Príncipe
In this paper, we investigate the use of adaptive extended Luenberger state estimators for general nonlinear and possibly time-varying systems. We identify the connection between the extended...
Justin C. Sanchez, Deniz Erdogmus, Jose C. Principe
Recently, several research groups demonstrated that linear models could estimate hand position using populations of action potentials collected in the pre-motor and motor cortical areas of a...
ABSTRACT: Traditionally, second-order statistics are used as the optimality criterion in almost any adaptive system training scenario, supervised or unsupervised, with great success, thanks to the...
Deniz Erdogmus, Yadunandana N. Rao, Jose C. Principe
SIPEX-G is a fast-converging, robust, gradient-based PCA algorithm that has been recently proposed by the authors. Its superior performance in synthetic and real data compared with its benchmark...
Deniz Erdogmus, Deniz Rende, Jose C. Principe, Tan F. Wong
Abstract. The minimum error entropy criterion was recently suggested in adaptive system training as an alternative to the mean-square-error criterion, and it was shown to produce better results in...
Deniz Erdogmus, Jose C. Principe
The error-entropy-minimization approach in adaptive system training is investigated. The effect of Parzen windowing on the location of the global minimum of entropy has been investigated. An...
Hegde, Anant, Erdogmus, Deniz, Shiau, Deng S., Principe, Jose C., Sackellares, Chris J.
Abnormal dynamical coupling between brain structures is believed to be primarily responsible for the generation of epileptic seizures and their propagation. In this study, we attempt to identify the...
Channel Selection and Feature Projection for Cognitive Load Estimation Using Ambulatory EEG
Lan, Tian, Erdogmus, Deniz, Adami, Andre, Mathan, Santosh, Pavel, Misha
We present an ambulatory cognitive state classification system to assess the subject's mental load based on EEG measurements. The ambulatory cognitive state estimator is utilized in the context of a...