# Thread Subject: negative eigenvalue in principal component analysis

 Subject: negative eigenvalue in principal component analysis From: aymer Date: 2 Mar, 2012 18:44:30 Message: 1 of 5 Hello there, I am trying to reconstruct a function using PCA. Here is what I do. I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chi-squared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative. The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues??? can someone kindly suggest a solution or some references... thanx in advance
 Subject: negative eigenvalue in principal component analysis From: Greg Heath Date: 4 Mar, 2012 20:34:14 Message: 2 of 5 "aymer" wrote in message ... > Hello there, > > I am trying to reconstruct a function using PCA. Here is what I do. > I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chi-squared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative. > > The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues??? > > can someone kindly suggest a solution or some references... > thanx in advance Negative eigenvalues with a significant magnitude indicate a serious model misspecification. You might rethink the equality assumption and/or use fewer original variables. Negative eigenvalues with insignificant magnitudes indicate a less serious model misspecification. Typically, it just indicates the use of too many variables that are highly correlated. Examine the coefficents of the negative eigenvalue eigenvectors as well as the higher magnitude values of the correlation coefficient matrix.   Hope this helps. Greg
 Subject: negative eigenvalue in principal component analysis From: aymer Date: 5 Mar, 2012 18:59:13 Message: 3 of 5 Hello Greg, Thank you for you reply. I generate my chi-square function and use John's Hessian function (available in matlab central) to evaluate the hessian matrix for it using some fiducial parameter values. Initially I used 25 parameters. (corresponding to 25 bins of my data range). Fisher matrix is just half of hessian (approximately) and covariance matrix is inverse of the fisher. When I evaluate the covariance matrix it gives me negative values on the diagonal elements, which is clearly wrong. So I think the problem is in the evaluation of fisher itself and this may be the reason for the negative eigen values. I tried using the same procedure for less parameters (using a subset of the data and binning it in just 3 bins and hence we have just three parameters), but I face the same problem. Any idea where I might be making a mistake?? thank you for your time.. "Greg Heath" wrote in message ... > "aymer" wrote in message ... > > Hello there, > > > > I am trying to reconstruct a function using PCA. Here is what I do. > > I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chi-squared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative. > > > > The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues??? > > > > can someone kindly suggest a solution or some references... > > thanx in advance > > Negative eigenvalues with a significant magnitude indicate a serious model misspecification. You might rethink the equality assumption and/or use fewer original variables. > > Negative eigenvalues with insignificant magnitudes indicate a less serious model misspecification. Typically, it just indicates the use of too many variables that are highly correlated. > > Examine the coefficents of the negative eigenvalue eigenvectors as well as the higher > magnitude values of the correlation coefficient matrix. > > Hope this helps. > > Greg
 Subject: negative eigenvalue in principal component analysis From: Greg Heath Date: 6 Mar, 2012 23:45:11 Message: 4 of 5 "aymer " wrote in message ... > Hello Greg, > > Thank you for you reply. I generate my chi-square function and use John's Hessian function (available in matlab central) to evaluate the hessian matrix for it using some fiducial parameter values. Initially I used 25 parameters. (corresponding to 25 bins of my data range). Fisher matrix is just half of hessian (approximately) and covariance matrix is inverse of the fisher. When I evaluate the covariance matrix it gives me negative values on the diagonal elements, which is clearly wrong. So I think the problem is in the evaluation of fisher itself and this may be the reason for the negative eigen values. I tried using the same procedure for less parameters (using a subset of the data and binning it in just 3 bins and hence we have just three parameters), but I face the same problem. > > Any idea where I might be making a mistake?? > thank you for your time.. > > "Greg Heath" wrote in message ... > > "aymer" wrote in message ... > > > Hello there, > > > > > > I am trying to reconstruct a function using PCA. Here is what I do. > > > I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chi-squared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative. > > > > > > The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues??? > > > > > > can someone kindly suggest a solution or some references... > > > thanx in advance > > > > Negative eigenvalues with a significant magnitude indicate a serious model misspecification. You might rethink the equality assumption and/or use fewer original variables. > > > > Negative eigenvalues with insignificant magnitudes indicate a less serious model misspecification. Typically, it just indicates the use of too many variables that are highly correlated. > > > > Examine the coefficents of the negative eigenvalue eigenvectors as well as the higher > > magnitude values of the correlation coefficient matrix. > > > > Hope this helps. > > > > Greg PLEASE DO NOT TOP POST! PLACE YOUR RESPONSES AT THE BOTTOM OF THE PAGE. Sorry, no help. Am familiar wth covariance, and eigenvalues but no Fisher. Greg
 Subject: negative eigenvalue in principal component analysis From: aymer Date: 8 Mar, 2012 17:53:34 Message: 5 of 5 "Greg Heath" wrote in message ... > "aymer " wrote in message ... > > Hello Greg, > > > > Thank you for you reply. I generate my chi-square function and use John's Hessian function (available in matlab central) to evaluate the hessian matrix for it using some fiducial parameter values. Initially I used 25 parameters. (corresponding to 25 bins of my data range). Fisher matrix is just half of hessian (approximately) and covariance matrix is inverse of the fisher. When I evaluate the covariance matrix it gives me negative values on the diagonal elements, which is clearly wrong. So I think the problem is in the evaluation of fisher itself and this may be the reason for the negative eigen values. I tried using the same procedure for less parameters (using a subset of the data and binning it in just 3 bins and hence we have just three parameters), but I face the same problem. > > > > Any idea where I might be making a mistake?? > > thank you for your time.. > > > > "Greg Heath" wrote in message ... > > > "aymer" wrote in message ... > > > > Hello there, > > > > > > > > I am trying to reconstruct a function using PCA. Here is what I do. > > > > I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chi-squared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative. > > > > > > > > The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues??? > > > > > > > > can someone kindly suggest a solution or some references... > > > > thanx in advance > > > > > > Negative eigenvalues with a significant magnitude indicate a serious model misspecification. You might rethink the equality assumption and/or use fewer original variables. > > > > > > Negative eigenvalues with insignificant magnitudes indicate a less serious model misspecification. Typically, it just indicates the use of too many variables that are highly correlated. > > > > > > Examine the coefficents of the negative eigenvalue eigenvectors as well as the higher > > > magnitude values of the correlation coefficient matrix. > > > > > > Hope this helps. > > > > > > Greg > > PLEASE DO NOT TOP POST! > PLACE YOUR RESPONSES AT THE BOTTOM OF THE PAGE. > > Sorry, no help. Am familiar wth covariance, and eigenvalues but no Fisher. > > Greg I'll keep that in mind Greg...sorry about that thanx for your time

## Tags for this Thread

### Add a New Tag:

Separated by commas
Ex.: root locus, bode

### What are tags?

A tag is like a keyword or category label associated with each thread. Tags make it easier for you to find threads of interest.

Anyone can tag a thread. Tags are public and visible to everyone.

Tag Activity for This Thread
Tag Applied By Date/Time
pca aymer 2 Mar, 2012 13:49:41
eigenvalues aymer 2 Mar, 2012 13:49:41
Feed for this Thread

Contact us