Thread Subject:

negative eigenvalue in principal component analysis

Hello there,

I am trying to reconstruct a function using PCA. Here is what I do.
I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chi-squared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative.

The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues???

can someone kindly suggest a solution or some references...

thanx in advance

"aymer" wrote in message <jir4ee$t1d$1@newscl01ah.mathworks.com>...
> Hello there,
>
> I am trying to reconstruct a function using PCA. Here is what I do.
> I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chi-squared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative.
>
> The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues???
>
> can someone kindly suggest a solution or some references...
> thanx in advance

Negative eigenvalues with a significant magnitude indicate a serious model misspecification. You might rethink the equality assumption and/or use fewer original variables.

Negative eigenvalues with insignificant magnitudes indicate a less serious model misspecification. Typically, it just indicates the use of too many variables that are highly correlated.

Examine the coefficents of the negative eigenvalue eigenvectors as well as the higher
magnitude values of the correlation coefficient matrix.
 
Hope this helps.

Greg

Hello Greg,

Thank you for you reply. I generate my chi-square function and use John's Hessian function (available in matlab central) to evaluate the hessian matrix for it using some fiducial parameter values. Initially I used 25 parameters. (corresponding to 25 bins of my data range). Fisher matrix is just half of hessian (approximately) and covariance matrix is inverse of the fisher. When I evaluate the covariance matrix it gives me negative values on the diagonal elements, which is clearly wrong. So I think the problem is in the evaluation of fisher itself and this may be the reason for the negative eigen values. I tried using the same procedure for less parameters (using a subset of the data and binning it in just 3 bins and hence we have just three parameters), but I face the same problem.

Any idea where I might be making a mistake??
thank you for your time..

"Greg Heath" <heath@alumni.brown.edu> wrote in message <jj0jk6$ntt$1@newscl01ah.mathworks.com>...
> "aymer" wrote in message <jir4ee$t1d$1@newscl01ah.mathworks.com>...
> > Hello there,
> >
> > I am trying to reconstruct a function using PCA. Here is what I do.
> > I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chi-squared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative.
> >
> > The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues???
> >
> > can someone kindly suggest a solution or some references...
> > thanx in advance
>
> Negative eigenvalues with a significant magnitude indicate a serious model misspecification. You might rethink the equality assumption and/or use fewer original variables.
>
> Negative eigenvalues with insignificant magnitudes indicate a less serious model misspecification. Typically, it just indicates the use of too many variables that are highly correlated.
>
> Examine the coefficents of the negative eigenvalue eigenvectors as well as the higher
> magnitude values of the correlation coefficient matrix.
>
> Hope this helps.
>
> Greg

"aymer " <remyansonu@gmail.com> wrote in message <jj32e1$ovd$1@newscl01ah.mathworks.com>...
> Hello Greg,
>
> Thank you for you reply. I generate my chi-square function and use John's Hessian function (available in matlab central) to evaluate the hessian matrix for it using some fiducial parameter values. Initially I used 25 parameters. (corresponding to 25 bins of my data range). Fisher matrix is just half of hessian (approximately) and covariance matrix is inverse of the fisher. When I evaluate the covariance matrix it gives me negative values on the diagonal elements, which is clearly wrong. So I think the problem is in the evaluation of fisher itself and this may be the reason for the negative eigen values. I tried using the same procedure for less parameters (using a subset of the data and binning it in just 3 bins and hence we have just three parameters), but I face the same problem.
>
> Any idea where I might be making a mistake??
> thank you for your time..
>
> "Greg Heath" <heath@alumni.brown.edu> wrote in message <jj0jk6$ntt$1@newscl01ah.mathworks.com>...
> > "aymer" wrote in message <jir4ee$t1d$1@newscl01ah.mathworks.com>...
> > > Hello there,
> > >
> > > I am trying to reconstruct a function using PCA. Here is what I do.
> > > I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chi-squared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative.
> > >
> > > The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues???
> > >
> > > can someone kindly suggest a solution or some references...
> > > thanx in advance
> >
> > Negative eigenvalues with a significant magnitude indicate a serious model misspecification. You might rethink the equality assumption and/or use fewer original variables.
> >
> > Negative eigenvalues with insignificant magnitudes indicate a less serious model misspecification. Typically, it just indicates the use of too many variables that are highly correlated.
> >
> > Examine the coefficents of the negative eigenvalue eigenvectors as well as the higher
> > magnitude values of the correlation coefficient matrix.
> >
> > Hope this helps.
> >
> > Greg

PLEASE DO NOT TOP POST!
PLACE YOUR RESPONSES AT THE BOTTOM OF THE PAGE.

Sorry, no help. Am familiar wth covariance, and eigenvalues but no Fisher.

Greg

"Greg Heath" <heath@alumni.brown.edu> wrote in message <jj67i7$i59$1@newscl01ah.mathworks.com>...
> "aymer " <remyansonu@gmail.com> wrote in message <jj32e1$ovd$1@newscl01ah.mathworks.com>...
> > Hello Greg,
> >
> > Thank you for you reply. I generate my chi-square function and use John's Hessian function (available in matlab central) to evaluate the hessian matrix for it using some fiducial parameter values. Initially I used 25 parameters. (corresponding to 25 bins of my data range). Fisher matrix is just half of hessian (approximately) and covariance matrix is inverse of the fisher. When I evaluate the covariance matrix it gives me negative values on the diagonal elements, which is clearly wrong. So I think the problem is in the evaluation of fisher itself and this may be the reason for the negative eigen values. I tried using the same procedure for less parameters (using a subset of the data and binning it in just 3 bins and hence we have just three parameters), but I face the same problem.
> >
> > Any idea where I might be making a mistake??
> > thank you for your time..
> >
> > "Greg Heath" <heath@alumni.brown.edu> wrote in message <jj0jk6$ntt$1@newscl01ah.mathworks.com>...
> > > "aymer" wrote in message <jir4ee$t1d$1@newscl01ah.mathworks.com>...
> > > > Hello there,
> > > >
> > > > I am trying to reconstruct a function using PCA. Here is what I do.
> > > > I divide my data range into N number of bins (at first attempt 25). I assume that my function is given by some constant number over each bin, i.e f(x)=sum(beta(i)). I reconstruct my theoretical predictions using this and construct chi-squared using data values. Now to find the fisher matrix , I take a fiducial model for this unknown parameters beta,I take them all to be equal to 1 (I read somewhere that the reconstruction does not depend much on these values). Next I find out the eigenvalues and eigen vectors of this fisher matrix using eig command. The problem is some of the eigen values are coming out to be negative.
> > > >
> > > > The errors in the principal components goes as 1/sqrt(eigenvalue). Is one supposed to take the magnitude of the eigenvalues???
> > > >
> > > > can someone kindly suggest a solution or some references...
> > > > thanx in advance
> > >
> > > Negative eigenvalues with a significant magnitude indicate a serious model misspecification. You might rethink the equality assumption and/or use fewer original variables.
> > >
> > > Negative eigenvalues with insignificant magnitudes indicate a less serious model misspecification. Typically, it just indicates the use of too many variables that are highly correlated.
> > >
> > > Examine the coefficents of the negative eigenvalue eigenvectors as well as the higher
> > > magnitude values of the correlation coefficient matrix.
> > >
> > > Hope this helps.
> > >
> > > Greg
>
> PLEASE DO NOT TOP POST!
> PLACE YOUR RESPONSES AT THE BOTTOM OF THE PAGE.
>
> Sorry, no help. Am familiar wth covariance, and eigenvalues but no Fisher.
>
> Greg

I'll keep that in mind Greg...sorry about that

thanx for your time