Analysis and convergence of weighted dimensionality reduction methods

Authors

  • Juan Carlos Riaño-Rojas Universidad Nacional de Colombia Sede Manizales
  • Flavio Augusto Prieto-Ortiz Universidad Nacional de Colombia Sede Bogotá
  • Edgar Nelson Sánchez-Camperos Instituto Politécnico Nacional
  • Carlos Daniel Acosta-Medina Universidad Nacional de Colombia Sede Manizales
  • Germán Augusto Castellanos-Domínguez Universidad Nacional de Colombia Sede Manizales

Keywords:

PCA, PPCA, WPCA, WRDA, dimensionality reduction

Abstract


We propose to use a Fisher type discriminant objective function addressed to weighted principal component analysis (WPCA) and weighted regularized discriminant analysis (WRDA) for dimensionality reduction. Additionally, two different proofs for the convergence of the method are obtained. First one analytically, by using the completeness theorem, and second one algebraically, employing spectral decomposition. The objective function depends on two parameters U matrix being the rotation and D diagonal matrix weight of relevant features, respectively. These parameters are computed iteratively, in order to maximize the reduction. Relevant features were obtained by determining the eigenvector associated to the most weighted eigenvalue onthe maximum value in U. Performance evaluation of the reduction methods was carried out on 70 benchmark databases. Results showed that weighted reduction methods presented the best behavior, PCA and PPCA lower than 17% while WPCA and WRDA higher than 45%. Particularly, WRDA method had the best performance in the 75% of the cases compared with the others studied here.
|Abstract
= 16 veces | PDF (ESPAÑOL (ESPAÑA))
= 15 veces|

Downloads

Download data is not yet available.

References

B. A. Olshausen, D. J. Field. “Emergence of simplecell receptive field properties by learning a sparse code for natural images”. Nature. Vol. 381. 1996. pp. 607- 609.

C. M. Bishop. Pattern recognition and machine learning. Ed. Springer. New York. 2006. pp. 1-738.

M. E. Tipping, C. M. Bishop. “Mixtures of probabilistic principal component analyzers”. Neural Computation. Vol. 11. 1999. pp. 443-482.

A. Sharma, K. K. Paliwal, G. C. Onwubolu. “Pattern Classification: An Improvement Using Combination of VQ and PCA Based Techniques”. American Journal of Applied Sciences. Vol. 2. 2005. pp. 1445-1455.

J. C. Burges. “Geometric Methods for Feature Selection and Dimensional Reduction” in: O. Maimon, L. Rokach. (editors). Data Mining and Knowledge Discovery Handbook: A Complete Guide for Practitioners and Researchers.. Ed. Springer. New York. 2006. pp. 59-91.

L. K. Saul, K. Q. Weinberger, J. H. Ham, F. Sha, D. D. Lee. “Spectral methods for dimensionality reduction”. In: O Chapelle, B. Schölkopf, A. Zien. (editores). Semisupervised Learning. Ed. MIT Press. Cambridge. Massachusetts. USA. 2006. pp. 279-294.

J. Venna. Dimensionality reduction for visual exploration of similarity structures. PhD thesis. Helsinki University of Technology. Helsinki. 2007. pp. 11-32.

L. V. Maaten, E. Postma, J. V. Herik. “Dimensionality Reduction: A Comparative Review”. Elsevier Journal of Machine Learning Research. Vol. 10. 2009. pp. 1-41.

L. Wolf, A. Shashua. “Feature Selection for Unsupervised and Supervised Inference: The Emergence of Sparsity in a Weight-Based Approach”. Journal of Machine Learning Research. Vol. 6. 2005. pp. 1855-1887.

A. Blum, P. Langley. “Selection of relevant features and examples in machine learning”. Artificial Intelligence. Vol. 97. 1997. pp. 245-271.

M. A. Turk, A. P. Pentland. “Face Recognition Using Eigenfaces”. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Maui. Hawaii. USA. Vol. 1. 1991. pp. 586-591.

G. Balcerowska, R. Siuda. “Inelastic background subtraction from a set of angle-dependent XPS spectra using PCA and polynomial approximation”. Vacuum. Vol. 54. 1999. pp.195-199.

W. Deng, J. Hu, J. Guo, W. Cai, D. Feng. “Robust,

accurate and efficient face recognition froma single training image: A uniform pursuit approach”. Pattern Recognition. Vol. 43. 2010. pp. 1748–1762.

Q. Zhao, H. Lu, D. Zhang. “A fast evolutionary pursuit algorithm based on linearly combining vectors”. Pattern Recognition. Vol. 39. 2006. pp. 310-312.

Published

2013-02-28

How to Cite

Riaño-Rojas, J. C., Prieto-Ortiz, F. A., Sánchez-Camperos, E. N., Acosta-Medina, C. D., & Castellanos-Domínguez, G. A. (2013). Analysis and convergence of weighted dimensionality reduction methods. Revista Facultad De Ingeniería Universidad De Antioquia, (56), 245–254. Retrieved from https://revistas.udea.edu.co/index.php/ingenieria/article/view/14674

Most read articles by the same author(s)