A generalized instance weighting principal component analysis framework
-
Ernest Domanaanmwi Ganaa
Department of Information and Communication Technology, Faculty of Applied Science and Technology, Dr. Hilla Limann Technical University, Wa P.O. Box 553, Ghana
Abstract
As a classical algorithm, principal component analysis (PCA) considers all data points as equally important in pursuing projections, although some data points may be more useful than others. This consequently makes traditional PCA sensitive to noisy or corrupt data. Another challenge faced by PCA is the problem of dealing with missing observations. These two challenges may significantly affect its performance negatively. In this study, we present a generalized instance weighting PCA (IWPCA) algorithm that innovatively addresses the problems of the traditional PCA outlined above. Instead of seeking orthogonal projections directly in PCA, data is embedded in the projection space to remove a random scaling factor in the projection space and therefore, solves the issue of the missing observations challenge. To achieve better projections, a sample instance is further introduced into the model. Two approaches are then used to weigh instances from geometric and data learning views. Thus, minimizing the impact of outliers and noisy instances in the proposed algorithm. This makes the proposed model a special form of the classical PCA where instances are weighed in pursuing projections, thus improving performance by enhancing projections. Extensive experimental evaluations indicate the proposed method has superior performance over all the comparative methods.
Copyright (c) 2026 Ernest Domanaanmwi Ganaa
This work is licensed under a Creative Commons Attribution 4.0 International License.
References
[1]Hsiao JY, Chuang SJ, Chen PY. A hybrid face recognition system based on multiple facial features. International Journal of Computers and Applications. 2016; 38(1): 1–8. doi: 10.1080/1206212X.2016.1188553
[2]Lin G, Tang N, Wang H. Locally principal component analysis based on L1‐norm maximisation. IET Image Processing. 2015; 9(2): 91–96. doi: 10.1049/iet-ipr.2013.0851
[3]Kong X, Hu C, Duan Z. Generalized Principal Component Analysis. In: Principal Component Analysis Networks and Algorithms. Springer; 2017. pp. 185–233. doi: 10.1007/978-981-10-2915-8_7
[4]Ganaa ED, Abeo TA, Mehta S, et al. Incomplete-Data Oriented Dimension Reduction via Instance Factoring PCA Framework. In: Zhao Y, Barnes N, Chen B, et al. (editors). Image and Graphics. Springer; 2019. pp. 479–490. doi: 10.1007/978-3-030-34113-8_40
[5]Cai D, He X, Zhou K, et al. Locality Sensitive Discriminant Analysis. In: Proceedings of the 20th International Joint Conference on Artificial Intelligence; 6–12 January 2007; Hyderabad, India. pp. 1713–1726.
[6]Li X, Chen M, Nie F, et al. Locality Adaptive Discriminant Analysis. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence; 19–25 August 2017; Melbourne, Australia. pp. 2201–2207. doi: 10.24963/ijcai.2017/306
[7]Huang KK, Dai DQ, Ren CX. Regularized coplanar discriminant analysis for dimensionality reduction. Pattern Recognition. 2017; 62: 87–98. doi: 10.1016/j.patcog.2016.08.024
[8]Li M, Luo X, Yang J, et al. Applying a Locally Linear Embedding Algorithm for Feature Extraction and Visualization of MI-EEG. Journal of Sensors. 2016; 2016: 1–9. doi: 10.1155/2016/7481946
[9]Feng G, Hu D, Zhou Z. A Direct Locality Preserving Projections (DLPP) Algorithm for Image Recognition. Neural Processing Letters. 2008; 27(3): 247–255. doi: 10.1007/s11063-008-9073-1
[10]Wang J, Wu Y, Li B, et al. Fast anchor graph preserving projections. Pattern Recognition. 2024; 146: 109996. doi: 10.1016/j.patcog.2023.109996
[11]Zhang F, Wang H, Qin W, et al. Generalized nonconvex regularization for tensor RPCA and its applications in visual inpainting. Applied Intelligence. 2023; 53(20): 23124–23146. doi: 10.1007/s10489-023-04744-9
[12]Xu H, Caramanis C, Sanghavi S. Robust PCA via Outlier Pursuit. IEEE Transactions on Information Theory. 2012; 58(5): 3047–3064. doi: 10.1109/TIT.2011.2173156
[13]Netrapalli P, Niranjan UN, Sanghavi S, et al. Non-convex robust pca. In: Proceedings of the Advances in Neural Information Processing Systems 27; 8–13 December 2014; Montreal, QC, Canada.
[14]Jamil NW, Sharif S. Outlier Map of Classical and Robust Principal Component Analysis. Applied Mathematics and Computational Intelligence (AMCI). 2025; 14(3): 37–46. doi: 10.58915/amci.v14i3.2071
[15]Ouermi TAJ, Li J, Johnson CR. A Fast Iterative Robust Principal Component Analysis Method. arXiv preprint. 2025; arXiv:2506.16013. doi: 10.48550/arXiv.2506.16013
[16]Nie F, Yuan J, Huang H. Optimal mean robust principal component analysis. International Conference on Machine Learning. 2014; 32(2): 1062–1070.
[17]Jana T, Raghav N, Majumdar A, et al. Exact Rotation Invariant Robust PCA. In: Proceedings of the 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP); 6–11 April 2025; Hyderabad, India. pp. 1–5. doi: 10.1109/ICASSP49660.2025.10889970
[18]Hu Z, Pan G, Wang Y, et al. Sparse Principal Component Analysis via Rotation and Truncation. In: Naik GR (editor). Advances in Principal Component Analysis. Springer; 2018. pp. 1–18. doi: 10.1007/978-981-10-6704-4_1
[19]Jenatton R, Obozinski G, Bach F. Structured sparse principal component analysis. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics. 2010; 9: 366–373.
[20]Jiang B, Ding C, Luo B, et al. Graph-Laplacian PCA: Closed-Form Solution and Robustness. In: Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Recognition; 23–28 June 2013; Portland, OR, USA. pp. 3492–3498. doi: 10.1109/CVPR.2013.448
[21]Kumar N, Singh S, Kumar A. Random permutation principal component analysis for cancelable biometric recognition. Applied Intelligence. 2018; 48(9): 2824–2836. doi: 10.1007/s10489-017-1117-7
[22]Zhao Q, Meng D, Xu Z, et al. Robust principal component analysis with complex noise. Proceedings of the 31st International Conference on Machine Learning. 2014; 32(2): 55–63.
[23]Tabaghi P, Khanzadeh M, Wang Y, et al. Principal Component Analysis in Space Forms. IEEE Transactions on Signal Processing. 2024; 72: 4428–4443. doi: 10.1109/TSP.2024.3457529
[24]Li X, Li P, Zhang H, et al. Pivotal-Aware Principal Component Analysis. IEEE Transactions on Neural Networks and Learning Systems. 2024; 35(9): 12201–12210. doi: 10.1109/TNNLS.2023.3252602
[25]Mi JX, Zhu Q, Lu J. Principal component analysis based on block-norm minimization. Applied Intelligence. 2019; 49(6): 2169–2177. doi: 10.1007/s10489-018-1382-0
[26]Hubert M, Rousseeuw PJ, Vanden Branden K. ROBPCA: A New Approach to Robust Principal Component Analysis. Technometrics. 2005; 47(1): 64–79. doi: 10.1198/004017004000000563
[27]Xu H, Caramanis C, Mannor S. Outlier-Robust PCA: The High-Dimensional Case. IEEE Transactions on Information Theory. 2013; 59(1): 546–572. doi: 10.1109/TIT.2012.2212415
[28]Leyder S, Raymaekers J, Verdonck T. Generalized spherical principal component analysis. Statistics and Computing. 2024; 34(3): 104. doi: 10.1007/s11222-024-10413-9
[29]Wang S, Nie F, Wang Z, et al. Max–Min Robust Principal Component Analysis. Neurocomputing. 2023; 521: 89–98. doi: 10.1016/j.neucom.2022.11.092
[30]Zhan J, Vaswani N. Robust PCA With Partial Subspace Knowledge. IEEE Transactions on Signal Processing. 2015; 63(13): 3332–3347. doi: 10.1109/TSP.2015.2421485
[31]Zhou P, Feng J. Outlier-Robust Tensor PCA. In: Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); 21–26 July 2017; Honolulu, HI, USA. pp. 3938–3946. doi: 10.1109/CVPR.2017.419
[32]Higuchi I, Eguchi S. Robust principal component analysis with adaptive selection for tuning parameters. Journal of Machine Learning Research. 2004; 5: 453–471.
[33]Candès EJ, Li X, Ma Y, et al. Robust principal component analysis? Journal of the ACM. 2011; 58(3): 1–37. doi: 10.1145/1970392.1970395
[34]Hubert M, Rousseeuw P, Verdonck T. Robust PCA for skewed data and its outlier map. Computational Statistics & Data Analysis. 2009; 53(6): 2264–2274. doi: 10.1016/j.csda.2008.05.027
[35]Xu X, Gao R, Qing Y, et al. Hyperspectral image mixed noise removal via tensor robust principal component analysis with tensor-ring decomposition. International Journal of Remote Sensing. 2023; 44(5): 1556–1578. doi: 10.1080/01431161.2023.2187720
[36]Zhou Q, Gao Q, Wang Q, et al. Sparse discriminant PCA based on contrastive learning and class-specificity distribution. Neural Networks. 2023; 167: 775–786. doi: 10.1016/j.neunet.2023.08.061
[37]Shen H, Huang JZ. Sparse principal component analysis via regularized low rank matrix approximation. Journal of Multivariate Analysis. 2008; 99(6): 1015–1034. doi: 10.1016/j.jmva.2007.06.007
[38]Chen X, Sun H. L2,1 ‐norm‐based sparse principle component analysis with trace norm regularised term. IET Image Processing. 2019; 13(6): 910–922. doi: 10.1049/iet-ipr.2018.5433
[39]Bertsimas D, Cory-Wright R, Pauphilet J. Solving large-scale sparse pca to certifiable (near) optimality. Journal of Machine Learning Research. 2022; 23(13): 1–35.
[40]Bertsimas D, Kitane DL. Sparse pca: a geometric approach. Journal of Machine Learning Research. 2023; 24(32): 1–33.
[41]Yi S, Lai Z, He Z, et al. Joint sparse principal component analysis. Pattern Recognition. 2017; 61: 524–536. doi: 10.1016/j.patcog.2016.08.025
[42]Van Deun K, Thorrez L, Coccia M, et al. Weighted sparse principal component analysis. Chemometrics and Intelligent Laboratory Systems. 2019; 195: 103875. doi: 10.1016/j.chemolab.2019.103875
[43]Del Pia A. Sparse PCA on fixed-rank matrices. Mathematical Programming. 2023; 198(1): 139–157. doi: 10.1007/s10107-022-01769-9
[44]Mackey L. Deflation methods for sparse pca. In: Proceedings of the Advances in Neural Information Processing Systems 21; 12–13 December 2008; Vancouver, BC, Canada.
[45]Khanna R, Ghosh J, Poldrack R, et al. A Deflation Method for Structured Probabilistic PCA. In: Proceedings of the 2017 SIAM International Conference on Data Mining; 27–29 April 2017; Houston, TX, USA. pp. 534–542. doi: 10.1137/1.9781611974973.60
[46]Ganaa ED, Abeo TA, Wang L, et al. Robust deflated principal component analysis via multiple instance factorings for dimension reduction in remote sensing images. Journal of Applied Remote Sensing. 2020; 14(3): 1. doi: 10.1117/1.JRS.14.032608
[47]Khanna R, Ghosh J, Poldrack R, et al. A Deflation Method for Probabilistic PCA. In: Proceedings of the 29th Annual Conference on Neural Information Processing Systems; 11–12 December 2015; Montreal, QC, Canada.
[48]Hassani S, Hanafi M, Qannari EM, et al. Deflation strategies for multi-block principal component analysis revisited. Chemometrics and Intelligent Laboratory Systems. 2013; 120: 154–168. doi: 10.1016/j.chemolab.2012.08.011
[49]Xu C, Yang M, Zhang J. Fast deflation sparse principal component analysis via subspace projections. Journal of Statistical Computation and Simulation. 2020; 90(8): 1399–1412. doi: 10.1080/00949655.2020.1728761
[50]Shi Q, Lu H, Cheung YM. Rank-One Matrix Completion with Automatic Rank Estimation via L1-Norm Regularization. IEEE Transactions on Neural Networks and Learning Systems. 2018; 29(10): 4744–4757. doi: 10.1109/TNNLS.2017.2766160
[51]Khishe M, Azar OP, Hashemzadeh E. Variable-length CNNs evolved by digitized chimp optimization algorithm for deep learning applications. Multimedia Tools and Applications. 2024; 83(1): 2589–2607. doi: 10.1007/s11042-023-15411-z
[52]Kosarirad H, Ghasempour Nejati M, Saffari A, et al. Feature Selection and Training Multilayer Perceptron Neural Networks Using Grasshopper Optimization Algorithm for Design Optimal Classifier of Big Data Sonar. Journal of Sensors. 2022; 2022: 1–14. doi: 10.1155/2022/9620555
[53]Dash CSK, Behera AK, Dehuri S, et al. An outliers detection and elimination framework in classification task of data mining. Decision Analytics Journal. 2023; 6: 100164. doi: 10.1016/j.dajour.2023.100164
[54]Ansari S, Nassif AB, Mahmoud S, et al. Impact of Outliers on Regression and Classification Models: An Empirical Analysis. In: Proceedings of the 2024 17th International Conference on Development in eSystem Engineering (DeSE); 6–8 November 2024; Khorfakkan, United Arab Emirates. pp. 211–218. doi: 10.1109/DeSE63988.2024.10912020
