Principal Component Analysis (PCA) can improve dramatically the treatment of large STEM spectrum-images by finding the directions (loadings) of highest data variance in the factor space and projecting the data on these directions. Loadings typically do not show clear physical meanings, thus the interpretation of PCA results is difficult. This work investigates the potential reasons for appearing such counterintuitive PCA outputs. The following reasons are identified: (i) missing the step of centering the data in the PCA pre-treatment, (ii) complexity of data variations inconsistent with the orthogonality restrictions of PCA, (iii) non-linearity caused either by chemical variations or by the peculiarities of the spectra formation, and (iv) inaccuracy in extracting major PCA components. In many cases, the PCA treatment can be altered in such a way that the intuitively clear loadings are delivered.