Real-time face recognition is one of the most challenging problems in face recognition. We propose a novel algorithm to address this problem based on a sparse representation based classification (SRC) framework. First, we remove the background through the background subtraction algorithm, extract the foreground, and then, extract the face, thereby reducing the workload of the latter algorithm to improve the operation speed of the algorithm. We have adopted a sparse illumination learning and transfer (SILT) with robustness. The illumination in SILT is learned by fitting illumination examples of auxiliary face images from one or more additional subjects with a sparsely-used illumination dictionary. By enforcing a sparse representation of the query image in the illumination dictionary, the SILT can effectively recover and transfer the illumination and pose information from the alignment stage to the recognition stage. The new algorithms significantly outperform the state of the art in the single-sample regime with less restriction. In particular, the single-sample face alignment accuracy is comparable to that of the well-known Deformable SRC algorithm using multiple gallery images per class. Furthermore, the face recognition accuracy exceeds those of the SRC and Extended SRC algorithms using hand labeled alignment initialization.