2014/12/21 by Abdulrahman Oladipupo Ibraheem, Ibraheem, Abdulrahman Oladipupo
Computer Science · Mathematics · #90-08 #Advanced Image and Video Retrieval Techniques #Artificial Intelligence (cs.AI) #Computer Vision and Pattern Recognition (cs.CV) #FOS: Computer and information sciences #FOS: Mathematics #Face and Expression Recognition #Image Retrieval and Classification Techniques #Machine Learning (stat.ML) #Neural and Evolutionary Computing (cs.NE) #Optimization and Control (math.OC) #cs.AI #cs.CV #cs.NE #math.OC #msc:90-08 #stat.ML
paper · pdf · doi:10.48550/arxiv.1412.6749
Eighteen pages in all, but much of the central ideas are covered in the first five and a half pages; most of the remaining pages are devoted to straightforward mathematical derivations, and the presentation of three algorithms. Manuscript contains no figures at this time
arxiv created 2014/12/21 · openalex publication_date 2014/12/21 · arxiv updated 2014/12/23 · openalex created_date 2025/10/10 · openalex updated_date 2026/07/28
By drawing on ideas from optimisation theory, artificial neural networks (ANN), graph embeddings and sparse representations, I develop a novel technique, termed SENNS (Sparse Extraction Neural NetworkS), aimed at addressing the feature extraction problem. The proposed method uses (preferably deep) ANNs for projecting input attribute vectors to an output space wherein pairwise distances are maximized for vectors belonging to different classes, but minimized for those belonging to the same class, while simultaneously enforcing sparsity on the ANN outputs. The vectors that result from the projection can then be used as features in any classifier of choice. Mathematically, I formulate the proposed method as the minimisation of an objective function which can be interpreted, in the ANN output space, as a negative factor of the sum of the squares of the pair-wise distances between output vectors belonging to different classes, added to a positive factor of the sum of squares of the pair-wise distances between output vectors belonging to the same classes, plus sparsity and weight decay terms. To derive an algorithm for minimizing the objective function via gradient descent, I use the multi-variate version of the chain rule to obtain the partial derivatives of the function with respect to ANN weights and biases, and find that each of the required partial derivatives can be expressed as a sum of six terms. As it turns out, four of those six terms can be computed using the standard back propagation algorithm; the fifth can be computed via a slight modification of the standard backpropagation algorithm; while the sixth one can be computed via simple arithmetic. Finally, I propose experiments on the ARABASE Arabic corpora of digits and letters, the CMU PIE database of faces, the MNIST digits database, and other standard machine learning databases.