Inicio  /  Applied Sciences  /  Vol: 13 Par: 4 (2023)  /  Artículo
ARTÍCULO
TITULO

Learning and Compressing: Low-Rank Matrix Factorization for Deep Neural Network Compression

Gaoyuan Cai    
Juhu Li    
Xuanxin Liu    
Zhibo Chen and Haiyan Zhang    

Resumen

Recently, the deep neural network (DNN) has become one of the most advanced and powerful methods used in classification tasks. However, the cost of DNN models is sometimes considerable due to the huge sets of parameters. Therefore, it is necessary to compress these models in order to reduce the parameters in weight matrices and decrease computational consumption, while maintaining the same level of accuracy. In this paper, in order to deal with the compression problem, we first combine the loss function and the compression cost function into a joint function, and optimize it as an optimization framework. Then we combine the CUR decomposition method with this joint optimization framework to obtain the low-rank approximation matrices. Finally, we narrow the gap between the weight matrices and the low-rank approximations to compress the DNN models on the image classification task. In this algorithm, we not only solve the optimal ranks by enumeration, but also obtain the compression result with low-rank characteristics iteratively. Experiments were carried out on three public datasets under classification tasks. Comparisons with baselines and current state-of-the-art results can conclude that our proposed low-rank joint optimization compression algorithm can achieve higher accuracy and compression ratios.

 Artículos similares

       
 
Can Cui, Jiwei Qin and Qiulin Ren    
Representation learning-based collaborative filtering (CF) methods address the linear relationship of user-items with dot products and cannot study the latent nonlinear relationship applied to implicit feedback. Matching function learning-based CF method... ver más
Revista: Applied Sciences

 
Shanshan Luo, Baoqing Li, Xiaobing Yuan and Huawei Liu    
The Discriminative Correlation Filter (DCF) has been universally recognized in visual object tracking, thanks to its excellent accuracy and high speed. Nevertheless, these DCF-based trackers perform poorly in long-term tracking. The reasons include the f... ver más
Revista: Applied Sciences

 
Ali Alqahtani, Xianghua Xie and Mark W. Jones    
Deep networks often possess a vast number of parameters, and their significant redundancy in parameterization has become a widely-recognized property. This presents significant challenges and restricts many deep learning applications, making the focus on... ver más
Revista: Informatics

 
Bardia Yousefi, Hamed Akbari, Michelle Hershman, Satoru Kawakita, Henrique C. Fernandes, Clemente Ibarra-Castanedo, Samad Ahadian and Xavier P. V. Maldague    
Early diagnosis of breast cancer unequivocally improves the survival rate of patients and is crucial for disease treatment. With the current developments in infrared imaging, breast screening using dynamic thermography seems to be a great complementary m... ver más
Revista: Applied Sciences

 
Zhangren Tu, Huiting Liu, Jiaying Zhan and Di Guo    
Multidimensional nuclear magnetic resonance (NMR) spectroscopy is one of the most crucial detection tools for molecular structure analysis and has been widely used in biomedicine and chemistry. However, the development of NMR spectroscopy is hampered by ... ver más
Revista: Applied Sciences