A TT-Tucker Decomposition-Based LC Convolutional Neural Network Compression Method Without Pre-Training

Tensor training (TT) decomposition and Tucker decomposition are two effective compression methods for convolutional neural networks. However, TT and Tucker decomposition face the problems of spatial structure information loss and high computational complexity respectively. To solve the above problems, this paper considered the information retention rate and resource occupancy of the network structure and proposed a LC convolutional neural network compressed method (TT-LC) without pre-training based on TT-Tucker decomposition, adopting the learning-compression (LC) algorithm constraint compression framework. The TT-LC method includes two parts: learning step and compression step. The learning step didn’t not need the pre-training process, and adopted the exponential cyclic learning rate method to improve the training accuracy. In the compression step, this paper selected the global optimal rank according to the advantages of TT and Tucker decomposition and the characteristics of Bayes rule, and used empirically variable Bayesian matrix factorization (EVBMF) and Bayesian optimization (BayesOpt) to select reasonable ranks to guide tensor decomposition. The TT-LC method was used to compress the trained model. TT-LC method not only reduces the loss rate of spatial structure information and computational complexity, but also solves the problem that the unreasonable rank selection of the tensor leads to the significant decrease in model accuracy. It can realize the double Bayesian rank selection and double compression of the model, and obtains the optimal compression model. Finally, experiments were carried out on CIFAR10 and CIFAR100 datasets using ResNets and VGG networks. The results show that for ResNet32 network, compared with the benchmark method, the proposed method achieved a compression rate of parameter quantity of 69.6% and a floating point computation compression rate of 66.7% with the accuracy of 92.22%.