Existing deep learning based convolutional code recognition methods still have shortcomings such as large parameter sizes and weak recognition performance. Aiming at this problem, a convolutional code recognition method based on joint learning of matrix transform features and code sequences was proposed. The received codeword sequence was arranged into a matrix form, and the soft information was used to eliminate the codewords with low reliability. Then, a new matrix transformation algorithm was used to obtain the feature matrix. During the recognition process, the original matrix of code words and the matrix of features were fed into a network model with a joint learning capability for multimodal data. The feature extraction fusion and convolution code recognition were completed in the neural network. Simulation results show that the recognition performance of the proposed method is significantly better than the existing recognition methods based on deep learning, especially for high bit rate convolutional codes. When the rate is low, the proposed method is also better than traditional methods. When the signal-to-noise ratio reaches 5 dB, the recognition rate of 25 convolutional codes with different parameters can reach 100%.