Abstract: In recent years, cosmic ray muon tomography technology has attracted extensive attention. It uses natural muons as the radiation source. Because of the high penetration ability and sensitivity to high-Z materials, cosmic ray muon tomography can realize nondestructive inspection of special nuclear materials. Therefore, it has broad application prospects in the field of container detection such as border defense and ports. Multiple Coulomb scattering occurs when muons pass through materials, and the scattering angle distribution is related to the atomic number of the material. In practical applications, such as container or cargo material smuggling detection and other scenes requiring timeliness, sometimes it is not necessary to get accurate reconstruction images, but only need to realize the discrimination of high atomic number materials in a relatively short time, and give an alarm. In view of this, we proposed a method to quickly realize material discrimination based on only a small amount of cosmic ray muon data without reconstructing the image, so as to better meet the needs of practical application scenarios. The machine learning method of convolutional neural network was applied first for material discrimination in muon tomography and the optimal model was obtained through processing the training data iteratively, and finally the results of the accuracy of material discrimination and its relation with the measurement time used were obtained. Above all, the Geant4 Monte Carlo simulation program was established. To simulate the real detection environment as much as possible, the natural source term of cosmic rays was introduced, and the models of the detector and the materials to be detected were built according to the actual size of the muon tomography facility. Further, the detection experiments of different materials were carried out at the muon tomography facility for special material tracking in China Institute of Atomic Energy, the muon tracks were reconstructed from the measured data, and the incidence and the scattering angles were calculated. The material discrimination model based on convolutional neural network was constructed for feature extraction to realize the classification and discrimination, and the residual error and the feature matrix were introduced to improve the accuracy of material discrimination. The experimental results show that, for the tungsten block with the size of 10 cm×10 cm×10 cm, the accuracy of material discrimination is up to 99.1% in 5 minutes’ measurement and 100.0% in 10 minutes’ measurement. This method based on convolutional neural network provides a new approach for material discrimination in muon scattering tomography.