Abstract: The analysis of Inconel 718 alloy O-ring of reactor pressure vessel (RPV) was conducted by finite element method (FEM) with two-dimensional nonlinear elastoplastic model. Based on the verification with experiments, the influences of the groove depth, groove gap on the flange ring slot on the sealing performance were investigated. The results show that the silver coating has no remarkable effect on deformation characteristics of O-ring, so the silver coating can be ignored when the O-ring is analyzed by FEM. The groove depth and groove gap are the key parameters which have important effect on the O-ring sealing performance. For the O-ring involved with RPV, the optimal values of the groove depth and the groove gap are around (11±0.25) mm and 0.6 mm respectively. In addition, the O-ring has high requirements to the manufacture and installation, and it is necessary to improve its dimensional accuracy in order to ensure its sealing performance, especially to keep the distance uniform between the O-ring and groove.