Abstract: The onset of flow instability (OFI) in a natural circulation loop with 3×3 rod bundle channels was studied experimentally and simulated by RELAP5 code. By fitting the experimental data, a new empirical equation for calculating the heat flux of OFI of the rod bundle channels in a natural circulation was obtained with the maximum relative error of 20.10%. The OFI was analyzed by the method of driving force in order to know its occurrence mechanism. Results indicate that the subcooling boiling at the outlet of the rod bundle channels leads to the increase of the density difference between the hot and cold parts in the natural circulation loop. Consequently, it contributes to the total driving force increase and the OFI occurrence. RELAP5 has good applicability to OFI calculation under low pressure natural circulation condition, and its result is less conservative than that of the experiment.