Abstract: To study the cooling capacity, the flow and heat transfer process of flow channel between the reactor pressure vessel and the insulation, a reactor pressure vessel external cooling (REPEC) experimental mechanism was performed by using the heating method. The experiment was simulated and analyzed by using the RELAP5 code. The RELAP5 results generally correspond with the experimental cases. With the increase of heat flux, inlet area and outlet area, the water circulation mass flow rate increases. But the increase of water circulation mass flow rate is limited when the degree of sub-cooling increases. The system appears two phase flow instability clearly on the condition of nearly saturated boiling.