Abstract: Small mobile reactors have the characteristic of being able to adapt to the multielement energy needs of different geographical environments and application scenarios, and are currently one of the important candidate solutions for solving diversified power demands in civilian field. Then the Brayton cycle is a commonly used form of thermoelectric conversion device for reactor power due to its advantages of fast start-up, simple system, high economic efficiency, and high efficiency. The gas cooled reactor coupled with Brayton cycle is a widely studied field by scholars both domestically and internationally. In order to investigate the key safety features of He-Xe gas cooled reactor systems coupled with Brayton cycle, this paper used RELAP5/MOD code to model the system based on its design parameters, and simulated and analyzed the system’s stability and potential accident conditions. The model established in this paper was composed of a He-Xe gas cooled reactor coupled with two sets of open Brayton power generation modules, where the He-Xe gas cooled reactor was the primary loop and the working fluid was He-Xe gas at 40 g/mol. The Brayton cycle was the secondary loop with air as the working fluid. The steady-state condition of the system was simulated, and the simulation parameters were compared with the original design values. The results show that maximum relative error is 3.08%, which proves the accuracy of the model. For the reactivity insertion accident caused by a control system failure resulting in an increase of 0.02 reactivity at a rate of 0.001 s⁻¹, when the reactor protection system fails, the reactor core hotspot temperature will reach 2 493.061 K 9 seconds after the accident, which exceeds the safe temperature of the system and will cause breakdown. When the reactor protection system is effective, the reactor shutdown protection will be triggered 11 seconds after the accident, and the reactor core temperature will be 2 001.581 K, then the reactor can shut down safely. For the loss of flow accident caused by a failure of the primary loop fan, the reactor core hotspot temperature will be 1 650.851 K, and the reactor will continue to operate in a new steady state thereafter. For the one of the secondary turbine faults caused a 40% decrease in mass flow rate in the loop, the reactor core hotspot temperature will be 1 507.644 K. After the system stabilizes, the reactor continue to operate at 33% rated power. The results indicate that the model established in the paper can accurately simulate various operating conditions of the Brayton He-Xe gas cooled reactor system, and the system is safe in the event of the three accidents mentioned above when the protection system is effective.