Abstract: The space nuclear power system, which integrates a gas-cooled reactor with a closed Brayton cycle, is a leading candidate for megawatt-level aerospace power system due to its high efficiency, substantial power output, and independence from sunlight. However, the system design faces significant challenges, including narrow design margins, high optimization complexity, strong component coupling, and rapid parameter responses. These challenges stem from stringent payload constraints during space launches, the trade-off between efficiency and mass optimization, and the unique conditions of the space environment. In China, the design and development of such systems are still in their start, with limited understanding of performance-influencing factors, system safety evaluation, and operational control. To address these challenges, this study proposed a refined mass calculation model for subsystems, developed an integrated design optimization code named Megrez, and modified the system analysis code RELAP5. The Megrez incorporated a flexible matrix-solving algorithm for thermodynamic cycle parameters, a physics-based mass estimation model for subsystems, and the NSGA-Ⅱ multi-objective optimization algorithm to achieve Pareto-optimal solutions for efficiency and mass. Validation results demonstrate that Megrez achieves high accuracy, with a maximum relative error of 1.1% in thermodynamic parameter calculations and a 2.1% relative error in total system mass estimation compared to the JIMO project design. The Megrez reaches a high computational efficiency, completing thermodynamic calculations in just 0.03 s, thereby facilitating extensive design space exploration. For transient analysis, the RELAP5 was modified to accommodate the unique characteristics of space nuclear power systems. Key improvements included expansion of the working fluid property library to support He-Xe mixtures, development of a transient model for turbomachinery that accounts for shaft work and flow work, optimization of numerical discretization for compact heat exchangers to mitigate staggered heat transfer errors, and refinement of convective and radiative heat transfer models for open-lattice gas-cooled reactors. The modified RELAP5 demonstrates high fidelity in simulating system transients, with key parameter relative errors below 3% compared to Megrez designs, CFD results and JIMO project benchmarks. Besides, during a negative reactivity insertion transient, the code accurately captures trends in reactor power and temperature responses, albeit with slightly faster dynamics due to differences in heat capacity assumptions. The integrated tools provide a robust foundation for the design and analysis of space nuclear power systems. Megrez enables efficient trade-off studies between system efficiency and mass, while the modified RELAP5 supports detailed transient and safety analyses. Future work will focus on multi-physics coupling, sensitivity studies of design parameters, intelligent control algorithms, and comprehensive accident scenario analyses. These advancements are critical for advancing the engineering application of space nuclear power technology and supporting future deep-space exploration missions.