Abstract: The formation and control of free surface are the most essential parts in the studies of windowless target in accelerator driven sub-critical system (ADS). Water model experiments and 360° full scale three dimensional simulations were conducted. The experimental study demonstrates that the free surface is significantly affected by the inlet flow velocity and outlet pressure. The length of free surface decreases in the second order with the increase of inlet flow velocity, while it decreases linearly with the outlet pressure. The structure and feature of flow field were investigated. The results show that the free surface is vulnerable to the vortex movement. Transient simulations were performed with volume of fluid (VOF) method, large eddy simulation (LES) and the pressure implicit with splitting of operators (PISO) algorithm. The simulation results agree qualitatively well with the experimental data related to both free surface flow and flow field. These simulation models and methods are proved to be applicable in the hydraulic simulations of liquid heavy metal target.