The reinforcement design for openings in reinforced concrete (RC) deep beams presents a notable challenge in structural engineering. This study centers on RC deep beams with square openings, aiming to assess the reliability of empirical design methods and introduce an innovative topology optimization-based approach for additional reinforcement design. Through experimental data acquisition and nonlinear finite-element simulations, the research systematically explores how key parameters, such as opening location, size, load-bearing capacity, and failure mechanism—impact structural performance. Findings reveal that the distance between openings and the "direct pressure path", along with opening size, are critical factors affecting force transfer in these beams. A multilevel rebar diameter topology optimization technique is presented, and the optimal initial reinforcement configuration for openings is determined useing simulation results, leading to the development of a novel design method for opening reinforcement. Numerical examples of deep beams under single-point loading are employed to compare the new method with the strut-and-tie model (STM). The results show that deep beams with openings designed via the proposed method have significant advantages in stiffness retention, ultimate load capacity, and ultimate elastoplastic deformation capacity. Moreover, the method effectively optimizes the distribution of tensile damage in concrete, enhances the load-bearing capacity around openings, and maximizes steel strength utilization. This research offers an improved approach for designing RC deep beams with square openings, providing valuable insights and methods to advance related fields by overcoming the limitations of traditional design approaches with a more efficient and reliable solution.