Nanoscale self-assembly, as a technique to transform two-dimensional (2D) planar patterns into three-dimensional (3D) nanoscale architectures, has achieved tremendous success in the past decade. However, an assembly process at nanoscale is easily affected by small unavoidable variations in sample conditions and reaction environment, resulting in a low yield. Recently, in-situ monitored self-assembly based on ion and electron irradiation has stood out as a promising candidate to overcome this limitation. The usage of ion and electron beam allows stress generation and real-time observation simultaneously, which significantly enhances the controllability of self-assembly. This enables the realization of various complex 3D nanostructures with a high yield. The additional dimension of the self-assembled 3D nanostructures opens the possibility to explore novel properties that cannot be demonstrated in 2D planar patterns. Here, we present a rapid review on the recent achievements and challenges in nanoscale self-assembly using electron and ion beam techniques, followed by a discussion of the novel optical properties achieved in the self-assembled 3D nanostructures.