Helical nanofilaments (HNFs) have attracted much interest because of their unique optical properties, but there have been many hurdles to overcome in using them for the practical applications due to their structural complexity. Here we demonstrate that the molecular configuration and layer conformation of a modulated HNF (HNFs(mod)) can be studied using a physicochemical confinement system. The layer directions affected by the chemical affinity between the mesogen and surface were drastically controlled in surface-modified nanochannels. Furthermore, an in situ experiment using grazing-incidence X-ray diffraction (GIXD) was carried out to investigate in detail the structural evolution through thermal transitions. The results demonstrate that the HNF(mod) structure can be perfectly controlled for functional HNF device applications, and a combined system with chemical and physical confinement effects will be helpful to better understand the fundamentals of soft matter.