The magnetic orders in Na2IrO3 and α-RuCl3, i.e., honeycomb systems with strong spin-orbit coupling and correlations, have been recently described by models with the dominant Kitaev interactions. In this work, we discuss how the orientation of the magnetic order parameter is selected in this class of models. We show that while the order-by-disorder mechanism in the models with solely Kitaev anisotropies always selects cubic axes as easy axes for magnetic ordering, the additional effect of other small bond-dependent anisotropies, such as, e.g., Γ terms, leads to a deviation of the order parameter from the cubic directions. We show that both the zigzag ground state and the face-diagonal orientation of the magnetic moments in Na2IrO3 can be obtained within the J1-K1-J2-K2-J3 model in the presence of perturbatively small Γ terms. We also show that the zigzag phase found in the nearest-neighbor Kitaev-Heisenberg model, relevant for α-RuCl3, has some stability against the Γ term.
Bibliographical noteFunding Information:
We acknowledge insightful discussions with C. Batista, G. Jackeli, M. Garst, G. Khaliullin, and I. Rousochatzakis. N.P. and Y.S. acknowledge the support from NSF Grant No. DMR-1511768. N.P. acknowledges the hospitality of KITP and partial support by the National Science Foundation under Grant No. NSF PHY11-25915.