State covariances of the linear systems satisfy certain constraints imposed by the underlying dynamics. These constraints dictate a particular structure of state covariances. On the other hand, sample covariances (e.g., obtained in experiments) almost always fail to have the required structure. In view of this, it is of interest to approximate sample covariances by positive semi-definite matrices of the required structure. The structured covariance least-squares approximation problem is formulated and the Lyapunov-type matrical linear constraint is converted into an equivalent set of trace constraints. Efficient quasi Newton and generalized Newton methods capable of solving the corresponding unconstrained dual problems with the large number of variables are developed.