SURE-tuned tapering estimation of large covariance matrices

Feng Yi, Hui Zou

Research output: Contribution to journalArticlepeer-review

13 Scopus citations

Abstract

Bandable covariance matrices are often used to model the dependence structure of variables that follow a nature order. It has been shown that the tapering covariance estimator attains the optimal minimax rates of convergence for estimating large bandable covariance matrices. The estimation risk critically depends on the choice of the tapering parameter. We develop a Stein's Unbiased Risk Estimation (SURE) theory for estimating the Frobenius risk of the tapering estimator. SURE tuning selects the minimizer of SURE curve as the chosen tapering parameter. An extensive Monte Carlo study shows that SURE tuning is often comparable to the oracle tuning and outperforms cross-validation. We further illustrate SURE tuning using rock sonar spectrum data. The real data analysis results are consistent with simulation findings.

Original languageEnglish (US)
Pages (from-to)339-351
Number of pages13
JournalComputational Statistics and Data Analysis
Volume58
Issue number1
DOIs
StatePublished - Feb 2013

Bibliographical note

Funding Information:
This work was supported in part by NSF grant DMS-0846068 . The authors thank the editor, AE and referees for their helpful comments. Appendix Proof of Lemma 1 We start with Stein’s identity ( □ Efron, 2004 ) (A.1) ( σ ˆ i j − σ i j ) 2 = ( σ ˆ i j − σ ̃ i j s ) 2 − ( σ ̃ i j s − σ i j ) 2 + 2 ( σ ˆ i j − σ i j ) ( σ ̃ i j s − σ i j ) . Taking expectation at both side of (A.1) and summing over i , j = 1 yield E ‖ Σ ̂ − Σ ‖ F 2 = E ‖ Σ ̂ − Σ ̃ s ‖ F 2 − ∑ i = 1 p ∑ j = 1 p var ( σ ̃ i j s ) + 2 ∑ i = 1 p ∑ j = 1 p cov ( σ ˆ i j , σ ̃ i j s ) . Note that E [ ( σ ˆ i j − σ i j ) ( σ ̃ i j s − σ i j ) ] = cov ( σ ˆ i j , σ ̃ i j s ) because E σ ̃ i j s = σ i j . Proof of Lemma 2 The estimators under consideration are translational invariant. Without loss of generality, we can let μ = E ( x ) = 0 . By straightforward calculation based on bivariate normal distribution, we have (A.2) E ( x i 2 x j 2 ) = σ i i σ j j + 2 σ i j 2 , which holds for both i = j and i ≠ j . (A.3) E ( ( σ ̃ i j s ) 2 ) = E ( ( n − 1 ) − 2 ( ∑ k = 1 n x k , i x k , j − n x ̄ i x ̄ j ) 2 ) = ( n − 1 ) − 2 { E ( ( ∑ k = 1 n x k , i x k , j ) 2 ) − 2 n − 1 ∑ k = 1 n E ( n x ̄ i n x ̄ j x k , i x k , j ) + n 2 E ( x ̄ i 2 x ̄ j 2 ) } . We also have (A.4) E ( ( n − 1 ∑ k = 1 n x k , i x k , j ) 2 ) = 1 n var ( x i x j ) + ( E ( x i x j ) ) 2 = 1 n ( σ i i σ j j + 2 σ i j 2 − σ i j 2 ) + σ i j 2 = 1 n σ i i σ j j + 1 + n n σ i j 2 . Note that X ̄ ∼ N ( 0 , Σ / n ) . Using (A.2) we have (A.5) n 2 E ( x ̄ i 2 x ̄ j 2 ) = 2 σ i j 2 + σ i i σ j j . (A.6) E ( n x ̄ i n x ̄ j x k , i x k , j ) = ∑ 1 ≤ l , l ′ ≤ n { I ( l = l ′ ≠ k ) E ( x l , i x l , j x k , i x k , j ) + I ( l = l ′ = k ) E ( x k , i 2 x k , j 2 ) } = ( n − 1 ) σ 12 2 + ( σ i i σ j j + 2 σ i j 2 ) . Substituting (A.4)–(A.6) into (A.3) gives (A.7) E ( ( σ ̃ i j s ) 2 ) = n σ i j 2 + σ i i σ j j n − 1 . Thus, var ( σ ̃ i j s ) = E ( ( σ ̃ i j s ) 2 ) − σ i j 2 = σ i j 2 + σ i i σ j j n − 1 . We now show □ (2.4) by deriving an expression for E ( σ ̃ i i s σ ̃ j j s ) . (A.8) ( n − 1 ) 2 E ( σ ̃ i i s σ ̃ j j s ) = ∑ 1 ≤ k , k ′ ≤ n E ( x k , i 2 x k ′ , j 2 ) − ∑ 1 ≤ k ′ ≤ n E ( x ̄ i 2 x k ′ , j 2 ) − ∑ 1 ≤ k ≤ n E ( x ̄ j 2 x k , i 2 ) + n 2 E ( x ̄ i 2 x ̄ j 2 ) . Repeatedly using (A.2) we have (A.9) ∑ 1 ≤ k , k ′ ≤ n E ( x k , i 2 x k ′ , j 2 ) = n 2 σ i i σ j j + 2 n σ i j 2 , (A.10) n 2 E ( x ̄ i 2 x k ′ , j 2 ) = ∑ 1 ≤ l , l ′ ≤ n { I ( l = l ′ ≠ k ′ ) E ( x l , i 2 x k ′ , j 2 ) + I ( l = l ′ = k ′ ) E ( x k ′ , i 2 x k ′ , j 2 ) } = n σ i i σ j j + 2 σ i j 2 , (A.11) n 2 E ( x ̄ j 2 x k , i 2 ) = n σ i i σ j j + 2 σ i j 2 . Substituting (A.5) and (A.9)–(A.11) into (A.8) gives (A.12) E ( σ ̃ i i s σ ̃ j j s ) = n + 1 n − 1 σ i i σ j j + 2 ( n + 2 ) n ( n − 1 ) σ i j 2 . Combining (A.7) and (A.12) gives (2.4) .

Keywords

  • Covariance matrix
  • Cross-validation
  • Frobenius norm
  • Operator norms
  • SURE
  • Tapering estimator

Fingerprint

Dive into the research topics of 'SURE-tuned tapering estimation of large covariance matrices'. Together they form a unique fingerprint.

Cite this