We analyze expert review and student performance data to evaluate the validity of the Cybersecurity Concept Inventory (CCI) for assessing student knowledge of core cybersecurity concepts after a first course on the topic. A panel of 12 experts in cybersecurity reviewed the CCI, and 142 students from six different institutions took the CCI as a pilot test. The panel reviewed each item of the CCI and the overwhelming majority rated every item as measuring appropriate cybersecurity knowledge. We administered the CCI to students taking a first cybersecurity course either online or proctored by the course instructor. We applied classical test theory to evaluate the quality of the CCI. This evaluation showed that the CCI is sufficiently reliable for measuring student knowledge of cybersecurity and that the CCI may be too difficult as a whole. We describe the results of the expert review and the pilot test and provide recommendations for the continued improvement of the CCI.
|Original language||English (US)|
|Title of host publication||2019 IEEE Frontiers in Education Conference, FIE 2019|
|Publisher||Institute of Electrical and Electronics Engineers Inc.|
|State||Published - Oct 2019|
|Event||49th IEEE Frontiers in Education Conference, FIE 2019 - Covington, United States|
Duration: Oct 16 2019 → Oct 19 2019
|Name||Proceedings - Frontiers in Education Conference, FIE|
|Conference||49th IEEE Frontiers in Education Conference, FIE 2019|
|Period||10/16/19 → 10/19/19|
Bibliographical noteFunding Information:
VII. ACKNOWLEDGMENTS We thank all of the experts and pilot trial participants. This work was supported in part by the U.S. Department of Defense under CAE-R grants H98230-15-1-0294, H98230-15-1-0273, H98230-17-1-0349, and H98230-17-1-0347; and by the National Science Foundation under SFS grant 1241576 and DGE grant 1820531.
© 2019 IEEE.
- Cybersecurity Assessment Tools (CATS) Project
- Cybersecurity Concept Inventory (CCI)
- Cybersecurity education
- assessment validation