2019/01/26 by Alan T. Sherman, Sherman, Alan T., Linda Oliva +27 · 1 citation
Engineering · #Biomedical and Engineering Education
paper · pdf · doi:10.48550/arxiv.1901.09286
For two days in February 2018, 17 cybersecurity educators and professionals\nfrom government and industry met in a "hackathon" to refine existing draft\nmultiple-choice test items, and to create new ones, for a Cybersecurity Concept\nInventory (CCI) and Cybersecurity Curriculum Assessment (CCA) being developed\nas part of the Cybersecurity Assessment Tools (CATS) Project. We report on the\nresults of the CATS Hackathon, discussing the methods we used to develop test\nitems, highlighting the evolution of a sample test item through this process,\nand offering suggestions to others who may wish to organize similar hackathons.\n Each test item embodies a scenario, question stem, and five answer choices.\nDuring the Hackathon, participants organized into teams to (1) Generate new\nscenarios and question stems, (2) Extend CCI items into CCA items, and generate\nnew answer choices for new scenarios and stems, and (3) Review and refine draft\nCCA test items.\n The CATS Project provides rigorous evidence-based instruments for assessing\nand evaluating educational practices; these instruments can help identify\npedagogies and content that are effective in teaching cybersecurity. The CCI\nmeasures how well students understand basic concepts in\ncybersecurity---especially adversarial thinking---after a first course in the\nfield. The CCA measures how well students understand core concepts after\ncompleting a full cybersecurity curriculum.\n