USENIX Workshop on Cyber Security Experimentation and Test (CSET)

What is CSET all about? For 12 years, the USENIX Workshop on Cyber Security Experimentation and Test (CSET) has been an important and lively space for presenting research and discussing “meta” topics related to reliability, validity, reproducibility, and scalability in cybersecurity and cybersecurity research, including cybersecurity evaluation and measurement, experiment design, benchmarks, datasets, tools, simulations, testbeds, and education.

Submissions that model a scientific approach to cybersecurity are especially encouraged, as well as those that advance the “infrastructure” of cybersecurity science. Significant challenges abound. For example, experiments should operate in realistic environments, yet identifying salient features and modeling them in testbeds is hard.

Repeatability is a worthy goal, but not always feasible. Few security-relevant datasets are publicly available for research use and little is understood about what “good datasets” look like.

Cybersecurity experiments and performance evaluations carry significant risks if not properly contained and controlled, yet often require some degree of interaction with the larger world in order to be useful; hence, ethical issues often arise. Finally, evidence-driven education practices and practices that leverage datasets are often absent in traditional computer science curricula.

Tackling these challenges helps promote evidence-based decision-making involving cybersecurity products and policies by industry, government, and individual users.