Badges
Submitted artifacts can be evaluated against the following badges:
USENIX Badges | Description |
---|---|
Artifacts Available: As per the new “Open-Science policy” of USENIX Security, all accepted papers are mandated to qualify for this badge. To earn this badge, the AEC must judge that the artifacts described in the “Open-Science section” of the paper have been made available for retrieval, permanently and publicly. The archived copy of the artifacts must be accessible via a long-term stable reference or DOI. For this purpose, we recommend Zenodo, but other valid hosting options include institutional and third-party digital repositories (e.g., FigShare, Dryad, or Software Heritage). Unlike previous iterations, software development repositories such as GitHub, GitLab, or personal web pages are not acceptable for this badge. Other than making the artifacts available, this badge does not mandate any further requirements on functionality, correctness, or documentation. | |
Artifacts Functional: To earn this badge, the AEC must judge that the artifacts conform to the expectations set by the paper in terms of functionality, usability, and relevance. In short, do the artifacts work, and are they useful for producing outcomes associated with the paper? The AEC will consider three aspects of the artifacts in particular: Documentation: are the artifacts sufficiently documented to enable them to be exercised by readers of the paper? Completeness: do the submitted artifacts include all of the key components described in the paper? Exercisability: do the submitted artifacts include the scripts and data needed to run the experiments described in the paper, and can the software be successfully executed? |
|
Results Reproduced: To earn this badge, the AEC must judge that they can use the submitted artifacts to obtain the main results presented in the paper. In short, could the AEC independently repeat the experiments and obtain results that support the main claims made by the paper? The goal of this effort is not to reproduce the results exactly but instead to generate results independently within an allowed tolerance such that the main claims of the paper are validated. |