Badges

Submitted artifacts could select to be evaluated against the following badges (inspired by ACM and USENIX):

SysTEX Badges Description
available badge Artifacts Available: When the AEC could verify that the artifacts were made available permanently and publicly via a stable URL. Any public hosting website is allowed, but not a personal webpage. When using a git repository (e.g., GitHub), a commit hash or tag is required for a stable URL. To earn this badge, no further requirements on documentation, completeness, or functionality are needed.
functional badge Artifacts Functional: When the AEC deems the artifact to be (i) complete (i.e., all key components described in the paper are included); (ii) reasonably well-documented; and (iii) exercisable (e.g., building of the software could be verified).
reproduced badge Artifacts Reusable: When the quality of the artifact significantly exceeds minimal functionality, e.g., the code can be ran and at least some results could be reproduced.

Note: As this is a new initiative, and to keep the reviewing load manageable, verifying full reproducibility as per ACM’s strict definition was put explicitly out of scope this year. The “Artifacts Reusable” badge is intended to recognize those artifacts that go beyond minimal expectations.