Evaluation

The two main goals of the TCHES 2023 artifact review process were to improve functionality and reusability of artifacts to enable reproducibility and extension by the scientific community.

Reproducibility, in the context of computational experiments, means that the scientific results claimed can be obtained by a different team using the original authors’ artifacts. The artifact review process for CHES 2023 did not include attempting to reproduce the experiment and verify the scientific claims in the accepted paper. Rather, the artifact review process for CHES 2023 aimed at ensuring sufficient functionality of the artifact to enable a research team to attempt to reproduce the results.

Reusability means that the artifacts are not just functional, but of sufficient quality that they could be extended and reused by others. Reusable artifacts have clear user and developer documentation, and are well-structured in ways that make them easy to modify or extend.

For more information, please see the TCHES 2023 Call for Artifacts.