Socrates: a unified framework for neural network testing, verification and repair.
Neural networks can be (arguably) viewed a different paradigm of programming, where logical reasoning is replaced with big data and optimization. Unlike traditional programs, however, neural networks are subject to bugs, e.g., adversarial samples and discriminatory instances. In this line of work, we aim to develop systematic theories, methods and tools to improve the quality of AI-systems.
SOCRATES is a unified platform for neural network testing, verification and repair. Unlike most existing neural network verification approaches which are scattered (i.e., each approach tackles some restricted classes of neural networks against certain particular properties), incomparable (i.e., each approach has its own assumptions and input format) and thus hard to apply, reuse or extend, SOCRATES aims at providing a unified platform for neural network verification. Specifically, it supports a standardized format for a variety of neural network models, an asseration language for property specification as well as many novel algorithms for verifying or falsifying neural network models, as well as repairing them. SOCRATES is still in active development. Any suggestions and collaborations are welcomed.
Details can be found at: https://socrates4nn.github.io/