TycheTyche, named after the Greek goddess of chance, aims to provide a comprehensive integrated development environment (IDE) for property-based testing libraries like Hypothesis, Bolero and more. It is implemented as an extension for Visual Studio Code. The Tyche extension is currently in early alpha; we presented a prototype implementation, as a demo at UIST 2023. MotivationCurrent property-based testing frameworks give insufficient feedback about the specific values that were used to test a given program and about the distributional trends in those values. In the worst case, this lack of visibility process may give users false confidence, encouraging them to believe their testing was thorough when, in fact, it had critical gaps. Tyche provides an interactive interface for understanding testing effectiveness, surfacing both "pre-testing" information about test inputs and their distributions and "post-testing" information like code coverage. Using TycheIn HypothesisTyche is supported natively in Hypothesis via its Observability
API. Simply run a Hypothesis test
with the You can add features to your Tyche visualization using Hypothesis's
In QuickCheckWe have an experimental library that adds limited support for Tyche to QuickCheck. As of now it needs to be installed via GitHub and it is not very fully featured, but we hope to make this support more robust over time. ElsewhereYou can implement support for Tyche in any PBT framework by having each property output data in a
specific JSON lines format. The Hypothesis
documentation has more information
about the details of the format. Once you have valid JSON lines files, simply run Please contact Harry Goldstein if you need help. DevelopingProject StructureThis project is two nested NPM projects. The outer project is the source for the whole extension.
The inner project, in the Running in Debug ModeWhen working on the webview itself, run:
This will allow you to connect to the webview itself via a browser and iterate quickly. When working on the extension as a whole, run:
Then press |