Patchwork [5,of,5,V5] testing: Add a 'continous' profile

login
register
mail settings
Submitter David MacIver
Date Feb. 24, 2016, 6:42 p.m.
Message ID <5de59028a47d7fc26921.1456339371@laser-shark>
Download mbox | patch
Permalink /patch/13361/
State Superseded
Headers show

Comments

David MacIver - Feb. 24, 2016, 6:42 p.m.
# HG changeset patch
# User David R. MacIver <david@drmaciver.com>
# Date 1456320006 0
#      Wed Feb 24 13:20:06 2016 +0000
# Node ID 5de59028a47d7fc2692151778f30c10595c6dc7b
# Parent  6847cd559c93ce671de93fde7d10aa4da911bb13
testing: Add a 'continous' profile

This gives a good way of letting Hypothesis run until it
finds an error, save that error, and be restarted without it
picking up on the old bug. This lets you run long-running
Hypothesis processes and then perform a manual deduplication
task on the bugs found at the end.

It's not an entirely satisfying way of using this, but anything
much better would require extensive changes to Hypothesis itself.

Patch

diff -r 6847cd559c93 -r 5de59028a47d tests/test-verify-repo-operations.py
--- a/tests/test-verify-repo-operations.py	Wed Feb 24 13:11:30 2016 +0000
+++ b/tests/test-verify-repo-operations.py	Wed Feb 24 13:20:06 2016 +0000
@@ -22,6 +22,7 @@ 
 import hypothesis.strategies as st
 from hypothesis import settings, note
 from hypothesis.configuration import set_hypothesis_home_dir
+from hypothesis.database import ExampleDatabase
 
 testdir = os.path.abspath(os.environ["TESTDIR"])
 
@@ -512,6 +513,23 @@ 
         with acceptableerrors("no shelved changes to apply"):
             self.hg("unshelve")
 
+class writeonlydatabase(ExampleDatabase):
+    def __init__(self, underlying):
+        super(ExampleDatabase, self).__init__()
+        self.underlying = underlying
+
+    def fetch(self, key):
+        return ()
+
+    def save(self, key, value):
+        self.underlying.save(key, value)
+
+    def delete(self, key, value):
+        self.underlying.delete(key, value)
+
+    def close(self):
+        self.underlying.close()
+
 settings.register_profile(
     'default',  settings(
         timeout=300,
@@ -530,6 +548,16 @@ 
     )
 )
 
+settings.register_profile(
+    'continuous', settings(
+        timeout=-1,
+        stateful_step_count=1000,
+        max_examples=10 ** 8,
+        max_iterations=10 ** 8,
+        database=writeonlydatabase(settings.default.database)
+    )
+)
+
 settings.load_profile(os.getenv('HYPOTHESIS_PROFILE', 'default'))
 
 verifyingtest = verifyingstatemachine.TestCase