Patchwork [5,of,5,V9] testing: Add a 'continuous' profile

login
register
mail settings
Submitter David MacIver
Date Feb. 26, 2016, 11:38 a.m.
Message ID <18680fed30581e7286a7.1456486720@laser-shark>
Download mbox | patch
Permalink /patch/13409/
State Accepted
Headers show

Comments

David MacIver - Feb. 26, 2016, 11:38 a.m.
# HG changeset patch
# User David R. MacIver <david@drmaciver.com>
# Date 1456320006 0
#      Wed Feb 24 13:20:06 2016 +0000
# Node ID 18680fed30581e7286a72b93af97de980a178d96
# Parent  a4034609a1cc665deac01aa7dc56a8da8a6b4397
testing: Add a 'continuous' profile

This gives a good way of letting Hypothesis run until it
finds an error, save that error, and be restarted without it
picking up on the old bug. This lets you run long-running
Hypothesis processes and then perform a manual deduplication
task on the bugs found at the end.

It's not an entirely satisfying way of using this, but anything
much better would require extensive changes to Hypothesis itself.
Martin von Zweigbergk - Feb. 26, 2016, 5:59 p.m.
Pushed to the clowncopter, thanks!

Patch

diff -r a4034609a1cc -r 18680fed3058 tests/test-verify-repo-operations.py
--- a/tests/test-verify-repo-operations.py	Wed Feb 24 13:11:30 2016 +0000
+++ b/tests/test-verify-repo-operations.py	Wed Feb 24 13:20:06 2016 +0000
@@ -41,6 +41,7 @@ 
     rule, RuleBasedStateMachine, Bundle, precondition)
 from hypothesis import settings, note, strategies as st
 from hypothesis.configuration import set_hypothesis_home_dir
+from hypothesis.database import ExampleDatabase
 
 testdir = os.path.abspath(os.environ["TESTDIR"])
 
@@ -527,6 +528,23 @@ 
         with acceptableerrors("no shelved changes to apply"):
             self.hg("unshelve")
 
+class writeonlydatabase(ExampleDatabase):
+    def __init__(self, underlying):
+        super(ExampleDatabase, self).__init__()
+        self.underlying = underlying
+
+    def fetch(self, key):
+        return ()
+
+    def save(self, key, value):
+        self.underlying.save(key, value)
+
+    def delete(self, key, value):
+        self.underlying.delete(key, value)
+
+    def close(self):
+        self.underlying.close()
+
 settings.register_profile(
     'default',  settings(
         timeout=300,
@@ -545,6 +563,16 @@ 
     )
 )
 
+settings.register_profile(
+    'continuous', settings(
+        timeout=-1,
+        stateful_step_count=1000,
+        max_examples=10 ** 8,
+        max_iterations=10 ** 8,
+        database=writeonlydatabase(settings.default.database)
+    )
+)
+
 settings.load_profile(os.getenv('HYPOTHESIS_PROFILE', 'default'))
 
 verifyingtest = verifyingstatemachine.TestCase