Patchwork [5,of,5,V6] testing: Add a 'continous' profile

login
register
mail settings
Submitter David MacIver
Date Feb. 25, 2016, 1:58 p.m.
Message ID <3422be049ea26a9b41b5.1456408728@laser-shark>
Download mbox | patch
Permalink /patch/13385/
State Superseded
Headers show

Comments

David MacIver - Feb. 25, 2016, 1:58 p.m.
# HG changeset patch
# User David R. MacIver <david@drmaciver.com>
# Date 1456320006 0
#      Wed Feb 24 13:20:06 2016 +0000
# Node ID 3422be049ea26a9b41b5863592a9d011ba0cefce
# Parent  058757b1826ff2b918edffc6d308b27272dd08d0
testing: Add a 'continous' profile

This gives a good way of letting Hypothesis run until it
finds an error, save that error, and be restarted without it
picking up on the old bug. This lets you run long-running
Hypothesis processes and then perform a manual deduplication
task on the bugs found at the end.

It's not an entirely satisfying way of using this, but anything
much better would require extensive changes to Hypothesis itself.

Patch

diff -r 058757b1826f -r 3422be049ea2 tests/test-verify-repo-operations.py
--- a/tests/test-verify-repo-operations.py	Wed Feb 24 13:11:30 2016 +0000
+++ b/tests/test-verify-repo-operations.py	Wed Feb 24 13:20:06 2016 +0000
@@ -31,6 +31,7 @@ 
     rule, RuleBasedStateMachine, Bundle, precondition)
 from hypothesis import settings, note, strategies as st
 from hypothesis.configuration import set_hypothesis_home_dir
+from hypothesis.database import ExampleDatabase
 
 testdir = os.path.abspath(os.environ["TESTDIR"])
 
@@ -522,6 +523,23 @@ 
         with acceptableerrors("no shelved changes to apply"):
             self.hg("unshelve")
 
+class writeonlydatabase(ExampleDatabase):
+    def __init__(self, underlying):
+        super(ExampleDatabase, self).__init__()
+        self.underlying = underlying
+
+    def fetch(self, key):
+        return ()
+
+    def save(self, key, value):
+        self.underlying.save(key, value)
+
+    def delete(self, key, value):
+        self.underlying.delete(key, value)
+
+    def close(self):
+        self.underlying.close()
+
 settings.register_profile(
     'default',  settings(
         timeout=300,
@@ -540,6 +558,16 @@ 
     )
 )
 
+settings.register_profile(
+    'continuous', settings(
+        timeout=-1,
+        stateful_step_count=1000,
+        max_examples=10 ** 8,
+        max_iterations=10 ** 8,
+        database=writeonlydatabase(settings.default.database)
+    )
+)
+
 settings.load_profile(os.getenv('HYPOTHESIS_PROFILE', 'default'))
 
 verifyingtest = verifyingstatemachine.TestCase