Submitter | Mads Kiilerich |
---|---|
Date | Jan. 20, 2013, 1:18 p.m. |
Message ID | <f62e594f1b67ea617c8d.1358687889@localhost6.localdomain6> |
Download | mbox | patch |
Permalink | /patch/692/ |
State | Accepted |
Commit | f0124a65e8f8d2498d76e3547cbb2fed866709cb |
Headers | show |
Comments
On Sun, 2013-01-20 at 14:18 +0100, Mads Kiilerich wrote: > # HG changeset patch > # User Mads Kiilerich <mads@kiilerich.com> > # Date 1358687872 -3600 > # Branch stable > # Node ID f62e594f1b67ea617c8d727834ee8c9895efe4ef > # Parent a0e2208420d688baa24bb52d9064e139a9dcc885 > run-tests.py: inherit PYTHONHASHSEED from environment if set > > This makes it possible to fix the seed by using for instance > PYTHONHASHSEED=7 ./run-tests.py ... > > This can be very convenient when trying to debug problems that are influenced > by hash values. Try different seed values until you find one that triggers the > bad behaviour and then keep that while debugging. > > It could also to some extent make sense for buildbots to set (different) fixed > values to make sure the test execution is stable. I'm not sure precisely what you mean here. I think there's value in running the buildbot with random seeds. However, it's also useful to be able to rerun a test with the same seed a buildbot uses. As there's no way to access the seed used for PYTHONHASHSEED=random, perhaps the buildbot or run-test should generate a seed and log it. > diff --git a/tests/run-tests.py b/tests/run-tests.py > --- a/tests/run-tests.py > +++ b/tests/run-tests.py > @@ -1252,7 +1252,7 @@ def main(): > os.environ['no_proxy'] = '' > os.environ['NO_PROXY'] = '' > os.environ['TERM'] = 'xterm' > - os.environ['PYTHONHASHSEED'] = 'random' > + os.environ['PYTHONHASHSEED'] = os.environ.get('PYTHONHASHSEED', 'random') > > # unset env related to hooks > for k in os.environ.keys(): > _______________________________________________ > Mercurial-devel mailing list > Mercurial-devel@selenic.com > http://selenic.com/mailman/listinfo/mercurial-devel
Matt Mackall wrote, On 01/20/2013 10:24 PM: > On Sun, 2013-01-20 at 14:18 +0100, Mads Kiilerich wrote: >> # HG changeset patch >> # User Mads Kiilerich <mads@kiilerich.com> >> # Date 1358687872 -3600 >> # Branch stable >> # Node ID f62e594f1b67ea617c8d727834ee8c9895efe4ef >> # Parent a0e2208420d688baa24bb52d9064e139a9dcc885 >> run-tests.py: inherit PYTHONHASHSEED from environment if set >> >> This makes it possible to fix the seed by using for instance >> PYTHONHASHSEED=7 ./run-tests.py ... >> >> This can be very convenient when trying to debug problems that are influenced >> by hash values. Try different seed values until you find one that triggers the >> bad behaviour and then keep that while debugging. >> >> It could also to some extent make sense for buildbots to set (different) fixed >> values to make sure the test execution is stable. > I'm not sure precisely what you mean here. I think there's value in > running the buildbot with random seeds. However, it's also useful to be > able to rerun a test with the same seed a buildbot uses. As there's no > way to access the seed used for PYTHONHASHSEED=random, perhaps the > buildbot or run-test should generate a seed and log it. Agreed. My thought was that every buildbot using a different fixed "random" number also would give good coverage and perhaps be less confusing ... and make sure that all test runs are fully reproducible. I guess either way could work ... but we should perhaps wait and see what kind of problems we end up having on the buildbot. Having a way to control it locally is however very convenient. /Mads (Mandatory references: http://dilbert.com/strips/comic/2001-10-25/ and http://xkcd.com/221/ ) >> diff --git a/tests/run-tests.py b/tests/run-tests.py >> --- a/tests/run-tests.py >> +++ b/tests/run-tests.py >> @@ -1252,7 +1252,7 @@ def main(): >> os.environ['no_proxy'] = '' >> os.environ['NO_PROXY'] = '' >> os.environ['TERM'] = 'xterm' >> - os.environ['PYTHONHASHSEED'] = 'random' >> + os.environ['PYTHONHASHSEED'] = os.environ.get('PYTHONHASHSEED', 'random') >> >> # unset env related to hooks >> for k in os.environ.keys(): >> _______________________________________________ >> Mercurial-devel mailing list >> Mercurial-devel@selenic.com >> http://selenic.com/mailman/listinfo/mercurial-devel >
On Sun, 2013-01-20 at 23:00 +0100, Mads Kiilerich wrote: > Matt Mackall wrote, On 01/20/2013 10:24 PM: > > On Sun, 2013-01-20 at 14:18 +0100, Mads Kiilerich wrote: > >> # HG changeset patch > >> # User Mads Kiilerich <mads@kiilerich.com> > >> # Date 1358687872 -3600 > >> # Branch stable > >> # Node ID f62e594f1b67ea617c8d727834ee8c9895efe4ef > >> # Parent a0e2208420d688baa24bb52d9064e139a9dcc885 > >> run-tests.py: inherit PYTHONHASHSEED from environment if set > >> > >> This makes it possible to fix the seed by using for instance > >> PYTHONHASHSEED=7 ./run-tests.py ... > >> > >> This can be very convenient when trying to debug problems that are influenced > >> by hash values. Try different seed values until you find one that triggers the > >> bad behaviour and then keep that while debugging. > >> > >> It could also to some extent make sense for buildbots to set (different) fixed > >> values to make sure the test execution is stable. > > I'm not sure precisely what you mean here. I think there's value in > > running the buildbot with random seeds. However, it's also useful to be > > able to rerun a test with the same seed a buildbot uses. As there's no > > way to access the seed used for PYTHONHASHSEED=random, perhaps the > > buildbot or run-test should generate a seed and log it. > > Agreed. > > My thought was that every buildbot using a different fixed "random" > number also would give good coverage and perhaps be less confusing ... > and make sure that all test runs are fully reproducible. Then we're prone to associate failures that are hash-related with platforms instead.
Patch
diff --git a/tests/run-tests.py b/tests/run-tests.py --- a/tests/run-tests.py +++ b/tests/run-tests.py @@ -1252,7 +1252,7 @@ def main(): os.environ['no_proxy'] = '' os.environ['NO_PROXY'] = '' os.environ['TERM'] = 'xterm' - os.environ['PYTHONHASHSEED'] = 'random' + os.environ['PYTHONHASHSEED'] = os.environ.get('PYTHONHASHSEED', 'random') # unset env related to hooks for k in os.environ.keys():