Patchwork [3,of,3] tests: split test-run-tests-rev.t for speed (RFC)

login
register
mail settings
Submitter timeless
Date May 26, 2016, 2:25 a.m.
Message ID <5a3ca5d9c46d83d93a1c.1464229551@gcc2-power8.osuosl.org>
Download mbox | patch
Permalink /patch/15210/
State Accepted
Headers show

Comments

timeless - May 26, 2016, 2:25 a.m.
# HG changeset patch
# User timeless <timeless@mozdev.org>
# Date 1462985159 0
#      Wed May 11 16:45:59 2016 +0000
# Node ID 5a3ca5d9c46d83d93a1c7efdf2932ad47802f190
# Parent  a66db77a12648b2f70486174be6ead8a0f597532
# EXP-Topic runtests
# Available At https://bitbucket.org/timeless/mercurial-crew
#              hg pull https://bitbucket.org/timeless/mercurial-crew -r 5a3ca5d9c46d
tests: split test-run-tests-rev.t for speed (RFC)

Each test in test-run-tests-rev.t is really slow, since it performs
a true clone of the repository.

It is possible to just create individual test files for each of these,
but it feels silly to have:
test-run-tests-rev-12.t
test-run-tests-rev-17.t

I felt bad enough about splitting test-run-tests-rev.t out of
test-run-tests.t -- But I did that because test-run-tests.t is
already moderately slow, and test-run-tests-rev.t is ~4 times
slower:

(py)[timeless@gcc2-power8 tests]$ ./run-tests.py -l test-run-tests*.t --allow-slow -j10 --time
..
# Ran 2 tests, 0 skipped, 0 warned, 0 failed.
# Producing time report
start   end     cuser   csys    real      Test
  0.000  57.540  42.610   3.250  57.540   test-run-tests.t
  0.000 197.810 412.280   9.260 197.810   test-run-tests-rev.t

-- This is with the -j2 split.

fwiw, it can get as "fast" as:
  0.000 141.940 290.500   4.160 141.940   test-run-tests-rev.t

There is a fair bit of variance in the times, although usually
the numbers seem to be closer to real 200...

Patch

diff -r a66db77a1264 -r 5a3ca5d9c46d tests/test-run-tests-rev.t
--- a/tests/test-run-tests-rev.t	Thu Apr 07 11:13:57 2016 +0000
+++ b/tests/test-run-tests-rev.t	Wed May 11 16:45:59 2016 +0000
@@ -2,39 +2,49 @@ 
 
 This file tests the behavior of run-tests.py --rev.
 
+  $ . "$TESTDIR/helper-runtests.sh"
   $ . "$TESTDIR/helpers-testrepo.sh"
 
 Avoid interference from actual test env:
 
-  $ unset HGTEST_JOBS
-  $ unset HGTEST_TIMEOUT
-  $ unset HGTEST_PORT
-  $ unset HGTEST_SHELL
+  $ echo "#require slow" > avoid-interference
+  $ echo "  \$ . '$TESTDIR/helper-runtests.sh'" >> avoid-interference
+  $ echo "  \$ . '$TESTDIR/helpers-testrepo.sh'" >> avoid-interference
 
 support for running a different version of mercurial
 note that it is typical for ~1/5 tests not to pass when using older
 versions of mercurial...
 
-  $ cat >> test-version.t <<EOF
-  >   $ hg version
-  >   Mercurial Distributed SCM (version 1.2)
-  >   
-  >   Copyright (C) 2005-2008 Matt Mackall <mpm@selenic.com> and others
-  >   This is free software; see the source for copying conditions. There is NO
-  >   warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
-  > EOF
-  $ run-tests.py --pure --rev 1.2 test-version.t
-  .
-  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
+  $ cp avoid-interference test-version-rev.t
+  $ cat >> test-version-rev.t <<EOT
+  >   $ cat >> test-version.t <<EOF
+  >   >   $ hg version
+  >   >   Mercurial Distributed SCM (version 1.2)
+  >   >   
+  >   >   Copyright (C) 2005-2008 Matt Mackall <mpm@selenic.com> and others
+  >   >   This is free software; see the source for copying conditions. There is NO
+  >   >   warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+  >   > EOF
+  >   $ run-tests.py --pure --rev 1.2 test-version.t
+  >   .
+  >   # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
+  > EOT
 
 To run tests at a version from the revision you select from --rev
 you need a test which introspects the source directory
 
-  $ cat > test-old-basic.t <<EOF
-  >   $ run-tests.py --pure --local ../../source/tests/test-basic.t
+  $ cp avoid-interference test-old-basic-rev.t
+  $ cat >> test-old-basic-rev.t <<EOT
+  >   $ cat > test-old-basic.t <<EOF
+  >   >   $ run-tests.py --pure --local ../../source/tests/test-basic.t
+  >   >   .
+  >   >   # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
+  >   > EOF
+  >   $ run-tests.py --pure --rev 1.7 test-old-basic.t
   >   .
   >   # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
-  > EOF
-  $ run-tests.py --pure --rev 1.7 test-old-basic.t
-  .
-  # Ran 1 tests, 0 skipped, 0 warned, 0 failed.
+  > EOT
+
+  $ run-tests.py -l -j2 --allow-slow test-version-rev.t test-old-basic-rev.t
+  ..
+  # Ran 2 tests, 0 skipped, 0 warned, 0 failed.