Patchwork [2,of,2] tests: ensure run-tests handles multiple lines of churn

login
register
mail settings
Submitter timeless@mozdev.org
Date March 21, 2016, 9:13 p.m.
Message ID <3ab81f2dc0dfedbc98f4.1458594830@waste.org>
Download mbox | patch
Permalink /patch/14020/
State Superseded
Commit 695c666f42ff9fea79057af3afdd942f9211c5d8
Headers show

Comments

timeless@mozdev.org - March 21, 2016, 9:13 p.m.
# HG changeset patch
# User timeless <timeless@mozdev.org>
# Date 1458593806 0
#      Mon Mar 21 20:56:46 2016 +0000
# Node ID 3ab81f2dc0dfedbc98f4704959b376f6b5431486
# Parent  d104b8e4e18c6f98267ec4bbc2bb4957e2571eeb
tests: ensure run-tests handles multiple lines of churn

1ad0ddf8cccc added the ability to remember lines (including their flags, like glob)
and tolerate them even if the output order varies.

This test ensures that the code will continue to work in the future.
Yuya Nishihara - March 24, 2016, 2:52 p.m.
On Mon, 21 Mar 2016 16:13:50 -0500, timeless wrote:
> # HG changeset patch
> # User timeless <timeless@mozdev.org>
> # Date 1458593806 0
> #      Mon Mar 21 20:56:46 2016 +0000
> # Node ID 3ab81f2dc0dfedbc98f4704959b376f6b5431486
> # Parent  d104b8e4e18c6f98267ec4bbc2bb4957e2571eeb
> tests: ensure run-tests handles multiple lines of churn

This somehow failed on Windows. Can you take a look?

> +test churn with globs
> +  $ cat > test-failure.t <<EOF
> +  >   $ echo "bar/baz"; echo "bar/bad"
> +  >   bar/bad (glob)
> +  >   bar/baz (glob)
> +  > EOF
> +  $ rt test-failure.t
> +  
> +  --- $TESTTMP/test-failure.t
> +  +++ $TESTTMP/test-failure.t.err
> +  @@ -1,3 +1,3 @@
> +     $ echo "bar/baz"; echo "bar/bad"
> +  +  bar/baz (glob)
> +     bar/bad (glob)
> +  -  bar/baz (glob)

-  +  bar/baz (glob)
-     bar/bad (glob)
+  -  bar/bad (glob)
   -  bar/baz (glob)
+  +  bar/baz
+  +  bar/bad
timeless - March 24, 2016, 9:42 p.m.
Yuya Nishihara wrote:
> This somehow failed on Windows. Can you take a look?

That's because of:
def linematch(el, l):
... Windows goes through this path (which is totally broken):
        if el == l or os.name == 'nt' and el[:-1] + b'\r\n' == l:
            return True
... Everyone else gets to reach this code (which is where the magic happens):
        if el.endswith(b" (glob)\n"):
            # ignore '(glob)' added to l by 'replacements'
            if l.endswith(b" (glob)\n"):
                l = l[:-8] + b"\n"
            return TTest.globmatch(el[:-8], l)
timeless - March 24, 2016, 9:51 p.m.
On Thu, Mar 24, 2016 at 5:42 PM, timeless <timeless@gmail.com> wrote:
> Yuya Nishihara wrote:
>> This somehow failed on Windows. Can you take a look?
>
> That's because of:
> def linematch(el, l):
ok, not this:
> ... Windows goes through this path (which is totally broken):
>         if el == l or os.name == 'nt' and el[:-1] + b'\r\n' == l:
>             return True
Everyone goes through here:
>         if el.endswith(b" (glob)\n"):
>             # ignore '(glob)' added to l by 'replacements'
>             if l.endswith(b" (glob)\n"):
>                 l = l[:-8] + b"\n"
The problem is in here:
>             return TTest.globmatch(el[:-8], l)

 def globmatch(el, l):
     # The only supported special characters are * and ? plus / which also
     # matches \ on windows. Escaping of these characters is supported.
     if el + b'\n' == l:
         if os.altsep:
             # matching on "/" is not needed for this line
             for pat in checkcodeglobpats:
                 if pat.match(el):
                     return True
On Windows, we trigger this:
            return b'-glob'
Everyone else hits this:
        return True

If I disable the os.altsep -> return b'-glob', then the test would pass.

Patch

diff --git a/tests/test-run-tests.t b/tests/test-run-tests.t
--- a/tests/test-run-tests.t
+++ b/tests/test-run-tests.t
@@ -67,6 +67,30 @@ 
 failing test
 ==================
 
+test churn with globs
+  $ cat > test-failure.t <<EOF
+  >   $ echo "bar/baz"; echo "bar/bad"
+  >   bar/bad (glob)
+  >   bar/baz (glob)
+  > EOF
+  $ rt test-failure.t
+  
+  --- $TESTTMP/test-failure.t
+  +++ $TESTTMP/test-failure.t.err
+  @@ -1,3 +1,3 @@
+     $ echo "bar/baz"; echo "bar/bad"
+  +  bar/baz (glob)
+     bar/bad (glob)
+  -  bar/baz (glob)
+  
+  ERROR: test-failure.t output changed
+  !
+  Failed test-failure.t: output changed
+  # Ran 1 tests, 0 skipped, 0 warned, 1 failed.
+  python hash seed: * (glob)
+  [1]
+
+basic failing test
   $ cat > test-failure.t << EOF
   >   $ echo babar
   >   rataxes