Patchwork run-tests: sort the skip, failure and error lists in the final output

login
register
mail settings
Submitter Matt Harbison
Date Jan. 31, 2019, 2:05 a.m.
Message ID <5380290f1f11face5e58.1548900345@Envy>
Download mbox | patch
Permalink /patch/38249/
State Accepted
Headers show

Comments

Matt Harbison - Jan. 31, 2019, 2:05 a.m.
# HG changeset patch
# User Matt Harbison <matt_harbison@yahoo.com>
# Date 1548894031 18000
#      Wed Jan 30 19:20:31 2019 -0500
# Node ID 5380290f1f11face5e58428cdb389eb7245969ba
# Parent  4a2c8ec90a28542cbccc155fc6ddf60fe0a8d1dc
run-tests: sort the skip, failure and error lists in the final output

This will help keep the lists consistent, for comparison across runs.
Yuya Nishihara - Jan. 31, 2019, 12:23 p.m.
On Wed, 30 Jan 2019 21:05:45 -0500, Matt Harbison wrote:
> # HG changeset patch
> # User Matt Harbison <matt_harbison@yahoo.com>
> # Date 1548894031 18000
> #      Wed Jan 30 19:20:31 2019 -0500
> # Node ID 5380290f1f11face5e58428cdb389eb7245969ba
> # Parent  4a2c8ec90a28542cbccc155fc6ddf60fe0a8d1dc
> run-tests: sort the skip, failure and error lists in the final output

Queued, thanks.

Patch

diff --git a/tests/run-tests.py b/tests/run-tests.py
--- a/tests/run-tests.py
+++ b/tests/run-tests.py
@@ -2258,14 +2258,17 @@  class TextTestRunner(unittest.TextTestRu
             self.stream.writeln('')
 
             if not self._runner.options.noskips:
-                for test, msg in self._result.skipped:
+                for test, msg in sorted(self._result.skipped,
+                                        key=lambda s: s[0].name):
                     formatted = 'Skipped %s: %s\n' % (test.name, msg)
                     msg = highlightmsg(formatted, self._result.color)
                     self.stream.write(msg)
-            for test, msg in self._result.failures:
+            for test, msg in sorted(self._result.failures,
+                                    key=lambda f: f[0].name):
                 formatted = 'Failed %s: %s\n' % (test.name, msg)
                 self.stream.write(highlightmsg(formatted, self._result.color))
-            for test, msg in self._result.errors:
+            for test, msg in sorted(self._result.errors,
+                                    key=lambda e: e[0].name):
                 self.stream.writeln('Errored %s: %s' % (test.name, msg))
 
             if self._runner.options.xunit:
diff --git a/tests/test-run-tests.t b/tests/test-run-tests.t
--- a/tests/test-run-tests.t
+++ b/tests/test-run-tests.t
@@ -324,8 +324,8 @@  basic failing test
   
   ERROR: test-failure-unicode.t output changed
   !
+  Failed test-failure-unicode.t: output changed
   Failed test-failure.t: output changed
-  Failed test-failure-unicode.t: output changed
   # Ran 3 tests, 0 skipped, 2 failed.
   python hash seed: * (glob)
   [1]
@@ -356,8 +356,8 @@  test --outputdir
   
   ERROR: test-failure-unicode.t output changed
   !
+  Failed test-failure-unicode.t: output changed
   Failed test-failure.t: output changed
-  Failed test-failure-unicode.t: output changed
   # Ran 3 tests, 0 skipped, 2 failed.
   python hash seed: * (glob)
   [1]
@@ -393,8 +393,8 @@  test --xunit support
   
   ERROR: test-failure-unicode.t output changed
   !
+  Failed test-failure-unicode.t: output changed
   Failed test-failure.t: output changed
-  Failed test-failure-unicode.t: output changed
   # Ran 3 tests, 0 skipped, 2 failed.
   python hash seed: * (glob)
   [1]