Patchwork [3,of,3,coverage] run-tests: report code coverage from source directory

login
register
mail settings
Submitter Gregory Szorc
Date March 28, 2015, 7:48 a.m.
Message ID <bebbe08335400096292c.1427528926@vm-ubuntu-main.gateway.sonic.net>
Download mbox | patch
Permalink /patch/8321/
State Accepted
Headers show

Comments

Gregory Szorc - March 28, 2015, 7:48 a.m.
# HG changeset patch
# User Gregory Szorc <gregory.szorc@gmail.com>
# Date 1427527290 25200
#      Sat Mar 28 00:21:30 2015 -0700
# Node ID bebbe08335400096292c194c2a860cc4318ff648
# Parent  6b3bac1eba5e9470690da017224fdcfb80609d45
run-tests: report code coverage from source directory

As part of testing code coverage output, I noticed some files were
being reported twice: there was an entry for the file in the install
location and for the file in the source tree. I'm not sure why this
is. But it resulted in under-reporting of coverage data since some
lines weren't getting covered in both locations.

I also noticed that files in the source directory and outside the
"mercurial" and "hgext" packages were getting included in the
coverage report. Cosmetically, this seemed odd to me. It's not
difficult to filter paths from the report. But I figure this data
can be useful (we could start reporting run-tests.py coverage,
for example).

This patch switches the coverage API to report code coverage from
the source directory. It registers a path alias so that data from
the install location is merged into data from the source directory.
We now get merged results for files that were being reported in
multiple locations.

Since code coverage reporting now relies on the profiled install
now being in sync with the source tree, an additional check to
disallow code coverage when --with-hg is specified has been added.
This should have been present before, as --local was previously
disallowed for the same reasons.

Merging the paths raises our aggregate line coverage from ~60 to
81%.
Augie Fackler - March 30, 2015, 2:51 p.m.
On Sat, Mar 28, 2015 at 12:48:46AM -0700, Gregory Szorc wrote:
> # HG changeset patch
> # User Gregory Szorc <gregory.szorc@gmail.com>
> # Date 1427527290 25200
> #      Sat Mar 28 00:21:30 2015 -0700
> # Node ID bebbe08335400096292c194c2a860cc4318ff648
> # Parent  6b3bac1eba5e9470690da017224fdcfb80609d45
> run-tests: report code coverage from source directory

Very, very nice. Queued with delight, as now I don't have to go
hunting down all these problems. Thank you!

>
> As part of testing code coverage output, I noticed some files were
> being reported twice: there was an entry for the file in the install
> location and for the file in the source tree. I'm not sure why this
> is. But it resulted in under-reporting of coverage data since some
> lines weren't getting covered in both locations.
>
> I also noticed that files in the source directory and outside the
> "mercurial" and "hgext" packages were getting included in the
> coverage report. Cosmetically, this seemed odd to me. It's not
> difficult to filter paths from the report. But I figure this data
> can be useful (we could start reporting run-tests.py coverage,
> for example).
>
> This patch switches the coverage API to report code coverage from
> the source directory. It registers a path alias so that data from
> the install location is merged into data from the source directory.
> We now get merged results for files that were being reported in
> multiple locations.
>
> Since code coverage reporting now relies on the profiled install
> now being in sync with the source tree, an additional check to
> disallow code coverage when --with-hg is specified has been added.
> This should have been present before, as --local was previously
> disallowed for the same reasons.
>
> Merging the paths raises our aggregate line coverage from ~60 to
> 81%.
>
> diff --git a/tests/run-tests.py b/tests/run-tests.py
> --- a/tests/run-tests.py
> +++ b/tests/run-tests.py
> @@ -259,8 +259,12 @@ def parseargs(args, parser):
>          # this needs some path mangling somewhere, I guess
>          parser.error("sorry, coverage options do not work when --local "
>                       "is specified")
>
> +    if options.anycoverage and options.with_hg:
> +        parser.error("sorry, coverage options do not work when --with-hg "
> +                     "is specified")
> +
>      global verbose
>      if options.verbose:
>          verbose = ''
>
> @@ -1562,8 +1566,9 @@ class TestRunner(object):
>      ]
>
>      def __init__(self):
>          self.options = None
> +        self._hgroot = None
>          self._testdir = None
>          self._hgtmp = None
>          self._installdir = None
>          self._bindir = None
> @@ -1867,8 +1872,9 @@ class TestRunner(object):
>
>          # Run installer in hg root
>          script = os.path.realpath(sys.argv[0])
>          hgroot = os.path.dirname(os.path.dirname(script))
> +        self._hgroot = hgroot
>          os.chdir(hgroot)
>          nohome = '--home=""'
>          if os.name == 'nt':
>              # The --home="" trick works only on OS where os.sep == '/'
> @@ -1991,11 +1997,15 @@ class TestRunner(object):
>
>          vlog('# Producing coverage report')
>          # chdir is the easiest way to get short, relative paths in the
>          # output.
> -        os.chdir(self._pythondir)
> +        os.chdir(self._hgroot)
>          covdir = os.path.join(self._installdir, '..', 'coverage')
>          cov = coverage(data_file=os.path.join(covdir, 'cov'))
> +
> +        # Map install directory paths back to source directory.
> +        cov.config.paths['srcdir'] = ['.', self._pythondir]
> +
>          cov.combine()
>
>          omit = [os.path.join(x, '*') for x in [self._bindir, self._testdir]]
>          cov.report(ignore_errors=True, omit=omit)
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel

Patch

diff --git a/tests/run-tests.py b/tests/run-tests.py
--- a/tests/run-tests.py
+++ b/tests/run-tests.py
@@ -259,8 +259,12 @@  def parseargs(args, parser):
         # this needs some path mangling somewhere, I guess
         parser.error("sorry, coverage options do not work when --local "
                      "is specified")
 
+    if options.anycoverage and options.with_hg:
+        parser.error("sorry, coverage options do not work when --with-hg "
+                     "is specified")
+
     global verbose
     if options.verbose:
         verbose = ''
 
@@ -1562,8 +1566,9 @@  class TestRunner(object):
     ]
 
     def __init__(self):
         self.options = None
+        self._hgroot = None
         self._testdir = None
         self._hgtmp = None
         self._installdir = None
         self._bindir = None
@@ -1867,8 +1872,9 @@  class TestRunner(object):
 
         # Run installer in hg root
         script = os.path.realpath(sys.argv[0])
         hgroot = os.path.dirname(os.path.dirname(script))
+        self._hgroot = hgroot
         os.chdir(hgroot)
         nohome = '--home=""'
         if os.name == 'nt':
             # The --home="" trick works only on OS where os.sep == '/'
@@ -1991,11 +1997,15 @@  class TestRunner(object):
 
         vlog('# Producing coverage report')
         # chdir is the easiest way to get short, relative paths in the
         # output.
-        os.chdir(self._pythondir)
+        os.chdir(self._hgroot)
         covdir = os.path.join(self._installdir, '..', 'coverage')
         cov = coverage(data_file=os.path.join(covdir, 'cov'))
+
+        # Map install directory paths back to source directory.
+        cov.config.paths['srcdir'] = ['.', self._pythondir]
+
         cov.combine()
 
         omit = [os.path.join(x, '*') for x in [self._bindir, self._testdir]]
         cov.report(ignore_errors=True, omit=omit)