Patchwork run-tests: added '--graph' functionality to store timing data in json file

login
register
mail settings
Submitter Anurag Goel
Date June 16, 2014, 7:55 p.m.
Message ID <c3dacef6c7c12ec1f3a0.1402948502@ubuntu.ubuntu-domain>
Download mbox | patch
Permalink /patch/5011/
State Changes Requested
Headers show

Comments

Anurag Goel - June 16, 2014, 7:55 p.m.
# HG changeset patch
# User anuraggoel <anurag.dsps@gmail.com>
# Date 1402941179 -19800
#      Mon Jun 16 23:22:59 2014 +0530
# Node ID c3dacef6c7c12ec1f3a0879af4a28dd72e4ac53b
# Parent  477bc39f2d88a7d38127a3509d3d85cc9b62faee
run-tests: added '--graph' functionality to store timing data in json file

This patch added a new functionality '--graph'. While testing, if '--graph'
is enabled then timing data gets stored in newly created "report.json" file,
in the format of "testname : testtime".

This "report.json" file will further accessed by html/javascript file to
plot graph between testname Vs testime.
Gregory Szorc - June 16, 2014, 8:52 p.m.
On 6/16/14, 12:55 PM, Anurag Goel wrote:
> # HG changeset patch
> # User anuraggoel <anurag.dsps@gmail.com>
> # Date 1402941179 -19800
> #      Mon Jun 16 23:22:59 2014 +0530
> # Node ID c3dacef6c7c12ec1f3a0879af4a28dd72e4ac53b
> # Parent  477bc39f2d88a7d38127a3509d3d85cc9b62faee
> run-tests: added '--graph' functionality to store timing data in json file
>
> This patch added a new functionality '--graph'. While testing, if '--graph'
> is enabled then timing data gets stored in newly created "report.json" file,
> in the format of "testname : testtime".
>
> This "report.json" file will further accessed by html/javascript file to
> plot graph between testname Vs testime.
>
> diff -r 477bc39f2d88 -r c3dacef6c7c1 tests/run-tests.py
> --- a/tests/run-tests.py	Fri Jun 13 14:45:23 2014 +0530
> +++ b/tests/run-tests.py	Mon Jun 16 23:22:59 2014 +0530
> @@ -58,6 +58,7 @@
>   import killdaemons as killmod
>   import Queue as queue
>   import unittest
> +import json

Unfortunately, json was added in Python 2.6. simplejson is commonly used 
as a drop-in replacement for json. Do we want to create a test-only 
dependency? W'd have to pin an old version of simplejson, as modern 
versions dropped 2.4 compatibility.
Sean Farley - June 16, 2014, 9:02 p.m.
Gregory Szorc writes:

> On 6/16/14, 12:55 PM, Anurag Goel wrote:
>> # HG changeset patch
>> # User anuraggoel <anurag.dsps@gmail.com>
>> # Date 1402941179 -19800
>> #      Mon Jun 16 23:22:59 2014 +0530
>> # Node ID c3dacef6c7c12ec1f3a0879af4a28dd72e4ac53b
>> # Parent  477bc39f2d88a7d38127a3509d3d85cc9b62faee
>> run-tests: added '--graph' functionality to store timing data in json file
>>
>> This patch added a new functionality '--graph'. While testing, if '--graph'
>> is enabled then timing data gets stored in newly created "report.json" file,
>> in the format of "testname : testtime".
>>
>> This "report.json" file will further accessed by html/javascript file to
>> plot graph between testname Vs testime.
>>
>> diff -r 477bc39f2d88 -r c3dacef6c7c1 tests/run-tests.py
>> --- a/tests/run-tests.py	Fri Jun 13 14:45:23 2014 +0530
>> +++ b/tests/run-tests.py	Mon Jun 16 23:22:59 2014 +0530
>> @@ -58,6 +58,7 @@
>>   import killdaemons as killmod
>>   import Queue as queue
>>   import unittest
>> +import json
>
> Unfortunately, json was added in Python 2.6. simplejson is commonly used 
> as a drop-in replacement for json. Do we want to create a test-only 
> dependency? W'd have to pin an old version of simplejson, as modern 
> versions dropped 2.4 compatibility.

The testsuite already has a defaultdict (added in Python 2.5). I think
it will suffice to just skip this test for earlier versions of Python.
Augie Fackler - June 17, 2014, 5:25 p.m.
On Mon, Jun 16, 2014 at 02:02:35PM -0700, Sean Farley wrote:
>
> Gregory Szorc writes:
>
> > On 6/16/14, 12:55 PM, Anurag Goel wrote:
> >> # HG changeset patch
> >> # User anuraggoel <anurag.dsps@gmail.com>
> >> # Date 1402941179 -19800
> >> #      Mon Jun 16 23:22:59 2014 +0530
> >> # Node ID c3dacef6c7c12ec1f3a0879af4a28dd72e4ac53b
> >> # Parent  477bc39f2d88a7d38127a3509d3d85cc9b62faee
> >> run-tests: added '--graph' functionality to store timing data in json file
> >>
> >> This patch added a new functionality '--graph'. While testing, if '--graph'
> >> is enabled then timing data gets stored in newly created "report.json" file,
> >> in the format of "testname : testtime".
> >>
> >> This "report.json" file will further accessed by html/javascript file to
> >> plot graph between testname Vs testime.
> >>
> >> diff -r 477bc39f2d88 -r c3dacef6c7c1 tests/run-tests.py
> >> --- a/tests/run-tests.py	Fri Jun 13 14:45:23 2014 +0530
> >> +++ b/tests/run-tests.py	Mon Jun 16 23:22:59 2014 +0530
> >> @@ -58,6 +58,7 @@
> >>   import killdaemons as killmod
> >>   import Queue as queue
> >>   import unittest
> >> +import json
> >
> > Unfortunately, json was added in Python 2.6. simplejson is commonly used
> > as a drop-in replacement for json. Do we want to create a test-only
> > dependency? W'd have to pin an old version of simplejson, as modern
> > versions dropped 2.4 compatibility.
>
> The testsuite already has a defaultdict (added in Python 2.5). I think
> it will suffice to just skip this test for earlier versions of Python.

Sounds reasonable to me.

(I did not review the patch as a whole.)

> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel
Pierre-Yves David - June 20, 2014, 4:41 p.m.
On 06/16/2014 12:55 PM, Anurag Goel wrote:
> # HG changeset patch
> # User anuraggoel <anurag.dsps@gmail.com>
> # Date 1402941179 -19800
> #      Mon Jun 16 23:22:59 2014 +0530
> # Node ID c3dacef6c7c12ec1f3a0879af4a28dd72e4ac53b
> # Parent  477bc39f2d88a7d38127a3509d3d85cc9b62faee
> run-tests: added '--graph' functionality to store timing data in json file

The option name does not make much sense. This is about exporting json. 
Not graph. What about a --json option (that could be later expanded to 
contains success data and others?)


> This patch added a new functionality '--graph'. While testing, if '--graph'
> is enabled then timing data gets stored in newly created "report.json" file,
> in the format of "testname : testtime".
>
> This "report.json" file will further accessed by html/javascript file to
> plot graph between testname Vs testime.

We need an example output of this json. Luckily, we also needs a tests. 
And such test should include and example.

> diff -r 477bc39f2d88 -r c3dacef6c7c1 tests/run-tests.py
> --- a/tests/run-tests.py	Fri Jun 13 14:45:23 2014 +0530
> +++ b/tests/run-tests.py	Mon Jun 16 23:22:59 2014 +0530
> @@ -58,6 +58,7 @@
>   import killdaemons as killmod
>   import Queue as queue
>   import unittest
> +import json

See previous comment on the json module. I'm okay on it working only on 
2.4 (gracefullly disabling the option on 2,4) as a start.

>   processlock = threading.Lock()
>
> @@ -185,6 +186,8 @@
>                " (default: $%s or %d)" % defaults['timeout'])
>       parser.add_option("--time", action="store_true",
>           help="time how long each test takes")
> +    parser.add_option("--graph", action="store_true",
> +        help="store timing data in 'report.json' file")

The output file should probably be configurable at some point. But this 
could (must?) come in a later changeset.

>       parser.add_option("--tmpdir", type="string",
>           help="run tests in the given temporary directory"
>                " (implies --keep-tmpdir)")
> @@ -1335,6 +1338,14 @@
>                   os.environ['PYTHONHASHSEED'])
>           if self._runner.options.time:
>               self.printtimes(result.times)
> +        if self._runner.options.graph:
> +            self.getgraph(result.times)
> +
> +    def getgraph(self, times):


This methods wants a doc string! (and a name about json, not graph)

> +        fp = open("report.json", "w")
> +        fp.writelines(("testreport =", (json.dumps({item[0]:item[1] for item in
> +                            times}, indent=2))))

are you aware of

   x = dict(*<list of pair>)

?

> +        fp.close()

Patch

diff -r 477bc39f2d88 -r c3dacef6c7c1 tests/run-tests.py
--- a/tests/run-tests.py	Fri Jun 13 14:45:23 2014 +0530
+++ b/tests/run-tests.py	Mon Jun 16 23:22:59 2014 +0530
@@ -58,6 +58,7 @@ 
 import killdaemons as killmod
 import Queue as queue
 import unittest
+import json
 
 processlock = threading.Lock()
 
@@ -185,6 +186,8 @@ 
              " (default: $%s or %d)" % defaults['timeout'])
     parser.add_option("--time", action="store_true",
         help="time how long each test takes")
+    parser.add_option("--graph", action="store_true",
+        help="store timing data in 'report.json' file")
     parser.add_option("--tmpdir", type="string",
         help="run tests in the given temporary directory"
              " (implies --keep-tmpdir)")
@@ -1335,6 +1338,14 @@ 
                 os.environ['PYTHONHASHSEED'])
         if self._runner.options.time:
             self.printtimes(result.times)
+        if self._runner.options.graph:
+            self.getgraph(result.times)
+
+    def getgraph(self, times):
+        fp = open("report.json", "w")
+        fp.writelines(("testreport =", (json.dumps({item[0]:item[1] for item in
+                            times}, indent=2))))
+        fp.close()
 
     def printtimes(self, times):
         self.stream.writeln('# Producing time report')