Patchwork [against-clowncopter] run-tests: add a --profile option

login
register
mail settings
Submitter Augie Fackler
Date May 14, 2015, 12:42 a.m.
Message ID <21735d2bcefb10605708.1431564170@augie-macbookair2.roam.corp.google.com>
Download mbox | patch
Permalink /patch/9065/
State Accepted
Headers show

Comments

Augie Fackler - May 14, 2015, 12:42 a.m.
# HG changeset patch
# User Augie Fackler <augie@google.com>
# Date 1431541326 25200
#      Wed May 13 11:22:06 2015 -0700
# Node ID 21735d2bcefb10605708a1dc640a6cfa590c1361
# Parent  130c305f79910395cbfb73a442cc512605f78bc5
run-tests: add a --profile option

While in the throes of a recent run-tests adventure, I found it useful
to have profiler output for the testrunner itself. Adding it was
simple enough and seems worth keeping around.
Pierre-Yves David - May 14, 2015, 1 a.m.
On 05/13/2015 05:42 PM, Augie Fackler wrote:
> # HG changeset patch
> # User Augie Fackler <augie@google.com>
> # Date 1431541326 25200
> #      Wed May 13 11:22:06 2015 -0700
> # Node ID 21735d2bcefb10605708a1dc640a6cfa590c1361
> # Parent  130c305f79910395cbfb73a442cc512605f78bc5
> run-tests: add a --profile option
>
> While in the throes of a recent run-tests adventure, I found it useful
> to have profiler output for the testrunner itself. Adding it was
> simple enough and seems worth keeping around.

Maybe? the amount of option in run-tests.py is already quite scary and 
one could hope that --profile would give --profile output for the tests 
themself.
Augie Fackler - May 14, 2015, 5:40 p.m.
On Wed, May 13, 2015 at 9:00 PM, Pierre-Yves David
<pierre-yves.david@ens-lyon.org> wrote:
>
>
> On 05/13/2015 05:42 PM, Augie Fackler wrote:
>>
>> # HG changeset patch
>> # User Augie Fackler <augie@google.com>
>> # Date 1431541326 25200
>> #      Wed May 13 11:22:06 2015 -0700
>> # Node ID 21735d2bcefb10605708a1dc640a6cfa590c1361
>> # Parent  130c305f79910395cbfb73a442cc512605f78bc5
>> run-tests: add a --profile option
>>
>> While in the throes of a recent run-tests adventure, I found it useful
>> to have profiler output for the testrunner itself. Adding it was
>> simple enough and seems worth keeping around.
>
>
> Maybe? the amount of option in run-tests.py is already quite scary and one
> could hope that --profile would give --profile output for the tests
> themself.

Maybe we should have a --profile-self and --profile-test flag then?

I'm not clear what profiling a test would look like. We'd have to
implement that, which seems unlikely to happen?

>
> --
> Pierre-Yves David
Pierre-Yves David - May 14, 2015, 6:13 p.m.
On 05/14/2015 10:40 AM, Augie Fackler wrote:
> On Wed, May 13, 2015 at 9:00 PM, Pierre-Yves David
> <pierre-yves.david@ens-lyon.org> wrote:
>>
>>
>> On 05/13/2015 05:42 PM, Augie Fackler wrote:
>>>
>>> # HG changeset patch
>>> # User Augie Fackler <augie@google.com>
>>> # Date 1431541326 25200
>>> #      Wed May 13 11:22:06 2015 -0700
>>> # Node ID 21735d2bcefb10605708a1dc640a6cfa590c1361
>>> # Parent  130c305f79910395cbfb73a442cc512605f78bc5
>>> run-tests: add a --profile option
>>>
>>> While in the throes of a recent run-tests adventure, I found it useful
>>> to have profiler output for the testrunner itself. Adding it was
>>> simple enough and seems worth keeping around.
>>
>>
>> Maybe? the amount of option in run-tests.py is already quite scary and one
>> could hope that --profile would give --profile output for the tests
>> themself.
>
> Maybe we should have a --profile-self and --profile-test flag then?
>
> I'm not clear what profiling a test would look like. We'd have to
> implement that, which seems unlikely to happen?

This does not seems so unlikely to me. gathering profile on each Python 
invocation or on the test script performance would not be so hard and 
provide useful information

--profile-self seems fine with me.
Sean Farley - May 15, 2015, 7:56 p.m.
Pierre-Yves David <pierre-yves.david@ens-lyon.org> writes:

> On 05/14/2015 10:40 AM, Augie Fackler wrote:
>> On Wed, May 13, 2015 at 9:00 PM, Pierre-Yves David
>> <pierre-yves.david@ens-lyon.org> wrote:
>>>
>>>
>>> On 05/13/2015 05:42 PM, Augie Fackler wrote:
>>>>
>>>> # HG changeset patch
>>>> # User Augie Fackler <augie@google.com>
>>>> # Date 1431541326 25200
>>>> #      Wed May 13 11:22:06 2015 -0700
>>>> # Node ID 21735d2bcefb10605708a1dc640a6cfa590c1361
>>>> # Parent  130c305f79910395cbfb73a442cc512605f78bc5
>>>> run-tests: add a --profile option
>>>>
>>>> While in the throes of a recent run-tests adventure, I found it useful
>>>> to have profiler output for the testrunner itself. Adding it was
>>>> simple enough and seems worth keeping around.
>>>
>>>
>>> Maybe? the amount of option in run-tests.py is already quite scary and one
>>> could hope that --profile would give --profile output for the tests
>>> themself.
>>
>> Maybe we should have a --profile-self and --profile-test flag then?
>>
>> I'm not clear what profiling a test would look like. We'd have to
>> implement that, which seems unlikely to happen?
>
> This does not seems so unlikely to me. gathering profile on each Python 
> invocation or on the test script performance would not be so hard and 
> provide useful information
>
> --profile-self seems fine with me.

What about --profile-runner? Seems more explicit than 'self' to me.
Pierre-Yves David - May 15, 2015, 7:57 p.m.
On 05/15/2015 12:56 PM, Sean Farley wrote:
>
> Pierre-Yves David <pierre-yves.david@ens-lyon.org> writes:
>
>> On 05/14/2015 10:40 AM, Augie Fackler wrote:
>>> On Wed, May 13, 2015 at 9:00 PM, Pierre-Yves David
>>> <pierre-yves.david@ens-lyon.org> wrote:
>>>>
>>>>
>>>> On 05/13/2015 05:42 PM, Augie Fackler wrote:
>>>>>
>>>>> # HG changeset patch
>>>>> # User Augie Fackler <augie@google.com>
>>>>> # Date 1431541326 25200
>>>>> #      Wed May 13 11:22:06 2015 -0700
>>>>> # Node ID 21735d2bcefb10605708a1dc640a6cfa590c1361
>>>>> # Parent  130c305f79910395cbfb73a442cc512605f78bc5
>>>>> run-tests: add a --profile option
>>>>>
>>>>> While in the throes of a recent run-tests adventure, I found it useful
>>>>> to have profiler output for the testrunner itself. Adding it was
>>>>> simple enough and seems worth keeping around.
>>>>
>>>>
>>>> Maybe? the amount of option in run-tests.py is already quite scary and one
>>>> could hope that --profile would give --profile output for the tests
>>>> themself.
>>>
>>> Maybe we should have a --profile-self and --profile-test flag then?
>>>
>>> I'm not clear what profiling a test would look like. We'd have to
>>> implement that, which seems unlikely to happen?
>>
>> This does not seems so unlikely to me. gathering profile on each Python
>> invocation or on the test script performance would not be so hard and
>> provide useful information
>>
>> --profile-self seems fine with me.
>
> What about --profile-runner? Seems more explicit than 'self' to me.

I like it. If augie is sold I'll rename and push.
Augie Fackler - May 15, 2015, 8:10 p.m.
> On May 15, 2015, at 15:57, Pierre-Yves David <pierre-yves.david@ens-lyon.org> wrote:
> 
> 
> 
> On 05/15/2015 12:56 PM, Sean Farley wrote:
>> 
>> Pierre-Yves David <pierre-yves.david@ens-lyon.org> writes:
>> 
>>> On 05/14/2015 10:40 AM, Augie Fackler wrote:
>>>> On Wed, May 13, 2015 at 9:00 PM, Pierre-Yves David
>>>> <pierre-yves.david@ens-lyon.org> wrote:
>>>>> 
>>>>> 
>>>>> On 05/13/2015 05:42 PM, Augie Fackler wrote:
>>>>>> 
>>>>>> # HG changeset patch
>>>>>> # User Augie Fackler <augie@google.com>
>>>>>> # Date 1431541326 25200
>>>>>> #      Wed May 13 11:22:06 2015 -0700
>>>>>> # Node ID 21735d2bcefb10605708a1dc640a6cfa590c1361
>>>>>> # Parent  130c305f79910395cbfb73a442cc512605f78bc5
>>>>>> run-tests: add a --profile option
>>>>>> 
>>>>>> While in the throes of a recent run-tests adventure, I found it useful
>>>>>> to have profiler output for the testrunner itself. Adding it was
>>>>>> simple enough and seems worth keeping around.
>>>>> 
>>>>> 
>>>>> Maybe? the amount of option in run-tests.py is already quite scary and one
>>>>> could hope that --profile would give --profile output for the tests
>>>>> themself.
>>>> 
>>>> Maybe we should have a --profile-self and --profile-test flag then?
>>>> 
>>>> I'm not clear what profiling a test would look like. We'd have to
>>>> implement that, which seems unlikely to happen?
>>> 
>>> This does not seems so unlikely to me. gathering profile on each Python
>>> invocation or on the test script performance would not be so hard and
>>> provide useful information
>>> 
>>> --profile-self seems fine with me.
>> 
>> What about --profile-runner? Seems more explicit than 'self' to me.
> 
> I like it. If augie is sold I'll rename and push.

I don't feel strongly. If you like it, then take it. I care more about the functionality than the color of this particular bikeshed.

> 
> 
> -- 
> Pierre-Yves David
Pierre-Yves David - May 15, 2015, 8:44 p.m.
On 05/15/2015 01:10 PM, Augie Fackler wrote:
>
>> On May 15, 2015, at 15:57, Pierre-Yves David <pierre-yves.david@ens-lyon.org> wrote:
>>
>>
>>
>> On 05/15/2015 12:56 PM, Sean Farley wrote:
>>>
>>> Pierre-Yves David <pierre-yves.david@ens-lyon.org> writes:
>>>
>>>> On 05/14/2015 10:40 AM, Augie Fackler wrote:
>>>>> On Wed, May 13, 2015 at 9:00 PM, Pierre-Yves David
>>>>> <pierre-yves.david@ens-lyon.org> wrote:
>>>>>>
>>>>>>
>>>>>> On 05/13/2015 05:42 PM, Augie Fackler wrote:
>>>>>>>
>>>>>>> # HG changeset patch
>>>>>>> # User Augie Fackler <augie@google.com>
>>>>>>> # Date 1431541326 25200
>>>>>>> #      Wed May 13 11:22:06 2015 -0700
>>>>>>> # Node ID 21735d2bcefb10605708a1dc640a6cfa590c1361
>>>>>>> # Parent  130c305f79910395cbfb73a442cc512605f78bc5
>>>>>>> run-tests: add a --profile option
>>>>>>>
>>>>>>> While in the throes of a recent run-tests adventure, I found it useful
>>>>>>> to have profiler output for the testrunner itself. Adding it was
>>>>>>> simple enough and seems worth keeping around.
>>>>>>
>>>>>>
>>>>>> Maybe? the amount of option in run-tests.py is already quite scary and one
>>>>>> could hope that --profile would give --profile output for the tests
>>>>>> themself.
>>>>>
>>>>> Maybe we should have a --profile-self and --profile-test flag then?
>>>>>
>>>>> I'm not clear what profiling a test would look like. We'd have to
>>>>> implement that, which seems unlikely to happen?
>>>>
>>>> This does not seems so unlikely to me. gathering profile on each Python
>>>> invocation or on the test script performance would not be so hard and
>>>> provide useful information
>>>>
>>>> --profile-self seems fine with me.
>>>
>>> What about --profile-runner? Seems more explicit than 'self' to me.
>>
>> I like it. If augie is sold I'll rename and push.
>
> I don't feel strongly. If you like it, then take it. I care more about the functionality than the color of this particular bikeshed.

Now that we've move the bikeshed off the middle of the road and picked a 
descent color. This is pushed to the clowcopter.

Patch

diff --git a/tests/run-tests.py b/tests/run-tests.py
--- a/tests/run-tests.py
+++ b/tests/run-tests.py
@@ -240,6 +240,8 @@  def getparser():
                       help='set the given config opt in the test hgrc')
     parser.add_option('--random', action="store_true",
                       help='run tests in random order')
+    parser.add_option('--profile', action='store_true',
+                      help='run statprof on run-tests')
 
     for option, (envvar, default) in defaults.items():
         defaults[option] = type(default)(os.environ.get(envvar, default))
@@ -1653,7 +1655,15 @@  class TestRunner(object):
 
             self._checktools()
             tests = self.findtests(args)
-            return self._run(tests)
+            if options.profile:
+                import statprof
+                statprof.start()
+            result = self._run(tests)
+            if options.profile:
+                statprof.stop()
+                statprof.display()
+            return result
+
         finally:
             os.umask(oldmask)