Patchwork [2,of,2] rust-cpython: using rustext.dagop.headrevs in revlog

login
register
mail settings
Submitter Georges Racinet
Date Jan. 16, 2019, 6:15 p.m.
Message ID <234c106610004aff2942.1547662509@ishtar>
Download mbox | patch
Permalink /patch/37836/
State New
Headers show

Comments

Georges Racinet - Jan. 16, 2019, 6:15 p.m.
# HG changeset patch
# User Georges Racinet <georges.racinet@octobus.net>
# Date 1547651966 -3600
#      Wed Jan 16 16:19:26 2019 +0100
# Node ID 234c106610004aff294268ccda82238164f91f64
# Parent  f32e7d90db76879096f216f23ebbe75772b0201e
# EXP-Topic revset.predicates
rust-cpython: using rustext.dagop.headrevs in revlog

As with the previous oxidation series, revlog plays the role
of the factory, either using its parents function, or passing the
index.

In most of the cases about heads revsetbenchmarks, this seems to be
either neutral or an improvement. There aren't many cases where it's
actually slower and they don't look too bad.

Result by revset
Georges Racinet - Jan. 16, 2019, 6:23 p.m.
On 1/16/19 7:15 PM, Georges Racinet wrote:
> # HG changeset patch
> # User Georges Racinet <georges.racinet@octobus.net>
> # Date 1547651966 -3600
> #      Wed Jan 16 16:19:26 2019 +0100
> # Node ID 234c106610004aff294268ccda82238164f91f64
> # Parent  f32e7d90db76879096f216f23ebbe75772b0201e
> # EXP-Topic revset.predicates
> rust-cpython: using rustext.dagop.headrevs in revlog

Note: this one depends on a previous series by Boris Feld, mamely on the
following changeset

# HG changeset patch
# User Boris Feld <boris.feld@octobus.net>
# Date 1547482251 -3600
#      Mon Jan 14 17:10:51 2019 +0100
# Node ID 39a2761dc425aaa14c00b69bce5a4c68a780ff50
# Parent  c9906eb8d45f2e80157ca441c1ad01987d8e198a

The measurements are taken with its later
8b13377cb4cb8789429693653091772029d54b15 (dagop: minor python
optimization to `headrevs`) as baseline.

Regards,



>
> As with the previous oxidation series, revlog plays the role
> of the factory, either using its parents function, or passing the
> index.
>
> In most of the cases about heads revsetbenchmarks, this seems to be
> either neutral or an improvement. There aren't many cases where it's
> actually slower and they don't look too bad.
>
> Result by revset
> ================
>
> Revision:
>  0) parent of this changeset: rust-cpython: binding for headrevs()
>  1) this changeset: rust-cpython: using rustext.dagop.headrevs in revlog
>
> revset #0: heads(commonancestors(last(head(), 2)))
>    plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
> 0) 0.002913      0.002845      0.002976      0.002953      0.002978      0.002894      0.002946      0.003127      0.002949      0.002977      0.003074
> 1) 0.002830      0.002900      0.002942      0.003066      0.002994      0.002881      0.003041      0.002973      0.002942      0.003008      0.003012
>
> revset #1: heads(commonancestors(head()))
>    plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
> 0) 0.124867      0.117328      0.117523      0.117500      0.114464      0.128686      0.121959      0.119129      0.114949      0.124387      0.119485
> 1) 0.125079      0.124376 106% 0.114437      0.118340      0.124021 108% 0.123225      0.126748      0.122144      0.126391 109% 0.116410  93% 0.121342
>
> revset #2: heads(all())
>    plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
> 0) 0.080521      0.079977      0.079199      0.079454      0.084119      0.079578      0.083369      0.080215      0.080788      0.079157      0.080142
> 1) 0.033069  41% 0.032773  40% 0.033060  41% 0.032708  41% 0.032987  39% 0.032458  40% 0.033019  39% 0.032306  40% 0.032341  40% 0.033317  42% 0.032645  40%
>
> revset #3: heads(-10000:-1)
>    plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
> 0) 0.008496      0.008642      0.008398      0.008665      0.008896      0.008857      0.008864      0.009749      0.009690      0.009445      0.008902
> 1) 0.002694  31% 0.002678  30% 0.002692  32% 0.002766  31% 0.002753  30% 0.002931  33% 0.003023  34% 0.002787  28% 0.002735  28% 0.002789  29% 0.002943  33%
>
> revset #4: (-5000:-1000) and heads(-10000:-1)
>    plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
> 0) 0.008835      0.008711      0.008920      0.008824      0.008948      0.009422      0.009073      0.008751      0.009244      0.009264      0.009489
> 1) 0.003033  34% 0.003027  34% 0.002885  32% 0.002911  32% 0.002850  31% 0.003068  32% 0.002971  32% 0.002890  33% 0.003239  35% 0.002973  32% 0.003068  32%
>
> revset #5: heads(matching(tip, "author"))
>    plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
> 0) 15.82091      15.47578      15.95506      15.74816      15.72207      15.72779      15.52043      15.65250      15.40250      15.46190      15.34036
> 1) 15.29257      15.39349      15.52757      15.41417      15.58991      15.42382      15.46773      15.48543      15.43644      15.76070      15.32104
>
> revset #6: heads(matching(tip, "author")) and -10000:-1
>    plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
> 0) 15.74241      15.63808      15.46853      15.77480      15.61004      15.80385      15.44905      15.38059      15.34111      15.45029      15.85194
> 1) 15.51474      15.45938      15.28755      15.77045      15.38245      15.40625      15.53653      15.31534      15.64134      15.43437      15.29366
>
> revset #7: (-10000:-1) and heads(matching(tip, "author"))
>    plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
> 0) 15.37988      15.59717      16.07863      15.29263      15.46472      15.84978      15.35538      15.50673      15.37396      15.42426      15.44425
> 1) 15.51504      15.38657      15.30847      15.39458      15.60453      15.26237      15.45240      15.34210      15.36410      15.53354      15.67735
>
> diff -r f32e7d90db76 -r 234c10661000 mercurial/revlog.py
> --- a/mercurial/revlog.py	Thu Jan 10 18:25:18 2019 +0100
> +++ b/mercurial/revlog.py	Wed Jan 16 16:19:26 2019 +0100
> @@ -1108,6 +1108,8 @@
>                  return self.index.headrevs()
>              except AttributeError:
>                  return self._headrevs()
> +        if rustext is not None:
> +            return rustext.dagop.headrevs(self.index, revs)
>          return dagop.headrevs(revs, self.parentrevs)
>  
>      def computephases(self, roots):
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Pulkit Goyal - Jan. 16, 2019, 6:40 p.m.
On Wed, Jan 16, 2019 at 11:55 PM Georges Racinet <
georges.racinet@octobus.net> wrote:

>
> On 1/16/19 7:15 PM, Georges Racinet wrote:
> > # HG changeset patch
> > # User Georges Racinet <georges.racinet@octobus.net>
> > # Date 1547651966 -3600
> > #      Wed Jan 16 16:19:26 2019 +0100
> > # Node ID 234c106610004aff294268ccda82238164f91f64
> > # Parent  f32e7d90db76879096f216f23ebbe75772b0201e
> > # EXP-Topic revset.predicates
> > rust-cpython: using rustext.dagop.headrevs in revlog
>
> Note: this one depends on a previous series by Boris Feld, mamely on the
> following changeset
>
> # HG changeset patch
> # User Boris Feld <boris.feld@octobus.net>
> # Date 1547482251 -3600
> #      Mon Jan 14 17:10:51 2019 +0100
> # Node ID 39a2761dc425aaa14c00b69bce5a4c68a780ff50
> # Parent  c9906eb8d45f2e80157ca441c1ad01987d8e198a
>
> The measurements are taken with its later
> 8b13377cb4cb8789429693653091772029d54b15 (dagop: minor python
> optimization to `headrevs`) as baseline.
>
> Regards,
>
>
>
> >
> > As with the previous oxidation series, revlog plays the role
> > of the factory, either using its parents function, or passing the
> > index.
> >
> > In most of the cases about heads revsetbenchmarks, this seems to be
> > either neutral or an improvement. There aren't many cases where it's
> > actually slower and they don't look too bad.
> >
> > Result by revset
> > ================
> >
> > Revision:
> >  0) parent of this changeset: rust-cpython: binding for headrevs()
> >  1) this changeset: rust-cpython: using rustext.dagop.headrevs in revlog
> >
> > revset #0: heads(commonancestors(last(head(), 2)))
> >    plain         min           max           first         last
> reverse       rev..rst      rev..ast      sort          sor..rst
> sor..ast
> > 0) 0.002913      0.002845      0.002976      0.002953      0.002978
> 0.002894      0.002946      0.003127      0.002949      0.002977
> 0.003074
> > 1) 0.002830      0.002900      0.002942      0.003066      0.002994
> 0.002881      0.003041      0.002973      0.002942      0.003008
> 0.003012
> >
> > revset #1: heads(commonancestors(head()))
> >    plain         min           max           first         last
> reverse       rev..rst      rev..ast      sort          sor..rst
> sor..ast
> > 0) 0.124867      0.117328      0.117523      0.117500      0.114464
> 0.128686      0.121959      0.119129      0.114949      0.124387
> 0.119485
> > 1) 0.125079      0.124376 106% 0.114437      0.118340      0.124021 108%
> 0.123225      0.126748      0.122144      0.126391 109% 0.116410  93%
> 0.121342
> >
> > revset #2: heads(all())
> >    plain         min           max           first         last
> reverse       rev..rst      rev..ast      sort          sor..rst
> sor..ast
> > 0) 0.080521      0.079977      0.079199      0.079454      0.084119
> 0.079578      0.083369      0.080215      0.080788      0.079157
> 0.080142
> > 1) 0.033069  41% 0.032773  40% 0.033060  41% 0.032708  41% 0.032987  39%
> 0.032458  40% 0.033019  39% 0.032306  40% 0.032341  40% 0.033317  42%
> 0.032645  40%
> >
> > revset #3: heads(-10000:-1)
> >    plain         min           max           first         last
> reverse       rev..rst      rev..ast      sort          sor..rst
> sor..ast
> > 0) 0.008496      0.008642      0.008398      0.008665      0.008896
> 0.008857      0.008864      0.009749      0.009690      0.009445
> 0.008902
> > 1) 0.002694  31% 0.002678  30% 0.002692  32% 0.002766  31% 0.002753  30%
> 0.002931  33% 0.003023  34% 0.002787  28% 0.002735  28% 0.002789  29%
> 0.002943  33%
> >
> > revset #4: (-5000:-1000) and heads(-10000:-1)
> >    plain         min           max           first         last
> reverse       rev..rst      rev..ast      sort          sor..rst
> sor..ast
> > 0) 0.008835      0.008711      0.008920      0.008824      0.008948
> 0.009422      0.009073      0.008751      0.009244      0.009264
> 0.009489
> > 1) 0.003033  34% 0.003027  34% 0.002885  32% 0.002911  32% 0.002850  31%
> 0.003068  32% 0.002971  32% 0.002890  33% 0.003239  35% 0.002973  32%
> 0.003068  32%
> >
> > revset #5: heads(matching(tip, "author"))
> >    plain         min           max           first         last
> reverse       rev..rst      rev..ast      sort          sor..rst
> sor..ast
> > 0) 15.82091      15.47578      15.95506      15.74816      15.72207
> 15.72779      15.52043      15.65250      15.40250      15.46190
> 15.34036
> > 1) 15.29257      15.39349      15.52757      15.41417      15.58991
> 15.42382      15.46773      15.48543      15.43644      15.76070
> 15.32104
> >
> > revset #6: heads(matching(tip, "author")) and -10000:-1
> >    plain         min           max           first         last
> reverse       rev..rst      rev..ast      sort          sor..rst
> sor..ast
> > 0) 15.74241      15.63808      15.46853      15.77480      15.61004
> 15.80385      15.44905      15.38059      15.34111      15.45029
> 15.85194
> > 1) 15.51474      15.45938      15.28755      15.77045      15.38245
> 15.40625      15.53653      15.31534      15.64134      15.43437
> 15.29366
> >
> > revset #7: (-10000:-1) and heads(matching(tip, "author"))
> >    plain         min           max           first         last
> reverse       rev..rst      rev..ast      sort          sor..rst
> sor..ast
> > 0) 15.37988      15.59717      16.07863      15.29263      15.46472
> 15.84978      15.35538      15.50673      15.37396      15.42426
> 15.44425
> > 1) 15.51504      15.38657      15.30847      15.39458      15.60453
> 15.26237      15.45240      15.34210      15.36410      15.53354
> 15.67735
> >
>
>
Nice work! Only reviewing the commit message as I don't know rust. Can you
mention on which repo these benchmarks were performed?
Georges Racinet - Jan. 16, 2019, 6:43 p.m.
On 1/16/19 7:40 PM, Pulkit Goyal wrote:
> Nice work! Only reviewing the commit message as I don't know rust. Can
> you mention on which repo these benchmarks were performed?


Thanks! Actually the results are better that I'd thought. The benchmarks
are done on a PyPy clone of last summer.

If you want to confirm them, just build with

   make local HGWITHRUSTEXT=cpython

and check that your python can import 'mercurial.rustext' to be sure.

Cheers,
Yuya Nishihara - Jan. 17, 2019, 1:40 p.m.
On Wed, 16 Jan 2019 19:15:09 +0100, Georges Racinet wrote:
> # HG changeset patch
> # User Georges Racinet <georges.racinet@octobus.net>
> # Date 1547651966 -3600
> #      Wed Jan 16 16:19:26 2019 +0100
> # Node ID 234c106610004aff294268ccda82238164f91f64
> # Parent  f32e7d90db76879096f216f23ebbe75772b0201e
> # EXP-Topic revset.predicates
> rust-cpython: using rustext.dagop.headrevs in revlog

> --- a/mercurial/revlog.py	Thu Jan 10 18:25:18 2019 +0100
> +++ b/mercurial/revlog.py	Wed Jan 16 16:19:26 2019 +0100
> @@ -1108,6 +1108,8 @@
>                  return self.index.headrevs()
>              except AttributeError:
>                  return self._headrevs()
> +        if rustext is not None:
> +            return rustext.dagop.headrevs(self.index, revs)
>          return dagop.headrevs(revs, self.parentrevs)

(CC: Boris)
Are we sure the given revs are not filtered?
Georges Racinet - Jan. 18, 2019, 11:33 a.m.
On 1/17/19 2:40 PM, Yuya Nishihara wrote:
> On Wed, 16 Jan 2019 19:15:09 +0100, Georges Racinet wrote:
>> # HG changeset patch
>> # User Georges Racinet <georges.racinet@octobus.net>
>> # Date 1547651966 -3600
>> #      Wed Jan 16 16:19:26 2019 +0100
>> # Node ID 234c106610004aff294268ccda82238164f91f64
>> # Parent  f32e7d90db76879096f216f23ebbe75772b0201e
>> # EXP-Topic revset.predicates
>> rust-cpython: using rustext.dagop.headrevs in revlog
>> --- a/mercurial/revlog.py	Thu Jan 10 18:25:18 2019 +0100
>> +++ b/mercurial/revlog.py	Wed Jan 16 16:19:26 2019 +0100
>> @@ -1108,6 +1108,8 @@
>>                  return self.index.headrevs()
>>              except AttributeError:
>>                  return self._headrevs()
>> +        if rustext is not None:
>> +            return rustext.dagop.headrevs(self.index, revs)
>>          return dagop.headrevs(revs, self.parentrevs)
> (CC: Boris)
> Are we sure the given revs are not filtered?

I just to talked to him directly about that, and the bottom line is that
no, it isn't.

I'm working right now on a v2 that will prefilter from changelog on
input revisions. In order not to take risks with the performance in the
pure Python case, I may have to pass a flag directly (we'll see).

Sorry for the hitch,
Yuya Nishihara - Jan. 18, 2019, 12:38 p.m.
On Fri, 18 Jan 2019 12:33:43 +0100, Georges Racinet wrote:
> On 1/17/19 2:40 PM, Yuya Nishihara wrote:
> > On Wed, 16 Jan 2019 19:15:09 +0100, Georges Racinet wrote:
> >> # HG changeset patch
> >> # User Georges Racinet <georges.racinet@octobus.net>
> >> # Date 1547651966 -3600
> >> #      Wed Jan 16 16:19:26 2019 +0100
> >> # Node ID 234c106610004aff294268ccda82238164f91f64
> >> # Parent  f32e7d90db76879096f216f23ebbe75772b0201e
> >> # EXP-Topic revset.predicates
> >> rust-cpython: using rustext.dagop.headrevs in revlog
> >> --- a/mercurial/revlog.py	Thu Jan 10 18:25:18 2019 +0100
> >> +++ b/mercurial/revlog.py	Wed Jan 16 16:19:26 2019 +0100
> >> @@ -1108,6 +1108,8 @@
> >>                  return self.index.headrevs()
> >>              except AttributeError:
> >>                  return self._headrevs()
> >> +        if rustext is not None:
> >> +            return rustext.dagop.headrevs(self.index, revs)
> >>          return dagop.headrevs(revs, self.parentrevs)
> > (CC: Boris)
> > Are we sure the given revs are not filtered?
> 
> I just to talked to him directly about that, and the bottom line is that
> no, it isn't.
> 
> I'm working right now on a v2 that will prefilter from changelog on
> input revisions. In order not to take risks with the performance in the
> pure Python case, I may have to pass a flag directly (we'll see).

Let's postpone it for 5.0 cycle. There are many unprocessed patches.

I think we can copy adbf8ca239e4, which filters input revs and uses
unfiltered parentrevs() instead.

Patch

================

Revision:
 0) parent of this changeset: rust-cpython: binding for headrevs()
 1) this changeset: rust-cpython: using rustext.dagop.headrevs in revlog

revset #0: heads(commonancestors(last(head(), 2)))
   plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
0) 0.002913      0.002845      0.002976      0.002953      0.002978      0.002894      0.002946      0.003127      0.002949      0.002977      0.003074
1) 0.002830      0.002900      0.002942      0.003066      0.002994      0.002881      0.003041      0.002973      0.002942      0.003008      0.003012

revset #1: heads(commonancestors(head()))
   plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
0) 0.124867      0.117328      0.117523      0.117500      0.114464      0.128686      0.121959      0.119129      0.114949      0.124387      0.119485
1) 0.125079      0.124376 106% 0.114437      0.118340      0.124021 108% 0.123225      0.126748      0.122144      0.126391 109% 0.116410  93% 0.121342

revset #2: heads(all())
   plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
0) 0.080521      0.079977      0.079199      0.079454      0.084119      0.079578      0.083369      0.080215      0.080788      0.079157      0.080142
1) 0.033069  41% 0.032773  40% 0.033060  41% 0.032708  41% 0.032987  39% 0.032458  40% 0.033019  39% 0.032306  40% 0.032341  40% 0.033317  42% 0.032645  40%

revset #3: heads(-10000:-1)
   plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
0) 0.008496      0.008642      0.008398      0.008665      0.008896      0.008857      0.008864      0.009749      0.009690      0.009445      0.008902
1) 0.002694  31% 0.002678  30% 0.002692  32% 0.002766  31% 0.002753  30% 0.002931  33% 0.003023  34% 0.002787  28% 0.002735  28% 0.002789  29% 0.002943  33%

revset #4: (-5000:-1000) and heads(-10000:-1)
   plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
0) 0.008835      0.008711      0.008920      0.008824      0.008948      0.009422      0.009073      0.008751      0.009244      0.009264      0.009489
1) 0.003033  34% 0.003027  34% 0.002885  32% 0.002911  32% 0.002850  31% 0.003068  32% 0.002971  32% 0.002890  33% 0.003239  35% 0.002973  32% 0.003068  32%

revset #5: heads(matching(tip, "author"))
   plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
0) 15.82091      15.47578      15.95506      15.74816      15.72207      15.72779      15.52043      15.65250      15.40250      15.46190      15.34036
1) 15.29257      15.39349      15.52757      15.41417      15.58991      15.42382      15.46773      15.48543      15.43644      15.76070      15.32104

revset #6: heads(matching(tip, "author")) and -10000:-1
   plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
0) 15.74241      15.63808      15.46853      15.77480      15.61004      15.80385      15.44905      15.38059      15.34111      15.45029      15.85194
1) 15.51474      15.45938      15.28755      15.77045      15.38245      15.40625      15.53653      15.31534      15.64134      15.43437      15.29366

revset #7: (-10000:-1) and heads(matching(tip, "author"))
   plain         min           max           first         last          reverse       rev..rst      rev..ast      sort          sor..rst      sor..ast
0) 15.37988      15.59717      16.07863      15.29263      15.46472      15.84978      15.35538      15.50673      15.37396      15.42426      15.44425
1) 15.51504      15.38657      15.30847      15.39458      15.60453      15.26237      15.45240      15.34210      15.36410      15.53354      15.67735

diff -r f32e7d90db76 -r 234c10661000 mercurial/revlog.py
--- a/mercurial/revlog.py	Thu Jan 10 18:25:18 2019 +0100
+++ b/mercurial/revlog.py	Wed Jan 16 16:19:26 2019 +0100
@@ -1108,6 +1108,8 @@ 
                 return self.index.headrevs()
             except AttributeError:
                 return self._headrevs()
+        if rustext is not None:
+            return rustext.dagop.headrevs(self.index, revs)
         return dagop.headrevs(revs, self.parentrevs)
 
     def computephases(self, roots):