Patchwork [1,of,6,V2] hg: make cachedlocalrepo filter newly cached repo as same as initial one

login
register
mail settings
Submitter Katsunori FUJIWARA
Date Dec. 16, 2015, 5:50 p.m.
Message ID <4c237821e543338dde6b.1450288249@feefifofum>
Download mbox | patch
Permalink /patch/12070/
State Changes Requested
Delegated to: Yuya Nishihara
Headers show

Comments

Katsunori FUJIWARA - Dec. 16, 2015, 5:50 p.m.
# HG changeset patch
# User FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
# Date 1450287754 -32400
#      Thu Dec 17 02:42:34 2015 +0900
# Node ID 4c237821e543338dde6bfd395c28884f065399ec
# Parent  7e8a883da1711008262150bb6f64131812de3e0b
hg: make cachedlocalrepo filter newly cached repo as same as initial one

Before this patch, 'cachedlocalrepo' always caches "visible" repoview
object, because 'hg.repository()' always returns "visible" repoview
object and 'cachedlocalrepo' uses it without any additional
processing.

If the client of 'cachedlocalrepo' wants "served" repoview, some
objects to be cached are discarded unintentionally.

    1. 'cachedlocalrepo' newly caches "visible" repoview object
       (call it VIEW1)

    2. 'cachedlocalrepo' returns VIEW1 to the client of it

    3. the client gets "served" repoview object by 'filtered("served")'
       on VIEW1 (call this "served" repoview VIEW2)

    4. accessing to 'repo.changelog' implies:
       - instantiation of changelog via 'localrepository.changelog'
       - instantiation of "filtered changelog" via 'repoview.changelog'

    5. "filtered changelog" above is cached in VIEW2

    6. VIEW2 is discarded after processing, because there is no
       reference to it

    7. 'cachedlocalrepo' returns VIEW1 cached at (1) above to the
       client at next 'fetch()'

    8. 'filtered("served")' on VIEW1 at the client side creates new
       "served" repoview again, because VIEW1 is "visible"
       (call this new "served" repoview VIEW3)

    9. accessing to 'repo.changelog' implies instantiation of filtered
       changelog again, because "filtered changelog" is cached in
       VIEW2 at (5), but not in VIEW3 currently used

    10. (go to (7) above)

As described above, "served" repoview object and "filtered changelog"
cached in it are discarded always, even if the repository itself
hasn't been changed since last access.

For example, in the case of 'hgweb_mod.hgweb', "newly caching" occurs,
when:

  - all cached objects are already assigned to another threads, or
    (in this case, repoview is created in 'cachedlocalrepo.copy()')

  - stat of '00changelog.i' is changed from last access
    (in this case, repoview is created in 'cachedlocalrepo.fetch()')

    once changes are pushed via HTTP, this always occurs.

The root cause of this inefficiency is that 'cachedlocalrepo' always
caches "visible" repoview object, even if the client of it wants
another view.

This patch makes 'cachedlocalrepo' filter newly cached repo as same as
initial one. It is assumed that initial repoview object should be
already filtered by expected view.

After this patch:

  - 'filtered("served")' on VIEW1 at (3)/(7) above returns VIEW1
    itself, because VIEW1 is now "served", and

  - VIEW2 and VIEW3 equal VIEW1

  - therefore, "filtered changelog" is cached in VIEW1, and reused
    intentionally
Yuya Nishihara - Dec. 20, 2015, 11:43 a.m.
On Thu, 17 Dec 2015 02:50:49 +0900, FUJIWARA Katsunori wrote:
> # HG changeset patch
> # User FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
> # Date 1450287754 -32400
> #      Thu Dec 17 02:42:34 2015 +0900
> # Node ID 4c237821e543338dde6bfd395c28884f065399ec
> # Parent  7e8a883da1711008262150bb6f64131812de3e0b
> hg: make cachedlocalrepo filter newly cached repo as same as initial one

This makes sense, but one nit.

> @@ -883,6 +884,8 @@ class cachedlocalrepo(object):
>              return self._repo, False
>  
>          self._repo = repository(self._repo.baseui, self._repo.url())
> +        if self._filtername:
> +            self._repo = self._repo.filtered(self._filtername)

If filtername is None, shouldn't it return an unfiltered repo?

(I haven't read PATCH 4-, they'll need a deeper review by transaction guru.)
Katsunori FUJIWARA - Dec. 20, 2015, 4:09 p.m.
At Sun, 20 Dec 2015 20:43:53 +0900,
Yuya Nishihara wrote:
> 
> On Thu, 17 Dec 2015 02:50:49 +0900, FUJIWARA Katsunori wrote:
> > # HG changeset patch
> > # User FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
> > # Date 1450287754 -32400
> > #      Thu Dec 17 02:42:34 2015 +0900
> > # Node ID 4c237821e543338dde6bfd395c28884f065399ec
> > # Parent  7e8a883da1711008262150bb6f64131812de3e0b
> > hg: make cachedlocalrepo filter newly cached repo as same as initial one
> 
> This makes sense, but one nit.
> 
> > @@ -883,6 +884,8 @@ class cachedlocalrepo(object):
> >              return self._repo, False
> >  
> >          self._repo = repository(self._repo.baseui, self._repo.url())
> > +        if self._filtername:
> > +            self._repo = self._repo.filtered(self._filtername)
> 
> If filtername is None, shouldn't it return an unfiltered repo?

'unfiltered()' on repoview just returns '_unfilteredrepo' of it, and
this doesn't cause redundant instantiation of repoview objects.

Therefore, this change works well at least for current hgweb_mode, the
only client of cachedlocalrepo: it invokes 'filtered()' or
'unfiltered()' explicitly on returned repoview at the beginning of
each serving to request of client.

Of course, it is fact that there are still some inefficiency around
hgweb_mode and cachedlocalrepo :-)


> (I haven't read PATCH 4-, they'll need a deeper review by transaction guru.)
> 

----------------------------------------------------------------------
[FUJIWARA Katsunori]                             foozy@lares.dti.ne.jp
Yuya Nishihara - Dec. 21, 2015, 1:44 p.m.
On Mon, 21 Dec 2015 01:09:57 +0900, FUJIWARA Katsunori wrote:
> At Sun, 20 Dec 2015 20:43:53 +0900,
> Yuya Nishihara wrote:
> > On Thu, 17 Dec 2015 02:50:49 +0900, FUJIWARA Katsunori wrote:
> > > # HG changeset patch
> > > # User FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
> > > # Date 1450287754 -32400
> > > #      Thu Dec 17 02:42:34 2015 +0900
> > > # Node ID 4c237821e543338dde6bfd395c28884f065399ec
> > > # Parent  7e8a883da1711008262150bb6f64131812de3e0b
> > > hg: make cachedlocalrepo filter newly cached repo as same as initial one
> > 
> > This makes sense, but one nit.
> > 
> > > @@ -883,6 +884,8 @@ class cachedlocalrepo(object):
> > >              return self._repo, False
> > >  
> > >          self._repo = repository(self._repo.baseui, self._repo.url())
> > > +        if self._filtername:
> > > +            self._repo = self._repo.filtered(self._filtername)
> > 
> > If filtername is None, shouldn't it return an unfiltered repo?
> 
> 'unfiltered()' on repoview just returns '_unfilteredrepo' of it, and
> this doesn't cause redundant instantiation of repoview objects.
> 
> Therefore, this change works well at least for current hgweb_mode, the
> only client of cachedlocalrepo: it invokes 'filtered()' or
> 'unfiltered()' explicitly on returned repoview at the beginning of
> each serving to request of client.
> 
> Of course, it is fact that there are still some inefficiency around
> hgweb_mode and cachedlocalrepo :-)

My concern is consistency of the API. Because this patch preserves the
filtering level of the given repo, if self._repo was initially unfiltered,
I expect fetch() would return unfiltered repo.

Patch

diff --git a/mercurial/hg.py b/mercurial/hg.py
--- a/mercurial/hg.py
+++ b/mercurial/hg.py
@@ -863,6 +863,7 @@  class cachedlocalrepo(object):
         assert isinstance(repo, localrepo.localrepository)
         self._repo = repo
         self._state, self.mtime = self._repostate()
+        self._filtername = repo.filtername
 
     def fetch(self):
         """Refresh (if necessary) and return a repository.
@@ -883,6 +884,8 @@  class cachedlocalrepo(object):
             return self._repo, False
 
         self._repo = repository(self._repo.baseui, self._repo.url())
+        if self._filtername:
+            self._repo = self._repo.filtered(self._filtername)
         self._state = state
         self.mtime = mtime
 
@@ -910,6 +913,8 @@  class cachedlocalrepo(object):
         completely independent of the original.
         """
         repo = repository(self._repo.baseui, self._repo.origroot)
+        if self._filtername:
+            repo = repo.filtered(self._filtername)
         c = cachedlocalrepo(repo)
         c._state = self._state
         c.mtime = self.mtime