Patchwork [8,of,8] obscache: use the obscache to compute the obsolete set

# HG changeset patch
# User Pierre-Yves David <pierre-yves.david@octobus.net>
# Date 1495198021 -7200
#      Fri May 19 14:47:01 2017 +0200
# Node ID f1082c7b4a2eb56f9bddfabacc7c5b3fefd19c75
# Parent  f41a5dbded63f3c721456187b88c49cd90f5ad53
# EXP-Topic obscache
# Available At https://www.mercurial-scm.org/repo/users/marmoute/mercurial/
#              hg pull https://www.mercurial-scm.org/repo/users/marmoute/mercurial/ -r f1082c7b4a2e
obscache: use the obscache to compute the obsolete set

Now that we have a cache and that the cache is kept up to date, we can use it to
speeds up the obsolete set computation. This way, we no longer need to load the
obsstore for most operation.

On the mercurial-core repository, this provide a significant speed up:

Running "hg  id -r ."
- before: 0.630 second (0.56s user 0.06s system 99% cpu 0.630)
- after:  0.129 second (0.11s user 0.02s system 98% cpu 0.129)

The and obsstore loading operation disapear from execution profile.


To keep the changeset simple it the handling of case were
the cache has not been kept up to date is pretty simple. That might introduce a
small performance impact during the transition in some case. This will get
improved in later changeset.

In addition the cache still needs to parse the full obsstore when updating.
There as known way to skip parsing the full obsstore for wrote operation too.
This will also get improved later.

On Fri, May 19, 2017 at 04:28:07PM +0200, Pierre-Yves David wrote:
> # HG changeset patch
> # User Pierre-Yves David <pierre-yves.david@octobus.net>
> # Date 1495198021 -7200
> #      Fri May 19 14:47:01 2017 +0200
> # Node ID f1082c7b4a2eb56f9bddfabacc7c5b3fefd19c75
> # Parent  f41a5dbded63f3c721456187b88c49cd90f5ad53
> # EXP-Topic obscache
> # Available At https://www.mercurial-scm.org/repo/users/marmoute/mercurial/
> #              hg pull https://www.mercurial-scm.org/repo/users/marmoute/mercurial/ -r f1082c7b4a2e
> obscache: use the obscache to compute the obsolete set

Very excited about this series, took the first two patches, but I had
some nits on some of the middle patches.

>
> Now that we have a cache and that the cache is kept up to date, we can use it to
> speeds up the obsolete set computation. This way, we no longer need to load the
> obsstore for most operation.
>
> On the mercurial-core repository, this provide a significant speed up:
>
> Running "hg  id -r ."
> - before: 0.630 second (0.56s user 0.06s system 99% cpu 0.630)
> - after:  0.129 second (0.11s user 0.02s system 98% cpu 0.129)
>
> The and obsstore loading operation disapear from execution profile.
>
>
> To keep the changeset simple it the handling of case were
> the cache has not been kept up to date is pretty simple. That might introduce a
> small performance impact during the transition in some case. This will get
> improved in later changeset.
>
> In addition the cache still needs to parse the full obsstore when updating.
> There as known way to skip parsing the full obsstore for wrote operation too.
> This will also get improved later.
>
> diff --git a/mercurial/obsolete.py b/mercurial/obsolete.py
> --- a/mercurial/obsolete.py
> +++ b/mercurial/obsolete.py
> @@ -1539,10 +1539,26 @@ def clearobscaches(repo):
>  def _computeobsoleteset(repo):
>      """the set of obsolete revisions"""
>      obs = set()
> -    getnode = repo.changelog.node
>      notpublic = repo._phasecache.getrevset(repo, (phases.draft, phases.secret))
> +    if not notpublic:
> +        # all changeset are public, none are obsolete
> +        return obs
> +
> +    # XXX There are a couple of case where the cache could not be up to date:
> +    #
> +    # 1) no transaction happened in the repository since the upgrade,
> +    # 2) both old and new client touches that repository
> +    #
> +    # recomputing the whole cache in these case is a bit slower that using the
> +    # good old version (parsing markers and checking them). We could add some
> +    # logic to fall back to the old way in these cases.
> +    obscache = repo.obsstore.obscache
> +    obscache.update(repo) # ensure it is up to date:
> +    isobs = obscache.get
> +
> +    # actually compute the obsolete set
>      for r in notpublic:
> -        if getnode(r) in repo.obsstore.successors:
> +        if isobs(r):
>              obs.add(r)
>      return obs
>
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel