Patchwork [3,of,3,v2] strip: include phases in bundle (BC)

login
register
mail settings
Submitter via Mercurial-devel
Date June 20, 2017, 5 p.m.
Message ID <64f74cbf6d070ddcf3a3.1497978010@martinvonz.svl.corp.google.com>
Download mbox | patch
Permalink /patch/21543/
State Accepted
Headers show

Comments

via Mercurial-devel - June 20, 2017, 5 p.m.
# HG changeset patch
# User Martin von Zweigbergk <martinvonz@google.com>
# Date 1497510952 25200
#      Thu Jun 15 00:15:52 2017 -0700
# Node ID 64f74cbf6d070ddcf3a373561ede2cf6ad333f36
# Parent  04fa2d783f067765f9d8cf0d1b02fae1819c232e
strip: include phases in bundle (BC)

Before this patch, unbundling a stripped changeset would make it a
draft (unless the parent was secret). This meant that one would lose
phase information when stripping and unbundling secret changesets. The
same thing was true for public changesets. While stripping public
changesets is generally rare, it's done frequently by e.g. the
narrowhg extension.

We also include the phases in the temporary bundle, just in case
stripping were to fail after that point, so the user can still restore
the repo including phase information. Before this patch, the phases
were left untouched during the bundling and unbundling of the
temporary bundle. Only at the end of the transaction would
phasecache.filterunknown() be called to remove phase roots that were
no longer valid. We now need to call that also after the first
stripping, i.e. before applying the temporary bundle. Otherwise
unbundling the temporary bundle will cause a read of the phase cache
which has stripped changesets in the cache and that fails.

Like with obsmarkers, we unconditionally include the phases in the
bundle when stripping (when using bundle2, such as when generaldelta
is enabled). The reason for doing that for strip but not for bundle is
that strip bundles are not meant to be shared outside the repo, so we
don't care as much about compatibility.
Pierre-Yves David - June 22, 2017, 4:03 p.m.
On 06/20/2017 07:00 PM, Martin von Zweigbergk via Mercurial-devel wrote:
> # HG changeset patch
> # User Martin von Zweigbergk <martinvonz@google.com>
> # Date 1497510952 25200
> #      Thu Jun 15 00:15:52 2017 -0700
> # Node ID 64f74cbf6d070ddcf3a373561ede2cf6ad333f36
> # Parent  04fa2d783f067765f9d8cf0d1b02fae1819c232e
> strip: include phases in bundle (BC)

That one seems fine (but see patch 2 for issue on patch 2)

If you get the energy to adds better support the new phase part in the 
'debugbundle' that would be nice. having better data will increase the 
tests strength.

> Before this patch, unbundling a stripped changeset would make it a
> draft (unless the parent was secret). This meant that one would lose
> phase information when stripping and unbundling secret changesets. The
> same thing was true for public changesets. While stripping public
> changesets is generally rare, it's done frequently by e.g. the
> narrowhg extension.
> 
> We also include the phases in the temporary bundle, just in case
> stripping were to fail after that point, so the user can still restore
> the repo including phase information. Before this patch, the phases
> were left untouched during the bundling and unbundling of the
> temporary bundle. Only at the end of the transaction would
> phasecache.filterunknown() be called to remove phase roots that were
> no longer valid. We now need to call that also after the first
> stripping, i.e. before applying the temporary bundle. Otherwise
> unbundling the temporary bundle will cause a read of the phase cache
> which has stripped changesets in the cache and that fails.
> 
> Like with obsmarkers, we unconditionally include the phases in the
> bundle when stripping (when using bundle2, such as when generaldelta
> is enabled). The reason for doing that for strip but not for bundle is
> that strip bundles are not meant to be shared outside the repo, so we
> don't care as much about compatibility.
> 
> diff --git a/mercurial/repair.py b/mercurial/repair.py
> --- a/mercurial/repair.py
> +++ b/mercurial/repair.py
> @@ -49,7 +49,11 @@
>           bundletype = "HG10UN"
>   
>       outgoing = discovery.outgoing(repo, missingroots=bases, missingheads=heads)
> -    contentopts = {'cg.version': cgversion, 'obsolescence': obsolescence}
> +    contentopts = {
> +        'cg.version': cgversion,
> +        'obsolescence': obsolescence,
> +        'phases': True,
> +    }
>       return bundle2.writenewbundle(repo.ui, repo, 'strip', name, bundletype,
>                                     outgoing, contentopts, vfs, compression=comp)
>   
> @@ -194,6 +198,7 @@
>               deleteobsmarkers(repo.obsstore, stripobsidx)
>               del repo.obsstore
>   
> +        repo._phasecache.filterunknown(repo)
>           if tmpbundlefile:
>               ui.note(_("adding branch\n"))
>               f = vfs.open(tmpbundlefile, "rb")
> diff --git a/tests/test-generaldelta.t b/tests/test-generaldelta.t
> --- a/tests/test-generaldelta.t
> +++ b/tests/test-generaldelta.t
> @@ -157,5 +157,6 @@
>     Stream params: sortdict([('Compression', 'BZ')])
>     changegroup -- "sortdict([('version', '02'), ('nbchanges', '1')])"
>         1c5d4dc9a8b8d6e1750966d343e94db665e7a1e9
> +  phases -- 'sortdict()'
>   
>     $ cd ..
> diff --git a/tests/test-obsolete.t b/tests/test-obsolete.t
> --- a/tests/test-obsolete.t
> +++ b/tests/test-obsolete.t
> @@ -1239,6 +1239,7 @@
>     obsmarkers -- 'sortdict()'
>         version: 1 (70 bytes)
>         f27abbcc1f77fb409cf9160482fe619541e2d605 0 {e008cf2834908e5d6b0f792a9d4b0e2272260fb8} (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
> +  phases -- 'sortdict()'
>   
>     $ hg pull .hg/strip-backup/e008cf283490-*-backup.hg
>     pulling from .hg/strip-backup/e008cf283490-39c978dc-backup.hg
> @@ -1283,6 +1284,7 @@
>         version: 1 (139 bytes)
>         e008cf2834908e5d6b0f792a9d4b0e2272260fb8 b0551702f918510f01ae838ab03a463054c67b46 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
>         f27abbcc1f77fb409cf9160482fe619541e2d605 0 {e008cf2834908e5d6b0f792a9d4b0e2272260fb8} (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
> +  phases -- 'sortdict()'
>   
>     $ hg unbundle .hg/strip-backup/e016b03fd86f-*-backup.hg
>     adding changesets
> diff --git a/tests/test-rebase-conflicts.t b/tests/test-rebase-conflicts.t
> --- a/tests/test-rebase-conflicts.t
> +++ b/tests/test-rebase-conflicts.t
> @@ -281,16 +281,18 @@
>     list of changesets:
>     e31216eec445e44352c5f01588856059466a24c9
>     2f2496ddf49d69b5ef23ad8cf9fb2e0e4faf0ac2
> -  bundle2-output-bundle: "HG20", (1 params) 1 parts total
> +  bundle2-output-bundle: "HG20", (1 params) 2 parts total
>     bundle2-output-part: "changegroup" (params: 1 mandatory 1 advisory) streamed payload
> +  bundle2-output-part: "phases" 24 bytes payload
>     saved backup bundle to $TESTTMP/issue4041/.hg/strip-backup/e31216eec445-15f7a814-backup.hg (glob)
>     3 changesets found
>     list of changesets:
>     4c9fbe56a16f30c0d5dcc40ec1a97bbe3325209c
>     19c888675e133ab5dff84516926a65672eaf04d9
>     2a7f09cac94c7f4b73ebd5cd1a62d3b2e8e336bf
> -  bundle2-output-bundle: "HG20", 1 parts total
> +  bundle2-output-bundle: "HG20", 2 parts total
>     bundle2-output-part: "changegroup" (params: 1 mandatory 1 advisory) streamed payload
> +  bundle2-output-part: "phases" 48 bytes payload
>     adding branch
>     bundle2-input-bundle: with-transaction
>     bundle2-input-part: "changegroup" (params: 1 mandatory 1 advisory) supported
> @@ -303,7 +305,9 @@
>     adding f1.txt revisions
>     added 2 changesets with 2 changes to 1 files
>     bundle2-input-part: total payload size 1686
> -  bundle2-input-bundle: 0 parts total
> +  bundle2-input-part: "phases" supported
> +  bundle2-input-part: total payload size 48
> +  bundle2-input-bundle: 1 parts total
>     updating the branch cache
>     invalid branchheads cache (served): tip differs
>     rebase completed
> diff --git a/tests/test-strip.t b/tests/test-strip.t
> --- a/tests/test-strip.t
> +++ b/tests/test-strip.t
> @@ -213,6 +213,7 @@
>     Stream params: sortdict([('Compression', 'BZ')])
>     changegroup -- "sortdict([('version', '02'), ('nbchanges', '1')])"
>         264128213d290d868c54642d13aeaa3675551a78
> +  phases -- 'sortdict()'
>     $ hg pull .hg/strip-backup/*
>     pulling from .hg/strip-backup/264128213d29-0b39d6bf-backup.hg
>     searching for changes
> @@ -839,8 +840,9 @@
>     list of changesets:
>     6625a516847449b6f0fa3737b9ba56e9f0f3032c
>     d8db9d1372214336d2b5570f20ee468d2c72fa8b
> -  bundle2-output-bundle: "HG20", (1 params) 1 parts total
> +  bundle2-output-bundle: "HG20", (1 params) 2 parts total
>     bundle2-output-part: "changegroup" (params: 1 mandatory 1 advisory) streamed payload
> +  bundle2-output-part: "phases" 24 bytes payload
>     saved backup bundle to $TESTTMP/issue4736/.hg/strip-backup/6625a5168474-345bb43d-backup.hg (glob)
>     updating the branch cache
>     invalid branchheads cache (served): tip differs
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
>
via Mercurial-devel - June 22, 2017, 5:07 p.m.
On Thu, Jun 22, 2017 at 9:03 AM, Pierre-Yves David
<pierre-yves.david@ens-lyon.org> wrote:
>
>
> On 06/20/2017 07:00 PM, Martin von Zweigbergk via Mercurial-devel wrote:
>>
>> # HG changeset patch
>> # User Martin von Zweigbergk <martinvonz@google.com>
>> # Date 1497510952 25200
>> #      Thu Jun 15 00:15:52 2017 -0700
>> # Node ID 64f74cbf6d070ddcf3a373561ede2cf6ad333f36
>> # Parent  04fa2d783f067765f9d8cf0d1b02fae1819c232e
>> strip: include phases in bundle (BC)
>
>
> That one seems fine (but see patch 2 for issue on patch 2)
>
> If you get the energy to adds better support the new phase part in the
> 'debugbundle' that would be nice. having better data will increase the tests
> strength.

Good idea. Will do.

Patch

diff --git a/mercurial/repair.py b/mercurial/repair.py
--- a/mercurial/repair.py
+++ b/mercurial/repair.py
@@ -49,7 +49,11 @@ 
         bundletype = "HG10UN"
 
     outgoing = discovery.outgoing(repo, missingroots=bases, missingheads=heads)
-    contentopts = {'cg.version': cgversion, 'obsolescence': obsolescence}
+    contentopts = {
+        'cg.version': cgversion,
+        'obsolescence': obsolescence,
+        'phases': True,
+    }
     return bundle2.writenewbundle(repo.ui, repo, 'strip', name, bundletype,
                                   outgoing, contentopts, vfs, compression=comp)
 
@@ -194,6 +198,7 @@ 
             deleteobsmarkers(repo.obsstore, stripobsidx)
             del repo.obsstore
 
+        repo._phasecache.filterunknown(repo)
         if tmpbundlefile:
             ui.note(_("adding branch\n"))
             f = vfs.open(tmpbundlefile, "rb")
diff --git a/tests/test-generaldelta.t b/tests/test-generaldelta.t
--- a/tests/test-generaldelta.t
+++ b/tests/test-generaldelta.t
@@ -157,5 +157,6 @@ 
   Stream params: sortdict([('Compression', 'BZ')])
   changegroup -- "sortdict([('version', '02'), ('nbchanges', '1')])"
       1c5d4dc9a8b8d6e1750966d343e94db665e7a1e9
+  phases -- 'sortdict()'
 
   $ cd ..
diff --git a/tests/test-obsolete.t b/tests/test-obsolete.t
--- a/tests/test-obsolete.t
+++ b/tests/test-obsolete.t
@@ -1239,6 +1239,7 @@ 
   obsmarkers -- 'sortdict()'
       version: 1 (70 bytes)
       f27abbcc1f77fb409cf9160482fe619541e2d605 0 {e008cf2834908e5d6b0f792a9d4b0e2272260fb8} (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
+  phases -- 'sortdict()'
 
   $ hg pull .hg/strip-backup/e008cf283490-*-backup.hg
   pulling from .hg/strip-backup/e008cf283490-39c978dc-backup.hg
@@ -1283,6 +1284,7 @@ 
       version: 1 (139 bytes)
       e008cf2834908e5d6b0f792a9d4b0e2272260fb8 b0551702f918510f01ae838ab03a463054c67b46 0 (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
       f27abbcc1f77fb409cf9160482fe619541e2d605 0 {e008cf2834908e5d6b0f792a9d4b0e2272260fb8} (Thu Jan 01 00:00:00 1970 +0000) {'user': 'test'}
+  phases -- 'sortdict()'
 
   $ hg unbundle .hg/strip-backup/e016b03fd86f-*-backup.hg
   adding changesets
diff --git a/tests/test-rebase-conflicts.t b/tests/test-rebase-conflicts.t
--- a/tests/test-rebase-conflicts.t
+++ b/tests/test-rebase-conflicts.t
@@ -281,16 +281,18 @@ 
   list of changesets:
   e31216eec445e44352c5f01588856059466a24c9
   2f2496ddf49d69b5ef23ad8cf9fb2e0e4faf0ac2
-  bundle2-output-bundle: "HG20", (1 params) 1 parts total
+  bundle2-output-bundle: "HG20", (1 params) 2 parts total
   bundle2-output-part: "changegroup" (params: 1 mandatory 1 advisory) streamed payload
+  bundle2-output-part: "phases" 24 bytes payload
   saved backup bundle to $TESTTMP/issue4041/.hg/strip-backup/e31216eec445-15f7a814-backup.hg (glob)
   3 changesets found
   list of changesets:
   4c9fbe56a16f30c0d5dcc40ec1a97bbe3325209c
   19c888675e133ab5dff84516926a65672eaf04d9
   2a7f09cac94c7f4b73ebd5cd1a62d3b2e8e336bf
-  bundle2-output-bundle: "HG20", 1 parts total
+  bundle2-output-bundle: "HG20", 2 parts total
   bundle2-output-part: "changegroup" (params: 1 mandatory 1 advisory) streamed payload
+  bundle2-output-part: "phases" 48 bytes payload
   adding branch
   bundle2-input-bundle: with-transaction
   bundle2-input-part: "changegroup" (params: 1 mandatory 1 advisory) supported
@@ -303,7 +305,9 @@ 
   adding f1.txt revisions
   added 2 changesets with 2 changes to 1 files
   bundle2-input-part: total payload size 1686
-  bundle2-input-bundle: 0 parts total
+  bundle2-input-part: "phases" supported
+  bundle2-input-part: total payload size 48
+  bundle2-input-bundle: 1 parts total
   updating the branch cache
   invalid branchheads cache (served): tip differs
   rebase completed
diff --git a/tests/test-strip.t b/tests/test-strip.t
--- a/tests/test-strip.t
+++ b/tests/test-strip.t
@@ -213,6 +213,7 @@ 
   Stream params: sortdict([('Compression', 'BZ')])
   changegroup -- "sortdict([('version', '02'), ('nbchanges', '1')])"
       264128213d290d868c54642d13aeaa3675551a78
+  phases -- 'sortdict()'
   $ hg pull .hg/strip-backup/*
   pulling from .hg/strip-backup/264128213d29-0b39d6bf-backup.hg
   searching for changes
@@ -839,8 +840,9 @@ 
   list of changesets:
   6625a516847449b6f0fa3737b9ba56e9f0f3032c
   d8db9d1372214336d2b5570f20ee468d2c72fa8b
-  bundle2-output-bundle: "HG20", (1 params) 1 parts total
+  bundle2-output-bundle: "HG20", (1 params) 2 parts total
   bundle2-output-part: "changegroup" (params: 1 mandatory 1 advisory) streamed payload
+  bundle2-output-part: "phases" 24 bytes payload
   saved backup bundle to $TESTTMP/issue4736/.hg/strip-backup/6625a5168474-345bb43d-backup.hg (glob)
   updating the branch cache
   invalid branchheads cache (served): tip differs