Patchwork [STABLE] merge: mark .hgsubstate as possibly dirty before submerge for consistency

login
register
mail settings
Submitter Katsunori FUJIWARA
Date Jan. 29, 2015, 8:15 p.m.
Message ID <6becb9dbca25057c6186.1422562503@juju>
Download mbox | patch
Permalink /patch/7575/
State Accepted
Commit 6becb9dbca25057c6186e255a48dd2c2ce5701a5
Headers show

Comments

Katsunori FUJIWARA - Jan. 29, 2015, 8:15 p.m.
# HG changeset patch
# User FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
# Date 1422561545 -32400
#      Fri Jan 30 04:59:05 2015 +0900
# Branch stable
# Node ID 6becb9dbca25057c6186e255a48dd2c2ce5701a5
# Parent  8a544fb645bb00995f5a0836b3a8aab0852c3e49
merge: mark .hgsubstate as possibly dirty before submerge for consistency

Before this patch, failure of updating subrepos may cause inconsistent
".hgsubstate". For example:

  1. dirstate entry for ".hgsubstate" of the parent repo is filled
     with valid size/date (via "hg state" or so)

  2. "hg update" is invoked at the parent repo

  3. ".hgsubstate" of the parent repo is updated on the filesystem as
     a part of "g"(et) action in "merge.applyupdates"

  4. it is assumed that size/date of ".hgsubstate" on the filesystem
     aren't changed from ones at (1)

     this is not so difficult condition, because just changing hash
     ids (every ids are same in length) in ".hgsubstate" doesn't
     change the file size of it

  5. "subrepo.submerge()" is invoked to update subrepos

  6. failure of updating in one of subrepos raises exception
     (e.g. "untracked file differs")

  7. "hg update" is aborted without updating dirstate of the parent repo

     dirstate entry for ".hgsubstate" still holds size/date at (1)

Then, ".hgsubstate" of the parent repo is treated as "CLEAN"
unexpectedly, because updating ".hgsubstate" at (3) doesn't change
size/date of it on the filesystem: see assumption at (4).

This inconsistent ".hgsubstate" status causes unexpected behavior, for
example:

  - "hg revert" forgets to revert ".hgsubstate"

  - "hg update" misunderstands that (not yet updated) subrepos diverge
    (then, it shows the prompt to confirm user's decision)

To avoid inconsistent ".hgsubstate" status above, this patch marks
".hgsubstate" as possibly dirty before "submerge" invocation.
"normallookup"-ed (= dirty) dirstate should be written out, even if
processing is aborted by failure.

This patch marks ".hgsubstate" as possibly dirty before "submerge",
also when it is removed or merged while merging, for safety. This
should prevent Mercurial from misunderstanding inconsistent
".hgsubstate" as clean.

To satisfy conditions at (1) and (4) above, this patch uses "hg status
--config debug.dirstate.delaywrite=2" (to fill valid size/date into
dirstate) and "touch" (to fix date of the file).
Matt Mackall - Jan. 29, 2015, 10 p.m.
On Fri, 2015-01-30 at 05:15 +0900, FUJIWARA Katsunori wrote:
> # HG changeset patch
> # User FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
> # Date 1422561545 -32400
> #      Fri Jan 30 04:59:05 2015 +0900
> # Branch stable
> # Node ID 6becb9dbca25057c6186e255a48dd2c2ce5701a5
> # Parent  8a544fb645bb00995f5a0836b3a8aab0852c3e49
> merge: mark .hgsubstate as possibly dirty before submerge for consistency

Queued for stable, thanks.
Katsunori FUJIWARA - Jan. 29, 2015, 11:10 p.m.
At Thu, 29 Jan 2015 16:00:46 -0600,
Matt Mackall wrote:
> 
> On Fri, 2015-01-30 at 05:15 +0900, FUJIWARA Katsunori wrote:
> > # HG changeset patch
> > # User FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
> > # Date 1422561545 -32400
> > #      Fri Jan 30 04:59:05 2015 +0900
> > # Branch stable
> > # Node ID 6becb9dbca25057c6186e255a48dd2c2ce5701a5
> > # Parent  8a544fb645bb00995f5a0836b3a8aab0852c3e49
> > merge: mark .hgsubstate as possibly dirty before submerge for consistency
> 
> Queued for stable, thanks.

I just focused on ".hgsubstate", because I accidentally met a test
failure for ".hgsubstate" inconsistency.

But after posting this patch, I hit on that this may not be safety
enough, because other normal files can also satisfy these conditions
at any failures (also other than "submerge": for example, I/O error at
writing merge status out)

In fact, should we do "normallookup" also for files to be
updated("g")/merged("m") for consistency, as a prevent for failures ?

> -- 
> Mathematics is the supreme nostalgia of our time.
> 
> 
> 


----------------------------------------------------------------------
[FUJIWARA Katsunori]                             foozy@lares.dti.ne.jp
Matt Mackall - Jan. 30, 2015, 10:46 p.m.
On Fri, 2015-01-30 at 08:10 +0900, FUJIWARA Katsunori wrote:
> At Thu, 29 Jan 2015 16:00:46 -0600,
> Matt Mackall wrote:
> > 
> > On Fri, 2015-01-30 at 05:15 +0900, FUJIWARA Katsunori wrote:
> > > # HG changeset patch
> > > # User FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
> > > # Date 1422561545 -32400
> > > #      Fri Jan 30 04:59:05 2015 +0900
> > > # Branch stable
> > > # Node ID 6becb9dbca25057c6186e255a48dd2c2ce5701a5
> > > # Parent  8a544fb645bb00995f5a0836b3a8aab0852c3e49
> > > merge: mark .hgsubstate as possibly dirty before submerge for consistency
> > 
> > Queued for stable, thanks.
> 
> I just focused on ".hgsubstate", because I accidentally met a test
> failure for ".hgsubstate" inconsistency.
> 
> But after posting this patch, I hit on that this may not be safety
> enough, because other normal files can also satisfy these conditions
> at any failures (also other than "submerge": for example, I/O error at
> writing merge status out)
> 
> In fact, should we do "normallookup" also for files to be
> updated("g")/merged("m") for consistency, as a prevent for failures ?

Scary, but not new. I don't want to get into this before the release.

Patch

diff --git a/mercurial/merge.py b/mercurial/merge.py
--- a/mercurial/merge.py
+++ b/mercurial/merge.py
@@ -741,7 +741,15 @@  def applyupdates(repo, actions, wctx, mc
 
     numupdates = sum(len(l) for m, l in actions.items() if m != 'k')
 
+    def dirtysubstate():
+        # mark '.hgsubstate' as possibly dirty forcibly, because
+        # modified '.hgsubstate' is misunderstood as clean,
+        # when both st_size/st_mtime of '.hgsubstate' aren't changed,
+        # even if "submerge" fails and '.hgsubstate' is inconsistent
+        repo.dirstate.normallookup('.hgsubstate')
+
     if [a for a in actions['r'] if a[0] == '.hgsubstate']:
+        dirtysubstate()
         subrepo.submerge(repo, wctx, mctx, wctx, overwrite)
 
     # remove in parallel (must come first)
@@ -760,6 +768,7 @@  def applyupdates(repo, actions, wctx, mc
     updated = len(actions['g'])
 
     if [a for a in actions['g'] if a[0] == '.hgsubstate']:
+        dirtysubstate()
         subrepo.submerge(repo, wctx, mctx, wctx, overwrite)
 
     # forget (manifest only, just log it) (must come first)
@@ -785,6 +794,7 @@  def applyupdates(repo, actions, wctx, mc
         z += 1
         progress(_updating, z, item=f, total=numupdates, unit=_files)
         if f == '.hgsubstate': # subrepo states need updating
+            dirtysubstate()
             subrepo.submerge(repo, wctx, mctx, wctx.ancestor(mctx),
                              overwrite)
             continue
diff --git a/tests/test-subrepo.t b/tests/test-subrepo.t
--- a/tests/test-subrepo.t
+++ b/tests/test-subrepo.t
@@ -758,6 +758,8 @@  test if untracked file is not overwritte
 
   $ echo issue3276_ok > repo/s/b
   $ hg -R repo2 push -f -q
+  $ touch -t 200001010000 repo/.hgsubstate
+  $ hg -R repo status --config debug.dirstate.delaywrite=2 repo/.hgsubstate
   $ hg -R repo update
   b: untracked file differs
   abort: untracked files in working directory differ from files in requested revision (in subrepo s)
@@ -766,6 +768,7 @@  test if untracked file is not overwritte
   $ cat repo/s/b
   issue3276_ok
   $ rm repo/s/b
+  $ touch -t 200001010000 repo/.hgsubstate
   $ hg -R repo revert --all
   reverting repo/.hgsubstate (glob)
   reverting subrepo s