Patchwork D11270: issue6528: also filter delta on the fly when applying a changegroup

login
register
mail settings
Submitter phabricator
Date Aug. 7, 2021, 2:08 p.m.
Message ID <differential-rev-PHID-DREV-ofdirewcl5klec3g5exj-req@mercurial-scm.org>
Download mbox | patch
Permalink /patch/49584/
State Superseded
Headers show

Comments

phabricator - Aug. 7, 2021, 2:08 p.m.
marmoute created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REVISION SUMMARY
  This ensure that corrupted clone does not spread corruption to "fixed" version.
  
  This might come at a performance cost, we will had a config option to control
  this behavior in the next changesets.

REPOSITORY
  rHG Mercurial

BRANCH
  stable

REVISION DETAIL
  https://phab.mercurial-scm.org/D11270

AFFECTED FILES
  mercurial/filelog.py
  mercurial/revlogutils/rewrite.py
  tests/bundles/issue6528.hg-v1
  tests/bundles/issue6528.hg-v2
  tests/test-issue6528.t

CHANGE DETAILS




To: marmoute, #hg-reviewers
Cc: mercurial-patches, mercurial-devel

Patch

diff --git a/tests/test-issue6528.t b/tests/test-issue6528.t
--- a/tests/test-issue6528.t
+++ b/tests/test-issue6528.t
@@ -431,3 +431,94 @@ 
   $ hg debug-repair-issue6528
   no affected revisions were found
   $ hg st
+
+  $ cd ..
+
+Applying a bad bundle should fix it on the fly
+----------------------------------------------
+
+from a v1 bundle
+~~~~~~~~~~~~~~~~
+
+  $ hg debugbundle  --spec "$TESTDIR"/bundles/issue6528.hg-v1
+  bzip2-v1
+
+  $ hg init unbundle-v1
+  $ cd unbundle-v1
+
+  $ hg unbundle "$TESTDIR"/bundles/issue6528.hg-v1
+  adding changesets
+  adding manifests
+  adding file changes
+  added 8 changesets with 12 changes to 4 files
+  new changesets f5a5a568022f:3beabb508514 (8 drafts)
+  (run 'hg update' to get a working copy)
+
+Check that revision were fixed on the fly
+
+  $ hg debugrevlogindex b.txt
+     rev linkrev nodeid       p1           p2
+       0       2 05b806ebe5ea 000000000000 000000000000
+       1       3 a58b36ad6b65 000000000000 05b806ebe5ea
+       2       6 216a5fe8b8ed 000000000000 000000000000
+       3       7 ea4f2f2463cc 000000000000 216a5fe8b8ed
+
+  $ hg debugrevlogindex D.txt
+     rev linkrev nodeid       p1           p2
+       0       6 2a8d3833f2fb 000000000000 000000000000
+       1       7 2a80419dfc31 000000000000 2a8d3833f2fb
+
+That we don't see the symptoms of the bug
+
+  $ hg up -- -1
+  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
+
+And that the repair command does not find anything to fix
+
+  $ hg debug-repair-issue6528
+  no affected revisions were found
+
+  $ cd ..
+
+from a v2 bundle
+~~~~~~~~~~~~~~~~
+
+  $ hg debugbundle --spec "$TESTDIR"/bundles/issue6528.hg-v2
+  bzip2-v2
+
+  $ hg init unbundle-v2
+  $ cd unbundle-v2
+
+  $ hg unbundle "$TESTDIR"/bundles/issue6528.hg-v2
+  adding changesets
+  adding manifests
+  adding file changes
+  added 8 changesets with 12 changes to 4 files
+  new changesets f5a5a568022f:3beabb508514 (8 drafts)
+  (run 'hg update' to get a working copy)
+
+Check that revision were fixed on the fly
+
+  $ hg debugrevlogindex b.txt
+     rev linkrev nodeid       p1           p2
+       0       2 05b806ebe5ea 000000000000 000000000000
+       1       3 a58b36ad6b65 000000000000 05b806ebe5ea
+       2       6 216a5fe8b8ed 000000000000 000000000000
+       3       7 ea4f2f2463cc 000000000000 216a5fe8b8ed
+
+  $ hg debugrevlogindex D.txt
+     rev linkrev nodeid       p1           p2
+       0       6 2a8d3833f2fb 000000000000 000000000000
+       1       7 2a80419dfc31 000000000000 2a8d3833f2fb
+
+That we don't see the symptoms of the bug
+
+  $ hg up -- -1
+  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
+
+And that the repair command does not find anything to fix
+
+  $ hg debug-repair-issue6528
+  no affected revisions were found
+
+  $ cd ..
diff --git a/tests/bundles/issue6528.hg-v2 b/tests/bundles/issue6528.hg-v2
new file mode 100644
index 0000000000000000000000000000000000000000..7c9c362ea561838727c608886d728bd3b311c21f
GIT binary patch
literal 2406
zc$@)d37PgtM=~$~000g{Z*6dLWpi_BZ*Dz8T0&ZAIWa+5CR15;_}hd407ZZQ|NsC0
z|NsC0|NsA`|NsB_=1%*6|NsB`=l}oz@BjaQ;0OEQK76-8D0t@TfChC%RSl}DX!4B)
zhKHyRC<D|3)EWS2Wc2}{G-;-Q01SiD8Vv#JJxx78(9xsH4FCgAQIG&M&;v~kGzLbE
z4@iIj13&-(000>@01W^D0000OBSS+#0000000E;w0000D0ilCcHcdSfMA1)E(lh`7
z10W3#PyjS&008vS003wJ8UO$Q0000013{*M0000H5>wJ?gkYE^CYUAw00J^F0Wg@D
z000Sq03m@e002w?OaK4?022W*8eo_rsX}NTlSIuCwJ_6DM$l=b#KEL(0jTvf^&Uji
z0hH6wBPIr#22r#FBS;>gWN0>kP3nypCy@=Qko7S%dLsnD$eIZxMvVb5iKO*U5gw)|
zssKGlsf0Z=fuJ%PX`^WLQKLblA)^p!pkxgOfEosYhz&9UggmFH14AItGzNsCjCN%@
zZh*(?B+4x8c<eE~p{5u*PL&c+MBf&rhLSQhVN2l4hixdX)~dQ{YUjs%%Jx8Qvht&H
z$6F|t)wH;)F))<~*50mE9xDPSpxm(zaY93)7zSMfoG>wrgPA0kD9I!Pkif~19<3$^
zCT2of@LO`JHro#Ch0cSv4L)QJfiCSSMfX94Fqs^3;HM){UD|Q0_P27Zun(BK1+tW7
zWR~BPUAP+=gS=brl;%np=(bN`xRbw4AtSmune4{k0w%~rH{rkv&H|aIgNKaO(C4y&
z-hib&cgkvNA2nKCV7a$#7Z`X|<hN_*i--cXIyuG_AkT>8n2u8eg#v^ob3?To2lO?~
zI`&rRG%XVTO!P3$J^`L%I4f4`*7g?x)0{_q8+VvBqsn;!^SG*aGUCy33cCE|ear>e
z0R}I4319#a#1*SUTOed4c!(f{kR}5r1c?kjpfK4c1d0YQ7!dj}OaW_(R4Y}?X+Ro;
ziVoutc=bvVeu5Ubg#<5Z(??_GX0Yv=MyBT*S8B?{_8*1Tu&pCYB+d;)rPhUMyfWLJ
zoS=dt223-C9``xUWWx?mdobpEp2NvM7`Y(neBPqw6X*a4pbN}g02jGk<H%;0D;d-!
zwn!2pRM8RDoL5OgTpA-~AXE;9kARl3=S-yJ^irACF&b>QBuk+x9Kmiebwn;;z%ZP6
zR|>4OBVBQ{B2z5WOA=_X_n#V2u8#APDp&(w?bJR@fpxT+c481`Xp>7e%D$7><}>H~
zM@eNvT~!UDj6z_ptBJ;gXs<X%xkiZ$xx*02C3DObxXWyCsC9SM35KsT7<$S{$>p)n
zsICNLeVz<5ECP7ZeigcCQA#T!PD3p0WVadS#=d>l6T$Erwy+{X7s|#$pwO3arob{-
z5xyvp9#?&2rI*;s;Bh?p2uw<K>O&f7YE!Zgs2@87-h!2JaUeCe6=I8tAmWl7FoON1
zEIa~D<a|gatObYxCMY4YB`7K591s@7><;TJE(<EYnrf1H*t{GP6;4bK44jWq@b_G&
zCyr2|nTo>0=|3Ox);{bp%ZqkN^2q`QT*Vsm+%(q>Tpnl*7KE>fS@q3v>wdMyntK}H
zzD&x%ze@870)j&00!d^CxU&e-sSGih(U2cyW9GSefp~!MLpB49FF`<)i99YEeS%>^
z(YzBq@5Q2j^GSBf(mq3Q<BkitHd<2TB{?wlwQwI4W+nw<CwLE3Ie|(!f~KY{QxwM@
z5loTOC<kO9**v#F)cDc}gGw;wB)a=4hMvSz)?%H)Awmfs1U?x`28q8dnxg!jyG~=a
z#S`*DAV1>mpDE!l=@%nVP%VMD#UI0rF7QY$<kqa<QGxb{k_2qH^i~ku+bB<sV0tF%
z`v)%0W_UB$alu`>-HV+!{EvwZ;q68flyKj19NDtRmF4bH&84*v4Zn;G6%_0;`^qoT
zu(Lb*`=Mdu0Jq0+R>%OMBn$}IqB-UMxHh?w(@=@YwFvqpt2wKx%qMEmZdyUXkPX6W
z8&JeFwLl=q5x}5@sf4(5lpxZOwY^z_G=chB3vzowT~`LTy2Pwk<)(O|1+r0=XATO2
zXa-V?UQ87iff;P)tfVSx9#R$GZfmB)8CGKDK*@(1-C#_K>7Kjombh2zZl4+p!#zlb
z<}fpXB}1~~U{$7M3j-;|l|pyMimtkDm|argaV;eyEz!e0Js4-i8fI}>eXP~7E|ijn
z(3?16i0Ha`X}GJ%9;W<U=V@3|68)jMFeQbc`-0|iIX&3mRZw0T12x(}N(Wujt^!(?
zXsL|_+A@%pu)jH~?Kp;a>Cr_5^u{!{oCzulT4z8gC1|=}cuY|SdASdSaK)H4`<}>M
zN?ur9%3T^<Qr<SyH;XX099h$@wRJ1+t_ep7PSs1VJ@vjvsus9x9C?u44+o6xxNabq
zR-_Ez#NxQyae{D)LQd3P$xI`T{i#%}4*Nlnr1@NWP)D>)Ik4bAIFjN(VJrq-|9TS}
z&@T%EwMip~m%8|=PtV!}dcNT<pjyc!c*ANOE_mLtCDI{#uOV>D=1h%{tj#S0)oPfv
zi<)`BOC4RiQwhMb{2}v#cx{jz#ppJ7iq-_`$POHsa}9G+1v|MP=u5h>rDjQ9$O=Ox
z0YYq2c8G+Q{SKgdovp-tS|6ozpDpxIANA>Y+D`>-4rhKgDkd8`xyJ6p#MXv%C3@q@
zS+$a9nG9m0YdVImWE1j;{G}#0Ap<~*X=*%VVkD64tXzzP=_!-yf{<>-y_;&5CXJC|
zj++JXt_Wk}ze>Ug9k#$`889c=(Y{i95`HwDTt$4esM|?)0^~3k(fzINy_sLRe?1$}
z3ip9)j+MPanua}f*LNIKgj+kR9SUSt4-3+7>r}@CP@5REuya@I`0*s@%OW);7eo$8
zk*Xe9Vd`GI(KKhj0ryVf*iejq7-@-4`y#tX1_eFsu|pbFvir%N7}vDK$_*-Pu^Yy0
z+4>3T5_K&J;w6sCB~f*cJUTLRj3tU<mmCB%Ld`c<<s7pZ_ZPf67Ho83*<`nLSr8-M
zO4qGx8b5_0nG50`ktM+^%9V5v(d0F0DC*hebGwa6{4uT?r6@wQYX=Y@$uLYsz4tB!
YV`Lg;^1IjIo`3OoBvXY62Vaf2K$mtm82|tP

literal 0
Hc$@<O00001

diff --git a/tests/bundles/issue6528.hg-v1 b/tests/bundles/issue6528.hg-v1
new file mode 100644
index 0000000000000000000000000000000000000000..fcc34db8300a7a93729057f6440ca2a760cfabeb
GIT binary patch
literal 2223
zc$@*R2vGM(M=>x$T4*^jL0KkKSwm`a9RL7ofB*mg|NsC0|NsB~^QHg)|M})l`+xuc
z|NH0vumA7=|9{{IdGM^MfCFV}QCk%x0E($hBz~y!ntGqAG(4uBru9sRsCt<*QR;hA
zdTM%r^#+YSKzfZmM$%;RnWX(Gpv+0-JxA(+>R~skVGk%ar=-(Os2-rtQaq-cNwkkm
z6Vg3R7)_-8D1ZO}13)wY00E;wXahh10gyBR8W=z{(Vzg(000dD&;S4ifuPU;2~Z{i
z69`oCK+pgH007VdpaVbyKmZy513&-(000000000000;m805k(Y000^^27ojG02u>7
z0ilEgO&S0V0002c4FCWDXc`Rw0ssI24FJ#p00xZ#pbY>320+jNXkh@;Mt}oA001-t
zKmY(527^EVB#})t8kwn)Cygnmp*=EBQ`9{}Pf3D$hMQ9$4H|lCdY)4?G}9meZ6i%G
z0C_+F4Wu;nJxw$K8Us&A(?dWS1Jw0BKy7}M!9NzY{cF7In#MC-DA~4Dsu160_Q5G+
zm((Xs;V4@CIvuU=)_3e%TG&N!jo^f3vD_zcF&k4*i?aF+Sj;T|;qy3_2D}`aRo%LZ
znP`Gr$jJzel|V4!xOYKtQB!p414ea2u325MJy|)-46GX<007Jk3=EkhuY|zl!!i<(
zsFU)(aB3F-UL{a0R7@7xFa(q^xRY)<8|pDf75H)osS9L!M)-*dZFw_xl>q+@H(Xem
z%`Teve}%HhVq9`O0nKQP(&=EuX55vfBg%?`0U)R<9LHv<i%S`*N{3{P>Zs5T%$8=%
zGl@zdMygtJSu>j)j6s7uwhoCl0fL-!Se!x0C4pi$rqzmLw1zwA9myDahd!N@z1g#E
zi}c^7=#cy8B^=L`$-yn|Cj4N;bcGnvbV*a2YiBd8q&d#a5DftLvDu7#P!E#KFzrJf
zDmO=M<YD+LyeBH41i#^oXQL(f*Bxl{9qm43%B&DJ*%)-QRU9*3Mmw3Lz_B@zPO!cg
zbyj!!lH*pxlMLa9z0Px)?B|T<J%^iwRR*@=CCiF+&W`fvE;*`c56WHMMFcYBLPG{>
z$_&&~US=VdZ-TX$xq&cKaysD|>vyv8PmGY2O(Z2%cpzBX5Gf&D@Knawkhm^*u%)xA
zu!S_juz@NZ>L5A^7PB`%Qy=pXYsDzM6L&;9EDI}{8g2%qSq#O8GFu{dn6YZ78C1mC
zIHw{v1A>{@JWEp-Vg-wFk%t&w6|!$|O-34_bj4L<+Tk?fNp2YrA4N)B%348-t2m4U
zt_ViksZ*U9Af$ltx4k9`?|ZMOf>lG>wwJ1;))5@*s7YZ+WC;34V7w~`T?7;Y%5J2f
zs_K%9K||kwYl;s5VTMvnkgNjc3wlA_?28Kq0&A$9=qwo7!Q|9gQ(+>odcINAI#LN3
z2dUN(bD;tisbhk;9zWlOwgpqRQkx$1mV$wV-Z&N*GEt^`c3KWfQbZ(W2wk7TWV*0f
zZHe;5(8i4|xQnm}ZUHl)W-!N4P`RA>x}!c5avJJWfGlO`q%6Z>YXTz-1*0fnEk&=2
zA{N}$J%GAqE!G=a4-_<!DYQ|jp#H+L*RR5rprM9X^OATg_K=eZ$hnA#Y)VR1nRD?a
zLfyaHb0WVn$&i`v8@o8mV6`NK5FRWXdMKmMKSQr9$=0nA&!TdkV5Rb+0|2xXq-hY?
zMDEC6d6yOlV>U3+6{{eRxitrbT@O`_*5^i4-W`c7I^Qik)++4WpwT7>!AZslUZZfW
z&kHQt26HpPp2LrP-5L~Jnk!tmU7E3nDRgrgTXDCf**7~8iZ6^D5vy|)LvmVJkB2aM
zoMx{g1u6(3aow$?(1b2Vsgyb7dvK^kq)96wx>Tf;sWV%Tt(iKE@NiZVv9~tkDecK{
zz`{DxP)p{uy7920Qm;_7<w~B?Q-L{P4D!-KFkJ^d14>y7oS{J=b3)=YtZ+m}KV$3J
zYPJ|r#YQlvY40Nn6R6eH!uqm`lrmw)OT$&jhA*9UQ8wjYW9^A(WlaNI_L(3uLa(}z
zo>5p-7n&J1z7d_4H;WORB6{N@B^sIRnW)>2;7gLClFHJYX(-0Z13Dd@1^7ilPoh>C
ze`?#*rD3Ft%W_RXw4t>E;|STrqEvKfMG#+vVN~?B*T#@p7^zHRu%B5(lLJ%7h|iUd
z0f~6y7>|@8B;#o?l3c0+Bhv*q77ilV5`^%3Cggdb!P*6o&nG{EF;gHEq#>Ov%W}w&
zT^1h-D@B7nVj{RQrevIOW5MHO!USQE1wb(fsfzJ1iD3V5F^M8AXKuw3rU``NX`xC$
zXUQYCDGP>VtHxU80iB)`L)}6Tj)wCA8kxyT`h~0tNI{RA3*E-Wlprr4`-X^n4{0x|
zhAD<!lF8be)uF@q3%RACQ>SZY+M_<NMD}gSzOpmZF&30rAkFb-nq!`_UBUXNAZ{=r
zhD~z@Dc^sFos6BD>Jrjxdj|Vxl3Com3?c-X@vMY1CUW2`4UylCcSb<DrP8%c4+%8G
z+zw3OLMH+(&1{mEDf=y<oj4IdUZJL&18j1^^n@L$PiSFw8$)N<g;B^JsI`P!*&yB@
z!H%%eW^7RsW5k#m^jnrGYs;i$OY<p~$YMmGOVU7S(|0&onW4HJ#8X!DW+4cKO;h}-
z!y7&^NKp*Qw@TN*7zcJ`hwM3ac+EO}l_8JUEum7CYfr@+Bk3Wec@%fq;z8tb2MjhO
zYV@N|D8)JiwIVLBC}Vg@>W%<tQefi~;SO5xoi#Iaf0@9qFWR#3wkOkA6cwdm`d|;d
zaZNFz9zWEXv$j3dsU{E9f8f!ZP_AtW78MK5pOE4X0Wz1Er;a(4*BP@a#E%&yhUn6-
xJnF3XnifqVVq}=)N6;7-Z9$BtmoZAEm>{L8)1}EDI#Yk*?ntK!5(sThL!c?%+ARP8

literal 0
Hc$@<O00001

diff --git a/mercurial/revlogutils/rewrite.py b/mercurial/revlogutils/rewrite.py
--- a/mercurial/revlogutils/rewrite.py
+++ b/mercurial/revlogutils/rewrite.py
@@ -29,6 +29,7 @@ 
     ENTRY_SIDEDATA_COMPRESSED_LENGTH,
     ENTRY_SIDEDATA_COMPRESSION_MODE,
     ENTRY_SIDEDATA_OFFSET,
+    REVIDX_ISCENSORED,
     REVLOGV0,
     REVLOGV1,
 )
@@ -36,6 +37,7 @@ 
 
 from .. import (
     error,
+    mdiff,
     pycompat,
     revlogutils,
     util,
@@ -719,6 +721,88 @@ 
                 _reorder_filelog_parents(repo, fl, sorted(to_fix))
 
 
+def filter_delta_issue6528(revlog, deltas_iter):
+    """filter incomind deltas to repaire issue 6528 on the fly"""
+    metadata_cache = {}
+
+    deltacomputer = deltas.deltacomputer(revlog)
+
+    for rev, d in enumerate(deltas_iter, len(revlog)):
+        (
+            node,
+            p1_node,
+            p2_node,
+            linknode,
+            deltabase,
+            delta,
+            flags,
+            sidedata,
+        ) = d
+
+        if not revlog.index.has_node(deltabase):
+            raise error.LookupError(
+                deltabase, revlog.radix, _(b'unknown parent')
+            )
+        base_rev = revlog.rev(deltabase)
+        if not revlog.index.has_node(p1_node):
+            raise error.LookupError(p1_node, revlog.radix, _(b'unknown parent'))
+        p1_rev = revlog.rev(p1_node)
+        if not revlog.index.has_node(p2_node):
+            raise error.LookupError(p2_node, revlog.radix, _(b'unknown parent'))
+        p2_rev = revlog.rev(p2_node)
+
+        is_censored = lambda: bool(flags & REVIDX_ISCENSORED)
+        delta_base = lambda: revlog.rev(delta_base)
+        delta_base = lambda: base_rev
+        parent_revs = lambda: (p1_rev, p2_rev)
+
+        def full_text():
+            # note: being able to reuse the full text computation in the
+            # underlying addrevision would be useful however this is a bit too
+            # intrusive the for the "quick" issue6528 we are writing before the
+            # 5.8 release
+            textlen = mdiff.patchedsize(revlog.size(base_rev), delta)
+
+            revinfo = revlogutils.revisioninfo(
+                node,
+                p1_node,
+                p2_node,
+                [None],
+                textlen,
+                (base_rev, delta),
+                flags,
+            )
+            # cached by the global "writing" context
+            assert revlog._writinghandles is not None
+            if revlog._inline:
+                fh = revlog._writinghandles[0]
+            else:
+                fh = revlog._writinghandles[1]
+            return deltacomputer.buildtext(revinfo, fh)
+
+        is_affected = _is_revision_affected_fast_inner(
+            is_censored,
+            delta_base,
+            lambda: delta,
+            full_text,
+            parent_revs,
+            rev,
+            metadata_cache,
+        )
+        if is_affected:
+            d = (
+                node,
+                p2_node,
+                p1_node,
+                linknode,
+                deltabase,
+                delta,
+                flags,
+                sidedata,
+            )
+        yield d
+
+
 def repair_issue6528(
     ui, repo, dry_run=False, to_report=None, from_report=None, paranoid=False
 ):
diff --git a/mercurial/filelog.py b/mercurial/filelog.py
--- a/mercurial/filelog.py
+++ b/mercurial/filelog.py
@@ -20,6 +20,7 @@ 
 from .utils import storageutil
 from .revlogutils import (
     constants as revlog_constants,
+    rewrite,
 )
 
 
@@ -158,6 +159,9 @@ 
             )
 
         with self._revlog._writing(transaction):
+
+            deltas = rewrite.filter_delta_issue6528(self._revlog, deltas)
+
             return self._revlog.addgroup(
                 deltas,
                 linkmapper,