Patchwork [3,of,8] clone: add an argument to determine if a clone should stream files

login
register
mail settings
Submitter Durham Goode
Date May 6, 2013, 7:36 p.m.
Message ID <e5c4416f076c1a99ea99.1367869013@dev350.prn1.facebook.com>
Download mbox | patch
Permalink /patch/1558/
State Superseded, archived
Headers show

Comments

Durham Goode - May 6, 2013, 7:36 p.m.
# HG changeset patch
# User Durham Goode <durham@fb.com>
# Date 1367429921 25200
#      Wed May 01 10:38:41 2013 -0700
# Node ID e5c4416f076c1a99ea99fe83da9a009a15ca33ea
# Parent  4da152c208d0f853bd03781a5d7a9b9cc8716a19
clone: add an argument to determine if a clone should stream files

This adds an argument to wireproto.stream to specify if a streaming clone should stream
file contents. This allows an extension to prevent file contents from being
streamed.
Matt Mackall - May 6, 2013, 7:59 p.m.
On Mon, 2013-05-06 at 12:36 -0700, Durham Goode wrote:
> # HG changeset patch
> # User Durham Goode <durham@fb.com>
> # Date 1367429921 25200
> #      Wed May 01 10:38:41 2013 -0700
> # Node ID e5c4416f076c1a99ea99fe83da9a009a15ca33ea
> # Parent  4da152c208d0f853bd03781a5d7a9b9cc8716a19
> clone: add an argument to determine if a clone should stream files
> 
> This adds an argument to wireproto.stream to specify if a streaming clone should stream
> file contents. This allows an extension to prevent file contents from being
> streamed.

Adding arguments to existing commands doesn't work with ssh unless the
command is already spec'ed to accept varargs, ie has a '*' here:

http://www.selenic.com/hg/file/a718a0ba6787/mercurial/wireproto.py#l638

..and we can't add '*' to existing commands.
Durham Goode - May 6, 2013, 8:39 p.m.
On 5/6/13 12:59 PM, "Matt Mackall" <mpm@selenic.com> wrote:

>On Mon, 2013-05-06 at 12:36 -0700, Durham Goode wrote:
>> # HG changeset patch
>> # User Durham Goode <durham@fb.com>
>> # Date 1367429921 25200
>> #      Wed May 01 10:38:41 2013 -0700
>> # Node ID e5c4416f076c1a99ea99fe83da9a009a15ca33ea
>> # Parent  4da152c208d0f853bd03781a5d7a9b9cc8716a19
>> clone: add an argument to determine if a clone should stream files
>> 
>> This adds an argument to wireproto.stream to specify if a streaming
>>clone should stream
>> file contents. This allows an extension to prevent file contents from
>>being
>> streamed.
>
>Adding arguments to existing commands doesn't work with ssh unless the
>command is already spec'ed to accept varargs, ie has a '*' here:
>
>http://www.selenic.com/hg/file/a718a0ba6787/mercurial/wireproto.py#l638
>
>..and we can't add '*' to existing commands.
>

I don't actually need this argument exposed via the wireproto.commands
network contract. The extension invokes this method with the appropriate
argument via another means.  Adding the optional argument doesn't seem to
prevent hg from calling the function using the old no-arguments network
contract.
Matt Mackall - May 6, 2013, 9:28 p.m.
On Mon, 2013-05-06 at 20:39 +0000, Durham Goode wrote:
> On 5/6/13 12:59 PM, "Matt Mackall" <mpm@selenic.com> wrote:
> 
> >On Mon, 2013-05-06 at 12:36 -0700, Durham Goode wrote:
> >> # HG changeset patch
> >> # User Durham Goode <durham@fb.com>
> >> # Date 1367429921 25200
> >> #      Wed May 01 10:38:41 2013 -0700
> >> # Node ID e5c4416f076c1a99ea99fe83da9a009a15ca33ea
> >> # Parent  4da152c208d0f853bd03781a5d7a9b9cc8716a19
> >> clone: add an argument to determine if a clone should stream files
> >> 
> >> This adds an argument to wireproto.stream to specify if a streaming
> >>clone should stream
> >> file contents. This allows an extension to prevent file contents from
> >>being
> >> streamed.
> >
> >Adding arguments to existing commands doesn't work with ssh unless the
> >command is already spec'ed to accept varargs, ie has a '*' here:
> >
> >http://www.selenic.com/hg/file/a718a0ba6787/mercurial/wireproto.py#l638
> >
> >..and we can't add '*' to existing commands.
> >
> 
> I don't actually need this argument exposed via the wireproto.commands
> network contract. The extension invokes this method with the appropriate
> argument via another means.  Adding the optional argument doesn't seem to
> prevent hg from calling the function using the old no-arguments network
> contract.

Hmm, ok. It might be better then to have a separate _stream method that
takes the extra parameter so it's clear that it can't be passed.

Patch

diff --git a/mercurial/store.py b/mercurial/store.py
--- a/mercurial/store.py
+++ b/mercurial/store.py
@@ -322,13 +322,16 @@ 
     def datafiles(self):
         return self._walk('data', True)
 
+    def topfiles(self):
+        return reversed(self._walk('', False))
+
     def walk(self):
         '''yields (unencoded, encoded, size)'''
         # yield data files first
         for x in self.datafiles():
             yield x
         # yield manifest before changelog
-        for x in reversed(self._walk('', False)):
+        for x in self.topfiles():
             yield x
 
     def copylist(self):
diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -523,7 +523,7 @@ 
 def _allowstream(ui):
     return ui.configbool('server', 'uncompressed', True, untrusted=True)
 
-def stream(repo, proto):
+def stream(repo, proto, includedatafiles=True):
     '''If the server supports streaming clone, it advertises the "stream"
     capability with a value representing the version and flags of the repo
     it is serving. Client checks to see if it understands the format.
@@ -544,7 +544,10 @@ 
         lock = repo.lock()
         try:
             repo.ui.debug('scanning\n')
-            for name, ename, size in repo.store.walk():
+            walkgen = repo.store.walk()
+            if not includedatafiles:
+                walkgen = repo.store.topfiles()
+            for name, ename, size in walkgen:
                 if size:
                     entries.append((name, size))
                     total_bytes += size