Patchwork [2,of,4] schemas: add wireproto implementations of schema passing

login
register
mail settings
Submitter Durham Goode
Date Aug. 7, 2013, 12:28 a.m.
Message ID <ef29e2e8de0198d5041f.1375835307@dev350.prn1.facebook.com>
Download mbox | patch
Permalink /patch/2012/
State Changes Requested
Headers show

Comments

Durham Goode - Aug. 7, 2013, 12:28 a.m.
# HG changeset patch
# User Durham Goode <durham@fb.com>
# Date 1375827497 25200
#      Tue Aug 06 15:18:17 2013 -0700
# Node ID ef29e2e8de0198d5041f60e9cbfc7b5a29fea93d
# Parent  f6fca02b697a67bec6e6ca2c9a48d7c7d7f1d077
schemas: add wireproto implementations of schema passing

This adds the wire protocol logic for requesting and transmitting
schema data. It is transmitted as a series of lines:
"%s %s\n" % (schema name, url)

'*' is used as the args for the api in case we want to change the signature
later.
Augie Fackler - Aug. 7, 2013, 1:24 p.m.
On Tue, Aug 06, 2013 at 05:28:27PM -0700, Durham Goode wrote:
> # HG changeset patch
> # User Durham Goode <durham@fb.com>
> # Date 1375827497 25200
> #      Tue Aug 06 15:18:17 2013 -0700
> # Node ID ef29e2e8de0198d5041f60e9cbfc7b5a29fea93d
> # Parent  f6fca02b697a67bec6e6ca2c9a48d7c7d7f1d077
> schemas: add wireproto implementations of schema passing

Why this instead of some pushkey love? This feels like a very good fit
for pushkey and a 3rd-party extension as a way to test the idea.

>
> This adds the wire protocol logic for requesting and transmitting
> schema data. It is transmitted as a series of lines:
> "%s %s\n" % (schema name, url)
>
> '*' is used as the args for the api in case we want to change the signature
> later.
>
> diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
> --- a/mercurial/wireproto.py
> +++ b/mercurial/wireproto.py
> @@ -293,6 +293,19 @@
>          f = self._callstream("getbundle", **opts)
>          return changegroupmod.unbundle10(self._decompress(f), 'UN')
>
> +    @batchable
> +    def schemas(self):
> +        f = future()
> +        yield {}, f
> +        d = f.value
> +        lines = d.splitlines()
> +        schemas = {}
> +        for line in lines:
> +            key, value = line.split(' ', 1)
> +            schemas[key] = value
> +
> +        yield schemas
> +
>      def unbundle(self, cg, heads, source):
>          '''Send cg (a readable file-like object representing the
>          changegroup to push, typically a chunkbuffer object) to the
> @@ -525,6 +538,13 @@
>                       encoding.tolocal(old), new)
>      return '%s\n' % int(r)
>
> +def schemas(repo, proto, others):
> +    schemas = repo.schemas
> +    serialized = ""
> +    for k, v in schemas.iteritems():
> +        serialized += "%s %s\n" % (k, v)
> +    return serialized
> +
>  def _allowstream(ui):
>      return ui.configbool('server', 'uncompressed', True, untrusted=True)
>
> @@ -660,6 +680,7 @@
>      'listkeys': (listkeys, 'namespace'),
>      'lookup': (lookup, 'key'),
>      'pushkey': (pushkey, 'namespace key old new'),
> +    'schemas' : (schemas, '*'),
>      'stream_out': (stream, ''),
>      'unbundle': (unbundle, 'heads'),
>  }
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel
Durham Goode - Aug. 7, 2013, 4:49 p.m.
On 8/7/13 6:24 AM, "Augie Fackler" <raf@durin42.com> wrote:

>On Tue, Aug 06, 2013 at 05:28:27PM -0700, Durham Goode wrote:
>> # HG changeset patch
>> # User Durham Goode <durham@fb.com>
>> # Date 1375827497 25200
>> #      Tue Aug 06 15:18:17 2013 -0700
>> # Node ID ef29e2e8de0198d5041f60e9cbfc7b5a29fea93d
>> # Parent  f6fca02b697a67bec6e6ca2c9a48d7c7d7f1d077
>> schemas: add wireproto implementations of schema passing
>
>Why this instead of some pushkey love? This feels like a very good fit
>for pushkey and a 3rd-party extension as a way to test the idea.

I didn't use pushkey because you never actually push schemas back to the
server.  But now that I look at the pushkey code more closely I see
there's precedence for that already (listing namespaces).  I'll switch to
that.

>
>>
>> This adds the wire protocol logic for requesting and transmitting
>> schema data. It is transmitted as a series of lines:
>> "%s %s\n" % (schema name, url)
>>
>> '*' is used as the args for the api in case we want to change the
>>signature
>> later.
>>
>> diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
>> --- a/mercurial/wireproto.py
>> +++ b/mercurial/wireproto.py
>> @@ -293,6 +293,19 @@
>>          f = self._callstream("getbundle", **opts)
>>          return changegroupmod.unbundle10(self._decompress(f), 'UN')
>>
>> +    @batchable
>> +    def schemas(self):
>> +        f = future()
>> +        yield {}, f
>> +        d = f.value
>> +        lines = d.splitlines()
>> +        schemas = {}
>> +        for line in lines:
>> +            key, value = line.split(' ', 1)
>> +            schemas[key] = value
>> +
>> +        yield schemas
>> +
>>      def unbundle(self, cg, heads, source):
>>          '''Send cg (a readable file-like object representing the
>>          changegroup to push, typically a chunkbuffer object) to the
>> @@ -525,6 +538,13 @@
>>                       encoding.tolocal(old), new)
>>      return '%s\n' % int(r)
>>
>> +def schemas(repo, proto, others):
>> +    schemas = repo.schemas
>> +    serialized = ""
>> +    for k, v in schemas.iteritems():
>> +        serialized += "%s %s\n" % (k, v)
>> +    return serialized
>> +
>>  def _allowstream(ui):
>>      return ui.configbool('server', 'uncompressed', True,
>>untrusted=True)
>>
>> @@ -660,6 +680,7 @@
>>      'listkeys': (listkeys, 'namespace'),
>>      'lookup': (lookup, 'key'),
>>      'pushkey': (pushkey, 'namespace key old new'),
>> +    'schemas' : (schemas, '*'),
>>      'stream_out': (stream, ''),
>>      'unbundle': (unbundle, 'heads'),
>>  }
>> _______________________________________________
>> Mercurial-devel mailing list
>> Mercurial-devel@selenic.com
>> http://selenic.com/mailman/listinfo/mercurial-devel
>
Matt Mackall - Aug. 7, 2013, 9:25 p.m.
On Wed, 2013-08-07 at 16:49 +0000, Durham Goode wrote:
> On 8/7/13 6:24 AM, "Augie Fackler" <raf@durin42.com> wrote:
> 
> >On Tue, Aug 06, 2013 at 05:28:27PM -0700, Durham Goode wrote:
> >> # HG changeset patch
> >> # User Durham Goode <durham@fb.com>
> >> # Date 1375827497 25200
> >> #      Tue Aug 06 15:18:17 2013 -0700
> >> # Node ID ef29e2e8de0198d5041f60e9cbfc7b5a29fea93d
> >> # Parent  f6fca02b697a67bec6e6ca2c9a48d7c7d7f1d077
> >> schemas: add wireproto implementations of schema passing
> >
> >Why this instead of some pushkey love? This feels like a very good fit
> >for pushkey and a 3rd-party extension as a way to test the idea.
> 
> I didn't use pushkey because you never actually push schemas back to the
> server.  But now that I look at the pushkey code more closely I see
> there's precedence for that already (listing namespaces).  I'll switch to
> that.

+1.
Martin Geisler - Aug. 9, 2013, 11:40 a.m.
Augie Fackler <raf@durin42.com> writes:

> On Tue, Aug 06, 2013 at 05:28:27PM -0700, Durham Goode wrote:
>> # HG changeset patch
>> # User Durham Goode <durham@fb.com>
>> # Date 1375827497 25200
>> #      Tue Aug 06 15:18:17 2013 -0700
>> # Node ID ef29e2e8de0198d5041f60e9cbfc7b5a29fea93d
>> # Parent  f6fca02b697a67bec6e6ca2c9a48d7c7d7f1d077
>> schemas: add wireproto implementations of schema passing
>
> Why this instead of some pushkey love? This feels like a very good fit
> for pushkey and a 3rd-party extension as a way to test the idea.

Pushkey is the approach taken by the projrc extension and it shows that
it's possible to write this as an extension.

Like Angel, I would like to see some core mechanism for distributing
config settings. It need not be the way projrc does it -- since it was
written as an extension, it (ab)uses the pushkey concept and it also
hooks into some low-level stuff to inject the settings.

Having better support for that in core would simplify the projrc
extension and perhaps make it possible to write a small extension for
distributing what you call "schemas" in case the full (and somewhat
scary) projrc functionality isn't needed.
Durham Goode - Aug. 9, 2013, 9:20 p.m.
On 8/9/13 4:40 AM, "Martin Geisler" <martin@geisler.net> wrote:

>Augie Fackler <raf@durin42.com> writes:
>
>> On Tue, Aug 06, 2013 at 05:28:27PM -0700, Durham Goode wrote:
>>> # HG changeset patch
>>> # User Durham Goode <durham@fb.com>
>>> # Date 1375827497 25200
>>> #      Tue Aug 06 15:18:17 2013 -0700
>>> # Node ID ef29e2e8de0198d5041f60e9cbfc7b5a29fea93d
>>> # Parent  f6fca02b697a67bec6e6ca2c9a48d7c7d7f1d077
>>> schemas: add wireproto implementations of schema passing
>>
>> Why this instead of some pushkey love? This feels like a very good fit
>> for pushkey and a 3rd-party extension as a way to test the idea.
>
>Pushkey is the approach taken by the projrc extension and it shows that
>it's possible to write this as an extension.
>
>Like Angel, I would like to see some core mechanism for distributing
>config settings. It need not be the way projrc does it -- since it was
>written as an extension, it (ab)uses the pushkey concept and it also
>hooks into some low-level stuff to inject the settings.
>
>Having better support for that in core would simplify the projrc
>extension and perhaps make it possible to write a small extension for
>distributing what you call "schemas" in case the full (and somewhat
>scary) projrc functionality isn't needed.

Just brainstorming, but what if we had a projrc-like extension where the
settings were kept in a hgrc style file (.hg/remoterc or whatever) but not
included in the normal config lookup.  So any time mercurial or an
extension wanted to look something up from it they would have to
explicitly run something like ui.configRemote(Š).  So it's a bit more
generic than schemas, but hooks/aliases/etc couldn't be slipped in (unless
you enabled a malicious extension).

That's basically what my schemas stuff is (a key/value map) but you'd look
it up via ui.configRemote(k) instead of repo.schemas[k]
Martin Geisler - Aug. 9, 2013, 10:56 p.m.
Durham Goode <durham@fb.com> writes:

> On 8/9/13 4:40 AM, "Martin Geisler" <martin@geisler.net> wrote:
>
>>Augie Fackler <raf@durin42.com> writes:
>>
>>> On Tue, Aug 06, 2013 at 05:28:27PM -0700, Durham Goode wrote:
>>>> # HG changeset patch
>>>> # User Durham Goode <durham@fb.com>
>>>> # Date 1375827497 25200
>>>> #      Tue Aug 06 15:18:17 2013 -0700
>>>> # Node ID ef29e2e8de0198d5041f60e9cbfc7b5a29fea93d
>>>> # Parent  f6fca02b697a67bec6e6ca2c9a48d7c7d7f1d077
>>>> schemas: add wireproto implementations of schema passing
>>>
>>> Why this instead of some pushkey love? This feels like a very good fit
>>> for pushkey and a 3rd-party extension as a way to test the idea.
>>
>>Pushkey is the approach taken by the projrc extension and it shows that
>>it's possible to write this as an extension.
>>
>>Like Angel, I would like to see some core mechanism for distributing
>>config settings. It need not be the way projrc does it -- since it was
>>written as an extension, it (ab)uses the pushkey concept and it also
>>hooks into some low-level stuff to inject the settings.
>>
>>Having better support for that in core would simplify the projrc
>>extension and perhaps make it possible to write a small extension for
>>distributing what you call "schemas" in case the full (and somewhat
>>scary) projrc functionality isn't needed.
>
> Just brainstorming, but what if we had a projrc-like extension where the
> settings were kept in a hgrc style file (.hg/remoterc or whatever) but not
> included in the normal config lookup.  So any time mercurial or an
> extension wanted to look something up from it they would have to
> explicitly run something like ui.configRemote(Š).  So it's a bit more
> generic than schemas, but hooks/aliases/etc couldn't be slipped in (unless
> you enabled a malicious extension).

That sounds nice and it would reduce the job of projrc to load some
(whitelisted) settings using ui.configremote.

Patch

diff --git a/mercurial/wireproto.py b/mercurial/wireproto.py
--- a/mercurial/wireproto.py
+++ b/mercurial/wireproto.py
@@ -293,6 +293,19 @@ 
         f = self._callstream("getbundle", **opts)
         return changegroupmod.unbundle10(self._decompress(f), 'UN')
 
+    @batchable
+    def schemas(self):
+        f = future()
+        yield {}, f
+        d = f.value
+        lines = d.splitlines()
+        schemas = {}
+        for line in lines:
+            key, value = line.split(' ', 1)
+            schemas[key] = value
+
+        yield schemas
+
     def unbundle(self, cg, heads, source):
         '''Send cg (a readable file-like object representing the
         changegroup to push, typically a chunkbuffer object) to the
@@ -525,6 +538,13 @@ 
                      encoding.tolocal(old), new)
     return '%s\n' % int(r)
 
+def schemas(repo, proto, others):
+    schemas = repo.schemas
+    serialized = ""
+    for k, v in schemas.iteritems():
+        serialized += "%s %s\n" % (k, v)
+    return serialized
+
 def _allowstream(ui):
     return ui.configbool('server', 'uncompressed', True, untrusted=True)
 
@@ -660,6 +680,7 @@ 
     'listkeys': (listkeys, 'namespace'),
     'lookup': (lookup, 'key'),
     'pushkey': (pushkey, 'namespace key old new'),
+    'schemas' : (schemas, '*'),
     'stream_out': (stream, ''),
     'unbundle': (unbundle, 'heads'),
 }