Patchwork [1,of,4,V2] schemas: add schemas to repositories

login
register
mail settings
Submitter Durham Goode
Date Aug. 8, 2013, 11:31 p.m.
Message ID <c683c90d00b124f985f4.1376004705@dev350.prn1.facebook.com>
Download mbox | patch
Permalink /patch/2097/
State Deferred
Headers show

Comments

Durham Goode - Aug. 8, 2013, 11:31 p.m.
# HG changeset patch
# User Durham Goode <durham@fb.com>
# Date 1375827328 25200
#      Tue Aug 06 15:15:28 2013 -0700
# Node ID c683c90d00b124f985f4d8b3b51c65210317eb38
# Parent  3e34e7b223d10bbe8814f82d7a1f53575fe09096
schemas: add schemas to repositories

This adds the concept of schemas to repositories. A schema is a key/value pair
indicating alternative means of accessing the remote repository. For example,
the largefiles extension might provide a schema such as:

largefiles = http://some.cdn.com/myrepo

When a user interacts with the repository, if the client knows how to handle
the largefiles schema, it will read the url and obtain largefiles from the
url. Thus letting you serve large files from a CDN.

The key difference between this and normal hgrc config options is that it is
copied across to the client during clone/pull. So if a client clones from
another client, they will also see the available schemas from the original
server. The logic for clone/pull will happen in a later patch.

This change enables an extension we're working on that will keep all of the
file history on the server, and clients will all have shallow repos.
Matt Mackall - Aug. 9, 2013, 1:51 a.m.
On Thu, 2013-08-08 at 16:31 -0700, Durham Goode wrote:
> # HG changeset patch
> # User Durham Goode <durham@fb.com>
> # Date 1375827328 25200
> #      Tue Aug 06 15:15:28 2013 -0700
> # Node ID c683c90d00b124f985f4d8b3b51c65210317eb38
> # Parent  3e34e7b223d10bbe8814f82d7a1f53575fe09096
> schemas: add schemas to repositories

I think persisting schemas to disk is a bit premature. Also, in general,
I think schemas will be a property of the server, not the client.
Sometimes the client will use the schema when it's not doing an 'online'
operation, but all the existing schemas (ssh, http(s), etc.) don't.

Let's start with a patch that adds the schema namespace and a (possibly
trivial) way to register a schema with it.
Augie Fackler - Aug. 9, 2013, 3:05 p.m.
On Thu, Aug 08, 2013 at 08:51:33PM -0500, Matt Mackall wrote:
> On Thu, 2013-08-08 at 16:31 -0700, Durham Goode wrote:
> > # HG changeset patch
> > # User Durham Goode <durham@fb.com>
> > # Date 1375827328 25200
> > #      Tue Aug 06 15:15:28 2013 -0700
> > # Node ID c683c90d00b124f985f4d8b3b51c65210317eb38
> > # Parent  3e34e7b223d10bbe8814f82d7a1f53575fe09096
> > schemas: add schemas to repositories
>
> I think persisting schemas to disk is a bit premature. Also, in general,
> I think schemas will be a property of the server, not the client.
> Sometimes the client will use the schema when it's not doing an 'online'
> operation, but all the existing schemas (ssh, http(s), etc.) don't.
>
> Let's start with a patch that adds the schema namespace and a (possibly
> trivial) way to register a schema with it.

I'd still like to see this done as an out-of-tree extension until we
can get a better sense of how it's going to work, rather than
immediately commiting the mainline hg project to keeping it around.

>
> --
> Mathematics is the supreme nostalgia of our time.
>
>
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel
Durham Goode - Aug. 9, 2013, 9:09 p.m.
On 8/9/13 8:05 AM, "Augie Fackler" <raf@durin42.com> wrote:

>On Thu, Aug 08, 2013 at 08:51:33PM -0500, Matt Mackall wrote:
>> On Thu, 2013-08-08 at 16:31 -0700, Durham Goode wrote:
>> > # HG changeset patch
>> > # User Durham Goode <durham@fb.com>
>> > # Date 1375827328 25200
>> > #      Tue Aug 06 15:15:28 2013 -0700
>> > # Node ID c683c90d00b124f985f4d8b3b51c65210317eb38
>> > # Parent  3e34e7b223d10bbe8814f82d7a1f53575fe09096
>> > schemas: add schemas to repositories
>>
>> I think persisting schemas to disk is a bit premature. Also, in general,
>> I think schemas will be a property of the server, not the client.
>> Sometimes the client will use the schema when it's not doing an 'online'
>> operation, but all the existing schemas (ssh, http(s), etc.) don't.
>>
>> Let's start with a patch that adds the schema namespace and a (possibly
>> trivial) way to register a schema with it.

Persisting the schema to disk is the most important part of this for me.
I need access to the schema information when users do any number of
operations (update, rebase, etc).  Are you saying don't persist them to
disk by default, but in my extension I can wrap clone/pull then fetch the
schemas, and register to disk the ones that I am interested in?

>
>I'd still like to see this done as an out-of-tree extension until we
>can get a better sense of how it's going to work, rather than
>immediately commiting the mainline hg project to keeping it around.

I can make that happen.

>
>>
>> --
>> Mathematics is the supreme nostalgia of our time.
>>
>>
>> _______________________________________________
>> Mercurial-devel mailing list
>> Mercurial-devel@selenic.com
>> http://selenic.com/mailman/listinfo/mercurial-devel
>

Patch

diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -661,6 +661,47 @@ 
             bt[bn] = self._branchtip(heads)
         return bt
 
+    @repofilecache('schemas')
+    def schemas(self):
+        try:
+            file = self.opener('schemas')
+        except IOError, inst:
+            if inst.errno != errno.ENOENT:
+                raise
+            return {}
+
+        try:
+            schemas = {}
+            lines = file.readlines()
+            for line in lines:
+                if line.find('=') > 0:
+                    k, v = line.split('=', 1)
+                    schemas[k.strip()] = v[:-1].strip()
+
+            return schemas
+        finally:
+            file.close()
+
+    def addschemas(self, schemas):
+        existingschemas = self.schemas
+        existingschemas.update(schemas)
+        try:
+            file = self.opener('schemas', 'w')
+        except IOError, inst:
+            if inst.errno != errno.ENOENT:
+                raise
+            return
+
+        try:
+            serialized = ""
+            for k, v in existingschemas.iteritems():
+                if len(k) > 0 and len(v) > 0:
+                    serialized += "%s = %s\n" % (k, v)
+
+            file.write(serialized)
+        finally:
+            file.close()
+
     def lookup(self, key):
         return self[key].node()