Patchwork [5,of,6,RFC] share: add support "full share" suport

login
register
mail settings
Submitter Angel Ezquerra
Date Dec. 24, 2014, 12:05 p.m.
Message ID <64ef59f0bf80cb4270af.1419422728@108.1.168.192.in-addr.arpa>
Download mbox | patch
Permalink /patch/7230/
State Superseded
Headers show

Comments

Angel Ezquerra - Dec. 24, 2014, 12:05 p.m.
# HG changeset patch
# User Angel Ezquerra <angel.ezquerra@gmail.com>
# Date 1419360788 -3600
#      Tue Dec 23 19:53:08 2014 +0100
# Node ID 64ef59f0bf80cb4270af791a482e6b450754f76c
# Parent  a0853216165f7995f5b81b44c188d4c7f5661c02
share: add support "full share" suport

Add a new --full flag that makes it possible to create "full shares".

The actual full commit message comes below, but first some notes and caveats:

# NOTES:
1. This is the first step on the "subrepo store" plan that we discussed during
the mercurial 3.2 sprint
2. This is RFC because I am not certain that this is the right way to do this.
Maybe there is a better, obvious way to do it. In that case I'd love to hear
about it
3. If this approach is not completely crazy, are there any files that I should
add to the fullshareexceptions list? (see below for context)
4. hg unshare is currently unsupported
5. This revision does not handle the locking of the non-store parts of the
repository properly. That will be handled on a separate patch (which in the
final form of this series should be folded into this revision).

# Actual commit message:

A "fully shared repository" is one shares _everything_ with its source
repository including bookmarks, mq patches and configuration files. The only
difference between the source repo and the fully shared copy is that their
working directory can be different and point to a different revision,
bookmark and branch.

In order to do so we replace the regular localrepo vfs object with an altvfs
object which points to the share source except for a few key files (wsharelock,
dirstate, branch, bookmarks.current, requires, sharedpath and sharedinfull).
The idea is to only keep the files that are strictly necessary to maintain
separate working directories in the shared repository, and share everything else
with the shared source repository.
Matt Mackall - Dec. 29, 2014, 7:09 p.m.
On Wed, 2014-12-24 at 13:05 +0100, Angel Ezquerra wrote:

> A "fully shared repository" is one shares _everything_ with its source
> repository including bookmarks, mq patches and configuration files.

Really curious what the story is for sharing mq state.
Angel Ezquerra - Dec. 29, 2014, 7:40 p.m.
On Mon, Dec 29, 2014 at 8:09 PM, Matt Mackall <mpm@selenic.com> wrote:
> On Wed, 2014-12-24 at 13:05 +0100, Angel Ezquerra wrote:
>
>> A "fully shared repository" is one shares _everything_ with its source
>> repository including bookmarks, mq patches and configuration files.
>
> Really curious what the story is for sharing mq state.

As you know this is related to my "subrepo cache" proposal with which
I want to make it possible to:

- Remove unused subrepos from the working directory
- Make the .hg folder of a repository self contained, even if the
repository has subrepos

To do so, the idea is to use full shares to create a "subrepo cache"
(or maybe a better name for it is a "subrepo store"?). The actual
subrepo repositories would be stored on the subrepo "store" while
their working directories pointing to a particular revision would
still be located in the parent repository's working directory.

In that context, it seems that the whole mq patches folder should be
kept on the subrepository source, not on the share. That way if you
delete the subrepo from the working directory you can get it back just
as it were by recreating the share later on.

Obviously this is not meant to let a user create two copies of the
same repository n which it can have separate mq series. That being
said the unionvfs class this is based upon could be used for to create
shares that do not share their mq patches. It is just a matter of
changing the selection function to include the patches folder (or any
other set of files) in the share or in the source. We could devise
different flags to select what to share.

Maybe these special "full shares" that are to be used for subrepos
should not be exposed on the hg share extension command line? I kind
of like them because they let you have the working directory in one
location and the actual repo somewhere else, but I can see how they
could be a bit dangerous unless you keep the repository source at the
null revision...

Sorry for the lengthy reply. I know you are busy but I don't know how
to better explain it...

Cheers,

Angel

Patch

diff --git a/hgext/share.py b/hgext/share.py
--- a/hgext/share.py
+++ b/hgext/share.py
@@ -16,10 +16,11 @@ 
 
 @command('share',
     [('U', 'noupdate', None, _('do not create a working copy')),
-     ('B', 'bookmarks', None, _('also share bookmarks'))],
-    _('[-U] [-B] SOURCE [DEST]'),
+     ('B', 'bookmarks', None, _('also share bookmarks')),
+     ('', 'full', None, _('share in full'))],
+    _('[-U] [-B] [--full] SOURCE [DEST]'),
     norepo=True)
-def share(ui, source, dest=None, noupdate=False, bookmarks=False):
+def share(ui, source, dest=None, noupdate=False, bookmarks=False, full=False):
     """create a new shared repository
 
     Initialize a new repository and working directory that shares its
@@ -37,7 +38,8 @@ 
        the broken clone to reset it to a changeset that still exists.
     """
 
-    return hg.share(ui, source, dest, not noupdate, bookmarks)
+    return hg.share(ui, source, dest, not noupdate, bookmarks=bookmarks,
+                    fullshare=full)
 
 @command('unshare', [], '')
 def unshare(ui, repo):
@@ -46,6 +48,8 @@ 
     Copy the store data to the repo and remove the sharedpath data.
     """
 
+    if repo.fullshare:
+        raise util.Abort(_("unsharing a full repository share is unsupported"))
     if not repo.shared():
         raise util.Abort(_("this is not a shared repo"))
 
diff --git a/mercurial/hg.py b/mercurial/hg.py
--- a/mercurial/hg.py
+++ b/mercurial/hg.py
@@ -158,7 +158,7 @@ 
         return ''
     return os.path.basename(os.path.normpath(path))
 
-def share(ui, source, dest=None, update=True, bookmarks=True):
+def share(ui, source, dest=None, update=True, bookmarks=True, fullshare=False):
     '''create a shared repository'''
 
     if not islocal(source):
@@ -201,6 +201,8 @@ 
     requirements += 'shared\n'
     destvfs.write('requires', requirements)
     destvfs.write('sharedpath', sharedpath)
+    if fullshare:
+        destvfs.write('sharedinfull', '')
 
     r = repository(ui, destwvfs.base)
 
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -189,6 +189,7 @@ 
         return self.requirements[:]
 
     def __init__(self, baseui, path=None, create=False):
+        self.fullshare = False
         self.wvfs = scmutil.vfs(path, expandpath=True, realpath=True)
         self.wopener = self.wvfs
         self.root = self.wvfs.base
@@ -256,13 +257,25 @@ 
 
         self.sharedpath = self.path
         try:
-            vfs = scmutil.vfs(self.vfs.read("sharedpath").rstrip('\n'),
-                              realpath=True)
+            fullshareexceptions = ('dirstate', 'branch', 'bookmarks.current',
+                                   'requires', 'sharedpath', 'sharedinfull')
+            vfs = scmutil.altvfs(self.vfs.read("sharedpath").rstrip('\n'),
+                              realpath=True, altbase=self.path,
+                              altpaths=fullshareexceptions)
             s = vfs.base
             if not vfs.exists():
                 raise error.RepoError(
                     _('.hg/sharedpath points to nonexistent directory %s') % s)
             self.sharedpath = s
+            self.fullshare = self.vfs.exists('sharedinfull')
+            if self.fullshare:
+                # full shares are those in which all files except for the
+                # requires, sharedpath and sharedinfull files must be read from
+                # the source repository
+                self.origpath = self.path
+                self.path = self.sharedpath
+                self.vfs = vfs
+                self.opener = self.vfs
         except IOError, inst:
             if inst.errno != errno.ENOENT:
                 raise
@@ -760,7 +773,9 @@ 
 
     def shared(self):
         '''the type of shared repository (None if not shared)'''
-        if self.sharedpath != self.path:
+        if self.fullshare:
+            return 'full'
+        elif self.sharedpath != self.path:
             return 'store'
         return None
 
diff --git a/tests/test-share.t b/tests/test-share.t
--- a/tests/test-share.t
+++ b/tests/test-share.t
@@ -297,6 +297,176 @@ 
      bm4                       5:92793bfc8cad
   $ cd ..
 
+create a full share
+
+  $ hg share --full repo1 repo-fullshare
+  updating working directory
+  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
+
+  $ cd repo-fullshare
+
+full shares share the whole set of revisions
+  $ hg log -G
+  @  changeset:   5:92793bfc8cad
+  |  bookmark:    bm4
+  |  tag:         tip
+  |  user:        test
+  |  date:        Thu Jan 01 00:00:00 1970 +0000
+  |  summary:     foo in b
+  |
+  o  changeset:   4:62f4ded848e4
+  |  bookmark:    bm3
+  |  parent:      2:c2e0ac586386
+  |  user:        test
+  |  date:        Thu Jan 01 00:00:00 1970 +0000
+  |  summary:     testing shared bookmarks
+  |
+  | o  changeset:   3:b87954705719
+  |/   bookmark:    bm1
+  |    user:        test
+  |    date:        Thu Jan 01 00:00:00 1970 +0000
+  |    summary:     testing shared bookmarks
+  |
+  o  changeset:   2:c2e0ac586386
+  |  user:        test
+  |  date:        Thu Jan 01 00:00:00 1970 +0000
+  |  summary:     another file
+  |
+  o  changeset:   1:8af4dc49db9e
+  |  user:        test
+  |  date:        Thu Jan 01 00:00:00 1970 +0000
+  |  summary:     change in shared clone
+  |
+  o  changeset:   0:d3873e73d99e
+     user:        test
+     date:        Thu Jan 01 00:00:00 1970 +0000
+     summary:     init
+  
+and bookmarks
+  $ hg bookmarks
+     bm1                       3:b87954705719
+     bm3                       4:62f4ded848e4
+     bm4                       5:92793bfc8cad
+  $ hg -R ../repo1 bookmarks
+   * bm1                       3:b87954705719
+     bm3                       4:62f4ded848e4
+     bm4                       5:92793bfc8cad
+
+the source and shared repo working directories can point to different revisions
+and bookmarks
+  $ hg update bm3
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  (activating bookmark bm3)
+  $ hg log -r .
+  changeset:   4:62f4ded848e4
+  bookmark:    bm3
+  parent:      2:c2e0ac586386
+  user:        test
+  date:        Thu Jan 01 00:00:00 1970 +0000
+  summary:     testing shared bookmarks
+  
+  $ hg -R ../repo1 log -r .
+  changeset:   3:b87954705719
+  bookmark:    bm1
+  user:        test
+  date:        Thu Jan 01 00:00:00 1970 +0000
+  summary:     testing shared bookmarks
+  
+they can also commit to different branches
+  $ hg branch createdinshare
+  marked working directory as branch createdinshare
+  (branches are permanent and global, did you want a bookmark?)
+  $ hg -R ../repo1 branch
+  default
+
+and commit on top of different revisions
+  $ cat a
+  more shared bookmarks
+  $ echo a > a
+  $ hg commit -m "created in full share"
+  $ echo a > ../repo1/a
+  $ hg commit -R ../repo1 -m "created in share source"
+  $ hg log -G
+  o  changeset:   7:e0fb5f85a10b
+  |  bookmark:    bm1
+  |  tag:         tip
+  |  parent:      3:b87954705719
+  |  user:        test
+  |  date:        Thu Jan 01 00:00:00 1970 +0000
+  |  summary:     created in share source
+  |
+  | @  changeset:   6:5ea6503932c4
+  | |  branch:      createdinshare
+  | |  bookmark:    bm3
+  | |  parent:      4:62f4ded848e4
+  | |  user:        test
+  | |  date:        Thu Jan 01 00:00:00 1970 +0000
+  | |  summary:     created in full share
+  | |
+  | | o  changeset:   5:92793bfc8cad
+  | |/   bookmark:    bm4
+  | |    user:        test
+  | |    date:        Thu Jan 01 00:00:00 1970 +0000
+  | |    summary:     foo in b
+  | |
+  | o  changeset:   4:62f4ded848e4
+  | |  parent:      2:c2e0ac586386
+  | |  user:        test
+  | |  date:        Thu Jan 01 00:00:00 1970 +0000
+  | |  summary:     testing shared bookmarks
+  | |
+  o |  changeset:   3:b87954705719
+  |/   user:        test
+  |    date:        Thu Jan 01 00:00:00 1970 +0000
+  |    summary:     testing shared bookmarks
+  |
+  o  changeset:   2:c2e0ac586386
+  |  user:        test
+  |  date:        Thu Jan 01 00:00:00 1970 +0000
+  |  summary:     another file
+  |
+  o  changeset:   1:8af4dc49db9e
+  |  user:        test
+  |  date:        Thu Jan 01 00:00:00 1970 +0000
+  |  summary:     change in shared clone
+  |
+  o  changeset:   0:d3873e73d99e
+     user:        test
+     date:        Thu Jan 01 00:00:00 1970 +0000
+     summary:     init
+  
+
+full shares should have a limited set of files in its .hg folder
+
+  $ ls .hg
+  bookmarks.current
+  branch
+  dirstate
+  requires
+  sharedinfull
+  sharedpath
+
+  $ ls ../repo1/.hg
+  00changelog.i
+  bookmarks
+  bookmarks.current
+  branch
+  cache
+  dirstate
+  last-message.txt
+  requires
+  store
+  undo.bookmarks
+  undo.branch
+  undo.desc
+  undo.dirstate
+
+unsharing a full share is unsupported
+
+  $ hg unshare
+  abort: unsharing a full repository share is unsupported
+  [255]
+
 Explicitly kill daemons to let the test exit on Windows
 
   $ "$TESTDIR/killdaemons.py" $DAEMON_PIDS