Patchwork [3,of,4,RFC] chgcache: report repo paths from worker to master

login
register
mail settings
Submitter Jun Wu
Date Feb. 9, 2017, 1:41 a.m.
Message ID <88c498ad9318df115d84.1486604463@x1c>
Download mbox | patch
Permalink /patch/18358/
State Changes Requested
Delegated to: Yuya Nishihara
Headers show

Comments

Jun Wu - Feb. 9, 2017, 1:41 a.m.
# HG changeset patch
# User Jun Wu <quark@fb.com>
# Date 1486603691 28800
#      Wed Feb 08 17:28:11 2017 -0800
# Node ID 88c498ad9318df115d8408412a38edbac7d92a6a
# Parent  8410c4a670ffff3bed4b459cf8d62bd32fdcb1e7
# Available At https://bitbucket.org/quark-zju/hg-draft
#              hg pull https://bitbucket.org/quark-zju/hg-draft -r 88c498ad9318
chgcache: report repo paths from worker to master

This patch makes chg workers report repo paths accessed to the chg master,
via the IPC utility added by the last patch.

In additional of repo paths, we also send out timestamps, so the master
server could avoid preloading a same repo if it was accessed frequently and
was queued many times in the ipc channel.
Yuya Nishihara - Feb. 13, 2017, 2:03 p.m.
On Wed, 8 Feb 2017 17:41:03 -0800, Jun Wu wrote:
> # HG changeset patch
> # User Jun Wu <quark@fb.com>
> # Date 1486603691 28800
> #      Wed Feb 08 17:28:11 2017 -0800
> # Node ID 88c498ad9318df115d8408412a38edbac7d92a6a
> # Parent  8410c4a670ffff3bed4b459cf8d62bd32fdcb1e7
> # Available At https://bitbucket.org/quark-zju/hg-draft
> #              hg pull https://bitbucket.org/quark-zju/hg-draft -r 88c498ad9318
> chgcache: report repo paths from worker to master

This and the last patch look good as a start.

Patch

diff --git a/hgext/chgcache.py b/hgext/chgcache.py
--- a/hgext/chgcache.py
+++ b/hgext/chgcache.py
@@ -13,4 +13,13 @@  from __future__ import absolute_import
 
 import socket
+import sys
+import time
+
+from mercurial import (
+    chgserver,
+    dispatch,
+    extensions,
+    localrepo,
+)
 
 class socketipc(object):
@@ -64,2 +73,38 @@  class socketipc(object):
             except socket.error:
                 pass
+
+# -- shared by both the forked worker, and the master -------------------------
+
+_ipc = socketipc()
+
+# -- only used by the forked worker process -----------------------------------
+
+_repopaths = set() # repo paths to send to the master after runcommand
+
+def _repoinit(orig, self, baseui, path=None, create=False):
+    if path and not create:
+        _repopaths.add(path)
+    orig(self, baseui, path, create)
+
+def _runcommand(orig, self):
+    result = orig(self)
+    now = time.time()
+    for path in _repopaths:
+        _ipc.send('%s %s' % (now, path))
+    return result
+
+# -----------------------------------------------------------------------------
+
+def extsetup(ui):
+    # this extension is a no-op if chg is not used
+    # TODO(quark): change "extsetup" to "chgsetup", and discard the hacker
+    # check, after the related chg change is done.
+    if 'chgunix' not in dispatch._earlygetopt(['--cmdserver'], sys.argv[:]):
+        return
+
+    # collect paths of repos being accessed
+    extensions.wrapfunction(localrepo.localrepository, '__init__', _repoinit)
+
+    # let chg worker report repo paths to chg master, after a command completes
+    caps = chgserver.chgcmdserver.capabilities
+    caps['runcommand'] = extensions.bind(_runcommand, caps['runcommand'])