Submitter | Tomasz Kleczek |
---|---|
Date | Dec. 11, 2012, 6:38 p.m. |
Message ID | <5d51bf818878db395e2e.1355251097@dev408.prn1.facebook.com> |
Download | mbox | patch |
Permalink | /patch/51/ |
State | Superseded |
Headers | show |
Comments
On Tue, Dec 11, 2012 at 8:38 PM, Tomasz Kleczek <tkleczek at fb.com> wrote: > # HG changeset patch > # User Tomasz Kleczek <tkleczek at fb.com> > # Date 1355245443 28800 > # Branch stable > # Node ID 5d51bf818878db395e2e1f840769a83254c3dcfd > # Parent 13c6bcb8dd900dc7dbf5e3da9ef68d56fed250b3 > speedy: stub history query server > > Introduce a server component to which clients delegate their > history requests. Sounds like you could use the command server for this by adding new commands to it and making it work with TCP sockets. Yuya already monkey patched it to work over Unix sockets so it's worth peeking at his stuff, and in general his work on chg might help you in some way. https://bitbucket.org/yuja/chg
Thanks. One problem that i see right away is that command server protocol doesn't allow you to send newlines in the data section. Communication between speedy client and server requires sending potentially big amount of binary data over the network and I would like to avoid overhead such as base64 encoding.. On Wed, Dec 12, 2012 at 1:57 PM, Idan Kamara <idankk86 at gmail.com> wrote: > On Tue, Dec 11, 2012 at 8:38 PM, Tomasz Kleczek <tkleczek at fb.com> wrote: > > # HG changeset patch > > # User Tomasz Kleczek <tkleczek at fb.com> > > # Date 1355245443 28800 > > # Branch stable > > # Node ID 5d51bf818878db395e2e1f840769a83254c3dcfd > > # Parent 13c6bcb8dd900dc7dbf5e3da9ef68d56fed250b3 > > speedy: stub history query server > > > > Introduce a server component to which clients delegate their > > history requests. > > Sounds like you could use the command server for this by > adding new commands to it and making it work with TCP sockets. > > Yuya already monkey patched it to work over Unix sockets so it's > worth peeking at his stuff, and in general his work on chg might > help you in some way. > > https://bitbucket.org/yuja/chg > _______________________________________________ > Mercurial-devel mailing list > Mercurial-devel at selenic.com > http://selenic.com/mailman/listinfo/mercurial-devel > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20121212/9f67e2a3/attachment.html>
On 2012-12-13 00:09, Tomasz K?eczek wrote: > Thanks. One problem that i see right away is that command server > protocol doesn't allow you to send newlines in the data section. > Communication between speedy client and server requires sending > potentially big amount of binary data over the network and I would > like to avoid overhead such as base64 encoding.. > Maybe the protocol should be updated to have a "binary chunk" capability? Sounds like something that would be useful for other server-side extensions anyway. Disclaimer: http://www.peralex.com/disclaimer.html
(you're top posting) On Thu, Dec 13, 2012 at 12:09 AM, Tomasz K?eczek <tomasz.kleczek at gmail.com> wrote: > Thanks. One problem that i see right away is that command server protocol > doesn't allow you to send newlines in the data section. > Communication between speedy client and server requires sending potentially > big amount of binary data over the network and I would > like to avoid overhead such as base64 encoding.. The data section has no restrictions on what it contains. You could be confused with the hello message that has a specific format (which uses new lines to separate fields). > > > On Wed, Dec 12, 2012 at 1:57 PM, Idan Kamara <idankk86 at gmail.com> wrote: >> >> On Tue, Dec 11, 2012 at 8:38 PM, Tomasz Kleczek <tkleczek at fb.com> wrote: >> > # HG changeset patch >> > # User Tomasz Kleczek <tkleczek at fb.com> >> > # Date 1355245443 28800 >> > # Branch stable >> > # Node ID 5d51bf818878db395e2e1f840769a83254c3dcfd >> > # Parent 13c6bcb8dd900dc7dbf5e3da9ef68d56fed250b3 >> > speedy: stub history query server >> > >> > Introduce a server component to which clients delegate their >> > history requests. >> >> Sounds like you could use the command server for this by >> adding new commands to it and making it work with TCP sockets. >> >> Yuya already monkey patched it to work over Unix sockets so it's >> worth peeking at his stuff, and in general his work on chg might >> help you in some way. >> >> https://bitbucket.org/yuja/chg >> _______________________________________________ >> Mercurial-devel mailing list >> Mercurial-devel at selenic.com >> http://selenic.com/mailman/listinfo/mercurial-devel > >
On 13 Dec 2012, at 3:07 AM, Idan Kamara wrote: > (you're top posting) > > On Thu, Dec 13, 2012 at 12:09 AM, Tomasz K?eczek > <tomasz.kleczek at gmail.com> wrote: >> Thanks. One problem that i see right away is that command server protocol >> doesn't allow you to send newlines in the data section. >> Communication between speedy client and server requires sending potentially >> big amount of binary data over the network and I would >> like to avoid overhead such as base64 encoding.. > > The data section has no restrictions on what it contains. > > You could be confused with the hello message that has a specific > format (which uses new lines to separate fields). +1 for reusing a protocol we've already got. pacem in terris / ??? / ?????? / ????????? / ?? Kevin R. Bullock
Patch
diff --git a/hgext/speedy/client.py b/hgext/speedy/client.py --- a/hgext/speedy/client.py +++ b/hgext/speedy/client.py @@ -5,19 +5,35 @@ from mercurial import extensions, commands from mercurial import revset +from mercurial import localrepo +from mercurial.i18n import _ +import server -def patchedauthor(repo, subset, x): +def patchedauthor(metapeer, repo, subset, x): """Return the revisions commited by user whose name match x + metapeer: proxy object used to query the metadata server + Used to monkey patch revset.author function. """ - # In the subsequent patches here we are going to forward the query - # to the server - return revset.author(repo, subset, x) + # We want to catch errors early and on client, if possible + pat = revset.getstring(x, _("author requires a string")) + revs = set(metapeer.author(pat)) + return [r for r in subset if r in revs] def _speedysetup(ui, repo): """Initialize speedy client.""" - revset.symbols['author'] = patchedauthor + # For now, local repo and server repo are the same. This is going + # to change soon + serverrepo = localrepo.localrepository(ui, path=repo.root) + mserver = server.makeserver(serverrepo) + + def wrapwithpeer(fun, peer): + def wrapper(*args, **kwargs): + return fun(peer, *args, **kwargs) + return wrapper + + revset.symbols['author'] = wrapwithpeer(patchedauthor, mserver) def uisetup(ui): # Perform patching and most of the initialization inside log wrapper, diff --git a/hgext/speedy/server.py b/hgext/speedy/server.py new file mode 100644 --- /dev/null +++ b/hgext/speedy/server.py @@ -0,0 +1,32 @@ +# Copyright 2012 Facebook +# +# This software may be used and distributed according to the terms of the +# GNU General Public License version 2 or any later version. + +"""Server component + +metaserver: contains all the logic behind query acceleration +""" + +from mercurial import revset + +class metaserver(object): + """Contains all the logic behind the query acceleration.""" + + def __init__(self, repo): + self.repo = repo + + def author(self, pat): + """Return a list of changes commited by a user that matches pattern. + + User matches pattern if their name has a `pat` substring (case + insensitive). + + Returns a list of revs. + """ + # This is going to be accelerated in the subsequent patches + return revset.author(self.repo, list(self.repo), ('symbol', pat)) + +def makeserver(repo): + """Return an initialized metaserver instance.""" + return metaserver(repo) diff --git a/tests/test-speedy.t b/tests/test-speedy.t --- a/tests/test-speedy.t +++ b/tests/test-speedy.t @@ -12,6 +12,11 @@ $ hg init localrepo $ cd localrepo + $ cat > .hg/hgrc <<EOF_END + > [speedy] + > client = True + > EOF_END + $ mkdir d1 $ echo chg0 > d1/chg0 $ hg commit -Am chg0 -u testuser1