Patchwork D3845: worker: support more return types in posix worker

login
register
mail settings
Submitter phabricator
Date July 1, 2018, 10:43 p.m.
Message ID <cffa3b69453c329587275ce1043fcd47@localhost.localdomain>
Download mbox | patch
Permalink /patch/32559/
State Not Applicable
Headers show

Comments

phabricator - July 1, 2018, 10:43 p.m.
hooper updated this revision to Diff 9401.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D3845?vs=9364&id=9401

REVISION DETAIL
  https://phab.mercurial-scm.org/D3845

AFFECTED FILES
  mercurial/worker.py

CHANGE DETAILS




To: hooper, #hg-reviewers
Cc: yuja, durin42, mercurial-devel
Yuya Nishihara - July 3, 2018, 12:37 p.m.
> +        while True:
> +            try:
> +                yield cbor.load(fp)
> +            except EOFError:
> +                break

Unfortunately this doesn't work because the cbor decoder doesn't care for
EOF. It tries to raise CBORDEcodeError and fail at `fp.tell()`. We'll have
to either fix the upstream cbor library or duplicate some parts to cborutil.
(or add an extra length field to feed a single chunk to `cbor.loads()`.)

This makes me feel that pickle is "okay" tool. @durin42, any idea?
phabricator - July 3, 2018, 12:38 p.m.
yuja added a comment.


  > +        while True:
  >  +            try:
  >  +                yield cbor.load(fp)
  >  +            except EOFError:
  >  +                break
  
  Unfortunately this doesn't work because the cbor decoder doesn't care for
  EOF. It tries to raise CBORDEcodeError and fail at `fp.tell()`. We'll have
  to either fix the upstream cbor library or duplicate some parts to cborutil.
  (or add an extra length field to feed a single chunk to `cbor.loads()`.)
  
  This makes me feel that pickle is "okay" tool. @durin42, any idea?

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D3845

To: hooper, #hg-reviewers
Cc: yuja, durin42, mercurial-devel

Patch

diff --git a/mercurial/worker.py b/mercurial/worker.py
--- a/mercurial/worker.py
+++ b/mercurial/worker.py
@@ -15,6 +15,9 @@ 
 import time
 
 from .i18n import _
+from .thirdparty import (
+    cbor,
+)
 from . import (
     encoding,
     error,
@@ -155,8 +158,8 @@ 
 
                 def workerfunc():
                     os.close(rfd)
-                    for i, item in func(*(staticargs + (pargs,))):
-                        os.write(wfd, '%d %s\n' % (i, item))
+                    for result in func(*(staticargs + (pargs,))):
+                        os.write(wfd, cbor.dumps(result))
                     return 0
 
                 ret = scmutil.callcatch(ui, workerfunc)
@@ -187,9 +190,15 @@ 
                 os.kill(os.getpid(), -status)
             sys.exit(status)
     try:
-        for line in util.iterfile(fp):
-            l = line.split(' ', 1)
-            yield int(l[0]), l[1][:-1]
+        while True:
+            try:
+                yield cbor.load(fp)
+            except EOFError:
+                break
+            except IOError as e:
+                if e.errno == errno.EINTR:
+                    continue
+                raise
     except: # re-raises
         killworkers()
         cleanup()