Patchwork D10696: remotefilelog: prevent file download to deadlock on windows

login
register
mail settings
Submitter phabricator
Date May 7, 2021, 1:02 p.m.
Message ID <differential-rev-PHID-DREV-5lwixkbdkowwj7nuveqe-req@mercurial-scm.org>
Download mbox | patch
Permalink /patch/49015/
State Superseded
Headers show

Comments

phabricator - May 7, 2021, 1:02 p.m.
klevesque created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REVISION SUMMARY
  When fetching a big file on Windows, the buffer gets full, which results on a
  deadlock. To prevent this, we fetch the data hunk by chunk until all the data
  has been retrieved.

REPOSITORY
  rHG Mercurial

BRANCH
  stable

REVISION DETAIL
  https://phab.mercurial-scm.org/D10696

AFFECTED FILES
  hgext/remotefilelog/fileserverclient.py

CHANGE DETAILS




To: klevesque, #hg-reviewers
Cc: mercurial-patches, mercurial-devel

Patch

diff --git a/hgext/remotefilelog/fileserverclient.py b/hgext/remotefilelog/fileserverclient.py
--- a/hgext/remotefilelog/fileserverclient.py
+++ b/hgext/remotefilelog/fileserverclient.py
@@ -495,7 +495,7 @@ 
                 _(b"connection closed early"),
             )
         size = int(line)
-        data = pipe.read(size)
+        data = _readhugestream(pipe, size)
         if len(data) != size:
             raise error.ResponseError(
                 _(b"error downloading file contents:"),
@@ -664,3 +664,21 @@ 
             b'excess remotefilelog fetching:\n%s\n',
             b''.join(pycompat.sysbytes(s) for s in traceback.format_stack()),
         )
+
+
+if pycompat.iswindows:
+    # read() deadlock on Windows when size is too big
+    def _readhugestream(stream, size):
+        maxchunksize = 4 * 1024
+        data = bytearray()
+
+        while len(data) < size:
+            datachunk = stream.read(min(size - len(data), maxchunksize))
+            data += datachunk
+            if not datachunk:
+                # Prevents a deadlock in case the pipe is empty
+                break
+        return pycompat.bytestr(data)
+else:
+    def _readhugestream(stream, size):
+        return stream.read(size)