Patchwork util: limit output chunk size in zlib decompression

login
register
mail settings
Submitter Gregory Szorc
Date Nov. 26, 2016, 5:07 p.m.
Message ID <378edaf9515996d0cfe0.1480180060@ubuntu-vm-main>
Download mbox | patch
Permalink /patch/17760/
State Accepted
Headers show

Comments

Gregory Szorc - Nov. 26, 2016, 5:07 p.m.
# HG changeset patch
# User Gregory Szorc <gregory.szorc@gmail.com>
# Date 1480180031 28800
#      Sat Nov 26 09:07:11 2016 -0800
# Node ID 378edaf9515996d0cfe04636b88980d8bd70a613
# Parent  906a7d8e969552536fffe0df7a5e63bf5d79b34b
util: limit output chunk size in zlib decompression

This is essentially a port of 65bd4b8e48bd, which was inadvertently
dropped by 8cd7d0fefd30.
Yuya Nishihara - Nov. 27, 2016, 9:08 a.m.
On Sat, 26 Nov 2016 09:07:40 -0800, Gregory Szorc wrote:
> # HG changeset patch
> # User Gregory Szorc <gregory.szorc@gmail.com>
> # Date 1480180031 28800
> #      Sat Nov 26 09:07:11 2016 -0800
> # Node ID 378edaf9515996d0cfe04636b88980d8bd70a613
> # Parent  906a7d8e969552536fffe0df7a5e63bf5d79b34b
> util: limit output chunk size in zlib decompression

Queued, thanks.

Patch

diff --git a/mercurial/util.py b/mercurial/util.py
--- a/mercurial/util.py
+++ b/mercurial/util.py
@@ -3124,7 +3124,10 @@  class _zlibengine(compressionengine):
         def gen():
             d = zlib.decompressobj()
             for chunk in filechunkiter(fh):
-                yield d.decompress(chunk)
+                while chunk:
+                    # Limit output size to limit memory.
+                    yield d.decompress(chunk, 2 ** 18)
+                    chunk = d.unconsumed_tail
 
         return chunkbuffer(gen())