Patchwork [3,of,3] obsolete: explode if metadata contains invalid UTF-8 sequence (API)

login
register
mail settings
Submitter Yuya Nishihara
Date July 16, 2018, 10:49 a.m.
Message ID <d113293cbe05e5026bf0.1531738151@mimosa>
Download mbox | patch
Permalink /patch/32862/
State Accepted
Headers show

Comments

Yuya Nishihara - July 16, 2018, 10:49 a.m.
# HG changeset patch
# User Yuya Nishihara <yuya@tcha.org>
# Date 1531647137 -32400
#      Sun Jul 15 18:32:17 2018 +0900
# Node ID d113293cbe05e5026bf0359ace3790ea2f3d2f95
# Parent  f34a53a6a9a0df0458669d5ce6a55a4af85e9cb6
obsolete: explode if metadata contains invalid UTF-8 sequence (API)

The current metadata API can be a source of bugs since it forces callers to
process encoding conversion by themselves. So let's make it reject bad data
as a last ditch. I assume there's no metadata field which is supposed to store
arbitrary BLOB like transplant_source.
Augie Fackler - July 16, 2018, 5:22 p.m.
On Mon, Jul 16, 2018 at 07:49:11PM +0900, Yuya Nishihara wrote:
> # HG changeset patch
> # User Yuya Nishihara <yuya@tcha.org>
> # Date 1531647137 -32400
> #      Sun Jul 15 18:32:17 2018 +0900
> # Node ID d113293cbe05e5026bf0359ace3790ea2f3d2f95
> # Parent  f34a53a6a9a0df0458669d5ce6a55a4af85e9cb6
> obsolete: explode if metadata contains invalid UTF-8 sequence (API)

queued, thanks

Patch

diff --git a/mercurial/obsolete.py b/mercurial/obsolete.py
--- a/mercurial/obsolete.py
+++ b/mercurial/obsolete.py
@@ -80,6 +80,7 @@  from . import (
     obsutil,
     phases,
     policy,
+    pycompat,
     util,
 )
 from .utils import dateutil
@@ -600,6 +601,16 @@  class obsstore(object):
             raise ValueError(_('in-marker cycle with %s') % node.hex(prec))
 
         metadata = tuple(sorted(metadata.iteritems()))
+        for k, v in metadata:
+            try:
+                # might be better to reject non-ASCII keys
+                k.decode('utf-8')
+                v.decode('utf-8')
+            except UnicodeDecodeError:
+                raise error.ProgrammingError(
+                    'obsstore metadata must be valid UTF-8 sequence '
+                    '(key = %r, value = %r)'
+                    % (pycompat.bytestr(k), pycompat.bytestr(v)))
 
         marker = (bytes(prec), tuple(succs), int(flag), metadata, date, parents)
         return bool(self.add(transaction, [marker]))