Patchwork [3,of,5,py3] ui: convert to/from Unicode on Python 3 in ui.editor()

login
register
mail settings
Submitter Augie Fackler
Date March 21, 2017, 1:56 a.m.
Message ID <70ec92cd06b503cdba45.1490061409@augie-macbookair2.roam.corp.google.com>
Download mbox | patch
Permalink /patch/19491/
State Changes Requested
Headers show

Comments

Augie Fackler - March 21, 2017, 1:56 a.m.
# HG changeset patch
# User Augie Fackler <augie@google.com>
# Date 1489901890 14400
#      Sun Mar 19 01:38:10 2017 -0400
# Node ID 70ec92cd06b503cdba456f069c0f65bd71987a6e
# Parent  4136d2f283afab8b143a5392144652c079dedbaf
ui: convert to/from Unicode on Python 3 in ui.editor()

I considered making this I/O be done in terms of bytes, but that would
cause an observable regression for Windows users, as non-binary-mode
open does EOL conversion there. There are probably new encoding
dragons lurking here, so we may want to switch to using binary mode
and doing EOL conversion ourselves.
Yuya Nishihara - March 21, 2017, 1:44 p.m.
On Mon, 20 Mar 2017 21:56:49 -0400, Augie Fackler wrote:
> # HG changeset patch
> # User Augie Fackler <augie@google.com>
> # Date 1489901890 14400
> #      Sun Mar 19 01:38:10 2017 -0400
> # Node ID 70ec92cd06b503cdba456f069c0f65bd71987a6e
> # Parent  4136d2f283afab8b143a5392144652c079dedbaf
> ui: convert to/from Unicode on Python 3 in ui.editor()
> 
> I considered making this I/O be done in terms of bytes, but that would
> cause an observable regression for Windows users, as non-binary-mode
> open does EOL conversion there. There are probably new encoding
> dragons lurking here, so we may want to switch to using binary mode
> and doing EOL conversion ourselves.

This is lossy conversion we should avoid, but I've taken this as it will
never break things on Python 2.

Another workaround is to abuse latin-1 unicode as a fat bytes on Python 3:

  f = fdopen(fd, 'w', encoding='latin-1')
  f.write(text.decode('latin-1'))

  f = open(name, 'r', encoding='latin-1')
  f.read().encode('latin-1')

Patch

diff --git a/mercurial/ui.py b/mercurial/ui.py
--- a/mercurial/ui.py
+++ b/mercurial/ui.py
@@ -1196,7 +1196,7 @@  class ui(object):
                                       dir=rdir)
         try:
             f = os.fdopen(fd, pycompat.sysstr("w"))
-            f.write(text)
+            f.write(encoding.strfromlocal(text))
             f.close()
 
             environ = {'HGUSER': user}
@@ -1219,7 +1219,7 @@  class ui(object):
                         blockedtag='editor')
 
             f = open(name)
-            t = f.read()
+            t = encoding.strtolocal(f.read())
             f.close()
         finally:
             os.unlink(name)