Patchwork [3,of,6] hgweb: set encoding to utf-8

login
register
mail settings
Submitter timeless@mozdev.org
Date Jan. 12, 2016, 6:38 p.m.
Message ID <198238d09e72b4d83f21.1452623886@waste.org>
Download mbox | patch
Permalink /patch/12699/
State Rejected
Delegated to: Yuya Nishihara
Headers show

Comments

timeless@mozdev.org - Jan. 12, 2016, 6:38 p.m.
# HG changeset patch
# User timeless <timeless@mozdev.org>
# Date 1452621820 0
#      Tue Jan 12 18:03:40 2016 +0000
# Node ID 198238d09e72b4d83f212120af8175a67e99dde2
# Parent  a4f6489df893272a532f9d9cf88b12fd0619efc6
hgweb: set encoding to utf-8
Matt Mackall - Jan. 12, 2016, 9:13 p.m.
On Tue, 2016-01-12 at 12:38 -0600, timeless wrote:
> # HG changeset patch
> # User timeless <timeless@mozdev.org>
> # Date 1452621820 0
> #      Tue Jan 12 18:03:40 2016 +0000
> # Node ID 198238d09e72b4d83f212120af8175a67e99dde2
> # Parent  a4f6489df893272a532f9d9cf88b12fd0619efc6
> hgweb: set encoding to utf-8

(I think you mean "serve" here, not hgweb, because you're only touching
commands.)

It's not an accident that we inherit the web server encoding from the locale.

If you have, say, a shift-jis or a cp1252 locale, this allows your filenames to
be displayed correctly when you run hg serve.

-- 
Mathematics is the supreme nostalgia of our time.
timeless - Jan. 13, 2016, 12:49 a.m.
On Tue, Jan 12, 2016 at 4:13 PM, Matt Mackall <mpm@selenic.com> wrote:
> On Tue, 2016-01-12 at 12:38 -0600, timeless wrote:
>> # HG changeset patch
>> # User timeless <timeless@mozdev.org>
>> # Date 1452621820 0
>> #      Tue Jan 12 18:03:40 2016 +0000
>> # Node ID 198238d09e72b4d83f212120af8175a67e99dde2
>> # Parent  a4f6489df893272a532f9d9cf88b12fd0619efc6
>> hgweb: set encoding to utf-8
>
> (I think you mean "serve" here, not hgweb, because you're only touching
> commands.)

I dithered over calling It serve (which I did originally) and hgweb.

> It's not an accident that we inherit the web server encoding from the locale.
>
> If you have, say, a shift-jis or a cp1252 locale, this allows your filenames to
> be displayed correctly when you run hg serve.

The current behavior results in garbage if your server is using
Japanese or similar localization.

This patch should still allow that if you pass --encoding to hg serve.

FWIW, To get localizations to display correctly will require more than
the series here, so it's ok if we don't take this step now. I can look
into it further later. This is my first round of looking at
serve/hgweb, I've mostly ignored it of late.
Yuya Nishihara - Jan. 13, 2016, 1:36 p.m.
On Tue, 12 Jan 2016 19:49:56 -0500, timeless wrote:
> On Tue, Jan 12, 2016 at 4:13 PM, Matt Mackall <mpm@selenic.com> wrote:
> > On Tue, 2016-01-12 at 12:38 -0600, timeless wrote:
> >> # HG changeset patch
> >> # User timeless <timeless@mozdev.org>
> >> # Date 1452621820 0
> >> #      Tue Jan 12 18:03:40 2016 +0000
> >> # Node ID 198238d09e72b4d83f212120af8175a67e99dde2
> >> # Parent  a4f6489df893272a532f9d9cf88b12fd0619efc6
> >> hgweb: set encoding to utf-8
> >
> > (I think you mean "serve" here, not hgweb, because you're only touching
> > commands.)
> 
> I dithered over calling It serve (which I did originally) and hgweb.
> 
> > It's not an accident that we inherit the web server encoding from the locale.
> >
> > If you have, say, a shift-jis or a cp1252 locale, this allows your filenames to
> > be displayed correctly when you run hg serve.
> 
> The current behavior results in garbage if your server is using
> Japanese or similar localization.

No. It works so long as you use Japanese environment and a repository is
in the same Japanese encoding. Please don't assume UTF-8 is capable of
everything.

> This patch should still allow that if you pass --encoding to hg serve.

Why we have to specify "--encoding cp932" only to hg serve?

Patch

diff --git a/mercurial/commands.py b/mercurial/commands.py
--- a/mercurial/commands.py
+++ b/mercurial/commands.py
@@ -6232,6 +6232,8 @@ 
     if opts["cmdserver"]:
         service = commandserver.createservice(ui, repo, opts)
     else:
+        if not "encoding" in opts:
+            encoding.encoding = 'UTF-8'
         service = hgweb.createservice(ui, repo, opts)
     return cmdutil.service(opts, initfn=service.init, runfn=service.run)
 
diff --git a/tests/test-hgweb-commands.t b/tests/test-hgweb-commands.t
--- a/tests/test-hgweb-commands.t
+++ b/tests/test-hgweb-commands.t
@@ -61,7 +61,7 @@ 
   $ get-with-headers.py 127.0.0.1:$HGPORT 'log/?style=atom'
   200 Script output follows
   
-  <?xml version="1.0" encoding="ascii"?>
+  <?xml version="1.0" encoding="UTF-8"?>
   <feed xmlns="http://www.w3.org/2005/Atom">
    <!-- Changelog -->
    <id>http://*:$HGPORT/</id> (glob)
@@ -247,7 +247,7 @@ 
   $ get-with-headers.py 127.0.0.1:$HGPORT 'log/?style=rss'
   200 Script output follows
   
-  <?xml version="1.0" encoding="ascii"?>
+  <?xml version="1.0" encoding="UTF-8"?>
   <rss version="2.0">
     <channel>
       <link>http://*:$HGPORT/</link> (glob)
@@ -421,7 +421,7 @@ 
   $ get-with-headers.py 127.0.0.1:$HGPORT 'log/1/?style=atom'
   200 Script output follows
   
-  <?xml version="1.0" encoding="ascii"?>
+  <?xml version="1.0" encoding="UTF-8"?>
   <feed xmlns="http://www.w3.org/2005/Atom">
    <!-- Changelog -->
    <id>http://*:$HGPORT/</id> (glob)
@@ -521,7 +521,7 @@ 
   $ get-with-headers.py 127.0.0.1:$HGPORT 'log/1/?style=rss'
   200 Script output follows
   
-  <?xml version="1.0" encoding="ascii"?>
+  <?xml version="1.0" encoding="UTF-8"?>
   <rss version="2.0">
     <channel>
       <link>http://*:$HGPORT/</link> (glob)
@@ -615,7 +615,7 @@ 
   $ get-with-headers.py 127.0.0.1:$HGPORT 'log/1/foo/?style=atom'
   200 Script output follows
   
-  <?xml version="1.0" encoding="ascii"?>
+  <?xml version="1.0" encoding="UTF-8"?>
   <feed xmlns="http://www.w3.org/2005/Atom">
    <id>http://*:$HGPORT/atom-log/tip/foo</id> (glob)
    <link rel="self" href="http://*:$HGPORT/atom-log/tip/foo"/> (glob)
@@ -670,7 +670,7 @@ 
   $ get-with-headers.py 127.0.0.1:$HGPORT 'log/1/foo/?style=rss'
   200 Script output follows
   
-  <?xml version="1.0" encoding="ascii"?>
+  <?xml version="1.0" encoding="UTF-8"?>
   <rss version="2.0">
     <channel>
       <link>http://*:$HGPORT/</link> (glob)
@@ -1501,7 +1501,7 @@ 
   $ get-with-headers.py 127.0.0.1:$HGPORT 'summary/?style=gitweb'
   200 Script output follows
   
-  <?xml version="1.0" encoding="ascii"?>
+  <?xml version="1.0" encoding="UTF-8"?>
   <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
   <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US" lang="en-US">
   <head>
@@ -1700,7 +1700,7 @@ 
   $ get-with-headers.py 127.0.0.1:$HGPORT 'graph/?style=gitweb'
   200 Script output follows
   
-  <?xml version="1.0" encoding="ascii"?>
+  <?xml version="1.0" encoding="UTF-8"?>
   <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
   <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US" lang="en-US">
   <head>
diff --git a/tests/test-hgweb-descend-empties.t b/tests/test-hgweb-descend-empties.t
--- a/tests/test-hgweb-descend-empties.t
+++ b/tests/test-hgweb-descend-empties.t
@@ -385,7 +385,7 @@ 
   $ get-with-headers.py 127.0.0.1:$HGPORT 'file?style=gitweb'
   200 Script output follows
   
-  <?xml version="1.0" encoding="ascii"?>
+  <?xml version="1.0" encoding="UTF-8"?>
   <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
   <html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en-US" lang="en-US">
   <head>
diff --git a/tests/test-hgweb-filelog.t b/tests/test-hgweb-filelog.t
--- a/tests/test-hgweb-filelog.t
+++ b/tests/test-hgweb-filelog.t
@@ -773,7 +773,7 @@ 
   $ (get-with-headers.py localhost:$HGPORT 'rss-log/tip/a')
   200 Script output follows
   
-  <?xml version="1.0" encoding="ascii"?>
+  <?xml version="1.0" encoding="UTF-8"?>
   <rss version="2.0">
     <channel>
       <link>http://*:$HGPORT/</link> (glob)
@@ -804,7 +804,7 @@ 
   $ (get-with-headers.py localhost:$HGPORT 'atom-log/tip/a')
   200 Script output follows
   
-  <?xml version="1.0" encoding="ascii"?>
+  <?xml version="1.0" encoding="UTF-8"?>
   <feed xmlns="http://www.w3.org/2005/Atom">
    <id>http://*:$HGPORT/atom-log/tip/a</id> (glob)
    <link rel="self" href="http://*:$HGPORT/atom-log/tip/a"/> (glob)
diff --git a/tests/test-hgweb-raw.t b/tests/test-hgweb-raw.t
--- a/tests/test-hgweb-raw.t
+++ b/tests/test-hgweb-raw.t
@@ -44,7 +44,7 @@ 
 
   $ cat getoutput.txt
   200 Script output follows
-  content-type: text/plain; charset="ascii"
+  content-type: text/plain; charset="UTF-8"
   content-length: 157
   content-disposition: inline; filename="some text%.txt"
   
diff --git a/tests/test-hgwebdir.t b/tests/test-hgwebdir.t
--- a/tests/test-hgwebdir.t
+++ b/tests/test-hgwebdir.t
@@ -746,7 +746,7 @@ 
   $ get-with-headers.py localhost:$HGPORT1 't/a?style=atom'
   200 Script output follows
   
-  <?xml version="1.0" encoding="ascii"?>
+  <?xml version="1.0" encoding="UTF-8"?>
   <feed xmlns="http://www.w3.org/2005/Atom">
    <!-- Changelog -->
    <id>http://*:$HGPORT1/t/a/</id> (glob)
@@ -803,7 +803,7 @@ 
   $ get-with-headers.py localhost:$HGPORT1 't/a/?style=atom'
   200 Script output follows
   
-  <?xml version="1.0" encoding="ascii"?>
+  <?xml version="1.0" encoding="UTF-8"?>
   <feed xmlns="http://www.w3.org/2005/Atom">
    <!-- Changelog -->
    <id>http://*:$HGPORT1/t/a/</id> (glob)