Patchwork [2,of,4] i18n: omit redundant warnings of GNU xgettext command

login
register
mail settings
Submitter Katsunori FUJIWARA
Date Aug. 2, 2018, 5:21 p.m.
Message ID <b2d92c607d9af9945a03.1533230510@blacknile>
Download mbox | patch
Permalink /patch/33116/
State Accepted
Headers show

Comments

Katsunori FUJIWARA - Aug. 2, 2018, 5:21 p.m.
# HG changeset patch
# User FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
# Date 1533224280 -32400
#      Fri Aug 03 00:38:00 2018 +0900
# Node ID b2d92c607d9af9945a03cd2e5d781021bfc80de1
# Parent  984894e4ad78ef5799a6974d079d45bcf5a64f05
# Available At https://bitbucket.org/foozy/mercurial-wip
#              hg pull https://bitbucket.org/foozy/mercurial-wip -r b2d92c607d9a
# EXP-Topic refactor-update-pot
i18n: omit redundant warnings of GNU xgettext command

At "make update-pot", GNU xgettext command shows many warning messages
like below, but these are meaningless, because i18n-ed messages with
unnamed arguments are intentional for Mercurial.

  FILE:LINE: warning: 'msgid' format string with unnamed arguments cannot be properly localized:
                      The translator cannot reorder the arguments.
                      Please consider using a format string with named arguments,
                      and a mapping instead of a tuple for the arguments.

This patch introduces i18n/hgxgettext.py, which omits such noisy
warnings of xgettext, in order to increase readability of "make
update-pot" output.

In order to omit multi-lines warning messages of xgettext like above,
this patch chooses "implementing specific command" instead of
"filtering with POSIX standard command (sed or so)", for
maintainability and extensibility.
Yuya Nishihara - Aug. 3, 2018, 1:09 p.m.
On Fri, 03 Aug 2018 02:21:50 +0900, FUJIWARA Katsunori wrote:
> # HG changeset patch
> # User FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
> # Date 1533224280 -32400
> #      Fri Aug 03 00:38:00 2018 +0900
> # Node ID b2d92c607d9af9945a03cd2e5d781021bfc80de1
> # Parent  984894e4ad78ef5799a6974d079d45bcf5a64f05
> # Available At https://bitbucket.org/foozy/mercurial-wip
> #              hg pull https://bitbucket.org/foozy/mercurial-wip -r b2d92c607d9a
> # EXP-Topic refactor-update-pot
> i18n: omit redundant warnings of GNU xgettext command
> 
> At "make update-pot", GNU xgettext command shows many warning messages
> like below, but these are meaningless, because i18n-ed messages with
> unnamed arguments are intentional for Mercurial.
> 
>   FILE:LINE: warning: 'msgid' format string with unnamed arguments cannot be properly localized:
>                       The translator cannot reorder the arguments.
>                       Please consider using a format string with named arguments,
>                       and a mapping instead of a tuple for the arguments.

Shouldn't we fix these instead? Translators can get around them, but the issue
pointed out by xgettext is real.

Patch

diff --git a/Makefile b/Makefile
--- a/Makefile
+++ b/Makefile
@@ -147,7 +147,8 @@  i18n/hg.pot: $(PYFILES) $(DOCFILES) i18n
         # Extracting with an explicit encoding of ISO-8859-1 will make
         # xgettext "parse" and ignore them.
 	$(PYFILESCMD) | xargs \
-	  xgettext --package-name "Mercurial" \
+	  $(PYTHON) i18n/hgxgettext.py \
+	  --package-name "Mercurial" \
 	  --msgid-bugs-address "<mercurial-devel@mercurial-scm.org>" \
 	  --copyright-holder "Matt Mackall <mpm@selenic.com> and others" \
 	  --from-code ISO-8859-1 --join --sort-by-file --add-comments=i18n: \
diff --git a/i18n/hgxgettext.py b/i18n/hgxgettext.py
new file mode 100755
--- /dev/null
+++ b/i18n/hgxgettext.py
@@ -0,0 +1,73 @@ 
+#!/usr/bin/env python
+#
+# hgxgettxt - spawn GNU xgettext, and omit redundant messages for Mercurial
+#
+# Copyright 2018 FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+
+from __future__ import absolute_import, print_function
+
+import os
+import re
+import sys
+
+# omittable message patterns:
+#
+# - 'format string with unnamed arguments':
+#   this is intentional implementation
+#
+omittable = re.compile('format string with unnamed arguments').search
+
+def itermessages(lines):
+    """Iterate messages of GNU xgettext command given via lines
+
+    This function assumes that below are part of the multi-lines
+    message.
+
+    - empty line
+    - starts with space character
+    """
+    pending = ''
+    for line in lines:
+        if not line.strip() or line[0].isspace():
+            pending += line
+        else:
+            if pending:
+                yield pending
+            pending = line
+    if pending:
+        yield pending
+
+def filteroutput(stdout, stderr):
+    for message in itermessages(iter(stderr.readline, '')):
+        if not omittable(message):
+            sys.stderr.write(message)
+
+    for line in iter(stdout.readline, ''):
+        sys.stdout.write(line)
+
+if __name__ == "__main__":
+    # in order to import the Mercurial modules from the source tree
+    # where hgxgettext.py is executed
+    sys.path.insert(0, os.getcwd())
+
+    from mercurial import pycompat
+    from mercurial.utils import procutil
+
+    # join and shellquote are needed to re-use procutil.popen4(), which
+    # implies subprocess.Popen() with shell=True
+    cmd = ' '.join(procutil.shellquote(e) for e in ['xgettext'] + sys.argv[1:])
+    stdin, stdout, stderr, proc = procutil.popen4(cmd)
+
+    filteroutput(stdout, stderr)
+
+    proc.wait()
+    rc = proc.returncode
+
+    # this comes from procutil.system()
+    if pycompat.sysplatform == 'OpenVMS' and rc & 1:
+        rc = 0
+
+    sys.exit(rc)