Patchwork [2,of,2] alias: expand "$@" as list of parameters quoted individually (BC) (issue4200)

login
register
mail settings
Submitter Siddharth Agarwal
Date Aug. 14, 2014, 6:54 a.m.
Message ID <38b9b34c3351935a54a2.1407999277@devbig136.prn2.facebook.com>
Download mbox | patch
Permalink /patch/5394/
State Accepted
Headers show

Comments

Siddharth Agarwal - Aug. 14, 2014, 6:54 a.m.
# HG changeset patch
# User Siddharth Agarwal <sid0@fb.com>
# Date 1407997312 25200
#      Wed Aug 13 23:21:52 2014 -0700
# Node ID 38b9b34c3351935a54a2d4da185452a753c98970
# Parent  c3fbbb727b54c25c93ef64a7353e62503e0fa441
alias: expand "$@" as list of parameters quoted individually (BC) (issue4200)

Before this patch, there was no way to pass in all the positional parameters as
separate words down to another command.

(1) $@ (without quotes) would expand to all the parameters separated by a space.
    This would work fine for arguments without spaces, but arguments with spaces
    in them would be split up by POSIX shells into separate words.
(2) '$@' (in single quotes) would expand to all the parameters within a pair of
    single quotes. POSIX shells would then treat the entire list of arguments
    as one word.
(3) "$@" (in double quotes) would expand similarly to (2).

With this patch, we expand "$@" (in double quotes) as all positional
parameters, quoted individually with util.shellquote, and separated by spaces.
Under standard field-splitting conditions, POSIX shells will tokenize each
argument into exactly one word.

This is a backwards-incompatible change, but the old behavior was arguably a
bug: Bourne-derived shells have expanded "$@" as a tokenized list of positional
parameters for a very long time. I could find this behavior specified in IEEE
Std 1003.1-2001, and this probably goes back to much further before that.
Siddharth Agarwal - Aug. 14, 2014, 7:02 a.m.
On 08/13/2014 11:54 PM, Siddharth Agarwal wrote:
> # HG changeset patch
> # User Siddharth Agarwal <sid0@fb.com>
> # Date 1407997312 25200
> #      Wed Aug 13 23:21:52 2014 -0700
> # Node ID 38b9b34c3351935a54a2d4da185452a753c98970
> # Parent  c3fbbb727b54c25c93ef64a7353e62503e0fa441
> alias: expand "$@" as list of parameters quoted individually (BC) (issue4200)

If this is too risky to be accepted, we can probably introduce another 
magic character for this. For example, $%, which isn't taken by bash or 
zsh. It'll also make this patch a lot simpler.

>
> Before this patch, there was no way to pass in all the positional parameters as
> separate words down to another command.
>
> (1) $@ (without quotes) would expand to all the parameters separated by a space.
>      This would work fine for arguments without spaces, but arguments with spaces
>      in them would be split up by POSIX shells into separate words.
> (2) '$@' (in single quotes) would expand to all the parameters within a pair of
>      single quotes. POSIX shells would then treat the entire list of arguments
>      as one word.
> (3) "$@" (in double quotes) would expand similarly to (2).
>
> With this patch, we expand "$@" (in double quotes) as all positional
> parameters, quoted individually with util.shellquote, and separated by spaces.
> Under standard field-splitting conditions, POSIX shells will tokenize each
> argument into exactly one word.
>
> This is a backwards-incompatible change, but the old behavior was arguably a
> bug: Bourne-derived shells have expanded "$@" as a tokenized list of positional
> parameters for a very long time. I could find this behavior specified in IEEE
> Std 1003.1-2001, and this probably goes back to much further before that.
>
> diff --git a/mercurial/dispatch.py b/mercurial/dispatch.py
> --- a/mercurial/dispatch.py
> +++ b/mercurial/dispatch.py
> @@ -331,6 +331,27 @@
>           args = shlex.split(cmd)
>       return args + givenargs
>   
> +def aliasinterpolate(name, args, cmd):
> +    '''interpolate args into cmd for shell aliases
> +
> +    This also handles $0, $@ and "$@".
> +    '''
> +    # util.interpolate can't deal with "$@" (with quotes) because it's only
> +    # built to match prefix + patterns.
> +    replacemap = dict(('$%d' % (i + 1), arg) for i, arg in enumerate(args))
> +    replacemap['$0'] = name
> +    replacemap['$$'] = '$'
> +    replacemap['$@'] = ' '.join(args)
> +    # Typical Unix shells interpolate "$@" (with quotes) as all the positional
> +    # parameters, separated out into words. Emulate the same behavior here by
> +    # quoting the arguments individually. POSIX shells will then typically
> +    # tokenize each argument into exactly one word.
> +    replacemap['"$@"'] = ' '.join(util.shellquote(arg) for arg in args)
> +    # escape '\$' for regex
> +    regex = '|'.join(replacemap.keys()).replace('$', r'\$')
> +    r = re.compile(regex)
> +    return r.sub(lambda x: replacemap[x.group()], cmd)
> +
>   class cmdalias(object):
>       def __init__(self, name, definition, cmdtable):
>           self.name = self.cmd = name
> @@ -376,10 +397,7 @@
>                                    % (int(m.groups()[0]), self.name))
>                           return ''
>                   cmd = re.sub(r'\$(\d+|\$)', _checkvar, self.definition[1:])
> -                replace = dict((str(i + 1), arg) for i, arg in enumerate(args))
> -                replace['0'] = self.name
> -                replace['@'] = ' '.join(args)
> -                cmd = util.interpolate(r'\$', replace, cmd, escape_prefix=True)
> +                cmd = aliasinterpolate(self.name, args, cmd)
>                   return util.system(cmd, environ=env, out=ui.fout)
>               self.fn = fn
>               return
> diff --git a/mercurial/help/config.txt b/mercurial/help/config.txt
> --- a/mercurial/help/config.txt
> +++ b/mercurial/help/config.txt
> @@ -229,8 +229,9 @@
>   Positional arguments like ``$1``, ``$2``, etc. in the alias definition
>   expand to the command arguments. Unmatched arguments are
>   removed. ``$0`` expands to the alias name and ``$@`` expands to all
> -arguments separated by a space. These expansions happen before the
> -command is passed to the shell.
> +arguments separated by a space. ``"$@"`` (with quotes) expands to all
> +arguments quoted individually and separated by a space. These expansions
> +happen before the command is passed to the shell.
>   
>   Shell aliases are executed in an environment where ``$HG`` expands to
>   the path of the Mercurial that was used to execute the alias. This is
> diff --git a/tests/test-alias.t b/tests/test-alias.t
> --- a/tests/test-alias.t
> +++ b/tests/test-alias.t
> @@ -30,6 +30,7 @@
>     > echo1 = !printf '\$1\n'
>     > echo2 = !printf '\$2\n'
>     > echo13 = !printf '\$1 \$3\n'
> +  > echotokens = !printf "%s\n" "\$@"
>     > count = !hg log -r "\$@" --template=. | wc -c | sed -e 's/ //g'
>     > mcount = !hg log \$@ --template=. | wc -c | sed -e 's/ //g'
>     > rt = root
> @@ -241,6 +242,22 @@
>     foo baz
>     $ hg echo2 foo
>     
> +  $ hg echotokens
> +
> +  $ hg echotokens foo 'bar $1 baz'
> +  foo
> +  bar $1 baz
> +  $ hg echotokens 'test $2' foo
> +  test $2
> +  foo
> +  $ hg echotokens 'test $@' foo '$@'
> +  test $@
> +  foo
> +  $@
> +  $ hg echotokens 'test "$@"' foo '"$@"'
> +  test "$@"
> +  foo
> +  "$@"
>     $ echo bar > bar
>     $ hg commit -qA -m bar
>     $ hg count .
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel
Augie Fackler - Aug. 14, 2014, 6:48 p.m.
On Thu, Aug 14, 2014 at 12:02:07AM -0700, Siddharth Agarwal wrote:
> On 08/13/2014 11:54 PM, Siddharth Agarwal wrote:
> ># HG changeset patch
> ># User Siddharth Agarwal <sid0@fb.com>
> ># Date 1407997312 25200
> >#      Wed Aug 13 23:21:52 2014 -0700
> ># Node ID 38b9b34c3351935a54a2d4da185452a753c98970
> ># Parent  c3fbbb727b54c25c93ef64a7353e62503e0fa441
> >alias: expand "$@" as list of parameters quoted individually (BC) (issue4200)
>
> If this is too risky to be accepted, we can probably introduce another magic
> character for this. For example, $%, which isn't taken by bash or zsh. It'll
> also make this patch a lot simpler.

I'm strongly in favor of $@, and folks in IRC also seemed similarly
enthusiastic. Queued.

>
> >
> >Before this patch, there was no way to pass in all the positional parameters as
> >separate words down to another command.
> >
> >(1) $@ (without quotes) would expand to all the parameters separated by a space.
> >     This would work fine for arguments without spaces, but arguments with spaces
> >     in them would be split up by POSIX shells into separate words.
> >(2) '$@' (in single quotes) would expand to all the parameters within a pair of
> >     single quotes. POSIX shells would then treat the entire list of arguments
> >     as one word.
> >(3) "$@" (in double quotes) would expand similarly to (2).
> >
> >With this patch, we expand "$@" (in double quotes) as all positional
> >parameters, quoted individually with util.shellquote, and separated by spaces.
> >Under standard field-splitting conditions, POSIX shells will tokenize each
> >argument into exactly one word.
> >
> >This is a backwards-incompatible change, but the old behavior was arguably a
> >bug: Bourne-derived shells have expanded "$@" as a tokenized list of positional
> >parameters for a very long time. I could find this behavior specified in IEEE
> >Std 1003.1-2001, and this probably goes back to much further before that.
> >
> >diff --git a/mercurial/dispatch.py b/mercurial/dispatch.py
> >--- a/mercurial/dispatch.py
> >+++ b/mercurial/dispatch.py
> >@@ -331,6 +331,27 @@
> >          args = shlex.split(cmd)
> >      return args + givenargs
> >+def aliasinterpolate(name, args, cmd):
> >+    '''interpolate args into cmd for shell aliases
> >+
> >+    This also handles $0, $@ and "$@".
> >+    '''
> >+    # util.interpolate can't deal with "$@" (with quotes) because it's only
> >+    # built to match prefix + patterns.
> >+    replacemap = dict(('$%d' % (i + 1), arg) for i, arg in enumerate(args))
> >+    replacemap['$0'] = name
> >+    replacemap['$$'] = '$'
> >+    replacemap['$@'] = ' '.join(args)
> >+    # Typical Unix shells interpolate "$@" (with quotes) as all the positional
> >+    # parameters, separated out into words. Emulate the same behavior here by
> >+    # quoting the arguments individually. POSIX shells will then typically
> >+    # tokenize each argument into exactly one word.
> >+    replacemap['"$@"'] = ' '.join(util.shellquote(arg) for arg in args)
> >+    # escape '\$' for regex
> >+    regex = '|'.join(replacemap.keys()).replace('$', r'\$')
> >+    r = re.compile(regex)
> >+    return r.sub(lambda x: replacemap[x.group()], cmd)
> >+
> >  class cmdalias(object):
> >      def __init__(self, name, definition, cmdtable):
> >          self.name = self.cmd = name
> >@@ -376,10 +397,7 @@
> >                                   % (int(m.groups()[0]), self.name))
> >                          return ''
> >                  cmd = re.sub(r'\$(\d+|\$)', _checkvar, self.definition[1:])
> >-                replace = dict((str(i + 1), arg) for i, arg in enumerate(args))
> >-                replace['0'] = self.name
> >-                replace['@'] = ' '.join(args)
> >-                cmd = util.interpolate(r'\$', replace, cmd, escape_prefix=True)
> >+                cmd = aliasinterpolate(self.name, args, cmd)
> >                  return util.system(cmd, environ=env, out=ui.fout)
> >              self.fn = fn
> >              return
> >diff --git a/mercurial/help/config.txt b/mercurial/help/config.txt
> >--- a/mercurial/help/config.txt
> >+++ b/mercurial/help/config.txt
> >@@ -229,8 +229,9 @@
> >  Positional arguments like ``$1``, ``$2``, etc. in the alias definition
> >  expand to the command arguments. Unmatched arguments are
> >  removed. ``$0`` expands to the alias name and ``$@`` expands to all
> >-arguments separated by a space. These expansions happen before the
> >-command is passed to the shell.
> >+arguments separated by a space. ``"$@"`` (with quotes) expands to all
> >+arguments quoted individually and separated by a space. These expansions
> >+happen before the command is passed to the shell.
> >  Shell aliases are executed in an environment where ``$HG`` expands to
> >  the path of the Mercurial that was used to execute the alias. This is
> >diff --git a/tests/test-alias.t b/tests/test-alias.t
> >--- a/tests/test-alias.t
> >+++ b/tests/test-alias.t
> >@@ -30,6 +30,7 @@
> >    > echo1 = !printf '\$1\n'
> >    > echo2 = !printf '\$2\n'
> >    > echo13 = !printf '\$1 \$3\n'
> >+  > echotokens = !printf "%s\n" "\$@"
> >    > count = !hg log -r "\$@" --template=. | wc -c | sed -e 's/ //g'
> >    > mcount = !hg log \$@ --template=. | wc -c | sed -e 's/ //g'
> >    > rt = root
> >@@ -241,6 +242,22 @@
> >    foo baz
> >    $ hg echo2 foo
> >+  $ hg echotokens
> >+
> >+  $ hg echotokens foo 'bar $1 baz'
> >+  foo
> >+  bar $1 baz
> >+  $ hg echotokens 'test $2' foo
> >+  test $2
> >+  foo
> >+  $ hg echotokens 'test $@' foo '$@'
> >+  test $@
> >+  foo
> >+  $@
> >+  $ hg echotokens 'test "$@"' foo '"$@"'
> >+  test "$@"
> >+  foo
> >+  "$@"
> >    $ echo bar > bar
> >    $ hg commit -qA -m bar
> >    $ hg count .
> >_______________________________________________
> >Mercurial-devel mailing list
> >Mercurial-devel@selenic.com
> >http://selenic.com/mailman/listinfo/mercurial-devel
>
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel

Patch

diff --git a/mercurial/dispatch.py b/mercurial/dispatch.py
--- a/mercurial/dispatch.py
+++ b/mercurial/dispatch.py
@@ -331,6 +331,27 @@ 
         args = shlex.split(cmd)
     return args + givenargs
 
+def aliasinterpolate(name, args, cmd):
+    '''interpolate args into cmd for shell aliases
+
+    This also handles $0, $@ and "$@".
+    '''
+    # util.interpolate can't deal with "$@" (with quotes) because it's only
+    # built to match prefix + patterns.
+    replacemap = dict(('$%d' % (i + 1), arg) for i, arg in enumerate(args))
+    replacemap['$0'] = name
+    replacemap['$$'] = '$'
+    replacemap['$@'] = ' '.join(args)
+    # Typical Unix shells interpolate "$@" (with quotes) as all the positional
+    # parameters, separated out into words. Emulate the same behavior here by
+    # quoting the arguments individually. POSIX shells will then typically
+    # tokenize each argument into exactly one word.
+    replacemap['"$@"'] = ' '.join(util.shellquote(arg) for arg in args)
+    # escape '\$' for regex
+    regex = '|'.join(replacemap.keys()).replace('$', r'\$')
+    r = re.compile(regex)
+    return r.sub(lambda x: replacemap[x.group()], cmd)
+
 class cmdalias(object):
     def __init__(self, name, definition, cmdtable):
         self.name = self.cmd = name
@@ -376,10 +397,7 @@ 
                                  % (int(m.groups()[0]), self.name))
                         return ''
                 cmd = re.sub(r'\$(\d+|\$)', _checkvar, self.definition[1:])
-                replace = dict((str(i + 1), arg) for i, arg in enumerate(args))
-                replace['0'] = self.name
-                replace['@'] = ' '.join(args)
-                cmd = util.interpolate(r'\$', replace, cmd, escape_prefix=True)
+                cmd = aliasinterpolate(self.name, args, cmd)
                 return util.system(cmd, environ=env, out=ui.fout)
             self.fn = fn
             return
diff --git a/mercurial/help/config.txt b/mercurial/help/config.txt
--- a/mercurial/help/config.txt
+++ b/mercurial/help/config.txt
@@ -229,8 +229,9 @@ 
 Positional arguments like ``$1``, ``$2``, etc. in the alias definition
 expand to the command arguments. Unmatched arguments are
 removed. ``$0`` expands to the alias name and ``$@`` expands to all
-arguments separated by a space. These expansions happen before the
-command is passed to the shell.
+arguments separated by a space. ``"$@"`` (with quotes) expands to all
+arguments quoted individually and separated by a space. These expansions
+happen before the command is passed to the shell.
 
 Shell aliases are executed in an environment where ``$HG`` expands to
 the path of the Mercurial that was used to execute the alias. This is
diff --git a/tests/test-alias.t b/tests/test-alias.t
--- a/tests/test-alias.t
+++ b/tests/test-alias.t
@@ -30,6 +30,7 @@ 
   > echo1 = !printf '\$1\n'
   > echo2 = !printf '\$2\n'
   > echo13 = !printf '\$1 \$3\n'
+  > echotokens = !printf "%s\n" "\$@"
   > count = !hg log -r "\$@" --template=. | wc -c | sed -e 's/ //g'
   > mcount = !hg log \$@ --template=. | wc -c | sed -e 's/ //g'
   > rt = root
@@ -241,6 +242,22 @@ 
   foo baz
   $ hg echo2 foo
   
+  $ hg echotokens
+  
+  $ hg echotokens foo 'bar $1 baz'
+  foo
+  bar $1 baz
+  $ hg echotokens 'test $2' foo
+  test $2
+  foo
+  $ hg echotokens 'test $@' foo '$@'
+  test $@
+  foo
+  $@
+  $ hg echotokens 'test "$@"' foo '"$@"'
+  test "$@"
+  foo
+  "$@"
   $ echo bar > bar
   $ hg commit -qA -m bar
   $ hg count .