Patchwork serve: introduce --get and --post for easy testing of hgweb

login
register
mail settings
Submitter Mads Kiilerich
Date Feb. 9, 2013, 11:22 p.m.
Message ID <86f6b4786d8d292843a9.1360452156@localhost.localdomain>
Download mbox | patch
Permalink /patch/914/
State Superseded, archived
Delegated to: Brodie Rao
Headers show

Comments

Mads Kiilerich - Feb. 9, 2013, 11:22 p.m.
# HG changeset patch
# User Mads Kiilerich <mads@kiilerich.com>
# Date 1359297965 -3600
# Node ID 86f6b4786d8d292843a9509b0c99f8e855ef4e9d
# Parent  19f90544863eadb00baf80ef21811d41b2beb54b
serve: introduce --get and --post for easy testing of hgweb

These options can be useful both in the test suite and for other kinds of hgweb
testing.
Brodie Rao - Feb. 10, 2013, 11:43 a.m.
(Resending, forgot to CC the list.)

On Sat, Feb 9, 2013 at 11:22 PM, Mads Kiilerich <mads@kiilerich.com> wrote:
> # HG changeset patch
> # User Mads Kiilerich <mads@kiilerich.com>
> # Date 1359297965 -3600
> # Node ID 86f6b4786d8d292843a9509b0c99f8e855ef4e9d
> # Parent  19f90544863eadb00baf80ef21811d41b2beb54b
> serve: introduce --get and --post for easy testing of hgweb
>
> These options can be useful both in the test suite and for other kinds of hgweb
> testing.
>
> diff --git a/mercurial/commands.py b/mercurial/commands.py
> --- a/mercurial/commands.py
> +++ b/mercurial/commands.py
> @@ -8,7 +8,7 @@
>  from node import hex, bin, nullid, nullrev, short
>  from lock import release
>  from i18n import _, gettext
> -import os, re, difflib, time, tempfile, errno
> +import os, re, difflib, time, tempfile, errno, sys
>  import hg, scmutil, util, revlog, extensions, copies, error, bookmarks
>  import patch, help, encoding, templatekw, discovery
>  import archival, changegroup, cmdutil, hbisect
> @@ -5251,7 +5251,9 @@
>      ('t', 'templates', '', _('web templates to use'), _('TEMPLATE')),
>      ('', 'style', '', _('template style to use'), _('STYLE')),
>      ('6', 'ipv6', None, _('use IPv6 in addition to IPv4')),
> -    ('', 'certificate', '', _('SSL certificate file'), _('FILE'))],
> +    ('', 'certificate', '', _('SSL certificate file'), _('FILE')),
> +    ('', 'get', '', _('just GET url'), _('URL')),
> +    ('', 'post', '', _('just POST url'), _('URL'))],

Do these options need to be exposed by default? I feel like they might
be better as debug options.

I also think the descriptions are a little too terse. Maybe something
like "GET specified URL and exit"?

Other than that, this looks useful.

>      _('[OPTION]...'))
>  def serve(ui, repo, **opts):
>      """start stand-alone webserver
> @@ -5275,6 +5277,9 @@
>      a port number of 0; in this case, the server will print the port
>      number it uses.
>
> +    --get and --post will process a single hgweb request and show the output
> +    without starting a server. Headers will be shown in verbose mode.
> +
>      Returns 0 on success.
>      """
>
> @@ -5320,6 +5325,45 @@
>
>      app = hgweb.hgweb(o, baseui=ui)
>
> +    get = opts.get('get')
> +    post = opts.get('post')
> +    if get or post:
> +        url = util.url(get or post)
> +
> +        environ = dict(os.environ.iteritems())
> +        environ['REQUEST_METHOD'] = get and 'GET' or 'POST'
> +        environ['wsgi.url_scheme'] = url.scheme or 'http'
> +        environ['SERVER_NAME'] = url.host or 'localhost'
> +        environ['SERVER_PORT'] = url.port or (url.scheme == 'https' and '443' or '80')
> +        environ['SCRIPT_NAME'] = ''
> +        environ['PATH_INFO'] = '/' + url.path
> +        environ['QUERY_STRING'] = url.query or ''
> +
> +        environ['wsgi.input'] = sys.stdin
> +        environ['wsgi.errors'] = sys.stderr
> +        environ['wsgi.version'] = (1, 0)
> +        environ['wsgi.multithread'] = False
> +        environ['wsgi.multiprocess'] = True
> +        environ['wsgi.run_once'] = True
> +
> +        def start_response(status, response_headers, exc_info=None):
> +            if exc_info:
> +                raise exc_info[0](exc_info[1], exc_info[2])
> +            # this is for stdout - don't use \r\n as HTTP mandates
> +            ui.note(status + '\n')
> +            for header in response_headers:
> +                if header[0] == 'ETag' and not ui.debugflag:
> +                    continue
> +                ui.note('%s: %s\n' % header)
> +            ui.note('\n')
> +            return ui.write
> +
> +        content = app(environ, start_response)
> +        for chunk in content:
> +            ui.write(chunk)
> +        getattr(content, 'close', lambda : None)()
> +        return
> +
>      class service(object):
>          def init(self):
>              util.setsignalhandler()
> diff --git a/tests/test-hgweb-commands.t b/tests/test-hgweb-commands.t
> --- a/tests/test-hgweb-commands.t
> +++ b/tests/test-hgweb-commands.t
> @@ -1366,73 +1366,46 @@
>
>    $ hg phase -fs 4
>    $ hg bookmark -r4 secret
> -  $ cat > hgweb.cgi <<HGWEB
> -  > from mercurial import demandimport; demandimport.enable()
> -  > from mercurial.hgweb import hgweb
> -  > from mercurial.hgweb import wsgicgi
> -  > app = hgweb('.', 'test')
> -  > wsgicgi.launch(app)
> -  > HGWEB
> -  $ . "$TESTDIR/cgienv"
> -  $ PATH_INFO=/bookmarks; export PATH_INFO
> -  $ QUERY_STRING='style=raw'
> -  $ python hgweb.cgi | grep -v ETag:
> -  Status: 200 Script output follows\r (esc)
> -  Content-Type: text/plain; charset=ascii\r (esc)
> -  \r (esc)
> +  $ hg serve --get 'http://server/bookmarks?style=raw' -v
> +  200 Script output follows
> +  Content-Type: text/plain; charset=ascii
> +
>
>  listbookmarks hides secret bookmarks
>
> -  $ PATH_INFO=/; export PATH_INFO
> -  $ QUERY_STRING='cmd=listkeys&namespace=bookmarks'
> -  $ python hgweb.cgi
> -  Status: 200 Script output follows\r (esc)
> -  Content-Type: application/mercurial-0.1\r (esc)
> -  Content-Length: 0\r (esc)
> -  \r (esc)
> +  $ hg serve --get 'http://server/?cmd=listkeys&namespace=bookmarks' -v
> +  200 Script output follows
> +  Content-Type: application/mercurial-0.1
> +  Content-Length: 0
> +
>
>  search works with filtering
>
> -  $ PATH_INFO=/log; export PATH_INFO
> -  $ QUERY_STRING='rev=babar'
> -  $ python hgweb.cgi > search
> -  $ grep Status search
> -  Status: 200 Script output follows\r (esc)
> +  $ hg serve --get 'http://server/log/?rev=babar/' -v | head -n1
> +  200 Script output follows
>
>  summary works with filtering (issue3810)
>
> -  $ PATH_INFO=/summary; export PATH_INFO
> -  $ QUERY_STRING='style=monoblue'; export QUERY_STRING
> -  $ python hgweb.cgi > summary.out
> -  $ grep "^Status" summary.out
> -  Status: 200 Script output follows\r (esc)
> +  $ hg serve --get 'http://server/summary?style=monoblue' -v | head -n1
> +  200 Script output follows
>
>  proper status for filtered revision
>
> -
>  (missing rev)
>
> -  $ PATH_INFO=/rev/5; export PATH_INFO
> -  $ QUERY_STRING='style=raw'
> -  $ python hgweb.cgi #> search
> -  Status: 404 Not Found\r (esc)
> -  ETag: *\r (glob) (esc)
> -  Content-Type: text/plain; charset=ascii\r (esc)
> -  \r (esc)
> +  $ hg serve --get 'http://server/rev/5?style=raw' -v
> +  404 Not Found
> +  Content-Type: text/plain; charset=ascii
> +
>
>    error: unknown revision '5'
>
> -
> -
>  (filtered rev)
>
> -  $ PATH_INFO=/rev/4; export PATH_INFO
> -  $ QUERY_STRING='style=raw'
> -  $ python hgweb.cgi #> search
> -  Status: 404 Not Found\r (esc)
> -  ETag: *\r (glob) (esc)
> -  Content-Type: text/plain; charset=ascii\r (esc)
> -  \r (esc)
> +  $ hg serve --get 'http://server/rev/4?style=raw' -v
> +  404 Not Found
> +  Content-Type: text/plain; charset=ascii
> +
>
>    error: unknown revision '4'
>
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel
Mads Kiilerich - Feb. 10, 2013, 12:18 p.m.
On 02/10/2013 12:43 PM, Brodie Rao wrote:
> (Resending, forgot to CC the list.)
>
> On Sat, Feb 9, 2013 at 11:22 PM, Mads Kiilerich <mads@kiilerich.com> wrote:
>> # HG changeset patch
>> # User Mads Kiilerich <mads@kiilerich.com>
>> # Date 1359297965 -3600
>> # Node ID 86f6b4786d8d292843a9509b0c99f8e855ef4e9d
>> # Parent  19f90544863eadb00baf80ef21811d41b2beb54b
>> serve: introduce --get and --post for easy testing of hgweb
>>
>> These options can be useful both in the test suite and for other kinds of hgweb
>> testing.
>>
>> diff --git a/mercurial/commands.py b/mercurial/commands.py
>> --- a/mercurial/commands.py
>> +++ b/mercurial/commands.py
>> @@ -8,7 +8,7 @@
>>   from node import hex, bin, nullid, nullrev, short
>>   from lock import release
>>   from i18n import _, gettext
>> -import os, re, difflib, time, tempfile, errno
>> +import os, re, difflib, time, tempfile, errno, sys
>>   import hg, scmutil, util, revlog, extensions, copies, error, bookmarks
>>   import patch, help, encoding, templatekw, discovery
>>   import archival, changegroup, cmdutil, hbisect
>> @@ -5251,7 +5251,9 @@
>>       ('t', 'templates', '', _('web templates to use'), _('TEMPLATE')),
>>       ('', 'style', '', _('template style to use'), _('STYLE')),
>>       ('6', 'ipv6', None, _('use IPv6 in addition to IPv4')),
>> -    ('', 'certificate', '', _('SSL certificate file'), _('FILE'))],
>> +    ('', 'certificate', '', _('SSL certificate file'), _('FILE')),
>> +    ('', 'get', '', _('just GET url'), _('URL')),
>> +    ('', 'post', '', _('just POST url'), _('URL'))],
> Do these options need to be exposed by default? I feel like they might
> be better as debug options.

Perhaps? But it is not that it is exposing Mercurial internals as most 
other debug commands do.

Showing them directly in the help makes them more directly available for 
people with problems with their hgweb configuration and with systematic 
debugging.

Other opinions?

/Mads
Sune Foldager - Feb. 10, 2013, 2:19 p.m.
On 2013-02-10 13:18, Mads Kiilerich wrote:
>On 02/10/2013 12:43 PM, Brodie Rao wrote:
>>(Resending, forgot to CC the list.)
>>
>>On Sat, Feb 9, 2013 at 11:22 PM, Mads Kiilerich <mads@kiilerich.com> wrote:
>>># HG changeset patch
>>># User Mads Kiilerich <mads@kiilerich.com>
>>># Date 1359297965 -3600
>>># Node ID 86f6b4786d8d292843a9509b0c99f8e855ef4e9d
>>># Parent  19f90544863eadb00baf80ef21811d41b2beb54b
>>>serve: introduce --get and --post for easy testing of hgweb
>>>
>>>These options can be useful both in the test suite and for other kinds of hgweb
>>>testing.
>>>
>>>diff --git a/mercurial/commands.py b/mercurial/commands.py
>>>--- a/mercurial/commands.py
>>>+++ b/mercurial/commands.py
>>>@@ -8,7 +8,7 @@
>>>  from node import hex, bin, nullid, nullrev, short
>>>  from lock import release
>>>  from i18n import _, gettext
>>>-import os, re, difflib, time, tempfile, errno
>>>+import os, re, difflib, time, tempfile, errno, sys
>>>  import hg, scmutil, util, revlog, extensions, copies, error, bookmarks
>>>  import patch, help, encoding, templatekw, discovery
>>>  import archival, changegroup, cmdutil, hbisect
>>>@@ -5251,7 +5251,9 @@
>>>      ('t', 'templates', '', _('web templates to use'), _('TEMPLATE')),
>>>      ('', 'style', '', _('template style to use'), _('STYLE')),
>>>      ('6', 'ipv6', None, _('use IPv6 in addition to IPv4')),
>>>-    ('', 'certificate', '', _('SSL certificate file'), _('FILE'))],
>>>+    ('', 'certificate', '', _('SSL certificate file'), _('FILE')),
>>>+    ('', 'get', '', _('just GET url'), _('URL')),
>>>+    ('', 'post', '', _('just POST url'), _('URL'))],
>>Do these options need to be exposed by default? I feel like they might
>>be better as debug options.
>
>Perhaps? But it is not that it is exposing Mercurial internals as 
>most other debug commands do.

But "just GET url" is a bit weird, I agree with Brodie. I mean, what does that even mean?

/Sune
Bryan O'Sullivan - Feb. 11, 2013, 9:53 p.m.
On Sat, Feb 9, 2013 at 3:22 PM, Mads Kiilerich <mads@kiilerich.com> wrote:

> serve: introduce --get and --post for easy testing of hgweb
>
> These options can be useful both in the test suite and for other kinds of
> hgweb
> testing.
>

Wouldn't it make sense to put this code in a Python file in the tests
directory? It seems quite inappropriate to add these options to hgweb
itself.
Matt Mackall - April 9, 2013, 10:35 p.m.
On Fri, 2013-03-01 at 11:49 -0500, Augie Fackler wrote:
> On Mar 1, 2013, at 11:19 AM, Kevin Bullock <kbullock+mercurial@ringworld.org> wrote:
> 
> > On 11 Feb 2013, at 3:53 PM, Bryan O'Sullivan wrote:
> > 
> >> On Sat, Feb 9, 2013 at 3:22 PM, Mads Kiilerich <mads@kiilerich.com> wrote:
> >>> serve: introduce --get and --post for easy testing of hgweb
> >>> 
> >>> These options can be useful both in the test suite and for other kinds of hgweb
> >>> testing.
> >>> 
> >> Wouldn't it make sense to put this code in a Python file in the tests directory? It seems quite inappropriate to add these options to hgweb itself.
> > 
> > At the sprint I suggested that it might be nice to hide these options behind a --verbose flag, but I'm not sure we have that capability at the moment.
> > 
> > I'm in favor of having them as flags on `hg serve`, though. I think it would be useful beyond our test suite.
> 
> Eh? Why?

Me too. I'm at about -.1 on this, lacking some story as to why we'd want
them outside the test suite. For the test suite itself, I don't think
it's quite warranted.

Patch

diff --git a/mercurial/commands.py b/mercurial/commands.py
--- a/mercurial/commands.py
+++ b/mercurial/commands.py
@@ -8,7 +8,7 @@ 
 from node import hex, bin, nullid, nullrev, short
 from lock import release
 from i18n import _, gettext
-import os, re, difflib, time, tempfile, errno
+import os, re, difflib, time, tempfile, errno, sys
 import hg, scmutil, util, revlog, extensions, copies, error, bookmarks
 import patch, help, encoding, templatekw, discovery
 import archival, changegroup, cmdutil, hbisect
@@ -5251,7 +5251,9 @@ 
     ('t', 'templates', '', _('web templates to use'), _('TEMPLATE')),
     ('', 'style', '', _('template style to use'), _('STYLE')),
     ('6', 'ipv6', None, _('use IPv6 in addition to IPv4')),
-    ('', 'certificate', '', _('SSL certificate file'), _('FILE'))],
+    ('', 'certificate', '', _('SSL certificate file'), _('FILE')),
+    ('', 'get', '', _('just GET url'), _('URL')),
+    ('', 'post', '', _('just POST url'), _('URL'))],
     _('[OPTION]...'))
 def serve(ui, repo, **opts):
     """start stand-alone webserver
@@ -5275,6 +5277,9 @@ 
     a port number of 0; in this case, the server will print the port
     number it uses.
 
+    --get and --post will process a single hgweb request and show the output
+    without starting a server. Headers will be shown in verbose mode.
+
     Returns 0 on success.
     """
 
@@ -5320,6 +5325,45 @@ 
 
     app = hgweb.hgweb(o, baseui=ui)
 
+    get = opts.get('get')
+    post = opts.get('post')
+    if get or post:
+        url = util.url(get or post)
+
+        environ = dict(os.environ.iteritems())
+        environ['REQUEST_METHOD'] = get and 'GET' or 'POST'
+        environ['wsgi.url_scheme'] = url.scheme or 'http'
+        environ['SERVER_NAME'] = url.host or 'localhost'
+        environ['SERVER_PORT'] = url.port or (url.scheme == 'https' and '443' or '80')
+        environ['SCRIPT_NAME'] = ''
+        environ['PATH_INFO'] = '/' + url.path
+        environ['QUERY_STRING'] = url.query or ''
+
+        environ['wsgi.input'] = sys.stdin
+        environ['wsgi.errors'] = sys.stderr
+        environ['wsgi.version'] = (1, 0)
+        environ['wsgi.multithread'] = False
+        environ['wsgi.multiprocess'] = True
+        environ['wsgi.run_once'] = True
+
+        def start_response(status, response_headers, exc_info=None):
+            if exc_info:
+                raise exc_info[0](exc_info[1], exc_info[2])
+            # this is for stdout - don't use \r\n as HTTP mandates
+            ui.note(status + '\n')
+            for header in response_headers:
+                if header[0] == 'ETag' and not ui.debugflag:
+                    continue
+                ui.note('%s: %s\n' % header)
+            ui.note('\n')
+            return ui.write
+
+        content = app(environ, start_response)
+        for chunk in content:
+            ui.write(chunk)
+        getattr(content, 'close', lambda : None)()
+        return
+
     class service(object):
         def init(self):
             util.setsignalhandler()
diff --git a/tests/test-hgweb-commands.t b/tests/test-hgweb-commands.t
--- a/tests/test-hgweb-commands.t
+++ b/tests/test-hgweb-commands.t
@@ -1366,73 +1366,46 @@ 
 
   $ hg phase -fs 4
   $ hg bookmark -r4 secret
-  $ cat > hgweb.cgi <<HGWEB
-  > from mercurial import demandimport; demandimport.enable()
-  > from mercurial.hgweb import hgweb
-  > from mercurial.hgweb import wsgicgi
-  > app = hgweb('.', 'test')
-  > wsgicgi.launch(app)
-  > HGWEB
-  $ . "$TESTDIR/cgienv"
-  $ PATH_INFO=/bookmarks; export PATH_INFO
-  $ QUERY_STRING='style=raw'
-  $ python hgweb.cgi | grep -v ETag:
-  Status: 200 Script output follows\r (esc)
-  Content-Type: text/plain; charset=ascii\r (esc)
-  \r (esc)
+  $ hg serve --get 'http://server/bookmarks?style=raw' -v
+  200 Script output follows
+  Content-Type: text/plain; charset=ascii
+  
 
 listbookmarks hides secret bookmarks
 
-  $ PATH_INFO=/; export PATH_INFO
-  $ QUERY_STRING='cmd=listkeys&namespace=bookmarks'
-  $ python hgweb.cgi
-  Status: 200 Script output follows\r (esc)
-  Content-Type: application/mercurial-0.1\r (esc)
-  Content-Length: 0\r (esc)
-  \r (esc)
+  $ hg serve --get 'http://server/?cmd=listkeys&namespace=bookmarks' -v
+  200 Script output follows
+  Content-Type: application/mercurial-0.1
+  Content-Length: 0
+  
 
 search works with filtering
 
-  $ PATH_INFO=/log; export PATH_INFO
-  $ QUERY_STRING='rev=babar'
-  $ python hgweb.cgi > search
-  $ grep Status search
-  Status: 200 Script output follows\r (esc)
+  $ hg serve --get 'http://server/log/?rev=babar/' -v | head -n1
+  200 Script output follows
 
 summary works with filtering (issue3810)
 
-  $ PATH_INFO=/summary; export PATH_INFO
-  $ QUERY_STRING='style=monoblue'; export QUERY_STRING
-  $ python hgweb.cgi > summary.out
-  $ grep "^Status" summary.out
-  Status: 200 Script output follows\r (esc)
+  $ hg serve --get 'http://server/summary?style=monoblue' -v | head -n1
+  200 Script output follows
 
 proper status for filtered revision
 
-
 (missing rev)
 
-  $ PATH_INFO=/rev/5; export PATH_INFO
-  $ QUERY_STRING='style=raw'
-  $ python hgweb.cgi #> search
-  Status: 404 Not Found\r (esc)
-  ETag: *\r (glob) (esc)
-  Content-Type: text/plain; charset=ascii\r (esc)
-  \r (esc)
+  $ hg serve --get 'http://server/rev/5?style=raw' -v
+  404 Not Found
+  Content-Type: text/plain; charset=ascii
+  
   
   error: unknown revision '5'
 
-
-
 (filtered rev)
 
-  $ PATH_INFO=/rev/4; export PATH_INFO
-  $ QUERY_STRING='style=raw'
-  $ python hgweb.cgi #> search
-  Status: 404 Not Found\r (esc)
-  ETag: *\r (glob) (esc)
-  Content-Type: text/plain; charset=ascii\r (esc)
-  \r (esc)
+  $ hg serve --get 'http://server/rev/4?style=raw' -v
+  404 Not Found
+  Content-Type: text/plain; charset=ascii
+  
   
   error: unknown revision '4'