Patchwork [6,of,6] spartan: render changesets server-side on /graph page

# HG changeset patch
# User Anton Shestakov <av6@dwimlabs.net>
# Date 1512385721 -28800
#      Mon Dec 04 19:08:41 2017 +0800
# Node ID aa14b97727c79991683a2ffd45986d18e3657576
# Parent  c85a282fa36ff57c7491700c54a1af9c69ded03c
# EXP-Topic hgweb-more-info
spartan: render changesets server-side on /graph page

On Mon, Dec 04, 2017 at 08:40:43PM +0800, Anton Shestakov wrote:
> # HG changeset patch
> # User Anton Shestakov <av6@dwimlabs.net>
> # Date 1512385721 -28800
> #      Mon Dec 04 19:08:41 2017 +0800
> # Node ID aa14b97727c79991683a2ffd45986d18e3657576
> # Parent  c85a282fa36ff57c7491700c54a1af9c69ded03c
> # EXP-Topic hgweb-more-info
> spartan: render changesets server-side on /graph page

queued, thanks

On Mon, Dec 4, 2017 at 1:28 PM, Augie Fackler <raf@durin42.com> wrote:

> On Mon, Dec 04, 2017 at 08:40:43PM +0800, Anton Shestakov wrote:
> > # HG changeset patch
> > # User Anton Shestakov <av6@dwimlabs.net>
> > # Date 1512385721 -28800
> > #      Mon Dec 04 19:08:41 2017 +0800
> > # Node ID aa14b97727c79991683a2ffd45986d18e3657576
> > # Parent  c85a282fa36ff57c7491700c54a1af9c69ded03c
> > # EXP-Topic hgweb-more-info
> > spartan: render changesets server-side on /graph page
>
> queued, thanks
>

Anton,

I'm not sure if it is this series or something before. But the graph view
of hgweb now loads substantially slower for me using my local copy of the
hg repo. Until recently, things would load about as fast as I could scroll
with my mouse wheel. Now there are delays of several hundred milliseconds.
Using `HGPROF=ls hg serve --profile` (the default stat profiler doesn't
work with hgweb), it looks like something is spending a lot of time
calculating visibility. Maybe it has something to do with rendering the
obsolescence state of each changeset?

These changes to hgweb are all great! I don't mean to discount the
improvements. If you need help tracking down the perf issue, I'd be happy
to help.

On Mon, 4 Dec 2017 21:46:55 -0800
Gregory Szorc <gregory.szorc@gmail.com> wrote:

> On Mon, Dec 4, 2017 at 1:28 PM, Augie Fackler <raf@durin42.com> wrote:
> 
> > On Mon, Dec 04, 2017 at 08:40:43PM +0800, Anton Shestakov wrote:
> > > # HG changeset patch
> > > # User Anton Shestakov <av6@dwimlabs.net>
> > > # Date 1512385721 -28800
> > > #      Mon Dec 04 19:08:41 2017 +0800
> > > # Node ID aa14b97727c79991683a2ffd45986d18e3657576
> > > # Parent  c85a282fa36ff57c7491700c54a1af9c69ded03c
> > > # EXP-Topic hgweb-more-info
> > > spartan: render changesets server-side on /graph page
> >
> > queued, thanks
> >
> 
> Anton,
> 
> I'm not sure if it is this series or something before. But the graph view
> of hgweb now loads substantially slower for me using my local copy of the
> hg repo. Until recently, things would load about as fast as I could scroll
> with my mouse wheel. Now there are delays of several hundred milliseconds.
> Using `HGPROF=ls hg serve --profile` (the default stat profiler doesn't
> work with hgweb), it looks like something is spending a lot of time
> calculating visibility. Maybe it has something to do with rendering the
> obsolescence state of each changeset?

This series definitely made hgweb slower, at least in the more
apparent way, simply because it moved the code that renders changesets
to the server side. So not only hgweb needs to produce more HTML, but
the browser needs to download more as well, which I suspect contributes
a sizable amount of time spend on /graph requests.

This is not the root of the problem though, because right now /graph
page doesn't render incrementally as you load more data: everything it
does under the hood when you scroll down to see the next page is to
increment revcount by 60 and redo all the work. So when you scroll
below the initial 60 entries, JS graph code fetches /graph?revcount=120
and replaces the current content with that. Then it goes to 180, 240
and so on. This all is a crude answer to the problem /graph has, AFAIU:
it needs to have graphmod.colored() to pass over all the revs in current
view to produce correct graph. Compare this with /log page which just
loads the next page without changing revcount, which is still as fast
as before phases, obsolescence and instabilities were added.

But not all is lost, obviously, I just need to figure out how to not
re-render and re-download every changeset for /graph if it just needs a
new page.

> These changes to hgweb are all great! I don't mean to discount the
> improvements.

Thanks! It's nice to hear that.

> If you need help tracking down the perf issue, I'd be happy to help.

I have no idea yet where can things be improved, apart from the obvious
problem described above, but having as many eyes on hgweb as possible is
definitely useful.