Patchwork [RFC] reflog: adds a reflog extension

login
register
mail settings
Submitter Durham Goode
Date Oct. 2, 2014, 1:45 a.m.
Message ID <942be96848993cf7ab5e.1412214345@dev2000.prn2.facebook.com>
Download mbox | patch
Permalink /patch/6081/
State Changes Requested
Headers show

Comments

Durham Goode - Oct. 2, 2014, 1:45 a.m.
# HG changeset patch
# User Durham Goode <durham@fb.com>
# Date 1412200597 25200
#      Wed Oct 01 14:56:37 2014 -0700
# Node ID 942be96848993cf7ab5ed529db9c1f39c6d43c30
# Parent  939ce500c92a3dcc0e10815242361ff70a6fcae9
reflog: adds a reflog extension

This adds an extension that tracks the locations of the working copy and
bookmarks over time. It's still a proof of concept, but I wanted to throw
it out there to start the bike shedding early (like finding a better name
than 'reflog'). We're close enough to the release that I don't think it should
go in before that.

Running `hg reflog` by default shows the previous locations of the working
copy (most recent first).

~/myrepo> hg reflog
35a5fcfee452 > rebase -d master
32eee5e2d406 > up .^
b5d6dab4f900 > up foo -C

Specifying a bookmark name shows the locations of that bookmark over time.

~/myrepo> hg reflog foo
d1a696044ec0 > rebase -d master
35a5fcfee452 > rebase -d master
32eee5e2d406 > book foo -f

--date and --user flags exist to show more information about each entry.

~/myrepo> hg reflog foo --user --date
d1a696044ec0 durham 2014-10-01 18:32:14 > rebase -d master
35a5fcfee452 durham 2014-10-01 17:28:54 > rebase -d master
32eee5e2d406 durham 2014-10-01 17:28:30 > book foo -f

It's currently stored as a single .hg/reflog file that is append only. Each
entry can store an arbitrary number of hashes (like storing 2 hashes for a merge
state working copy), which means we could also potentially use this to track
heads in branches as well.
Gregory Szorc - Oct. 2, 2014, 2:56 a.m.
On 10/1/14 6:45 PM, Durham Goode wrote:
> # HG changeset patch
> # User Durham Goode <durham@fb.com>
> # Date 1412200597 25200
> #      Wed Oct 01 14:56:37 2014 -0700
> # Node ID 942be96848993cf7ab5ed529db9c1f39c6d43c30
> # Parent  939ce500c92a3dcc0e10815242361ff70a6fcae9
> reflog: adds a reflog extension
>
> This adds an extension that tracks the locations of the working copy and
> bookmarks over time. It's still a proof of concept, but I wanted to throw
> it out there to start the bike shedding early (like finding a better name
> than 'reflog'). We're close enough to the release that I don't think it should
> go in before that.
>
> Running `hg reflog` by default shows the previous locations of the working
> copy (most recent first).
>
> ~/myrepo> hg reflog
> 35a5fcfee452 > rebase -d master
> 32eee5e2d406 > up .^
> b5d6dab4f900 > up foo -C
>
> Specifying a bookmark name shows the locations of that bookmark over time.
>
> ~/myrepo> hg reflog foo
> d1a696044ec0 > rebase -d master
> 35a5fcfee452 > rebase -d master
> 32eee5e2d406 > book foo -f
>
> --date and --user flags exist to show more information about each entry.
>
> ~/myrepo> hg reflog foo --user --date
> d1a696044ec0 durham 2014-10-01 18:32:14 > rebase -d master
> 35a5fcfee452 durham 2014-10-01 17:28:54 > rebase -d master
> 32eee5e2d406 durham 2014-10-01 17:28:30 > book foo -f
>
> It's currently stored as a single .hg/reflog file that is append only. Each
> entry can store an arbitrary number of hashes (like storing 2 hashes for a merge
> state working copy), which means we could also potentially use this to track
> heads in branches as well.

Very nice!

Do you think it's worth adding "local/working" terminology to help 
distinguish it from the inevitable support for "remote reflogs" or 
"store reflogs?"

FWIW, I'm rewriting Mozilla's reflog implementation as an extension and 
adding support for data transfer via the wire protocol. However, I'm 
currently aiming for backwards compatibility (SQLite - which I know 
won't fly upstream) and only support for a single remote (again, 
optimized for our current server-side use). However, I'd eventually like 
to integrate client-side aggregation of reflogs from multiple remotes. I 
know it's scope bloat, but if this reflog extension becomes officialish, 
I'd like to see a path to supporting store and remote reflogs in the 
same extension.

Gregory
Durham Goode - Oct. 2, 2014, 5:16 a.m.
On 10/1/14, 7:56 PM, Gregory Szorc wrote:
> On 10/1/14 6:45 PM, Durham Goode wrote:
>> # HG changeset patch
>> # User Durham Goode <durham@fb.com>
>> # Date 1412200597 25200
>> #      Wed Oct 01 14:56:37 2014 -0700
>> # Node ID 942be96848993cf7ab5ed529db9c1f39c6d43c30
>> # Parent  939ce500c92a3dcc0e10815242361ff70a6fcae9
>> reflog: adds a reflog extension
> Very nice!
>
> Do you think it's worth adding "local/working" terminology to help 
> distinguish it from the inevitable support for "remote reflogs" or 
> "store reflogs?"
I'm not sure I understand what "remote reflogs" and "store reflogs" are? 
Could you provide examples?
Mike Hommey - Oct. 2, 2014, 5:28 a.m.
On Wed, Oct 01, 2014 at 10:16:20PM -0700, Durham Goode wrote:
> 
> On 10/1/14, 7:56 PM, Gregory Szorc wrote:
> >On 10/1/14 6:45 PM, Durham Goode wrote:
> >># HG changeset patch
> >># User Durham Goode <durham@fb.com>
> >># Date 1412200597 25200
> >>#      Wed Oct 01 14:56:37 2014 -0700
> >># Node ID 942be96848993cf7ab5ed529db9c1f39c6d43c30
> >># Parent  939ce500c92a3dcc0e10815242361ff70a6fcae9
> >>reflog: adds a reflog extension
> >Very nice!
> >
> >Do you think it's worth adding "local/working" terminology to help
> >distinguish it from the inevitable support for "remote reflogs" or "store
> >reflogs?"
> I'm not sure I understand what "remote reflogs" and "store reflogs" are?
> Could you provide examples?

He is talking about pushlog, which is some kind of reflog. And he would
like to make them pullable. ("what changesets are push heads" is a
valuable information)

Mike
Gregory Szorc - Oct. 2, 2014, 5:32 a.m.
On 10/1/14 10:16 PM, Durham Goode wrote:
>
> On 10/1/14, 7:56 PM, Gregory Szorc wrote:
>> On 10/1/14 6:45 PM, Durham Goode wrote:
>>> # HG changeset patch
>>> # User Durham Goode <durham@fb.com>
>>> # Date 1412200597 25200
>>> #      Wed Oct 01 14:56:37 2014 -0700
>>> # Node ID 942be96848993cf7ab5ed529db9c1f39c6d43c30
>>> # Parent  939ce500c92a3dcc0e10815242361ff70a6fcae9
>>> reflog: adds a reflog extension
>> Very nice!
>>
>> Do you think it's worth adding "local/working" terminology to help
>> distinguish it from the inevitable support for "remote reflogs" or
>> "store reflogs?"
> I'm not sure I understand what "remote reflogs" and "store reflogs" are?
> Could you provide examples?

"store reflogs" are essentially logs of when changesets were introduced 
to a store. Conceptually a log of when addchangegroup() is called. If 
deployed on a server, this is a "pushlog."

"remote reflogs" could be two things a) the "store reflogs" from a 
remote, existing locally b) a reflog listing heads, etc of remotes at 
various times. I was referring mostly to "a". "b" ventures into remote 
tracking territory. But it's arguably two sides of the same coin, as 
remote tracking can be done by injecting log events of pull operations 
into the locally-stored "remote reflog."
Matt Mackall - Oct. 2, 2014, 5:25 p.m.
On Wed, 2014-10-01 at 18:45 -0700, Durham Goode wrote:

> --date and --user flags exist to show more information about each entry.
> 
> ~/myrepo> hg reflog foo --user --date
> d1a696044ec0 durham 2014-10-01 18:32:14 > rebase -d master
> 35a5fcfee452 durham 2014-10-01 17:28:54 > rebase -d master
> 32eee5e2d406 durham 2014-10-01 17:28:30 > book foo -f

Since you're writing a new list-some-things command, you should use a
formatter:

http://mercurial.selenic.com/wiki/GenericTemplatingPlan

And once you have that, consider having just normal, verbose, and
templated output rather than adding options.
Augie Fackler - Oct. 2, 2014, 5:43 p.m.
On Wed, Oct 01, 2014 at 06:45:45PM -0700, Durham Goode wrote:
> # HG changeset patch
> # User Durham Goode <durham@fb.com>
> # Date 1412200597 25200
> #      Wed Oct 01 14:56:37 2014 -0700
> # Node ID 942be96848993cf7ab5ed529db9c1f39c6d43c30
> # Parent  939ce500c92a3dcc0e10815242361ff70a6fcae9
> reflog: adds a reflog extension

Perhaps this could live at 'bookmark --log'? Given that we don't have
anything called a ref, the name reflog is kind of a bummer.

Other options:
bmlog
booklog

(I'm bad at naming things. I think I like bookmark --log the best?)

>
> This adds an extension that tracks the locations of the working copy and
> bookmarks over time. It's still a proof of concept, but I wanted to throw
> it out there to start the bike shedding early (like finding a better name
> than 'reflog'). We're close enough to the release that I don't think it should
> go in before that.
>
> Running `hg reflog` by default shows the previous locations of the working
> copy (most recent first).
>
> ~/myrepo> hg reflog
> 35a5fcfee452 > rebase -d master
> 32eee5e2d406 > up .^
> b5d6dab4f900 > up foo -C
>
> Specifying a bookmark name shows the locations of that bookmark over time.
>
> ~/myrepo> hg reflog foo
> d1a696044ec0 > rebase -d master
> 35a5fcfee452 > rebase -d master
> 32eee5e2d406 > book foo -f
>
> --date and --user flags exist to show more information about each entry.
>
> ~/myrepo> hg reflog foo --user --date
> d1a696044ec0 durham 2014-10-01 18:32:14 > rebase -d master
> 35a5fcfee452 durham 2014-10-01 17:28:54 > rebase -d master
> 32eee5e2d406 durham 2014-10-01 17:28:30 > book foo -f
>
> It's currently stored as a single .hg/reflog file that is append only. Each
> entry can store an arbitrary number of hashes (like storing 2 hashes for a merge
> state working copy), which means we could also potentially use this to track
> heads in branches as well.
>
> diff --git a/hgext/reflog.py b/hgext/reflog.py
> new file mode 100644
> --- /dev/null
> +++ b/hgext/reflog.py
> @@ -0,0 +1,152 @@
> +# reflog.py
> +#
> +# Copyright 2013 Facebook, Inc.
> +#
> +# This software may be used and distributed according to the terms of the
> +# GNU General Public License version 2 or any later version.
> +
> +from mercurial import util, cmdutil, commands, hg, scmutil, localrepo
> +from mercurial import bookmarks, dispatch, dirstate
> +from mercurial.extensions import wrapcommand, wrapfunction
> +from mercurial.node import nullid, hex
> +from mercurial.i18n import _
> +import errno, os, getpass, time
> +
> +cmdtable = {}
> +command = cmdutil.command(cmdtable)
> +testedwith = 'internal'
> +
> +bookmarktype = 'bookmark'
> +workingcopyparenttype = 'workingcopyparent'
> +
> +def extsetup(ui):
> +    wrapfunction(dispatch, '_parse', recordcommand)
> +    wrapfunction(bookmarks.bmstore, 'write', recordbookmarks)
> +    wrapfunction(dirstate.dirstate, 'write', recorddirstateparents)
> +
> +def reposetup(ui, repo):
> +    if isinstance(repo, localrepo.localrepository):
> +        repo.reflog = Reflog(repo, currentcommand)
> +        repo.dirstate.repo = repo
> +
> +currentcommand = ''
> +def recordcommand(orig, ui, args):
> +    """Records the current command line args for later logging to the reflog."""
> +    global currentcommand
> +    currentcommand = ' '.join(args)
> +    return orig(ui, args)
> +
> +def recordbookmarks(orig, self):
> +    """Records all bookmark changes to the reflog."""
> +    repo = self._repo
> +    oldmarks = bookmarks.bmstore(repo)
> +    for mark, value in self.iteritems():
> +        if value != oldmarks.get(mark):
> +            repo.reflog.addentry(bookmarktype, mark, value)
> +    return orig(self)
> +
> +def recorddirstateparents(orig, self):
> +    """Records all dirstate parent changes to the reflog."""
> +    oldparents = [nullid]
> +    try:
> +        fp = self._opener("dirstate")
> +        st = fp.read(40)
> +        fp.close()
> +        l = len(st)
> +        if l == 40:
> +            oldparents = [st[:20]]
> +            oldparents.append(st[20:40])
> +    except IOError, err:
> +        pass
> +    if oldparents != self.parents():
> +        hashes = [hash for hash in self.parents() if hash != nullid]
> +        self.repo.reflog.addentry(workingcopyparenttype, '.', hashes)
> +    return orig(self)
> +
> +@command('reflog',
> +    [('', 'all', None, 'show history for all refs'),
> +     ('', 'date', None, 'include timestamp information'),
> +     ('', 'user', None, 'include user information'),
> +     ], '[OPTION]... [REFNAME]')
> +def reflog(ui, repo, *args, **opts):
> +    """show the previous position of bookmarks and the working copy
> +
> +    The reflog is used to see the previous commits that bookmarks and the
> +    working copy pointed to. By default it shows the previous locations of the
> +    working copy.  Passing a bookmark name will show all the previous
> +    positions of that bookmark. Passing --all will show the previous
> +    locations of all bookmarks and the working copy.
> +
> +    By default the reflog only shows the commit hash and the command that was
> +    running at that time. --date will also show the timestamp of the entry, and
> +    --user will show the user name of the user who was executing the command.
> +    """
> +    refname = '.'
> +    if args:
> +        refname = args[0]
> +    if opts.get('all'):
> +        refname = None
> +
> +    for entry in repo.reflog.iter(refnamecond=refname):
> +        timestamp, user, command, reftype, refname, hashes = entry
> +        output = ','.join([hash[:12] for hash in hashes])
> +        if opts.get('user'):
> +            output += ' %s' % user
> +        if opts.get('date'):
> +            timestruct = time.localtime(timestamp)
> +            timestring = time.strftime('%Y-%m-%d %H:%M:%S', timestruct)
> +            output += ' %s' % timestring
> +        output += ' > %s' % command
> +        ui.status('%s\n' % output)
> +
> +class Reflog(object):
> +    def __init__(self, repo, command):
> +        self.repo = repo
> +        self.command = command
> +        self.user = getpass.getuser()
> +        self.path = repo.join('reflog')
> +
> +    def __iter__(self):
> +        return self._read()
> +
> +    def iter(self, reftypecond=None, refnamecond=None):
> +        for entry in self._read():
> +            time, user, command, reftype, refname, hashes = entry
> +            if reftypecond and reftype != reftypecond:
> +                continue
> +            if refnamecond and refname != refnamecond:
> +                continue
> +            yield entry
> +
> +    def _read(self):
> +        if not os.path.exists(self.path):
> +            raise StopIteration()
> +
> +        f = open(self.path, 'r')
> +        try:
> +            raw = f.read()
> +        finally:
> +            f.close()
> +
> +        lines = reversed(raw.split('\0'))
> +        for line in lines:
> +            if not line:
> +                continue
> +            time, user, command, reftype, refname, hashes = line.split('\n')
> +            time = int(time)
> +            hashes = hashes.split(',')
> +            yield (time, user, command, reftype, refname, hashes)
> +
> +    def addentry(self, reftype, refname, hashes):
> +        if isinstance(hashes, str):
> +            hashes = [hashes]
> +
> +        date = str(int(time.time()))
> +        hashes = ','.join([hex(hash) for hash in hashes])
> +        data = (date, self.user, self.command, reftype, refname, hashes)
> +        data = '\n'.join(data)
> +        f = open(self.path, 'a+')
> +        try:
> +            f.write(data + '\0')
> +        finally:
> +            f.close()
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel
Matt Mackall - Oct. 2, 2014, 5:59 p.m.
On Thu, 2014-10-02 at 13:43 -0400, Augie Fackler wrote:
> On Wed, Oct 01, 2014 at 06:45:45PM -0700, Durham Goode wrote:
> > # HG changeset patch
> > # User Durham Goode <durham@fb.com>
> > # Date 1412200597 25200
> > #      Wed Oct 01 14:56:37 2014 -0700
> > # Node ID 942be96848993cf7ab5ed529db9c1f39c6d43c30
> > # Parent  939ce500c92a3dcc0e10815242361ff70a6fcae9
> > reflog: adds a reflog extension
> 
> Perhaps this could live at 'bookmark --log'? Given that we don't have
> anything called a ref, the name reflog is kind of a bummer.
> 
> Other options:
> bmlog
> booklog
> 
> (I'm bad at naming things. I think I like bookmark --log the best?)

That is considerably less horrible.
Durham Goode - Oct. 2, 2014, 6:45 p.m.
On 10/2/14, 10:59 AM, Matt Mackall wrote:
> On Thu, 2014-10-02 at 13:43 -0400, Augie Fackler wrote:
>> On Wed, Oct 01, 2014 at 06:45:45PM -0700, Durham Goode wrote:
>>> # HG changeset patch
>>> # User Durham Goode <durham@fb.com>
>>> # Date 1412200597 25200
>>> #      Wed Oct 01 14:56:37 2014 -0700
>>> # Node ID 942be96848993cf7ab5ed529db9c1f39c6d43c30
>>> # Parent  939ce500c92a3dcc0e10815242361ff70a6fcae9
>>> reflog: adds a reflog extension
>> Perhaps this could live at 'bookmark --log'? Given that we don't have
>> anything called a ref, the name reflog is kind of a bummer.
>>
>> Other options:
>> bmlog
>> booklog
>>
>> (I'm bad at naming things. I think I like bookmark --log the best?)
> That is considerably less horrible.
>
The only problem is that it also tracks the working copy parent (and 
potentially branch heads) as well.  How would they fit in to bookmark --log?

dsop had an interesting idea a while back.  He said it'd be nice if 
there was a command that could restore your repo to the state it was in 
at a previous point in time (heads, bookmarks, working copy parent, 
etc).  Almost like a timemachine.  'hg timemachine', or something 
involving the word 'time', might be more descriptive of the actual 
usecase that reflog is used for (i.e. going back in time) and not limit 
us to a bookmark specific command.
David Soria Parra - Oct. 2, 2014, 7:03 p.m.
Durham Goode <durham@fb.com> writes:

> On 10/2/14, 10:59 AM, Matt Mackall wrote:
>> On Thu, 2014-10-02 at 13:43 -0400, Augie Fackler wrote:
>>> On Wed, Oct 01, 2014 at 06:45:45PM -0700, Durham Goode wrote:
>>>> # HG changeset patch
>>>> # User Durham Goode <durham@fb.com>
>>>> # Date 1412200597 25200
>>>> #      Wed Oct 01 14:56:37 2014 -0700
>>>> # Node ID 942be96848993cf7ab5ed529db9c1f39c6d43c30
>>>> # Parent  939ce500c92a3dcc0e10815242361ff70a6fcae9
>>>> reflog: adds a reflog extension
>>> Perhaps this could live at 'bookmark --log'? Given that we don't have
>>> anything called a ref, the name reflog is kind of a bummer.
>>>
>>> Other options:
>>> bmlog
>>> booklog
>>>
>>> (I'm bad at naming things. I think I like bookmark --log the best?)
>> That is considerably less horrible.
>>
> The only problem is that it also tracks the working copy parent (and
> potentially branch heads) as well.  How would they fit in to bookmark
> --log?
>
> dsop had an interesting idea a while back.  He said it'd be nice if
> there was a command that could restore your repo to the state it was
> in at a previous point in time (heads, bookmarks, working copy parent,
> etc).  Almost like a timemachine.  'hg timemachine', or something
> involving the word 'time', might be more descriptive of the actual
> usecase that reflog is used for (i.e. going back in time) and not
> limit us to a bookmark specific command.

Unless we have that restore functionality, I think timemachine might not
suitable. I would go for something like hg statelog.
Jordi Gutiérrez Hermoso - Oct. 6, 2014, 1:54 a.m.
On Wed, 2014-10-01 at 18:45 -0700, Durham Goode wrote:
> # HG changeset patch
> # User Durham Goode <durham@fb.com>
> # Date 1412200597 25200
> #      Wed Oct 01 14:56:37 2014 -0700
> # Node ID 942be96848993cf7ab5ed529db9c1f39c6d43c30
> # Parent  939ce500c92a3dcc0e10815242361ff70a6fcae9
> reflog: adds a reflog extension
> 
> This adds an extension that tracks the locations of the working copy and
> bookmarks over time. It's still a proof of concept, but I wanted to throw
> it out there to start the bike shedding early (like finding a better name
> than 'reflog'). We're close enough to the release that I don't think it should
> go in before that.
> 
> Running `hg reflog` by default shows the previous locations of the working
> copy (most recent first).
> 
> ~/myrepo> hg reflog
> 35a5fcfee452 > rebase -d master
> 32eee5e2d406 > up .^
> b5d6dab4f900 > up foo -C
> 
> Specifying a bookmark name shows the locations of that bookmark over time.
> 
> ~/myrepo> hg reflog foo
> d1a696044ec0 > rebase -d master
> 35a5fcfee452 > rebase -d master
> 32eee5e2d406 > book foo -f
> 
> --date and --user flags exist to show more information about each entry.
> 
> ~/myrepo> hg reflog foo --user --date
> d1a696044ec0 durham 2014-10-01 18:32:14 > rebase -d master
> 35a5fcfee452 durham 2014-10-01 17:28:54 > rebase -d master
> 32eee5e2d406 durham 2014-10-01 17:28:30 > book foo -f

I'm a little bothered by the UI. Like you said in a later email, the
goal here is to undo bookmark or pwd motions. The fact that the
information to undo is in a log should be mostly irrelevant for the
user. In that case, I propose that what you're currently calling
`reflog` should be something like `debugstatelog` and the `bookmark`
and `update` commands should use this statelog to grow --undo and
possibly --redo flags, e.g.

    # Move bookmark master to whatever state it was before its current
    # state
    hg bookmark master --undo

    # Move whatever was the last moved bookmark to whatever its
    # previous state was
    hg bookmark --undo

    # Move dirstate and active bookmark to whatever they were before
    # the last time an `update` command was invoked
    hg update --undo

Can we use your extension or perhaps blackbox to also provide an
--undo flag for `update`? Should your extension be rolled into
blackbox?
Durham Goode - Oct. 6, 2014, 5:42 p.m.
On 10/5/14, 6:54 PM, Jordi Gutiérrez Hermoso wrote:
> On Wed, 2014-10-01 at 18:45 -0700, Durham Goode wrote:
>> # HG changeset patch
>> # User Durham Goode <durham@fb.com>
>> # Date 1412200597 25200
>> #      Wed Oct 01 14:56:37 2014 -0700
>> # Node ID 942be96848993cf7ab5ed529db9c1f39c6d43c30
>> # Parent  939ce500c92a3dcc0e10815242361ff70a6fcae9
>> reflog: adds a reflog extension
>>
> I'm a little bothered by the UI. Like you said in a later email, the
> goal here is to undo bookmark or pwd motions. The fact that the
> information to undo is in a log should be mostly irrelevant for the
> user. In that case, I propose that what you're currently calling
> `reflog` should be something like `debugstatelog` and the `bookmark`
> and `update` commands should use this statelog to grow --undo and
> possibly --redo flags, e.g.
>
>      # Move bookmark master to whatever state it was before its current
>      # state
>      hg bookmark master --undo
>
>      # Move whatever was the last moved bookmark to whatever its
>      # previous state was
>      hg bookmark --undo
>
>      # Move dirstate and active bookmark to whatever they were before
>      # the last time an `update` command was invoked
>      hg update --undo
>
> Can we use your extension or perhaps blackbox to also provide an
> --undo flag for `update`? Should your extension be rolled into
> blackbox?
>
>
Undo and redo have significant connotations (and large expectations) for 
a user, so I don't want to use that terminology.  Also, in vanilla hg 
there is no way to undo many things without digging through the backup 
bundles, so I don't think the log can be actionable by default.

I'm not sure about putting it in blackbox.  Blackbox is about recording 
a series of actions in the repo.  The 'ref/book/thing-log' is about 
recording previous states of the repository.  If we keep this small and 
separate, I think it'll be easier to roll into core later.
Jordi Gutiérrez Hermoso - Oct. 6, 2014, 6:19 p.m.
On Mon, 2014-10-06 at 10:42 -0700, Durham Goode wrote:
> On 10/5/14, 6:54 PM, Jordi Gutiérrez Hermoso wrote:
> > On Wed, 2014-10-01 at 18:45 -0700, Durham Goode wrote:
> >> # HG changeset patch
> >> # User Durham Goode <durham@fb.com>
> >> # Date 1412200597 25200
> >> #      Wed Oct 01 14:56:37 2014 -0700
> >> # Node ID 942be96848993cf7ab5ed529db9c1f39c6d43c30
> >> # Parent  939ce500c92a3dcc0e10815242361ff70a6fcae9
> >> reflog: adds a reflog extension
> >>
> > I'm a little bothered by the UI. Like you said in a later email, the
> > goal here is to undo bookmark or pwd motions. The fact that the
> > information to undo is in a log should be mostly irrelevant for the
> > user. In that case, I propose that what you're currently calling
> > `reflog` should be something like `debugstatelog` and the `bookmark`
> > and `update` commands should use this statelog to grow --undo and
> > possibly --redo flags, e.g.
> >
> >      # Move bookmark master to whatever state it was before its current
> >      # state
> >      hg bookmark master --undo
> >
> >      # Move whatever was the last moved bookmark to whatever its
> >      # previous state was
> >      hg bookmark --undo
> >
> >      # Move dirstate and active bookmark to whatever they were before
> >      # the last time an `update` command was invoked
> >      hg update --undo
> >
> > Can we use your extension or perhaps blackbox to also provide an
> > --undo flag for `update`? Should your extension be rolled into
> > blackbox?
> >
> >
> Undo and redo have significant connotations (and large expectations) for 
> a user, so I don't want to use that terminology.

I think it's expected to expect that `hg bookmark --undo` only moves
or recreates bookmarks, which is the same thing that `hg bookmark`
does. Same for `hg update`, all it does is update to a different
revision. What user expectations are you afraid of?

> Also, in vanilla hg there is no way to undo many things without
> digging through the backup bundles, so I don't think the log can be
> actionable by default.

I think this is an acceptable compromise. "Cannot find revision X,
trying revision Y instead", where Y is the earliest revision in the
log that was found.

What I am trying to get at here is that I think we can do better than
`git reflog`. Or perhaps now that everyone is used to `git reflog`,
it's the gold standard and we should just duplicate it exactly. Do you
think we can do any better than `git reflog`?

> I'm not sure about putting it in blackbox. Blackbox is about
> recording a series of actions in the repo. The 'ref/book/thing-log'
> is about recording previous states of the repository.

Aren't these just two ways to record the same information?

- Jordi G. H.
Matt Mackall - Oct. 6, 2014, 6:52 p.m.
On Mon, 2014-10-06 at 14:19 -0400, Jordi Gutiérrez Hermoso wrote:
> On Mon, 2014-10-06 at 10:42 -0700, Durham Goode wrote:
> > On 10/5/14, 6:54 PM, Jordi Gutiérrez Hermoso wrote:
> > > On Wed, 2014-10-01 at 18:45 -0700, Durham Goode wrote:
> > >> # HG changeset patch
> > >> # User Durham Goode <durham@fb.com>
> > >> # Date 1412200597 25200
> > >> #      Wed Oct 01 14:56:37 2014 -0700
> > >> # Node ID 942be96848993cf7ab5ed529db9c1f39c6d43c30
> > >> # Parent  939ce500c92a3dcc0e10815242361ff70a6fcae9
> > >> reflog: adds a reflog extension
> > >>
> > > I'm a little bothered by the UI. Like you said in a later email, the
> > > goal here is to undo bookmark or pwd motions. The fact that the
> > > information to undo is in a log should be mostly irrelevant for the
> > > user. In that case, I propose that what you're currently calling
> > > `reflog` should be something like `debugstatelog` and the `bookmark`
> > > and `update` commands should use this statelog to grow --undo and
> > > possibly --redo flags, e.g.
> > >
> > >      # Move bookmark master to whatever state it was before its current
> > >      # state
> > >      hg bookmark master --undo
> > >
> > >      # Move whatever was the last moved bookmark to whatever its
> > >      # previous state was
> > >      hg bookmark --undo
> > >
> > >      # Move dirstate and active bookmark to whatever they were before
> > >      # the last time an `update` command was invoked
> > >      hg update --undo
> > >
> > > Can we use your extension or perhaps blackbox to also provide an
> > > --undo flag for `update`? Should your extension be rolled into
> > > blackbox?
> > >
> > >
> > Undo and redo have significant connotations (and large expectations) for 
> > a user, so I don't want to use that terminology.
> 
> I think it's expected to expect that `hg bookmark --undo` only moves
> or recreates bookmarks, which is the same thing that `hg bookmark`
> does.

If you think that, you are clearly not supporting a thousand ex-Git
users who are mis-applying their vague understanding of Git branches to
Mercurial bookmarks. Because those users really do expect mysterious
DWIM magic that they can't really describe to happen at various stages.
And --undo is the ultimate DWIM feature name.

> > I'm not sure about putting it in blackbox. Blackbox is about
> > recording a series of actions in the repo. The 'ref/book/thing-log'
> > is about recording previous states of the repository.
> 
> Aren't these just two ways to record the same information?

Nope. Blackbox is free-form human-readable text that's automatically
trimmed at random intervals. It's in no way intended to be reliably read
by machines.
Jordi Gutiérrez Hermoso - Oct. 6, 2014, 8:51 p.m.
On Mon, 2014-10-06 at 13:52 -0500, Matt Mackall wrote:
> On Mon, 2014-10-06 at 14:19 -0400, Jordi Gutiérrez Hermoso wrote:
> > On Mon, 2014-10-06 at 10:42 -0700, Durham Goode wrote:
> > > On 10/5/14, 6:54 PM, Jordi Gutiérrez Hermoso wrote:
> > > > On Wed, 2014-10-01 at 18:45 -0700, Durham Goode wrote:
> > > >> # HG changeset patch
> > > >> # User Durham Goode <durham@fb.com>
> > > >> # Date 1412200597 25200
> > > >> #      Wed Oct 01 14:56:37 2014 -0700
> > > >> # Node ID 942be96848993cf7ab5ed529db9c1f39c6d43c30
> > > >> # Parent  939ce500c92a3dcc0e10815242361ff70a6fcae9
> > > >> reflog: adds a reflog extension
> > > >>
> > > > I'm a little bothered by the UI. Like you said in a later email, the
> > > > goal here is to undo bookmark or pwd motions. The fact that the
> > > > information to undo is in a log should be mostly irrelevant for the
> > > > user. In that case, I propose that what you're currently calling
> > > > `reflog` should be something like `debugstatelog` and the `bookmark`
> > > > and `update` commands should use this statelog to grow --undo and
> > > > possibly --redo flags, e.g.
> > > >
> > > >      # Move bookmark master to whatever state it was before its current
> > > >      # state
> > > >      hg bookmark master --undo
> > > >
> > > >      # Move whatever was the last moved bookmark to whatever its
> > > >      # previous state was
> > > >      hg bookmark --undo
> > > >
> > > >      # Move dirstate and active bookmark to whatever they were before
> > > >      # the last time an `update` command was invoked
> > > >      hg update --undo
> > > >
> > > > Can we use your extension or perhaps blackbox to also provide an
> > > > --undo flag for `update`? Should your extension be rolled into
> > > > blackbox?
> > > >
> > > >
> > > Undo and redo have significant connotations (and large expectations) for 
> > > a user, so I don't want to use that terminology.
> > 
> > I think it's expected to expect that `hg bookmark --undo` only moves
> > or recreates bookmarks, which is the same thing that `hg bookmark`
> > does.
> 
> If you think that, you are clearly not supporting a thousand ex-Git
> users who are mis-applying their vague understanding of Git branches to
> Mercurial bookmarks. 

If this is about appeasing git users, then I suppose giving hg a git
reflog command and exactly a git reflog command is all we can hope
for.
Durham Goode - Oct. 6, 2014, 9:14 p.m.
On 10/6/14, 1:51 PM, Jordi Gutiérrez Hermoso wrote:
> On Mon, 2014-10-06 at 13:52 -0500, Matt Mackall wrote:
>> On Mon, 2014-10-06 at 14:19 -0400, Jordi Gutiérrez Hermoso wrote:
>>> On Mon, 2014-10-06 at 10:42 -0700, Durham Goode wrote:
>>>> On 10/5/14, 6:54 PM, Jordi Gutiérrez Hermoso wrote:
>>>>> On Wed, 2014-10-01 at 18:45 -0700, Durham Goode wrote:
>>>>>> # HG changeset patch
>>>>>> # User Durham Goode <durham@fb.com>
>>>>>> # Date 1412200597 25200
>>>>>> #      Wed Oct 01 14:56:37 2014 -0700
>>>>>> # Node ID 942be96848993cf7ab5ed529db9c1f39c6d43c30
>>>>>> # Parent  939ce500c92a3dcc0e10815242361ff70a6fcae9
>>>>>> reflog: adds a reflog extension
>>>>>>
>>>>> I'm a little bothered by the UI. Like you said in a later email, the
>>>>> goal here is to undo bookmark or pwd motions. The fact that the
>>>>> information to undo is in a log should be mostly irrelevant for the
>>>>> user. In that case, I propose that what you're currently calling
>>>>> `reflog` should be something like `debugstatelog` and the `bookmark`
>>>>> and `update` commands should use this statelog to grow --undo and
>>>>> possibly --redo flags, e.g.
>>>>>
>>>>>       # Move bookmark master to whatever state it was before its current
>>>>>       # state
>>>>>       hg bookmark master --undo
>>>>>
>>>>>       # Move whatever was the last moved bookmark to whatever its
>>>>>       # previous state was
>>>>>       hg bookmark --undo
>>>>>
>>>>>       # Move dirstate and active bookmark to whatever they were before
>>>>>       # the last time an `update` command was invoked
>>>>>       hg update --undo
>>>>>
>>>>> Can we use your extension or perhaps blackbox to also provide an
>>>>> --undo flag for `update`? Should your extension be rolled into
>>>>> blackbox?
>>>>>
>>>>>
>>>> Undo and redo have significant connotations (and large expectations) for
>>>> a user, so I don't want to use that terminology.
>>> I think it's expected to expect that `hg bookmark --undo` only moves
>>> or recreates bookmarks, which is the same thing that `hg bookmark`
>>> does.
>> If you think that, you are clearly not supporting a thousand ex-Git
>> users who are mis-applying their vague understanding of Git branches to
>> Mercurial bookmarks.
> If this is about appeasing git users, then I suppose giving hg a git
> reflog command and exactly a git reflog command is all we can hope
> for.
It’s about building a nice, easy road for people to follow to the 
greener pastures of changeset evolution. Part of that is making 
migration easy, and reflog is a good starting point since it's an 
already well defined concept (albeit with a rough UI).  We have plans 
for a 'hg rewind' command that will allow the undo-esque behavior you're 
talking about in a more friendly way.
Jordi Gutiérrez Hermoso - Oct. 6, 2014, 11:41 p.m.
On Mon, 2014-10-06 at 14:14 -0700, Durham Goode wrote:
> We have plans for a 'hg rewind' command that will allow the
> undo-esque behavior you're talking about in a more friendly way.

I like this name. I think it also maps to a corresponding git concept,
but also a more universal concept. Good naming choice.

Patch

diff --git a/hgext/reflog.py b/hgext/reflog.py
new file mode 100644
--- /dev/null
+++ b/hgext/reflog.py
@@ -0,0 +1,152 @@ 
+# reflog.py
+#
+# Copyright 2013 Facebook, Inc.
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+
+from mercurial import util, cmdutil, commands, hg, scmutil, localrepo
+from mercurial import bookmarks, dispatch, dirstate
+from mercurial.extensions import wrapcommand, wrapfunction
+from mercurial.node import nullid, hex
+from mercurial.i18n import _
+import errno, os, getpass, time
+
+cmdtable = {}
+command = cmdutil.command(cmdtable)
+testedwith = 'internal'
+
+bookmarktype = 'bookmark'
+workingcopyparenttype = 'workingcopyparent'
+
+def extsetup(ui):
+    wrapfunction(dispatch, '_parse', recordcommand)
+    wrapfunction(bookmarks.bmstore, 'write', recordbookmarks)
+    wrapfunction(dirstate.dirstate, 'write', recorddirstateparents)
+
+def reposetup(ui, repo):
+    if isinstance(repo, localrepo.localrepository):
+        repo.reflog = Reflog(repo, currentcommand)
+        repo.dirstate.repo = repo
+
+currentcommand = ''
+def recordcommand(orig, ui, args):
+    """Records the current command line args for later logging to the reflog."""
+    global currentcommand
+    currentcommand = ' '.join(args)
+    return orig(ui, args)
+
+def recordbookmarks(orig, self):
+    """Records all bookmark changes to the reflog."""
+    repo = self._repo
+    oldmarks = bookmarks.bmstore(repo)
+    for mark, value in self.iteritems():
+        if value != oldmarks.get(mark):
+            repo.reflog.addentry(bookmarktype, mark, value) 
+    return orig(self)
+
+def recorddirstateparents(orig, self):
+    """Records all dirstate parent changes to the reflog."""
+    oldparents = [nullid]
+    try:
+        fp = self._opener("dirstate")
+        st = fp.read(40)
+        fp.close()
+        l = len(st)
+        if l == 40:
+            oldparents = [st[:20]]
+            oldparents.append(st[20:40])
+    except IOError, err:
+        pass
+    if oldparents != self.parents():
+        hashes = [hash for hash in self.parents() if hash != nullid]
+        self.repo.reflog.addentry(workingcopyparenttype, '.', hashes)
+    return orig(self)
+
+@command('reflog',
+    [('', 'all', None, 'show history for all refs'),
+     ('', 'date', None, 'include timestamp information'),
+     ('', 'user', None, 'include user information'),
+     ], '[OPTION]... [REFNAME]')
+def reflog(ui, repo, *args, **opts):
+    """show the previous position of bookmarks and the working copy
+
+    The reflog is used to see the previous commits that bookmarks and the
+    working copy pointed to. By default it shows the previous locations of the
+    working copy.  Passing a bookmark name will show all the previous
+    positions of that bookmark. Passing --all will show the previous
+    locations of all bookmarks and the working copy.
+
+    By default the reflog only shows the commit hash and the command that was
+    running at that time. --date will also show the timestamp of the entry, and
+    --user will show the user name of the user who was executing the command.
+    """
+    refname = '.'
+    if args:
+        refname = args[0]
+    if opts.get('all'):
+        refname = None
+
+    for entry in repo.reflog.iter(refnamecond=refname):
+        timestamp, user, command, reftype, refname, hashes = entry
+        output = ','.join([hash[:12] for hash in hashes])
+        if opts.get('user'):
+            output += ' %s' % user
+        if opts.get('date'):
+            timestruct = time.localtime(timestamp)
+            timestring = time.strftime('%Y-%m-%d %H:%M:%S', timestruct)
+            output += ' %s' % timestring
+        output += ' > %s' % command
+        ui.status('%s\n' % output)
+
+class Reflog(object):
+    def __init__(self, repo, command):
+        self.repo = repo
+        self.command = command
+        self.user = getpass.getuser()
+        self.path = repo.join('reflog')
+
+    def __iter__(self):
+        return self._read()
+
+    def iter(self, reftypecond=None, refnamecond=None):
+        for entry in self._read():
+            time, user, command, reftype, refname, hashes = entry
+            if reftypecond and reftype != reftypecond:
+                continue
+            if refnamecond and refname != refnamecond:
+                continue
+            yield entry
+
+    def _read(self):
+        if not os.path.exists(self.path):
+            raise StopIteration()
+
+        f = open(self.path, 'r')
+        try:
+            raw = f.read()
+        finally:
+            f.close()
+
+        lines = reversed(raw.split('\0'))
+        for line in lines:
+            if not line:
+                continue
+            time, user, command, reftype, refname, hashes = line.split('\n')
+            time = int(time)
+            hashes = hashes.split(',')
+            yield (time, user, command, reftype, refname, hashes)
+
+    def addentry(self, reftype, refname, hashes):
+        if isinstance(hashes, str):
+            hashes = [hashes]
+
+        date = str(int(time.time()))
+        hashes = ','.join([hex(hash) for hash in hashes])
+        data = (date, self.user, self.command, reftype, refname, hashes)
+        data = '\n'.join(data)
+        f = open(self.path, 'a+')
+        try:
+            f.write(data + '\0')
+        finally:
+            f.close()