Patchwork D2897: fix: new extension for automatically modifying file contents

login
register
mail settings
Submitter phabricator
Date March 19, 2018, 7:40 p.m.
Message ID <differential-rev-PHID-DREV-ijiltq5a2op6vzvl3wqc-req@phab.mercurial-scm.org>
Download mbox | patch
Permalink /patch/29637/
State Superseded
Headers show

Comments

phabricator - March 19, 2018, 7:40 p.m.
hooper created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This change implements most of the corresponding proposal as discussed at the
  4.4 and 4.6 sprints: https://www.mercurial-scm.org/wiki/AutomaticFormattingPlan
  
  This change notably does not include parallel execution of the formatter/fixer
  tools. It does allow for implementing that without affecting other areas of the
  code.
  
  I believe the test coverage to be good, but this is a hotbed of corner cases.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2897

AFFECTED FILES
  hgext/fix.py
  tests/test-doctest.py
  tests/test-fix-clang-format.t
  tests/test-fix-topology.t
  tests/test-fix.t

CHANGE DETAILS




To: hooper, #hg-reviewers
Cc: mercurial-devel
phabricator - March 20, 2018, 7:12 a.m.
pulkit added a comment.


  Haven't reviewed the patch but I think `fix` is too generic for this use case. What about `format`? Sorry for not having this idea when we had couple of discussion on it.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2897

To: hooper, #hg-reviewers
Cc: pulkit, mercurial-devel
phabricator - March 20, 2018, 8:52 p.m.
hooper added a comment.


  In https://phab.mercurial-scm.org/D2897#46656, @pulkit wrote:
  
  > Haven't reviewed the patch but I think `fix` is too generic for this use case. What about `format`? Sorry for not having this idea when we had couple of discussion on it.
  
  
  I will do a final pass to rename it if desired, but I would like to wait for any other feedback before doing so.
  
  I think "format" is accurate for how we expect this to be used, but I also think it may be overly specific. I don't have any good alternatives in mind. "fix" is problematic because it is confusing in contexts like commit messages.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2897

To: hooper, #hg-reviewers
Cc: pulkit, mercurial-devel
phabricator - March 26, 2018, 7:21 p.m.
pulkit added a comment.


  Can you add a test where fixing of one parent commit leads to a child commit becoming empty. For example:
  
  Commit 1: echo foo> a
  Commit 2: echo Foo > a
  
  And I run a lexer on commit 1 and 2, to change everything to uppercase.

INLINE COMMENTS

> fix.py:460
> +        ctxmandatory = varnames[2] == 'changectx'
> +        if ctxmandatory:
> +            return context.memfilectx(repo, ctx, path, data, islink, isexec,

we can drop this if-else now since things are moved to core.

> test-fix-topology.t:18
> +#testcases no-evolution evolution
> +#if evolution
> +  $ cat >> $HGRCPATH <<EOF

In core we do '#if obsstore-on' or '#if obsstore-off'.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2897

To: hooper, #hg-reviewers
Cc: pulkit, mercurial-devel
phabricator - March 26, 2018, 10:11 p.m.
hooper marked 2 inline comments as done.
hooper added a comment.


  In https://phab.mercurial-scm.org/D2897#47695, @pulkit wrote:
  
  > Can you add a test where fixing of one parent commit leads to a child commit becoming empty. For example:
  >
  > Commit 1: echo foo> a
  >  Commit 2: echo Foo > a
  >
  > And I run a lexer on commit 1 and 2, to change everything to uppercase.
  
  
  Done.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2897

To: hooper, #hg-reviewers
Cc: pulkit, mercurial-devel
phabricator - March 28, 2018, 9:35 a.m.
pulkit added subscribers: indygreg, yuja, martinvonz, krbullock, durin42.
pulkit added a comment.


  The code except of minor nits looks good to me. I will like others to chime in on whether we should rename the command and extension to `format` because `fix` is too generic. I will push it once we get the naming and these comments sorted out.
  cc: @yuja @indygreg @martinvonz @durin42 @krbullock

INLINE COMMENTS

> test-fix.t:813
> +
> +It's also possible that the child needs absolutely no changes, but we still
> +need to replace it to update its parent. If we simply skip replacing it because

This should respect `experimental.evolution.allowunstable` and error out if set to False.

> test-fix.t:838
> +
> +Similar to the case above, the child revision may become empty as a result of
> +fixing its parent. We should still create an empty replacement child.

I am not sure how it should interact with `ui.allowemptycommit`. Maybe add a TODO here about this.

> test-fix.t:857
> +  
> +  @  1 edit foo
> +  |   foo.whole |  2 +-

When the working directory parent get obsoleted because of `fix`, we should update to it's successor similar to what other commands do.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2897

To: hooper, #hg-reviewers
Cc: durin42, krbullock, martinvonz, yuja, indygreg, pulkit, mercurial-devel
phabricator - March 29, 2018, 1:35 p.m.
yuja added a comment.


  In https://phab.mercurial-scm.org/D2897#47875, @pulkit wrote:
  
  > The code except of minor nits looks good to me. I will like others to chime in on whether we should rename the command and extension to `format` because `fix` is too generic.
  
  
  I agree the extension name `fix` is  too generic, but I have no strong opinion.
  It could be `format`, `formatter`, `extfmt`, `extformatter`, etc., but I guess
  this isn't just for "formatting" in Google? And `hg fix` is easy to type. ;)

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2897

To: hooper, #hg-reviewers
Cc: durin42, krbullock, martinvonz, yuja, indygreg, pulkit, mercurial-devel
phabricator - March 29, 2018, 8:42 p.m.
hooper added a comment.


  In https://phab.mercurial-scm.org/D2897#47944, @yuja wrote:
  
  > In https://phab.mercurial-scm.org/D2897#47875, @pulkit wrote:
  >
  > > The code except of minor nits looks good to me. I will like others to chime in on whether we should rename the command and extension to `format` because `fix` is too generic.
  >
  >
  > I agree the extension name `fix` is  too generic, but I have no strong opinion.
  >  It could be `format`, `formatter`, `extfmt`, `extformatter`, etc., but I guess
  >  this isn't just for "formatting" in Google? And `hg fix` is easy to type. ;)
  
  
  One possibility to consider is a config that runs a linter tool on the changed lines, but makes no edits. In the proposal I also gave the example of a config to sort the lines in a file. Another thought was spell checking/fixing. Implying that the command is only useful for source code formatting is selling the feature short, but still I wouldn't know what else to call it. We had picked "fix" because our users are familiar with the term.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2897

To: hooper, #hg-reviewers
Cc: durin42, krbullock, martinvonz, yuja, indygreg, pulkit, mercurial-devel
phabricator - March 29, 2018, 9:36 p.m.
hooper added inline comments.

INLINE COMMENTS

> pulkit wrote in test-fix.t:813
> This should respect `experimental.evolution.allowunstable` and error out if set to False.

Maybe the comment is poorly worded. I mean that an incorrect implementation could create an orphan in this case, even though both commits were meant to be replaced.

I'm adding a separate test case for aborting if the user asks to create an orphan. I imitated an abort from histedit, which seems like it should be a utility function (checknodescendants).

> pulkit wrote in test-fix.t:857
> When the working directory parent get obsoleted because of `fix`, we should update to it's successor similar to what other commands do.

Would it abort or merge if the working copy is dirty? Is that behavior expected to end up in cleanupnodes? It could also just force --working-dir. It's similar to creating an orphan if we don't fix the dirty working copy.

One of my goals is to avoid any need for merging or conflict resolution in this command.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2897

To: hooper, #hg-reviewers
Cc: durin42, krbullock, martinvonz, yuja, indygreg, pulkit, mercurial-devel
phabricator - March 30, 2018, 6:55 p.m.
pulkit accepted this revision.
pulkit added inline comments.

INLINE COMMENTS

> hooper wrote in test-fix.t:857
> Would it abort or merge if the working copy is dirty? Is that behavior expected to end up in cleanupnodes? It could also just force --working-dir. It's similar to creating an orphan if we don't fix the dirty working copy.
> 
> One of my goals is to avoid any need for merging or conflict resolution in this command.

> One of my goals is to avoid any need for merging or conflict resolution in this command.

That makes sense. I am good with current behavior then.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D2897

To: hooper, #hg-reviewers, pulkit
Cc: durin42, krbullock, martinvonz, yuja, indygreg, pulkit, mercurial-devel

Patch

diff --git a/tests/test-fix.t b/tests/test-fix.t
new file mode 100644
--- /dev/null
+++ b/tests/test-fix.t
@@ -0,0 +1,927 @@ 
+Set up the config with two simple fixers: one that fixes specific line ranges,
+and one that always fixes the whole file. They both "fix" files by converting
+letters to uppercase. They use different file extensions, so each test case can
+choose which behavior to use by naming files.
+
+  $ cat >> $HGRCPATH <<EOF
+  > [extensions]
+  > fix =
+  > [experimental]
+  > evolution.createmarkers=True
+  > evolution.allowunstable=True
+  > [fix]
+  > uppercase-whole-file:command=sed -e 's/.*/\U&/'
+  > uppercase-whole-file:fileset=set:**.whole
+  > uppercase-changed-lines:command=sed
+  > uppercase-changed-lines:linerange=-e '{first},{last} s/.*/\U&/'
+  > uppercase-changed-lines:fileset=set:**.changed
+  > EOF
+
+Help text for fix.
+
+  $ hg help fix
+  hg fix [OPTION]... [FILE]...
+  
+  rewrite file content in changesets or working directory
+  
+      Runs any configured tools to fix the content of files. Only affects files
+      with changes, unless file arguments are provided. Only affects changed
+      lines of files, unless the --whole flag is used. Some tools may always
+      affect the whole file regardless of --whole.
+  
+      If revisions are specified with --rev, those revisions will be checked,
+      and they may be replaced with new revisions that have fixed file content.
+      It is desirable to specify all descendants of each specified revision, so
+      that the fixes propagate to the descendants. If all descendants are fixed
+      at the same time, no merging, rebasing, or evolution will be required.
+  
+      If --working-dir is used, files with uncommitted changes in the working
+      copy will be fixed. If the checked-out revision is also fixed, the working
+      directory will update to the replacement revision.
+  
+      When determining what lines of each file to fix at each revision, the
+      whole set of revisions being fixed is considered, so that fixes to earlier
+      revisions are not forgotten in later ones. The --base flag can be used to
+      override this default behavior, though it is not usually desirable to do
+      so.
+  
+  (use 'hg help -e fix' to show help for the fix extension)
+  
+  options ([+] can be repeated):
+  
+      --base REV [+] revisions to diff against (overrides automatic selection,
+                     and applies to every revision being fixed)
+   -r --rev REV [+]  revisions to fix
+   -w --working-dir  fix the working directory
+      --whole        always fix every line of a file
+  
+  (some details hidden, use --verbose to show complete help)
+
+  $ hg help -e fix
+  fix extension - rewrite file content in changesets or working copy
+  (EXPERIMENTAL)
+  
+  Provides a command that runs configured tools on the contents of modified
+  files, writing back any fixes to the working copy or replacing changesets.
+  
+  Here is an example configuration that causes 'hg fix' to apply automatic
+  formatting fixes to modified lines in C++ code:
+  
+    [fix]
+    clang-format:command=clang-format --assume-filename={rootpath}
+    clang-format:linerange=--lines={first}:{last}
+    clang-format:fileset=set:**.cpp or **.hpp
+  
+  The :command suboption forms the first part of the shell command that will be
+  used to fix a file. The content of the file is passed on standard input, and
+  the fixed file content is expected on standard output. If there is any output
+  on standard error, the file will not be affected. Some values may be
+  substituted into the command:
+  
+    {rootpath}  The path of the file being fixed, relative to the repo root
+    {basename}  The name of the file being fixed, without the directory path
+  
+  If the :linerange suboption is set, the tool will only be run if there are
+  changed lines in a file. The value of this suboption is appended to the shell
+  command once for every range of changed lines in the file. Some values may be
+  substituted into the command:
+  
+    {first}   The 1-based line number of the first line in the modified range
+    {last}    The 1-based line number of the last line in the modified range
+  
+  The :fileset suboption determines which files will be passed through each
+  configured tool. See 'hg help fileset' for possible values. If there are file
+  arguments to 'hg fix', the intersection of these filesets is used.
+  
+  There is also a configurable limit for the maximum size of file that will be
+  processed by 'hg fix':
+  
+    [fix]
+    maxfilesize=2MB
+  
+  list of commands:
+  
+   fix           rewrite file content in changesets or working directory
+  
+  (use 'hg help -v -e fix' to show built-in aliases and global options)
+
+There is no default behavior in the absence of --rev and --working-dir.
+
+  $ hg init badusage
+  $ cd badusage
+
+  $ hg fix
+  abort: no changesets specified
+  (use --rev or --working-dir)
+  [255]
+  $ hg fix --whole
+  abort: no changesets specified
+  (use --rev or --working-dir)
+  [255]
+  $ hg fix --base 0
+  abort: no changesets specified
+  (use --rev or --working-dir)
+  [255]
+
+Fixing a public revision isn't allowed. It should abort early enough that
+nothing happens, even to the working directory.
+
+  $ printf "hello\n" > hello.whole
+  $ hg commit -Am "hello"
+  adding hello.whole
+  $ hg phase -r 0 --public
+  $ hg fix -r 0
+  abort: can't fix immutable changeset 0:6470986d2e7b
+  [255]
+  $ hg fix -r 0 --working-dir
+  abort: can't fix immutable changeset 0:6470986d2e7b
+  [255]
+  $ hg cat -r tip hello.whole
+  hello
+  $ cat hello.whole
+  hello
+
+  $ cd ..
+
+Fixing a clean working directory should do nothing. Even the --whole flag
+shouldn't cause any clean files to be fixed. Specifying a clean file explicitly
+should only fix it if the fixer always fixes the whole file. The combination of
+an explicit filename and --whole should format the entire file regardless.
+
+  $ hg init fixcleanwdir
+  $ cd fixcleanwdir
+
+  $ printf "hello\n" > hello.changed
+  $ printf "world\n" > hello.whole
+  $ hg commit -Am "foo"
+  adding hello.changed
+  adding hello.whole
+  $ hg fix --working-dir
+  $ hg diff
+  $ hg fix --working-dir --whole
+  $ hg diff
+  $ hg fix --working-dir *
+  $ cat *
+  hello
+  WORLD
+  $ hg revert --all --no-backup
+  reverting hello.whole
+  $ hg fix --working-dir * --whole
+  $ cat *
+  HELLO
+  WORLD
+
+The same ideas apply to fixing a revision, so we create a revision that doesn't
+modify either of the files in question and try fixing it. This also tests that
+we ignore a file that doesn't match any configured fixer.
+
+  $ hg revert --all --no-backup
+  reverting hello.changed
+  reverting hello.whole
+  $ printf "unimportant\n" > some.file
+  $ hg commit -Am "some other file"
+  adding some.file
+
+  $ hg fix -r .
+  $ hg cat -r tip *
+  hello
+  world
+  unimportant
+  $ hg fix -r . --whole
+  $ hg cat -r tip *
+  hello
+  world
+  unimportant
+  $ hg fix -r . *
+  $ hg cat -r tip *
+  hello
+  WORLD
+  unimportant
+  $ hg fix -r . * --whole --config experimental.evolution.allowdivergence=true
+  2 new content-divergent changesets
+  $ hg cat -r tip *
+  HELLO
+  WORLD
+  unimportant
+
+  $ cd ..
+
+Fixing the working directory should still work if there are no revisions.
+
+  $ hg init norevisions
+  $ cd norevisions
+
+  $ printf "something\n" > something.whole
+  $ hg add
+  adding something.whole
+  $ hg fix --working-dir
+  $ cat something.whole
+  SOMETHING
+
+  $ cd ..
+
+Test the effect of fixing the working directory for each possible status, with
+and without providing explicit file arguments.
+
+  $ hg init implicitlyfixstatus
+  $ cd implicitlyfixstatus
+
+  $ printf "modified\n" > modified.whole
+  $ printf "removed\n" > removed.whole
+  $ printf "deleted\n" > deleted.whole
+  $ printf "clean\n" > clean.whole
+  $ printf "ignored.whole" > .hgignore
+  $ hg commit -Am "stuff"
+  adding .hgignore
+  adding clean.whole
+  adding deleted.whole
+  adding modified.whole
+  adding removed.whole
+
+  $ printf "modified!!!\n" > modified.whole
+  $ printf "unknown\n" > unknown.whole
+  $ printf "ignored\n" > ignored.whole
+  $ printf "added\n" > added.whole
+  $ hg add added.whole
+  $ hg remove removed.whole
+  $ rm deleted.whole
+
+  $ hg status --all
+  M modified.whole
+  A added.whole
+  R removed.whole
+  ! deleted.whole
+  ? unknown.whole
+  I ignored.whole
+  C .hgignore
+  C clean.whole
+
+  $ hg fix --working-dir
+
+  $ hg status --all
+  M modified.whole
+  A added.whole
+  R removed.whole
+  ! deleted.whole
+  ? unknown.whole
+  I ignored.whole
+  C .hgignore
+  C clean.whole
+
+  $ cat *.whole
+  ADDED
+  clean
+  ignored
+  MODIFIED!!!
+  unknown
+
+  $ printf "modified!!!\n" > modified.whole
+  $ printf "added\n" > added.whole
+  $ hg fix --working-dir *.whole
+
+  $ hg status --all
+  M clean.whole
+  M modified.whole
+  A added.whole
+  R removed.whole
+  ! deleted.whole
+  ? unknown.whole
+  I ignored.whole
+  C .hgignore
+
+It would be better if this also fixed the unknown file.
+  $ cat *.whole
+  ADDED
+  CLEAN
+  ignored
+  MODIFIED!!!
+  unknown
+
+  $ cd ..
+
+Test that incremental fixing works on files with additions, deletions, and
+changes in multiple line ranges. Note that deletions do not generally cause
+neighboring lines to be fixed, so we don't return a line range for purely
+deleted sections. In the future we should support a :deletion config that
+allows fixers to know where deletions are located.
+
+  $ hg init incrementalfixedlines
+  $ cd incrementalfixedlines
+
+  $ printf "a\nb\nc\nd\ne\nf\ng\n" > foo.txt
+  $ hg commit -Am "foo"
+  adding foo.txt
+  $ printf "zz\na\nc\ndd\nee\nff\nf\ngg\n" > foo.txt
+
+  $ hg --config "fix.fail:command=echo" \
+  >    --config "fix.fail:linerange={first}:{last}" \
+  >    --config "fix.fail:fileset=foo.txt" \
+  >    fix --working-dir
+  $ cat foo.txt
+  1:1 4:6 8:8
+
+  $ cd ..
+
+Test that --whole fixes all lines regardless of the diffs present.
+
+  $ hg init wholeignoresdiffs
+  $ cd wholeignoresdiffs
+
+  $ printf "a\nb\nc\nd\ne\nf\ng\n" > foo.changed
+  $ hg commit -Am "foo"
+  adding foo.changed
+  $ printf "zz\na\nc\ndd\nee\nff\nf\ngg\n" > foo.changed
+  $ hg fix --working-dir --whole
+  $ cat foo.changed
+  ZZ
+  A
+  C
+  DD
+  EE
+  FF
+  F
+  GG
+
+  $ cd ..
+
+We should do nothing with symlinks, and their targets should be unaffected. Any
+other behavior would be more complicated to implement and harder to document.
+
+#if symlink
+  $ hg init dontmesswithsymlinks
+  $ cd dontmesswithsymlinks
+
+  $ printf "hello\n" > hello.whole
+  $ ln -s hello.whole hellolink
+  $ hg add
+  adding hello.whole
+  adding hellolink
+  $ hg fix --working-dir hellolink
+  $ hg status
+  A hello.whole
+  A hellolink
+
+  $ cd ..
+#endif
+
+We should allow fixers to run on binary files, even though this doesn't sound
+like a common use case. There's not much benefit to disallowing it, and users
+can add "and not binary()" to their filesets if needed. The Mercurial
+philosophy is generally to not handle binary files specially anyway.
+
+  $ hg init cantouchbinaryfiles
+  $ cd cantouchbinaryfiles
+
+  $ printf "hello\0\n" > hello.whole
+  $ hg add
+  adding hello.whole
+  $ hg fix --working-dir 'set:binary()'
+  $ cat hello.whole
+  HELLO\x00 (esc)
+
+  $ cd ..
+
+We have a config for the maximum size of file we will attempt to fix. This can
+be helpful to avoid running unsuspecting fixer tools on huge inputs, which
+could happen by accident without a well considered configuration. A more
+precise configuration could use the size() fileset function if one global limit
+is undesired.
+
+  $ hg init maxfilesize
+  $ cd maxfilesize
+
+  $ printf "this file is huge\n" > hello.whole
+  $ hg add
+  adding hello.whole
+  $ hg --config fix.maxfilesize=10 fix --working-dir
+  ignoring file larger than 10 bytes: hello.whole
+  $ cat hello.whole
+  this file is huge
+
+  $ cd ..
+
+If we specify a file to fix, other files should be left alone, even if they
+have changes.
+
+  $ hg init fixonlywhatitellyouto
+  $ cd fixonlywhatitellyouto
+
+  $ printf "fix me!\n" > fixme.whole
+  $ printf "not me.\n" > notme.whole
+  $ hg add
+  adding fixme.whole
+  adding notme.whole
+  $ hg fix --working-dir fixme.whole
+  $ cat *.whole
+  FIX ME!
+  not me.
+
+  $ cd ..
+
+Specifying a directory name should fix all its files and subdirectories.
+
+  $ hg init fixdirectory
+  $ cd fixdirectory
+
+  $ mkdir -p dir1/dir2
+  $ printf "foo\n" > foo.whole
+  $ printf "bar\n" > dir1/bar.whole
+  $ printf "baz\n" > dir1/dir2/baz.whole
+  $ hg add
+  adding dir1/bar.whole
+  adding dir1/dir2/baz.whole
+  adding foo.whole
+  $ hg fix --working-dir dir1
+  $ cat foo.whole dir1/bar.whole dir1/dir2/baz.whole
+  foo
+  BAR
+  BAZ
+
+  $ cd ..
+
+Fixing a file in the working directory that needs no fixes should not actually
+write back to the file, so for example the mtime shouldn't change.
+
+  $ hg init donttouchunfixedfiles
+  $ cd donttouchunfixedfiles
+
+  $ printf "NO FIX NEEDED\n" > foo.whole
+  $ hg add
+  adding foo.whole
+  $ OLD_MTIME=`stat -c %Y foo.whole`
+  $ sleep 1 # mtime has a resolution of one second.
+  $ hg fix --working-dir
+  $ NEW_MTIME=`stat -c %Y foo.whole`
+  $ test $OLD_MTIME = $NEW_MTIME
+
+  $ cd ..
+
+When a fixer prints to stderr, we assume that it has failed. We should show the
+error messages to the user, and we should not let the failing fixer affect the
+file it was fixing (many code formatters might emit error messages on stderr
+and nothing on stdout, which would cause us the clear the file). We show the
+user which fixer failed and which revision, but we assume that the fixer will
+print the filename if it is relevant.
+
+  $ hg init showstderr
+  $ cd showstderr
+
+  $ printf "hello\n" > hello.txt
+  $ hg add
+  adding hello.txt
+  $ hg --config "fix.fail:command=printf 'HELLO\n' ; \
+  >                               printf '{rootpath}: some\nerror' >&2" \
+  >    --config "fix.fail:fileset=hello.txt" \
+  >    fix --working-dir
+  [wdir] fail: hello.txt: some
+  [wdir] fail: error
+  $ cat hello.txt
+  hello
+
+  $ cd ..
+
+Fixing the working directory and its parent revision at the same time should
+check out the replacement revision for the parent. This prevents any new
+uncommitted changes from appearing. We test this for a clean working directory
+and a dirty one. In both cases, all lines/files changed since the grandparent
+will be fixed. The grandparent is the "baserev" for both the parent and the
+working copy.
+
+  $ hg init fixdotandcleanwdir
+  $ cd fixdotandcleanwdir
+
+  $ printf "hello\n" > hello.whole
+  $ printf "world\n" > world.whole
+  $ hg commit -Am "the parent commit"
+  adding hello.whole
+  adding world.whole
+
+  $ hg parents --template '{rev} {desc}\n'
+  0 the parent commit
+  $ hg fix --working-dir -r .
+  $ hg parents --template '{rev} {desc}\n'
+  1 the parent commit
+  $ hg cat -r . *.whole
+  HELLO
+  WORLD
+  $ cat *.whole
+  HELLO
+  WORLD
+  $ hg status
+
+  $ cd ..
+
+Same test with a dirty working copy.
+
+  $ hg init fixdotanddirtywdir
+  $ cd fixdotanddirtywdir
+
+  $ printf "hello\n" > hello.whole
+  $ printf "world\n" > world.whole
+  $ hg commit -Am "the parent commit"
+  adding hello.whole
+  adding world.whole
+
+  $ printf "hello,\n" > hello.whole
+  $ printf "world!\n" > world.whole
+
+  $ hg parents --template '{rev} {desc}\n'
+  0 the parent commit
+  $ hg fix --working-dir -r .
+  $ hg parents --template '{rev} {desc}\n'
+  1 the parent commit
+  $ hg cat -r . *.whole
+  HELLO
+  WORLD
+  $ cat *.whole
+  HELLO,
+  WORLD!
+  $ hg status
+  M hello.whole
+  M world.whole
+
+  $ cd ..
+
+When we have a chain of commits that change mutually exclusive lines of code,
+we should be able to do incremental fixing that causes each commit in the chain
+to include fixes made to the previous commits. This prevents children from
+backing out the fixes made in their parents. A dirty working directory is
+conceptually similar to another commit in the chain.
+
+  $ hg init incrementallyfixchain
+  $ cd incrementallyfixchain
+
+  $ cat > file.changed <<EOF
+  > first
+  > second
+  > third
+  > fourth
+  > fifth
+  > EOF
+  $ hg commit -Am "the common ancestor (the baserev)"
+  adding file.changed
+  $ cat > file.changed <<EOF
+  > first (changed)
+  > second
+  > third
+  > fourth
+  > fifth
+  > EOF
+  $ hg commit -Am "the first commit to fix"
+  $ cat > file.changed <<EOF
+  > first (changed)
+  > second
+  > third (changed)
+  > fourth
+  > fifth
+  > EOF
+  $ hg commit -Am "the second commit to fix"
+  $ cat > file.changed <<EOF
+  > first (changed)
+  > second
+  > third (changed)
+  > fourth
+  > fifth (changed)
+  > EOF
+
+  $ hg fix -r . -r '.^' --working-dir
+
+  $ hg parents --template '{rev}\n'
+  4
+  $ hg cat -r '.^^' file.changed
+  first
+  second
+  third
+  fourth
+  fifth
+  $ hg cat -r '.^' file.changed
+  FIRST (CHANGED)
+  second
+  third
+  fourth
+  fifth
+  $ hg cat -r . file.changed
+  FIRST (CHANGED)
+  second
+  THIRD (CHANGED)
+  fourth
+  fifth
+  $ cat file.changed
+  FIRST (CHANGED)
+  second
+  THIRD (CHANGED)
+  fourth
+  FIFTH (CHANGED)
+
+  $ cd ..
+
+If we incrementally fix a merge commit, we should fix any lines that changed
+versus either parent. You could imagine only fixing the intersection or some
+other subset, but this is necessary if either parent is being fixed. It
+prevents us from forgetting fixes made in either parent.
+
+  $ hg init incrementallyfixmergecommit
+  $ cd incrementallyfixmergecommit
+
+  $ printf "a\nb\nc\n" > file.changed
+  $ hg commit -Am "ancestor"
+  adding file.changed
+
+  $ printf "aa\nb\nc\n" > file.changed
+  $ hg commit -m "change a"
+
+  $ hg checkout '.^'
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ printf "a\nb\ncc\n" > file.changed
+  $ hg commit -m "change c"
+  created new head
+
+  $ hg merge
+  merging file.changed
+  0 files updated, 1 files merged, 0 files removed, 0 files unresolved
+  (branch merge, don't forget to commit)
+  $ hg commit -m "merge"
+  $ hg cat -r . file.changed
+  aa
+  b
+  cc
+
+  $ hg fix -r . --working-dir
+  $ hg cat -r . file.changed
+  AA
+  b
+  CC
+
+  $ cd ..
+
+Abort fixing revisions if there is an unfinished operation. We don't want to
+make things worse by editing files or stripping/obsoleting things. Also abort
+fixing the working directory if there are unresolved merge conflicts.
+
+  $ hg init abortunresolved
+  $ cd abortunresolved
+
+  $ echo "foo1" > foo.whole
+  $ hg commit -Am "foo 1"
+  adding foo.whole
+
+  $ hg update null
+  0 files updated, 0 files merged, 1 files removed, 0 files unresolved
+  $ echo "foo2" > foo.whole
+  $ hg commit -Am "foo 2"
+  adding foo.whole
+  created new head
+
+  $ hg --config extensions.rebase= rebase -r 1 -d 0
+  rebasing 1:c3b6dc0e177a "foo 2" (tip)
+  merging foo.whole
+  warning: conflicts while merging foo.whole! (edit, then use 'hg resolve --mark')
+  unresolved conflicts (see hg resolve, then hg rebase --continue)
+  [1]
+
+  $ hg --config extensions.rebase= fix --working-dir
+  abort: unresolved conflicts
+  (use 'hg resolve')
+  [255]
+
+  $ hg --config extensions.rebase= fix -r .
+  abort: rebase in progress
+  (use 'hg rebase --continue' or 'hg rebase --abort')
+  [255]
+
+When fixing a file that was renamed, we should diff against the source of the
+rename for incremental fixing and we should correctly reproduce the rename in
+the replacement revision.
+
+  $ hg init fixrenamecommit
+  $ cd fixrenamecommit
+
+  $ printf "a\nb\nc\n" > source.changed
+  $ hg commit -Am "source revision"
+  adding source.changed
+  $ hg move source.changed dest.changed
+  $ printf "a\nb\ncc\n" > dest.changed
+  $ hg commit -m "dest revision"
+
+  $ hg fix -r .
+  $ hg log -r tip --copies --template "{file_copies}\n"
+  dest.changed (source.changed)
+  $ hg cat -r tip dest.changed
+  a
+  b
+  CC
+
+  $ cd ..
+
+When fixing revisions that remove files we must ensure that the replacement
+actually removes the file, whereas it could accidentally leave it unchanged or
+write an empty string to it.
+
+  $ hg init fixremovedfile
+  $ cd fixremovedfile
+
+  $ printf "foo\n" > foo.whole
+  $ printf "bar\n" > bar.whole
+  $ hg commit -Am "add files"
+  adding bar.whole
+  adding foo.whole
+  $ hg remove bar.whole
+  $ hg commit -m "remove file"
+  $ hg status --change .
+  R bar.whole
+  $ hg fix -r . foo.whole
+  $ hg status --change tip
+  M foo.whole
+  R bar.whole
+
+  $ cd ..
+
+If fixing a revision finds no fixes to make, no replacement revision should be
+created.
+
+  $ hg init nofixesneeded
+  $ cd nofixesneeded
+
+  $ printf "FOO\n" > foo.whole
+  $ hg commit -Am "add file"
+  adding foo.whole
+  $ hg log --template '{rev}\n'
+  0
+  $ hg fix -r .
+  $ hg log --template '{rev}\n'
+  0
+
+  $ cd ..
+
+If fixing a commit reverts all the changes in the commit, we replace it with a
+commit that changes no files.
+
+  $ hg init nochangesleft
+  $ cd nochangesleft
+
+  $ printf "FOO\n" > foo.whole
+  $ hg commit -Am "add file"
+  adding foo.whole
+  $ printf "foo\n" > foo.whole
+  $ hg commit -m "edit file"
+  $ hg status --change .
+  M foo.whole
+  $ hg fix -r .
+  $ hg status --change tip
+
+  $ cd ..
+
+If we fix a parent and child revision together, the child revision must be
+replaced if the parent is replaced, even if the diffs of the child needed no
+fixes. However, we're free to not replace revisions that need no fixes and have
+no ancestors that are replaced.
+
+  $ hg init mustreplacechild
+  $ cd mustreplacechild
+
+  $ printf "FOO\n" > foo.whole
+  $ hg commit -Am "add foo"
+  adding foo.whole
+  $ printf "foo\n" > foo.whole
+  $ hg commit -m "edit foo"
+  $ printf "BAR\n" > bar.whole
+  $ hg commit -Am "add bar"
+  adding bar.whole
+
+  $ hg log --graph --template '{node|shortest} {files}'
+  @  bc05 bar.whole
+  |
+  o  4fd2 foo.whole
+  |
+  o  f9ac foo.whole
+  
+  $ hg fix -r 0:2
+  $ hg log --graph --template '{node|shortest} {files}'
+  o  3801 bar.whole
+  |
+  o  38cc
+  |
+  | @  bc05 bar.whole
+  | |
+  | x  4fd2 foo.whole
+  |/
+  o  f9ac foo.whole
+  
+
+  $ cd ..
+
+It's also possible that the child needs absolutely no changes, but we still
+need to replace it to update its parent. If we simply skip replacing it because
+it has no file content changes, this test will create an orphan.
+
+  $ hg init mustreplacechildevenifnop
+  $ cd mustreplacechildevenifnop
+
+  $ printf "Foo\n" > foo.whole
+  $ hg commit -Am "add a bad foo"
+  adding foo.whole
+  $ printf "FOO\n" > foo.whole
+  $ hg commit -m "add a good foo"
+  $ hg fix -r . -r '.^'
+  $ hg log --graph --template '{rev} {desc}'
+  o  3 add a good foo
+  |
+  o  2 add a bad foo
+  
+  @  1 add a good foo
+  |
+  x  0 add a bad foo
+  
+
+  $ cd ..
+
+Fixing a secret commit should replace it with another secret commit.
+
+  $ hg init fixsecretcommit
+  $ cd fixsecretcommit
+
+  $ printf "foo\n" > foo.whole
+  $ hg commit -Am "add foo" --secret
+  adding foo.whole
+  $ hg fix -r .
+  $ hg log --template '{rev} {phase}\n'
+  1 secret
+  0 secret
+
+  $ cd ..
+
+We should also preserve phase when fixing a draft commit while the user has
+their default set to secret.
+
+  $ hg init respectphasesnewcommit
+  $ cd respectphasesnewcommit
+
+  $ printf "foo\n" > foo.whole
+  $ hg commit -Am "add foo"
+  adding foo.whole
+  $ hg --config phases.newcommit=secret fix -r .
+  $ hg log --template '{rev} {phase}\n'
+  1 draft
+  0 draft
+
+  $ cd ..
+
+Debug output should show what fixer commands are being subprocessed, which is
+useful for anyone trying to set up a new config.
+
+  $ hg init debugoutput
+  $ cd debugoutput
+
+  $ printf "foo\nbar\nbaz\n" > foo.changed
+  $ hg commit -Am "foo"
+  adding foo.changed
+  $ printf "Foo\nbar\nBaz\n" > foo.changed
+  $ hg --debug fix --working-dir
+  subprocess: sed -e '1,1 s/.*/\U&/' -e '3,3 s/.*/\U&/'
+
+  $ cd ..
+
+Fixing an obsolete revision can cause divergence, so we abort unless the user
+configures to allow it. This is not yet smart enough to know whether there is a
+successor, but even then it is not likely intentional or idiomatic to fix an
+obsolete revision.
+
+  $ hg init abortobsoleterev
+  $ cd abortobsoleterev
+
+  $ printf "foo\n" > foo.changed
+  $ hg commit -Am "foo"
+  adding foo.changed
+  $ hg debugobsolete `hg parents --template '{node}'`
+  obsoleted 1 changesets
+  $ hg --hidden fix -r 0
+  abort: fixing obsolete revision could cause divergence
+  [255]
+
+  $ hg --hidden fix -r 0 --config experimental.evolution.allowdivergence=true
+  $ hg cat -r tip foo.changed
+  FOO
+
+  $ cd ..
+
+Test all of the available substitution values for fixer commands.
+
+  $ hg init substitution
+  $ cd substitution
+
+  $ mkdir foo
+  $ printf "hello\ngoodbye\n" > foo/bar
+  $ hg add
+  adding foo/bar
+  $ hg --config "fix.fail:command=printf '%s\n' '{rootpath}' '{basename}'" \
+  >    --config "fix.fail:linerange='{first}' '{last}'" \
+  >    --config "fix.fail:fileset=foo/bar" \
+  >    fix --working-dir
+  $ cat foo/bar
+  foo/bar
+  bar
+  1
+  2
+
+  $ cd ..
+
diff --git a/tests/test-fix-topology.t b/tests/test-fix-topology.t
new file mode 100644
--- /dev/null
+++ b/tests/test-fix-topology.t
@@ -0,0 +1,252 @@ 
+Tests for the fix extension's behavior around non-trivial history topologies.
+Looks for correct incremental fixing and reproduction of parent/child
+relationships. We indicate fixed file content by uppercasing it.
+
+  $ cat >> $HGRCPATH <<EOF
+  > [extensions]
+  > fix =
+  > [fix]
+  > uppercase-whole-file:command=sed -e 's/.*/\U&/'
+  > uppercase-whole-file:fileset=set:**
+  > EOF
+
+This tests the only behavior that should really be affected by obsolescence, so
+we'll test it with evolution off and on. This only changes the revision
+numbers, if all is well.
+
+#testcases no-evolution evolution
+#if evolution
+  $ cat >> $HGRCPATH <<EOF
+  > [experimental]
+  > evolution.createmarkers=True
+  > evolution.allowunstable=True
+  > EOF
+#endif
+
+Setting up the test topology. Scroll down to see the graph produced. We make it
+clear which files were modified in each revision. It's enough to test at the
+file granularity, because that demonstrates which baserevs were diffed against.
+The computation of changed lines is orthogonal and tested separately.
+
+  $ hg init repo
+  $ cd repo
+
+  $ printf "aaaa\n" > a
+  $ hg commit -Am "change A"
+  adding a
+  $ printf "bbbb\n" > b
+  $ hg commit -Am "change B"
+  adding b
+  $ printf "cccc\n" > c
+  $ hg commit -Am "change C"
+  adding c
+  $ hg checkout 0
+  0 files updated, 0 files merged, 2 files removed, 0 files unresolved
+  $ printf "dddd\n" > d
+  $ hg commit -Am "change D"
+  adding d
+  created new head
+  $ hg merge -r 2
+  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  (branch merge, don't forget to commit)
+  $ printf "eeee\n" > e
+  $ hg commit -Am "change E"
+  adding e
+  $ hg checkout 0
+  0 files updated, 0 files merged, 4 files removed, 0 files unresolved
+  $ printf "ffff\n" > f
+  $ hg commit -Am "change F"
+  adding f
+  created new head
+  $ hg checkout 0
+  0 files updated, 0 files merged, 1 files removed, 0 files unresolved
+  $ printf "gggg\n" > g
+  $ hg commit -Am "change G"
+  adding g
+  created new head
+  $ hg merge -r 5
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  (branch merge, don't forget to commit)
+  $ printf "hhhh\n" > h
+  $ hg commit -Am "change H"
+  adding h
+  $ hg merge -r 4
+  4 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  (branch merge, don't forget to commit)
+  $ printf "iiii\n" > i
+  $ hg commit -Am "change I"
+  adding i
+  $ hg checkout 2
+  0 files updated, 0 files merged, 6 files removed, 0 files unresolved
+  $ printf "jjjj\n" > j
+  $ hg commit -Am "change J"
+  adding j
+  created new head
+  $ hg checkout 7
+  3 files updated, 0 files merged, 3 files removed, 0 files unresolved
+  $ printf "kkkk\n" > k
+  $ hg add
+  adding k
+
+  $ hg log --graph --template '{rev} {desc}\n'
+  o  9 change J
+  |
+  | o    8 change I
+  | |\
+  | | @    7 change H
+  | | |\
+  | | | o  6 change G
+  | | | |
+  | | o |  5 change F
+  | | |/
+  | o |  4 change E
+  |/| |
+  | o |  3 change D
+  | |/
+  o |  2 change C
+  | |
+  o |  1 change B
+  |/
+  o  0 change A
+  
+
+Fix all but the root revision and its four children.
+
+#if evolution
+  $ hg fix -r '2|4|7|8|9' --working-dir
+#else
+  $ hg fix -r '2|4|7|8|9' --working-dir
+  saved backup bundle to * (glob)
+#endif
+
+The five revisions remain, but the other revisions were fixed and replaced. All
+parent pointers have been accurately set to reproduce the previous topology
+(though it is rendered in a slightly different order now).
+
+#if evolution
+  $ hg log --graph --template '{rev} {desc}\n'
+  o  14 change J
+  |
+  | o    13 change I
+  | |\
+  | | @    12 change H
+  | | |\
+  | o | |  11 change E
+  |/| | |
+  o | | |  10 change C
+  | | | |
+  | | | o  6 change G
+  | | | |
+  | | o |  5 change F
+  | | |/
+  | o /  3 change D
+  | |/
+  o /  1 change B
+  |/
+  o  0 change A
+  
+  $ C=10
+  $ E=11
+  $ H=12
+  $ I=13
+  $ J=14
+#else
+  $ hg log --graph --template '{rev} {desc}\n'
+  o  9 change J
+  |
+  | o    8 change I
+  | |\
+  | | @    7 change H
+  | | |\
+  | o | |  6 change E
+  |/| | |
+  o | | |  5 change C
+  | | | |
+  | | | o  4 change G
+  | | | |
+  | | o |  3 change F
+  | | |/
+  | o /  2 change D
+  | |/
+  o /  1 change B
+  |/
+  o  0 change A
+  
+  $ C=5
+  $ E=6
+  $ H=7
+  $ I=8
+  $ J=9
+#endif
+
+Change C is a root of the set being fixed, so all we fix is what has changed
+since its parent. That parent, change B, is its baserev.
+
+  $ hg cat -r $C 'set:**'
+  aaaa
+  bbbb
+  CCCC
+
+Change E is a merge with only one parent being fixed. Its baserevs are the
+unfixed parent plus the baserevs of the other parent. This evaluates to changes
+B and D. We now have to decide what it means to incrementally fix a merge
+commit. We choose to fix anything that has changed versus any baserev. Only the
+undisturbed content of the common ancestor, change A, is unfixed.
+
+  $ hg cat -r $E 'set:**'
+  aaaa
+  BBBB
+  CCCC
+  DDDD
+  EEEE
+
+Change H is a merge with neither parent being fixed. This is essentially
+equivalent to the previous case because there is still only one baserev for
+each parent of the merge.
+
+  $ hg cat -r $H 'set:**'
+  aaaa
+  FFFF
+  GGGG
+  HHHH
+
+Change I is a merge that has four baserevs; two from each parent. We handle
+multiple baserevs in the same way regardless of how many came from each parent.
+So, fixing change H will fix any files that were not exactly the same in each
+baserev.
+
+  $ hg cat -r $I 'set:**'
+  aaaa
+  BBBB
+  CCCC
+  DDDD
+  EEEE
+  FFFF
+  GGGG
+  HHHH
+  IIII
+
+Change J is a simple case with one baserev, but its baserev is not its parent,
+change C. Its baserev is its grandparent, change B.
+
+  $ hg cat -r $J 'set:**'
+  aaaa
+  bbbb
+  CCCC
+  JJJJ
+
+The working copy was dirty, so it is treated much like a revision. The baserevs
+for the working copy are inherited from its parent, change H, because it is
+also being fixed.
+
+  $ cat *
+  aaaa
+  FFFF
+  GGGG
+  HHHH
+  KKKK
+
+Change A was never a baserev because none of its children were to be fixed.
+
+  $ cd ..
+
diff --git a/tests/test-fix-clang-format.t b/tests/test-fix-clang-format.t
new file mode 100644
--- /dev/null
+++ b/tests/test-fix-clang-format.t
@@ -0,0 +1,34 @@ 
+#require clang-format
+
+Test that a simple "hg fix" configuration for clang-format works.
+
+  $ cat >> $HGRCPATH <<EOF
+  > [extensions]
+  > fix =
+  > [experimental]
+  > evolution.createmarkers=True
+  > evolution.allowunstable=True
+  > [fix]
+  > clang-format:command=clang-format --style=Google --assume-filename={rootpath}
+  > clang-format:linerange=--lines={first}:{last}
+  > clang-format:fileset=set:**.cpp or **.hpp
+  > EOF
+
+  $ hg init repo
+  $ cd repo
+
+  $ printf "void foo(){int x=2;}\n" > foo.cpp
+  $ printf "void\nfoo();\n" > foo.hpp
+  $ hg commit -Am "foo commit"
+  adding foo.cpp
+  adding foo.hpp
+  $ hg cat -r tip *
+  void foo(){int x=2;}
+  void
+  foo();
+  $ hg fix -r tip
+  $ hg cat -r tip *
+  void foo() { int x = 2; }
+  void foo();
+
+  $ cd ..
diff --git a/tests/test-doctest.py b/tests/test-doctest.py
--- a/tests/test-doctest.py
+++ b/tests/test-doctest.py
@@ -75,6 +75,7 @@ 
 testmod('hgext.convert.filemap')
 testmod('hgext.convert.p4')
 testmod('hgext.convert.subversion')
+testmod('hgext.fix')
 testmod('hgext.mq')
 # Helper scripts in tests/ that have doctests:
 testmod('drawdag')
diff --git a/hgext/fix.py b/hgext/fix.py
new file mode 100644
--- /dev/null
+++ b/hgext/fix.py
@@ -0,0 +1,544 @@ 
+# fix - rewrite file content in changesets and working copy
+#
+# Copyright 2018 Google LLC.
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+"""rewrite file content in changesets or working copy (EXPERIMENTAL)
+
+Provides a command that runs configured tools on the contents of modified files,
+writing back any fixes to the working copy or replacing changesets.
+
+Here is an example configuration that causes :hg:`fix` to apply automatic
+formatting fixes to modified lines in C++ code::
+
+  [fix]
+  clang-format:command=clang-format --assume-filename={rootpath}
+  clang-format:linerange=--lines={first}:{last}
+  clang-format:fileset=set:**.cpp or **.hpp
+
+The :command suboption forms the first part of the shell command that will be
+used to fix a file. The content of the file is passed on standard input, and the
+fixed file content is expected on standard output. If there is any output on
+standard error, the file will not be affected. Some values may be substituted
+into the command::
+
+  {rootpath}  The path of the file being fixed, relative to the repo root
+  {basename}  The name of the file being fixed, without the directory path
+
+If the :linerange suboption is set, the tool will only be run if there are
+changed lines in a file. The value of this suboption is appended to the shell
+command once for every range of changed lines in the file. Some values may be
+substituted into the command::
+
+  {first}   The 1-based line number of the first line in the modified range
+  {last}    The 1-based line number of the last line in the modified range
+
+The :fileset suboption determines which files will be passed through each
+configured tool. See :hg:`help fileset` for possible values. If there are file
+arguments to :hg:`fix`, the intersection of these filesets is used.
+
+There is also a configurable limit for the maximum size of file that will be
+processed by :hg:`fix`::
+
+  [fix]
+  maxfilesize=2MB
+
+"""
+
+from __future__ import absolute_import
+
+import collections
+import itertools
+import os
+import re
+import subprocess
+import sys
+
+from mercurial.i18n import _
+from mercurial.node import nullrev
+from mercurial.node import wdirrev
+
+from mercurial import (
+    cmdutil,
+    context,
+    copies,
+    error,
+    match,
+    mdiff,
+    merge,
+    posix,
+    registrar,
+    scmutil,
+    util,
+)
+
+# Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
+# extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
+# be specifying the version(s) of Mercurial they are tested with, or
+# leave the attribute unspecified.
+testedwith = 'ships-with-hg-core'
+
+cmdtable = {}
+command = registrar.command(cmdtable)
+
+configtable = {}
+configitem = registrar.configitem(configtable)
+
+# Register the suboptions allowed for each configured fixer.
+FIXER_ATTRS = ('command', 'linerange', 'fileset')
+
+for key in FIXER_ATTRS:
+    configitem('fix', '.*(:%s)?' % key, default=None, generic=True)
+
+# A good default size allows most source code files to be fixed, but avoids
+# letting fixer tools choke on huge inputs, which could be surprising to the
+# user.
+configitem('fix', 'maxfilesize', default='2MB')
+
+@command('fix',
+    [('', 'base', [], _('revisions to diff against (overrides automatic '
+                        'selection, and applies to every revision being '
+                        'fixed)'), _('REV')),
+     ('r', 'rev', [], _('revisions to fix'), _('REV')),
+     ('w', 'working-dir', False, _('fix the working directory')),
+     ('', 'whole', False, _('always fix every line of a file'))],
+    _('[OPTION]... [FILE]...'))
+def fix(ui, repo, *pats, **opts):
+    """rewrite file content in changesets or working directory
+
+    Runs any configured tools to fix the content of files. Only affects files
+    with changes, unless file arguments are provided. Only affects changed lines
+    of files, unless the --whole flag is used. Some tools may always affect the
+    whole file regardless of --whole.
+
+    If revisions are specified with --rev, those revisions will be checked, and
+    they may be replaced with new revisions that have fixed file content.  It is
+    desirable to specify all descendants of each specified revision, so that the
+    fixes propagate to the descendants. If all descendants are fixed at the same
+    time, no merging, rebasing, or evolution will be required.
+
+    If --working-dir is used, files with uncommitted changes in the working copy
+    will be fixed. If the checked-out revision is also fixed, the working
+    directory will update to the replacement revision.
+
+    When determining what lines of each file to fix at each revision, the whole
+    set of revisions being fixed is considered, so that fixes to earlier
+    revisions are not forgotten in later ones. The --base flag can be used to
+    override this default behavior, though it is not usually desirable to do so.
+    """
+    with repo.wlock(), repo.lock():
+        revstofix = getrevstofix(ui, repo, opts)
+        basectxs = getbasectxs(repo, opts, revstofix)
+        workqueue, numitems = getworkqueue(ui, repo, pats, opts, revstofix,
+                                           basectxs)
+        filedata = collections.defaultdict(dict)
+        replacements = {}
+        fixers = getfixers(ui)
+        # Some day this loop can become a worker pool, but for now it's easier
+        # to fix everything serially in topological order.
+        for rev, path in sorted(workqueue):
+            ctx = repo[rev]
+            olddata = ctx[path].data()
+            newdata = fixfile(ui, opts, fixers, ctx, path, basectxs[rev])
+            if newdata != olddata:
+                filedata[rev][path] = newdata
+            numitems[rev] -= 1
+            if not numitems[rev]:
+                if rev == wdirrev:
+                    writeworkingdir(repo, ctx, filedata[rev], replacements)
+                else:
+                    replacerev(ui, repo, ctx, filedata[rev], replacements)
+                del filedata[rev]
+
+        replacements = {prec: [succ] for prec, succ in replacements.iteritems()}
+        scmutil.cleanupnodes(repo, replacements, 'fix')
+
+def getworkqueue(ui, repo, pats, opts, revstofix, basectxs):
+    """"Constructs the list of files to be fixed at specific revisions
+
+    It is up to the caller how to consume the work items, and the only
+    dependence between them is that replacement revisions must be committed in
+    topological order. Each work item represents a file in the working copy or
+    in some revision that should be fixed and written back to the working copy
+    or into a replacement revision.
+    """
+    workqueue = []
+    numitems = collections.defaultdict(int)
+    maxfilesize = ui.configbytes('fix', 'maxfilesize')
+    for rev in revstofix:
+        fixctx = repo[rev]
+        match = scmutil.match(fixctx, pats, opts)
+        for path in pathstofix(ui, repo, pats, opts, match, basectxs[rev],
+                               fixctx):
+            if path not in fixctx:
+                continue
+            fctx = fixctx[path]
+            if fctx.islink():
+                continue
+            if fctx.size() > maxfilesize:
+                ui.warn(_('ignoring file larger than %s: %s\n') %
+                        (util.bytecount(maxfilesize), path))
+                continue
+            workqueue.append((rev, path))
+            numitems[rev] += 1
+    return workqueue, numitems
+
+def getrevstofix(ui, repo, opts):
+    """Returns the set of revision numbers that should be fixed"""
+    revs = set(scmutil.revrange(repo, opts['rev']))
+    for rev in revs:
+        checkfixablectx(ui, repo, repo[rev])
+    if revs:
+        cmdutil.checkunfinished(repo)
+    if opts.get('working_dir'):
+        revs.add(wdirrev)
+        if list(merge.mergestate.read(repo).unresolved()):
+            raise error.Abort('unresolved conflicts', hint="use 'hg resolve'")
+    if not revs:
+        raise error.Abort(
+            'no changesets specified', hint='use --rev or --working-dir')
+    return revs
+
+def checkfixablectx(ui, repo, ctx):
+    """Aborts if the revision shouldn't be replaced with a fixed one."""
+    if not ctx.mutable():
+        raise error.Abort('can\'t fix immutable changeset %s' %
+                          (scmutil.formatchangeid(ctx),))
+    if ctx.obsolete():
+        # It would be better to actually check if the revision has a successor.
+        allowdivergence = ui.configbool('experimental',
+                                        'evolution.allowdivergence')
+        if not allowdivergence:
+            raise error.Abort('fixing obsolete revision could cause divergence')
+
+def pathstofix(ui, repo, pats, opts, match, basectxs, fixctx):
+    """Returns the set of files that should be fixed in a context
+
+    The result depends on the base contexts; we include any file that has
+    changed relative to any of the base contexts. Base contexts should be
+    ancestors of the context being fixed.
+    """
+    files = set()
+    for basectx in basectxs:
+        stat = repo.status(
+            basectx, fixctx, match=match, clean=bool(pats), unknown=bool(pats))
+        files.update(
+            set(itertools.chain(stat.added, stat.modified, stat.clean,
+                                stat.unknown)))
+    return files
+
+def lineranges(opts, path, basectxs, fixctx, content2):
+    """Returns the set of line ranges that should be fixed in a file
+
+    Of the form [(10, 20), (30, 40)].
+
+    This depends on the given base contexts; we must consider lines that have
+    changed versus any of the base contexts, and whether the file has been
+    renamed versus any of them.
+
+    Another way to understand this is that we exclude line ranges that are
+    common to the file in all base contexts.
+    """
+    if opts.get('whole'):
+        # Return a range containing all lines. Rely on the diff implementation's
+        # idea of how many lines are in the file, instead of reimplementing it.
+        return difflineranges('', content2)
+
+    rangeslist = []
+    for basectx in basectxs:
+        basepath = copies.pathcopies(basectx, fixctx).get(path, path)
+        if basepath in basectx:
+            content1 = basectx[basepath].data()
+        else:
+            content1 = ''
+        rangeslist.extend(difflineranges(content1, content2))
+    return unionranges(rangeslist)
+
+def unionranges(rangeslist):
+    """Return the union of some closed intervals
+
+    >>> unionranges([])
+    []
+    >>> unionranges([(1, 100)])
+    [(1, 100)]
+    >>> unionranges([(1, 100), (1, 100)])
+    [(1, 100)]
+    >>> unionranges([(1, 100), (2, 100)])
+    [(1, 100)]
+    >>> unionranges([(1, 99), (1, 100)])
+    [(1, 100)]
+    >>> unionranges([(1, 100), (40, 60)])
+    [(1, 100)]
+    >>> unionranges([(1, 49), (50, 100)])
+    [(1, 100)]
+    >>> unionranges([(1, 48), (50, 100)])
+    [(1, 48), (50, 100)]
+    >>> unionranges([(1, 2), (3, 4), (5, 6)])
+    [(1, 6)]
+    """
+    rangeslist = sorted(set(rangeslist))
+    unioned = []
+    if rangeslist:
+        unioned, rangeslist = [rangeslist[0]], rangeslist[1:]
+    for a, b in rangeslist:
+        c, d = unioned[-1]
+        if a > d + 1:
+            unioned.append((a, b))
+        else:
+            unioned[-1] = (c, max(b, d))
+    return unioned
+
+def difflineranges(content1, content2):
+    """Return list of line number ranges in content2 that differ from content1.
+
+    Line numbers are 1-based. The numbers are the first and last line contained
+    in the range. Single-line ranges have the same line number for the first and
+    last line. Excludes any empty ranges that result from lines that are only
+    present in content1. Relies on mdiff's idea of where the line endings are in
+    the string.
+
+    >>> lines = lambda s: '\\n'.join([c for c in s])
+    >>> difflineranges2 = lambda a, b: difflineranges(lines(a), lines(b))
+    >>> difflineranges2('', '')
+    []
+    >>> difflineranges2('a', '')
+    []
+    >>> difflineranges2('', 'A')
+    [(1, 1)]
+    >>> difflineranges2('a', 'a')
+    []
+    >>> difflineranges2('a', 'A')
+    [(1, 1)]
+    >>> difflineranges2('ab', '')
+    []
+    >>> difflineranges2('', 'AB')
+    [(1, 2)]
+    >>> difflineranges2('abc', 'ac')
+    []
+    >>> difflineranges2('ab', 'aCb')
+    [(2, 2)]
+    >>> difflineranges2('abc', 'aBc')
+    [(2, 2)]
+    >>> difflineranges2('ab', 'AB')
+    [(1, 2)]
+    >>> difflineranges2('abcde', 'aBcDe')
+    [(2, 2), (4, 4)]
+    >>> difflineranges2('abcde', 'aBCDe')
+    [(2, 4)]
+    """
+    ranges = []
+    for lines, kind in mdiff.allblocks(content1, content2):
+        firstline, lastline = lines[2:4]
+        if kind == '!' and firstline != lastline:
+            ranges.append((firstline + 1, lastline))
+    return ranges
+
+def getbasectxs(repo, opts, revstofix):
+    """Returns a map of the base contexts for each revision
+
+    The base contexts determine which lines are considered modified when we
+    attempt to fix just the modified lines in a file.
+    """
+    # The --base flag overrides the usual logic, and we give every revision
+    # exactly the set of baserevs that the user specified.
+    if opts.get('base'):
+        revs = set(scmutil.revrange(repo, [opts.get('base')]))
+        if not revs:
+            revs = {nullrev}
+        return {rev: revs for rev in revstofix}
+
+    # Proceed in topological order so that we can easily determine each
+    # revision's baserevs by looking at its parents and their baserevs.
+    basectxs = collections.defaultdict(set)
+    for rev in sorted(revstofix):
+        ctx = repo[rev]
+        for pctx in ctx.parents():
+            if pctx.rev() in basectxs:
+                basectxs[rev].update(basectxs[pctx.rev()])
+            else:
+                basectxs[rev].add(pctx)
+    return basectxs
+
+def fixfile(ui, opts, fixers, fixctx, path, basectxs):
+    """Run any configured fixers that should affect the file in this context
+
+    Returns the file content that results from applying the fixers in some order
+    starting with the file's content in the fixctx. Fixers that support line
+    ranges will affect lines that have changed relative to any of the basectxs
+    (i.e. they will only avoid lines that are common to all basectxs).
+    """
+    newdata = fixctx[path].data()
+    for fixername, fixer in fixers.iteritems():
+        if fixer.affects(opts, fixctx, path):
+            ranges = lineranges(opts, path, basectxs, fixctx, newdata)
+            command = fixer.command(path, ranges)
+            if command is None:
+                continue
+            ui.debug('subprocess: %s\n' % (command,))
+            proc = subprocess.Popen(
+                command,
+                shell=True,
+                cwd='/',
+                stdin=subprocess.PIPE,
+                stdout=subprocess.PIPE,
+                stderr=subprocess.PIPE)
+            newerdata, stderr = proc.communicate(newdata)
+            if stderr:
+                showstderr(ui, fixctx.rev(), fixername, stderr)
+            else:
+                newdata = newerdata
+    return newdata
+
+def showstderr(ui, rev, fixername, stderr):
+    """Writes the lines of the stderr string as warnings on the ui
+
+    Uses the revision number and fixername to give more context to each line of
+    the error message. Doesn't include file names, since those take up a lot of
+    space and would tend to be included in the error message if they were
+    relevant.
+    """
+    for line in re.split('[\r\n]+', stderr):
+        if line:
+            ui.warn(('['))
+            if rev is None:
+                ui.warn(_('wdir'), label='evolve.rev')
+            else:
+                ui.warn((str(rev)), label='evolve.rev')
+            ui.warn(('] %s: %s\n') % (fixername, line))
+
+def writeworkingdir(repo, ctx, filedata, replacements):
+    """Write new content to the working copy and check out the new p1 if any
+
+    We check out a new revision if and only if we fixed something in both the
+    working directory and its parent revision. This avoids the need for a full
+    update/merge, and means that the working directory simply isn't affected
+    unless the --working-dir flag is given.
+
+    Directly updates the dirstate for the affected files.
+    """
+    for path, data in filedata.iteritems():
+        fctx = ctx[path]
+        fctx.write(data, fctx.flags())
+        if repo.dirstate[path] == 'n':
+            repo.dirstate.normallookup(path)
+
+    oldparentnodes = repo.dirstate.parents()
+    newparentnodes = [replacements.get(n, n) for n in oldparentnodes]
+    if newparentnodes != oldparentnodes:
+        repo.setparents(*newparentnodes)
+
+def replacerev(ui, repo, ctx, filedata, replacements):
+    """Commit a new revision like the given one, but with file content changes
+
+    "ctx" is the original revision to be replaced by a modified one.
+
+    "filedata" is a dict that maps paths to their new file content. All other
+    paths will be recreated from the original revision without changes.
+    "filedata" may contain paths that didn't exist in the original revision;
+    they will be added.
+
+    "replacements" is a dict that maps a single node to a single node, and it is
+    updated to indicate the original revision is replaced by the newly created
+    one. No entry is added if the replacement's node already exists.
+
+    The new revision has the same parents as the old one, unless those parents
+    have already been replaced, in which case those replacements are the parents
+    of this new revision. Thus, if revisions are replaced in topological order,
+    there is no need to rebase them into the original topology later.
+    """
+
+    p1rev, p2rev = repo.changelog.parentrevs(ctx.rev())
+    p1ctx, p2ctx = repo[p1rev], repo[p2rev]
+    newp1node = replacements.get(p1ctx.node(), p1ctx.node())
+    newp2node = replacements.get(p2ctx.node(), p2ctx.node())
+
+    def memfilectx(repo, ctx, path, data, islink, isexec, copied):
+        varnames = context.memfilectx.__init__.__code__.co_varnames
+        ctxmandatory = varnames[2] == 'changectx'
+        if ctxmandatory:
+            return context.memfilectx(repo, ctx, path, data, islink, isexec,
+                                      copied)
+        else:
+            return context.memfilectx(repo, path, data, islink, isexec, copied)
+
+    def filectxfn(repo, memctx, path):
+        if path not in ctx:
+            return None
+        fctx = ctx[path]
+        copied = fctx.renamed()
+        if copied:
+            copied = copied[0]
+        return memfilectx(
+            repo,
+            ctx=memctx,
+            path=fctx.path(),
+            data=filedata.get(path, fctx.data()),
+            islink=fctx.islink(),
+            isexec=fctx.isexec(),
+            copied=copied)
+
+    overrides = {('phases', 'new-commit'): ctx.phase()}
+    with ui.configoverride(overrides, source='fix'):
+        memctx = context.memctx(
+            repo,
+            parents=(newp1node, newp2node),
+            text=ctx.description(),
+            files=set(ctx.files()) | set(filedata.keys()),
+            filectxfn=filectxfn,
+            user=ctx.user(),
+            date=ctx.date(),
+            extra=ctx.extra(),
+            branch=ctx.branch(),
+            editor=None)
+        sucnode = memctx.commit()
+        prenode = ctx.node()
+        if prenode == sucnode:
+            ui.debug('node %s already existed\n' % (ctx.hex()))
+        else:
+            replacements[ctx.node()] = sucnode
+
+def getfixers(ui):
+    """Returns a map of configured fixer tools indexed by their names
+
+    Each value is a Fixer object with methods that implement the behavior of the
+    fixer's config suboptions. Does not validate the config values.
+    """
+    result = {}
+    for name in fixernames(ui):
+        result[name] = Fixer()
+        attrs = ui.configsuboptions('fix', name)[1]
+        for key in FIXER_ATTRS:
+            setattr(result[name], '_' + key, attrs.get(key, ''))
+    return result
+
+def fixernames(ui):
+    """Returns the names of [fix] config options that have suboptions"""
+    names = set()
+    for k, v in ui.configitems('fix'):
+        if ':' in k:
+            names.add(k.split(':', 1)[0])
+    return names
+
+class Fixer(object):
+    """Wraps the raw config values for a fixer with methods"""
+
+    def affects(self, opts, fixctx, path):
+        """Should this fixer run on the file at the given path and context?"""
+        return scmutil.match(fixctx, [self._fileset], opts)(path)
+
+    def command(self, path, ranges):
+        """A shell command to use to invoke this fixer on the given file/lines
+
+        May return None if there is no appropriate command to run for the given
+        parameters.
+        """
+        parts = [self._command.format(rootpath=path,
+                                      basename=os.path.basename(path))]
+        if self._linerange:
+            if not ranges:
+                # No line ranges to fix, so don't run the fixer.
+                return None
+            for first, last in ranges:
+                parts.append(self._linerange.format(first=first, last=last))
+        return ' '.join(parts)