Patchwork D6359: test: change test's diff generation to use mdiff for nicer output

login
register
mail settings
Submitter phabricator
Date June 10, 2019, 8:56 p.m.
Message ID <445edfed6209f88b6072840b6b0aeeee@localhost.localdomain>
Download mbox | patch
Permalink /patch/40416/
State Not Applicable
Headers show

Comments

phabricator - June 10, 2019, 8:56 p.m.
sangeet259 updated this revision to Diff 15433.
sangeet259 edited the summary of this revision.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST UPDATE
  https://phab.mercurial-scm.org/D6359?vs=15052&id=15433

REVISION DETAIL
  https://phab.mercurial-scm.org/D6359

AFFECTED FILES
  tests/run-tests.py
  tests/test-push.t

CHANGE DETAILS




To: sangeet259, #hg-reviewers
Cc: pulkit, mercurial-devel

Patch

diff --git a/tests/test-push.t b/tests/test-push.t
--- a/tests/test-push.t
+++ b/tests/test-push.t
@@ -8,343 +8,10 @@ 
   $ hg init test-revflag
   $ hg -R test-revflag unbundle "$TESTDIR/bundles/remote.hg"
   adding changesets
-  adding manifests
-  adding file changes
-  added 9 changesets with 7 changes to 4 files (+1 heads)
-  new changesets bfaf4b5cbf01:916f1afdef90 (9 drafts)
-  (run 'hg heads' to see heads, 'hg merge' to merge)
-
-  $ for i in 0 1 2 3 4 5 6 7 8; do
-  >    echo
-  >    hg init test-revflag-"$i"
-  >    hg -R test-revflag push -r "$i" test-revflag-"$i"
-  >    hg -R test-revflag-"$i" verify
-  > done
-  
-  pushing to test-revflag-0
-  searching for changes
-  adding changesets
-  adding manifests
-  adding file changes
-  added 1 changesets with 1 changes to 1 files
-  checking changesets
-  checking manifests
-  crosschecking files in changesets and manifests
-  checking files
-  checked 1 changesets with 1 changes to 1 files
-  
-  pushing to test-revflag-1
-  searching for changes
-  adding changesets
-  adding manifests
-  adding file changes
-  added 2 changesets with 2 changes to 1 files
-  checking changesets
-  checking manifests
-  crosschecking files in changesets and manifests
-  checking files
-  checked 2 changesets with 2 changes to 1 files
-  
-  pushing to test-revflag-2
-  searching for changes
-  adding changesets
-  adding manifests
-  adding file changes
-  added 3 changesets with 3 changes to 1 files
-  checking changesets
-  checking manifests
-  crosschecking files in changesets and manifests
-  checking files
-  checked 3 changesets with 3 changes to 1 files
-  
-  pushing to test-revflag-3
-  searching for changes
-  adding changesets
-  adding manifests
-  adding file changes
-  added 4 changesets with 4 changes to 1 files
-  checking changesets
-  checking manifests
-  crosschecking files in changesets and manifests
-  checking files
-  checked 4 changesets with 4 changes to 1 files
-  
-  pushing to test-revflag-4
-  searching for changes
-  adding changesets
-  adding manifests
-  adding file changes
-  added 2 changesets with 2 changes to 1 files
-  checking changesets
-  checking manifests
-  crosschecking files in changesets and manifests
-  checking files
-  checked 2 changesets with 2 changes to 1 files
-  
-  pushing to test-revflag-5
-  searching for changes
-  adding changesets
-  adding manifests
-  adding file changes
-  added 3 changesets with 3 changes to 1 files
-  checking changesets
-  checking manifests
-  crosschecking files in changesets and manifests
-  checking files
-  checked 3 changesets with 3 changes to 1 files
-  
-  pushing to test-revflag-6
-  searching for changes
-  adding changesets
-  adding manifests
-  adding file changes
-  added 4 changesets with 5 changes to 2 files
-  checking changesets
-  checking manifests
-  crosschecking files in changesets and manifests
-  checking files
-  checked 4 changesets with 5 changes to 2 files
-  
-  pushing to test-revflag-7
-  searching for changes
-  adding changesets
-  adding manifests
-  adding file changes
-  added 5 changesets with 6 changes to 3 files
-  checking changesets
-  checking manifests
-  crosschecking files in changesets and manifests
-  checking files
-  checked 5 changesets with 6 changes to 3 files
-  
-  pushing to test-revflag-8
-  searching for changes
-  adding changesets
-  adding manifests
-  adding file changes
-  added 5 changesets with 5 changes to 2 files
-  checking changesets
-  checking manifests
-  crosschecking files in changesets and manifests
-  checking files
-  checked 5 changesets with 5 changes to 2 files
-
-  $ cd test-revflag-8
-
-  $ hg pull ../test-revflag-7
-  pulling from ../test-revflag-7
-  searching for changes
-  adding changesets
+  Yeh line aise hi daalte hain
+  Man kar raha hai
+  So je baat
   adding manifests
   adding file changes
-  added 4 changesets with 2 changes to 3 files (+1 heads)
-  new changesets c70afb1ee985:faa2e4234c7a
+  added 8 changesets with 7 changes to 4 files (+34 heads)
   (run 'hg heads' to see heads, 'hg merge' to merge)
-
-  $ hg verify
-  checking changesets
-  checking manifests
-  crosschecking files in changesets and manifests
-  checking files
-  checked 9 changesets with 7 changes to 4 files
-
-  $ cd ..
-
-Test server side validation during push
-=======================================
-
-  $ hg init test-validation
-  $ cd test-validation
-
-  $ cat > .hg/hgrc <<EOF
-  > [server]
-  > validate=1
-  > EOF
-
-  $ echo alpha > alpha
-  $ echo beta > beta
-  $ hg addr
-  adding alpha
-  adding beta
-  $ hg ci -m 1
-
-  $ cd ..
-  $ hg clone test-validation test-validation-clone
-  updating to branch default
-  2 files updated, 0 files merged, 0 files removed, 0 files unresolved
-
-#if reporevlogstore
-
-Test spurious filelog entries:
-
-  $ cd test-validation-clone
-  $ echo blah >> beta
-  $ cp .hg/store/data/beta.i tmp1
-  $ hg ci -m 2
-  $ cp .hg/store/data/beta.i tmp2
-  $ hg -q rollback
-  $ mv tmp2 .hg/store/data/beta.i
-  $ echo blah >> beta
-  $ hg ci -m '2 (corrupt)'
-
-Expected to fail:
-
-  $ hg verify
-  checking changesets
-  checking manifests
-  crosschecking files in changesets and manifests
-  checking files
-   beta@1: dddc47b3ba30 not in manifests
-  checked 2 changesets with 4 changes to 2 files
-  1 integrity errors encountered!
-  (first damaged changeset appears to be 1)
-  [1]
-
-  $ hg push
-  pushing to $TESTTMP/test-validation
-  searching for changes
-  adding changesets
-  adding manifests
-  adding file changes
-  transaction abort!
-  rollback completed
-  abort: received spurious file revlog entry
-  [255]
-
-  $ hg -q rollback
-  $ mv tmp1 .hg/store/data/beta.i
-  $ echo beta > beta
-
-Test missing filelog entries:
-
-  $ cp .hg/store/data/beta.i tmp
-  $ echo blah >> beta
-  $ hg ci -m '2 (corrupt)'
-  $ mv tmp .hg/store/data/beta.i
-
-Expected to fail:
-
-  $ hg verify
-  checking changesets
-  checking manifests
-  crosschecking files in changesets and manifests
-  checking files
-   beta@1: manifest refers to unknown revision dddc47b3ba30
-  checked 2 changesets with 2 changes to 2 files
-  1 integrity errors encountered!
-  (first damaged changeset appears to be 1)
-  [1]
-
-  $ hg push
-  pushing to $TESTTMP/test-validation
-  searching for changes
-  adding changesets
-  adding manifests
-  adding file changes
-  transaction abort!
-  rollback completed
-  abort: missing file data for beta:dddc47b3ba30e54484720ce0f4f768a0f4b6efb9 - run hg verify
-  [255]
-
-  $ cd ..
-
-#endif
-
-Test push hook locking
-=====================
-
-  $ hg init 1
-
-  $ echo '[ui]' >> 1/.hg/hgrc
-  $ echo 'timeout = 10' >> 1/.hg/hgrc
-
-  $ echo foo > 1/foo
-  $ hg --cwd 1 ci -A -m foo
-  adding foo
-
-  $ hg clone 1 2
-  updating to branch default
-  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
-
-  $ hg clone 2 3
-  updating to branch default
-  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
-
-  $ cat <<EOF > $TESTTMP/debuglocks-pretxn-hook.sh
-  > hg debuglocks
-  > true
-  > EOF
-  $ echo '[hooks]' >> 2/.hg/hgrc
-  $ echo "pretxnchangegroup.a = sh $TESTTMP/debuglocks-pretxn-hook.sh" >> 2/.hg/hgrc
-  $ echo 'changegroup.push = hg push -qf ../1' >> 2/.hg/hgrc
-
-  $ echo bar >> 3/foo
-  $ hg --cwd 3 ci -m bar
-
-  $ hg --cwd 3 push ../2 --config devel.legacy.exchange=bundle1
-  pushing to ../2
-  searching for changes
-  adding changesets
-  adding manifests
-  adding file changes
-  added 1 changesets with 1 changes to 1 files
-  lock:  user *, process * (*s) (glob)
-  wlock: free
-
-  $ hg --cwd 1 --config extensions.strip= strip tip -q
-  $ hg --cwd 2 --config extensions.strip= strip tip -q
-  $ hg --cwd 3 push ../2 # bundle2+
-  pushing to ../2
-  searching for changes
-  adding changesets
-  adding manifests
-  adding file changes
-  added 1 changesets with 1 changes to 1 files
-  lock:  user *, process * (*s) (glob)
-  wlock: user *, process * (*s) (glob)
-
-Test bare push with multiple race checking options
---------------------------------------------------
-
-  $ hg init test-bare-push-no-concurrency
-  $ hg init test-bare-push-unrelated-concurrency
-  $ hg -R test-revflag push -r 0 test-bare-push-no-concurrency --config server.concurrent-push-mode=strict
-  pushing to test-bare-push-no-concurrency
-  searching for changes
-  adding changesets
-  adding manifests
-  adding file changes
-  added 1 changesets with 1 changes to 1 files
-  $ hg -R test-revflag push -r 0 test-bare-push-unrelated-concurrency --config server.concurrent-push-mode=check-related
-  pushing to test-bare-push-unrelated-concurrency
-  searching for changes
-  adding changesets
-  adding manifests
-  adding file changes
-  added 1 changesets with 1 changes to 1 files
-
-SEC: check for unsafe ssh url
-
-  $ cat >> $HGRCPATH << EOF
-  > [ui]
-  > ssh = sh -c "read l; read l; read l"
-  > EOF
-
-  $ hg -R test-revflag push 'ssh://-oProxyCommand=touch${IFS}owned/path'
-  pushing to ssh://-oProxyCommand%3Dtouch%24%7BIFS%7Downed/path
-  abort: potentially unsafe url: 'ssh://-oProxyCommand=touch${IFS}owned/path'
-  [255]
-  $ hg -R test-revflag push 'ssh://%2DoProxyCommand=touch${IFS}owned/path'
-  pushing to ssh://-oProxyCommand%3Dtouch%24%7BIFS%7Downed/path
-  abort: potentially unsafe url: 'ssh://-oProxyCommand=touch${IFS}owned/path'
-  [255]
-  $ hg -R test-revflag push 'ssh://fakehost|touch${IFS}owned/path'
-  pushing to ssh://fakehost%7Ctouch%24%7BIFS%7Downed/path
-  abort: no suitable response from remote hg!
-  [255]
-  $ hg -R test-revflag push 'ssh://fakehost%7Ctouch%20owned/path'
-  pushing to ssh://fakehost%7Ctouch%20owned/path
-  abort: no suitable response from remote hg!
-  [255]
-
-  $ [ ! -f owned ] || echo 'you got owned'
diff --git a/tests/run-tests.py b/tests/run-tests.py
--- a/tests/run-tests.py
+++ b/tests/run-tests.py
@@ -87,6 +87,10 @@ 
     except NameError:
         pass
 
+tabsplitter = re.compile(br'(\t+|[^\t]+)')
+wordsplitter = re.compile(br'(\t+| +|[a-zA-Z0-9_\x80-\xff]+|'
+                          b'[^ \ta-zA-Z0-9_\x80-\xff])')
+
 processlock = threading.Lock()
 
 pygmentspresent = False
@@ -210,6 +214,30 @@ 
 # For Windows support
 wifexited = getattr(os, "WIFEXITED", lambda x: False)
 
+_unified_diff = difflib.unified_diff
+if PYTHON3:
+    import functools
+    _unified_diff = functools.partial(difflib.diff_bytes, difflib.unified_diff)
+
+try:
+    from mercurial import mdiff, ui, color
+
+    # load the ui profile for the user
+    uip = ui.ui.load()
+    try:
+        inlineDiff = uip._config('diff','word-diff')
+    except Exception as e:
+        print(e)
+        inlineDiff = False
+    # mdiff.new_diff may not be present if the mercurial version installed
+    # is not up to date hence this try block will throw exception AttributeError
+    # and if Mercurial itself is not installed in the system then it will throw
+    # an ImportError
+    _diff = mdiff.new_diff
+except (ImportError, AttributeError):
+    print("Falling back to unified_diff")
+    _diff = _unified_diff
+
 # Whether to use IPv6
 def checksocketfamily(name, port=20058):
     """return true if we can listen on localhost using family=name
@@ -598,15 +626,10 @@ 
             except OSError:
                 pass
 
-_unified_diff = difflib.unified_diff
-if PYTHON3:
-    import functools
-    _unified_diff = functools.partial(difflib.diff_bytes, difflib.unified_diff)
-
 def getdiff(expected, output, ref, err):
     servefail = False
     lines = []
-    for line in _unified_diff(expected, output, ref, err):
+    for line in _diff(expected, output, ref, err):
         if line.startswith(b'+++') or line.startswith(b'---'):
             line = line.replace(b'\\', b'/')
             if line.endswith(b' \n'):
@@ -1915,8 +1938,53 @@ 
                 servefail, lines = getdiff(expected, got,
                                            test.refpath, test.errpath)
                 self.stream.write('\n')
-                for line in lines:
-                    line = highlightdiff(line, self.color)
+
+                i = 0
+                import pudb;pu.db
+                while i < len(lines):
+
+                    lines = lines[i:]
+                    line = lines[0]
+
+                    if line.startswith(('- ','+ ')) and inlineDiff:
+                        hunklines1=[]
+                        for line in lines:
+                            if line.startswith(('- ','+ ')):
+                                i = i+1
+                                hunklines1.append(line)
+                            else:
+                                break
+                        hunktokens=[]
+
+                        """yield tokens for a list of lines in a single hunk, with inline colors"""
+                        # prepare deleted, and inserted content
+                        a = ''
+                        b = ''
+                        for line in hunklines1:
+                            if line[0:1] == '-':
+                                a += line[1:]
+                            elif line[0:1] == '+':
+                                b += line[1:]
+                            else:
+                                raise error.ProgrammingError('unexpected hunk line: %s' % line)
+                        if not a or not b:
+                            if not a:
+                                line = b
+                            if not b :
+                                line = a
+                        else:
+                            i, line = processInlineHunks(i,lines[i:])
+
+
+                        
+                            # treat and print the hunklines separately
+                            # get the tokens
+                            # call a function
+
+                    else:
+                        line = highlightdiff(line, self.color)
+                        i = i+1
+
                     if PYTHON3:
                         self.stream.flush()
                         self.stream.buffer.write(line)
@@ -3248,6 +3316,108 @@ 
                 print("WARNING: Did not find prerequisite tool: %s " %
                       p.decode("utf-8"))
 
+def processInlineHunks(i, lines):
+    hunklines1=[]
+    for line in lines:
+        if line.startswith(('- ','+ ')):
+            i = i+1
+            hunklines1.append(line)
+        else:
+            break
+    hunktokens=[]
+
+    """yield tokens for a list of lines in a single hunk, with inline colors"""
+    # prepare deleted, and inserted content
+    a = ''
+    b = ''
+    for line in hunklines1:
+        if line[0:1] == '-':
+            a += line[1:]
+        elif line[0:1] == '+':
+            b += line[1:]
+        else:
+            raise error.ProgrammingError('unexpected hunk line: %s' % line)
+    if not a or not b:
+        return
+    # re-split the content into words
+    al = wordsplitter.findall(a)
+    bl = wordsplitter.findall(b)
+    # re-arrange the words to lines since the diff algorithm is line-based
+    aln = [s if s == '\n' else s + '\n' for s in al]
+    bln = [s if s == '\n' else s + '\n' for s in bl]
+    an = ''.join(aln)
+    bn = ''.join(bln)
+    # run the diff algorithm, prepare atokens and btokens
+    atokens = []
+    btokens = []
+    blocks = mdiff.allblocks(an, bn, lines1=aln, lines2=bln)
+    for (a1, a2, b1, b2), btype in blocks:
+        changed = btype == '!'
+        for token in mdiff.splitnewlines(''.join(al[a1:a2])):
+            atokens.append((changed, token))
+        for token in mdiff.splitnewlines(''.join(bl[b1:b2])):
+            btokens.append((changed, token))
+
+    # yield deleted tokens, then inserted ones
+    for prefix, label, tokens in [('-', 'diff.deleted', atokens),
+                                  ('+', 'diff.inserted', btokens)]:
+        nextisnewline = True
+        for changed, token in tokens:
+            if nextisnewline:
+                hunktokens.append((prefix, label))
+                nextisnewline = False
+            # special handling line end
+            isendofline = token.endswith('\n')
+            if isendofline:
+                chomp = token[:-1] # chomp
+                if chomp.endswith('\r'):
+                    chomp = chomp[:-1]
+                endofline = token[len(chomp):]
+                token = chomp.rstrip() # detect spaces at the end
+                endspaces = chomp[len(token):]
+            # scan tabs
+            for maybetab in tabsplitter.findall(token):
+                if b'\t' == maybetab[0:1]:
+                    currentlabel = 'diff.tab'
+                else:
+                    if changed:
+                        currentlabel = label + '.changed'
+                    else:
+                        currentlabel = label + '.unchanged'
+                hunktokens.append((maybetab, currentlabel))
+            if isendofline:
+                if endspaces:
+                    hunktokens.append(endspaces, 'diff.trailingwhitespace')
+                hunktokens.append((endofline, ''))
+                nextisnewline = True
+
+    # iterate through hunktokens and print
+    final_string = ""
+    uip._colormode = 'auto'
+
+    uip._styles = {
+        'diff.changed': 'white',
+        'diff.deleted': 'red',
+        'diff.deleted.changed': 'red bold underline',
+        'diff.deleted.unchanged': 'red',
+        'diff.diffline': 'bold',
+        'diff.extended': 'cyan bold',
+        'diff.file_a': 'red bold',
+        'diff.file_b': 'green bold',
+        'diff.hunk': 'magenta',
+        'diff.inserted': 'green',
+        'diff.inserted.changed': 'green bold underline',
+        'diff.inserted.unchanged': 'green',
+        'diff.tab': '',
+        'diff.trailingwhitespace': 'bold red_background',
+    }
+    for msg, label in hunktokens:
+        txt = color.colorlabel(uip,msg,label)
+        final_string = final_string + " " + txt
+    final_string = final_string.lstrip()
+
+    return i,final_string
+
 def aggregateexceptions(path):
     exceptioncounts = collections.Counter()
     testsbyfailure = collections.defaultdict(set)