Patchwork [3,of,3,highlight] highlight: don't highlight files that have fancy linebreaks (issue4291)

login
register
mail settings
Submitter Augie Fackler
Date Dec. 17, 2014, 1:47 a.m.
Message ID <70d411d79a2f77d91279.1418780820@130.17.16.172.in-addr.arpa>
Download mbox | patch
Permalink /patch/7140/
State Superseded
Delegated to: Pierre-Yves David
Headers show

Comments

Augie Fackler - Dec. 17, 2014, 1:47 a.m.
# HG changeset patch
# User Augie Fackler <raf@durin42.com>
# Date 1418764197 18000
#      Tue Dec 16 16:09:57 2014 -0500
# Node ID 70d411d79a2f77d9127988acf236cb3f6c349ee9
# Parent  8d5d9431112bbf7ef694437217f0cd919d6856fd
highlight: don't highlight files that have fancy linebreaks (issue4291)

Spotted this on a file with a form feed in the cpython repository. I'm
not sure how to make the lines line up correctly, so for now we'll
disable highlighting if the file contains such an exotic linebreak.

Patch

diff --git a/hgext/highlight/highlight.py b/hgext/highlight/highlight.py
--- a/hgext/highlight/highlight.py
+++ b/hgext/highlight/highlight.py
@@ -34,7 +34,13 @@  def pygmentize(field, fctx, style, tmpl)
 
     # Pygments is best used with Unicode strings:
     # <http://pygments.org/docs/unicode/>
+    numlines = len(text.splitlines())
     text = text.decode(encoding.encoding, 'replace')
+    if numlines != len(text.splitlines()):
+        # Some characters count as line separators to unicode but not
+        # bytes, notably U+000C FORM FEED. If one of those is present,
+        # the lines won't line up and highlight will omit lines.
+        return
 
     # To get multi-line strings right, we can't format line-by-line
     try:
diff --git a/tests/test-highlight.t b/tests/test-highlight.t
--- a/tests/test-highlight.t
+++ b/tests/test-highlight.t
@@ -186,7 +186,7 @@  hgweb filerevision, html
   </body>
   </html>
   
-file with exotic linebreak is not highlighted (currently broken)
+file with exotic linebreak is not highlighted
   $ ("$TESTDIR/get-with-headers.py" localhost:$HGPORT 'file/tip/with-ff.py') \
   >     | sed "s/class=\"k\"/class=\"kn\"/g" | sed "s/class=\"mf\"/class=\"mi\"/g"
   200 Script output follows
@@ -271,8 +271,8 @@  file with exotic linebreak is not highli
   <div class="sourcefirst linewraptoggle">line wrap: <a class="linewraplink" href="javascript:toggleLinewrap()">on</a></div>
   <div class="sourcefirst"> line source</div>
   <pre class="sourcelines stripes4 wrap">
-  <span id="l1"><span class="n">x</span> <span class="o">=</span> <span class="mi">1</span></span><a href="#l1"></a>
-  <span id="l2"><span class="n">x</span><span class="o">=</span><span class="mi">2</span></span><a href="#l2"></a></pre>
+  <span id="l1">x = 1</span><a href="#l1"></a>
+  <span id="l2">x=2\x0cx=&quot;three&quot;</span><a href="#l2"></a></pre> (esc)
   <div class="sourcelast"></div>
   </div>
   </div>