Patchwork [7,of,7,v2] tests: add test-check-execute.t

login
register
mail settings
Submitter timeless@mozdev.org
Date Dec. 22, 2015, 11:18 a.m.
Message ID <eba93bae90fd0489a293.1450783086@waste.org>
Download mbox | patch
Permalink /patch/12245/
State Superseded
Commit b3eba79b7e0414861cac8e427606e0784ace459b
Delegated to: Yuya Nishihara
Headers show

Comments

timeless@mozdev.org - Dec. 22, 2015, 11:18 a.m.
# HG changeset patch
# User timeless <timeless@mozdev.org>
# Date 1450782356 0
#      Tue Dec 22 11:05:56 2015 +0000
# Node ID eba93bae90fd0489a2935caf1a917ddcf3c721a2
# Parent  e394811e1af32bd177e2429baea8c7d623491ff0
tests: add test-check-execute.t

Try to prevent people from adding files with incorrect execute bits
Yuya Nishihara - Dec. 23, 2015, 10:23 a.m.
On Tue, 22 Dec 2015 05:18:06 -0600, timeless wrote:
> # HG changeset patch
> # User timeless <timeless@mozdev.org>
> # Date 1450782356 0
> #      Tue Dec 22 11:05:56 2015 +0000
> # Node ID eba93bae90fd0489a2935caf1a917ddcf3c721a2
> # Parent  e394811e1af32bd177e2429baea8c7d623491ff0
> tests: add test-check-execute.t

Pushed 1-6 to the clowncopter, thanks.

> --- /dev/null
> +++ b/tests/test-check-execute.t
> @@ -0,0 +1,34 @@
> +#require test-repo

It needs execbit to be skipped on Windows.

> +look for python scripts without the execute bit
> +
> +  $ hg files 'set:(**.py) and not exec()' 2> /dev/null | \
> +  > xargs -n1 egrep -n -H '(env |/)python'|grep ':1:' | \
> +  > cat

You can use grep() fileset.
timeless - Dec. 28, 2015, 6:26 p.m.
Yuya Nishihara wrote:
>> +#require test-repo
>
> It needs execbit to be skipped on Windows.

I don't think so.

I tested w/ a FAT filesystem on OS X, and the set:exec() fileset
properly returned only files w/ +x from the repo, even though they all
appear to have +x according to ls.

>> +look for python scripts without the execute bit
>> +
>> +  $ hg files 'set:(**.py) and not exec()' 2> /dev/null | \
>> +  > xargs -n1 egrep -n -H '(env |/)python'|grep ':1:' | \
>> +  > cat
>
> You can use grep() fileset.

I can't. grep() doesn't give me line numbers.
I'm not asking "does this file have `env python`/`python`, I'm asking
"does the first line of this file have that" -- I'm doing that using
`-n` and :1: (the -H is to handle the case where there's only one
file).

Technically, a file could have " blah env python :1: " and that'd be a
false positive, but I'd rather cross that bridge if it ever
happens....
Yuya Nishihara - Dec. 30, 2015, 2:31 a.m.
On Mon, 28 Dec 2015 13:26:41 -0500, timeless wrote:
> Yuya Nishihara wrote:
> >> +#require test-repo
> >
> > It needs execbit to be skipped on Windows.
> 
> I don't think so.
> 
> I tested w/ a FAT filesystem on OS X, and the set:exec() fileset
> properly returned only files w/ +x from the repo, even though they all
> appear to have +x according to ls.

That only applies to committed files. If you add a new file on FAT, it can't
be marked as an executable.

That said, do you think Windows people should be blamed if they add an
executable with no executable bit? If so, no "#require execbit" would make
sense.

> >> +look for python scripts without the execute bit
> >> +
> >> +  $ hg files 'set:(**.py) and not exec()' 2> /dev/null | \
> >> +  > xargs -n1 egrep -n -H '(env |/)python'|grep ':1:' | \
> >> +  > cat
> >
> > You can use grep() fileset.
> 
> I can't. grep() doesn't give me line numbers.
> I'm not asking "does this file have `env python`/`python`, I'm asking
> "does the first line of this file have that" -- I'm doing that using
> `-n` and :1: (the -H is to handle the case where there's only one
> file).

grep() uses re.search(), in which MULTILINE is off by default. "^" only
matches the first line. You can also use "\A".

https://docs.python.org/2.7/library/re.html

FWIW, '(env |/)python' doesn't work on Windows because of "/".
timeless - Dec. 30, 2015, 3:50 a.m.
On Tue, Dec 29, 2015 at 9:31 PM, Yuya Nishihara <yuya@tcha.org> wrote:
>> I tested w/ a FAT filesystem on OS X, and the set:exec() fileset
>> properly returned only files w/ +x from the repo, even though they all
>> appear to have +x according to ls.
>
> That only applies to committed files. If you add a new file on FAT, it can't
> be marked as an executable.

> That said, do you think Windows people should be blamed if they add an
> executable with no executable bit? If so, no "#require execbit" would make
> sense.

To be honest, I consider this a bug in hg. I should be able to set the
execute information while using a fat file system. It's just metadata.
And we can clearly record and report it.

That said, I guess I'm ok w/ doing a require execbit for now...

>> >> +look for python scripts without the execute bit
>> >> +
>> >> +  $ hg files 'set:(**.py) and not exec()' 2> /dev/null | \
>> >> +  > xargs -n1 egrep -n -H '(env |/)python'|grep ':1:' | \
>> >> +  > cat
>> >
>> > You can use grep() fileset.
>>
>> I can't. grep() doesn't give me line numbers.
>> I'm not asking "does this file have `env python`/`python`, I'm asking
>> "does the first line of this file have that" -- I'm doing that using
>> `-n` and :1: (the -H is to handle the case where there's only one
>> file).
>
> grep() uses re.search(), in which MULTILINE is off by default. "^" only
> matches the first line. You can also use "\A".
>
> https://docs.python.org/2.7/library/re.html
>
> FWIW, '(env |/)python' doesn't work on Windows because of "/".

ok. Unqueue this and I'll revisit it...

Patch

diff --git a/tests/test-check-execute.t b/tests/test-check-execute.t
new file mode 100644
--- /dev/null
+++ b/tests/test-check-execute.t
@@ -0,0 +1,34 @@ 
+#require test-repo
+
+  $ cd "`dirname "$TESTDIR"`"
+
+look for python scripts without the execute bit
+
+  $ hg files 'set:(**.py) and not exec()' 2> /dev/null | \
+  > xargs -n1 egrep -n -H '(env |/)python'|grep ':1:' | \
+  > cat
+
+look for python scripts with execute bit but not shbang
+
+  $ hg files 'set:(**.py) and exec()' 2> /dev/null | \
+  > xargs -n1 egrep -n -L '^#!.*(env |/)python' | \
+  > cat
+
+look for shell scripts with execute bit but not shbang
+
+  $ hg files 'set:(**.sh) and exec()' 2> /dev/null | \
+  > xargs -n1 egrep -n -L '^#!.*(env |/)(ba|)sh' | \
+  > cat
+
+look for shell scripts with execute bit and no shbang on first line
+
+  $ for a in `hg files 'set:exec() and (**.sh)' 2> /dev/null`; \
+  > do head -1 $a|grep '#!' > /dev/null|| echo $a; \
+  > done
+
+look for non scripts with no shbang
+
+  $ for a in `hg files 'set:exec() and not (**.sh) and not (**.py)' 2> /dev/null`; \
+  > do head -1 $a|grep '#!' > /dev/null|| echo $a; \
+  > done
+