Patchwork D7295: pytype: add a (very slow) test that executes pytype

login
register
mail settings
Submitter phabricator
Date Nov. 6, 2019, 11:01 p.m.
Message ID <differential-rev-PHID-DREV-ja2vr3fd2becf6h26ylj-req@mercurial-scm.org>
Download mbox | patch
Permalink /patch/42846/
State New
Headers show

Comments

phabricator - Nov. 6, 2019, 11:01 p.m.
durin42 created this revision.
Herald added subscribers: mercurial-devel, mjpieters.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This doesn't actually pass yet, but I wanted to share it early so
  others can play with pytype.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7295

AFFECTED FILES
  tests/test-check-pytype.t

CHANGE DETAILS




To: durin42, #hg-reviewers
Cc: mjpieters, mercurial-devel
phabricator - Nov. 8, 2019, 4:52 p.m.
This revision now requires changes to proceed.
indygreg added a comment.
indygreg requested changes to this revision.


  This can't land per commit message, so setting status accordingly.
  
  I'm extremely supportive of adding type checking, whether it be pytype or mypy. I don't think it matters much. And TBH I wouldn't be surprised if we end up with both once we drop Python 2 support and start using inline annotations heavily!
  
  As for how to start testing, I think we should do this like Python 3 and try to make individual files "clean." Maintain a list of clean files and we ratchet from there.
  
  Since type checking is slow (but there are state files we can reuse to speed things up), we'll need to figure out how to make this work in CI. But I have no doubt we can figure something out. Out of curiosity, how long does pytype take to run in a clean source directory, without any state files?

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7295/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7295

To: durin42, #hg-reviewers, indygreg
Cc: indygreg, mjpieters, mercurial-devel
phabricator - Nov. 19, 2019, 4:28 p.m.
mharbison72 added inline comments.

INLINE COMMENTS

> test-check-pytype.t:22
> +  >    -x mercurial/lsprof.py \
> +  >    -x mercurial/policy.py
> +  >    -x mercurial/pycompat.py \

Needs a trailing slash here.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7295/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7295

To: durin42, #hg-reviewers, indygreg
Cc: mharbison72, dlax, indygreg, mjpieters, mercurial-devel
phabricator - Nov. 22, 2019, 10:43 p.m.
mharbison72 added a comment.


  In D7295#107482 <https://phab.mercurial-scm.org/D7295#107482>, @indygreg wrote:
  
  > Since type checking is slow (but there are state files we can reuse to speed things up), we'll need to figure out how to make this work in CI. But I have no doubt we can figure something out. Out of curiosity, how long does pytype take to run in a clean source directory, without any state files?
  
  It takes 15-20 minutes on a 2018 Mac mini.  In addition to the disabled files here, I've disabled:
  
    mercurial/httppeer.py
    mercurial/repoview.py
    mercurial/localrepo.py
    mercurial/revlog.py
    mercurial/merge.py
    mercurial/testing/storage.py
  
  But I've also added back these, and sprinkled various disable comments around:
  
    mercurial/bundlerepo.py
    mercurial/error.py
    mercurial/exchange.py
    mercurial/policy.py
    mercurial/pycompat.py
    mercurial/scmwindows.py
    mercurial/windows.py
    mercurial/wireprotoframing.py
  
  Even with the state files, it seems to take 10-15 minutes or so.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7295/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7295

To: durin42, #hg-reviewers, indygreg
Cc: mharbison72, dlax, indygreg, mjpieters, mercurial-devel
phabricator - Nov. 23, 2019, 5:48 p.m.
durin42 added a comment.


  In D7295#110254 <https://phab.mercurial-scm.org/D7295#110254>, @mharbison72 wrote:
  
  > In D7295#107482 <https://phab.mercurial-scm.org/D7295#107482>, @indygreg wrote:
  >
  >> Since type checking is slow (but there are state files we can reuse to speed things up), we'll need to figure out how to make this work in CI. But I have no doubt we can figure something out. Out of curiosity, how long does pytype take to run in a clean source directory, without any state files?
  >
  > It takes 15-20 minutes on a 2018 Mac mini.  In addition to the disabled files here, I've disabled:
  >
  >   mercurial/httppeer.py
  >   mercurial/repoview.py
  >   mercurial/localrepo.py
  >   mercurial/revlog.py
  >   mercurial/merge.py
  >   mercurial/testing/storage.py
  >
  > But I've also added back these, and sprinkled various disable comments around:
  >
  >   mercurial/bundlerepo.py
  >   mercurial/error.py
  >   mercurial/exchange.py
  >   mercurial/policy.py
  >   mercurial/pycompat.py
  >   mercurial/scmwindows.py
  >   mercurial/windows.py
  >   mercurial/wireprotoframing.py
  >
  > Even with the state files, it seems to take 10-15 minutes or so.
  
  Ooh, can you send your extra suppressions? For the most part I've been tackling this as a side project when I'm waiting on compiles.
  
  As to doing individual files: that's well and good, and might be a viable approach for mypy. That said, my sense has been that mypy is faster and significantly less useful, since we don't get cross-file analysis of what's going on. I _think_ pytype gets faster as we add more annotations, so my rough thought thus far as been to try and get some subset of the codebase clean, transitively, and then try to start stamping some types on important functions and interfaces to constrain things. Where I'd _really_ like to get is to be able to migrate the `@command` decorators and friends to use strings instead of bytes, and be able to recommend typechecking as a way for extension authors to clean things up, but that'll take a while.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7295/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7295

To: durin42, #hg-reviewers, indygreg
Cc: mharbison72, dlax, indygreg, mjpieters, mercurial-devel

Patch

diff --git a/tests/test-check-pytype.t b/tests/test-check-pytype.t
new file mode 100644
--- /dev/null
+++ b/tests/test-check-pytype.t
@@ -0,0 +1,33 @@ 
+#require pytype py3 slow
+
+  $ cd $RUNTESTDIR/..
+
+Many of the individual files that are excluded here confuse pytype
+because they do a mix of Python 2 and Python 3 things
+conditionally. There's no good way to help it out with that as far as
+I can tell, so let's just hide those files from it for now. We should
+endeavor to empty this list out over time, as some of these are
+probably hiding real problems.
+
+  $ pytype mercurial \
+  >   -x mercurial/encoding.py \
+  >   -x mercurial/error.py \
+  >   -x mercurial/pycompat.py \
+  >   -x mercurial/urllibcompat.py \
+  >   -x mercurial/lsprof.py \
+  >   -x mercurial/i18n.py \
+  >   -x mercurial/sslutil.py \
+  >   -x mercurial/scmwindows.py \
+  >   -x mercurial/keepalive.py \
+  >   -x mercurial/windows.py \
+  >   -x mercurial/wireprotoframing.py \
+  >   -x mercurial/utils/stringutil.py \
+  >   -x mercurial/hgweb/server.py \
+  >   -x mercurial/hgweb/wsgicgi.py \
+  >   -x mercurial/minirst.py \
+  >   -x mercurial/chgserver.py \
+  >   -x mercurial/bundlerepo.py \
+  >   -x mercurial/interfaces \
+  >   -x mercurial/cffi \
+  >   -x mercurial/pure \
+  >   -x mercurial/thirdparty