Patchwork D7452: debugextensions: gracefully handle missing __file__ attributes

login
register
mail settings
Submitter phabricator
Date Nov. 17, 2019, 6:36 a.m.
Message ID <differential-rev-PHID-DREV-z3tr6b3q72npyhkvub25-req@mercurial-scm.org>
Download mbox | patch
Permalink /patch/43322/
State Superseded
Headers show

Comments

phabricator - Nov. 17, 2019, 6:36 a.m.
mharbison72 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This was crashing PyOxidizer.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7452

AFFECTED FILES
  mercurial/debugcommands.py

CHANGE DETAILS




To: mharbison72, #hg-reviewers
Cc: mercurial-devel
phabricator - Nov. 17, 2019, 6:42 a.m.
mharbison72 added a comment.


  Interesting failure trying to load an external copy of evolve (e5d92ac69a9c):
  
    $ ./build/apps/hg/x86_64-pc-windows-msvc/debug/hg.exe debugextensions -v
    *** failed to import extension evolve from C:/Users/Matt/hg-evolve/hgext3rd\evolve: cannot import name 'compat' from 'hgext_evolve' (C:/Users/Matt/hg-evolve/hgext3rd\evolve\__init__.py)
    Traceback (most recent call last):
      File "mercurial.extensions", line 294, in loadall
        load(ui, name, path, loadingtime)
      File "mercurial.extensions", line 211, in load
        mod = _importext(name, path, bind(_reportimporterror, ui))
      File "mercurial.extensions", line 119, in _importext
        mod = loadpath(path, b'hgext.%s' % name)
      File "mercurial.extensions", line 95, in loadpath
        return imp.load_module(module_name, fd, fpath, desc)
      File "imp", line 244, in load_module
        return load_package(name, filename)
      File "imp", line 216, in load_package
        return _load(spec)
      File "<frozen importlib._bootstrap>", line 696, in _load
      File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
      File "<frozen importlib._bootstrap_external>", line 728, in exec_module
      File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
      File "C:/Users/Matt/hg-evolve/hgext3rd\evolve\__init__.py", line 289, in <module>
        from . import (
    ImportError: cannot import name 'compat' from 'hgext_evolve' (C:/Users/Matt/hg-evolve/hgext3rd\evolve\__init__.py)
    absorb
      location: c:\Users\Matt\hg3\hg\build\apps\hg\x86_64-pc-windows-msvc\debug\hg.exe
      bundled: yes
  
  It loads fine when using `PipInstallSimple` to bundle it into the executable.  Not sure why the name is `hgext_evolve` and not `hgext3rd_evolve`.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7452/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7452

To: mharbison72, #hg-reviewers
Cc: mercurial-devel
phabricator - Nov. 17, 2019, 11:28 p.m.
This revision is now accepted and ready to land.
indygreg added a comment.
indygreg accepted this revision.


  The failure might be due to PyOxidizer not supporting namespace packages. I'm pretty sure that isn't implemented and nobody has asked for it yet. Workaround would be to bundle the Mercurial modules in the binary.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7452/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7452

To: mharbison72, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel
phabricator - Nov. 18, 2019, 3:49 a.m.
mharbison72 added a comment.


  In D7452#109475 <https://phab.mercurial-scm.org/D7452#109475>, @indygreg wrote:
  
  > The failure might be due to PyOxidizer not supporting namespace packages. I'm pretty sure that isn't implemented and nobody has asked for it yet. Workaround would be to bundle the Mercurial modules in the binary.
  
  OK, I filed a feature request.
  
  I think the modules //are// bundled into the binary for what I'm hacking on:
  
    #BUILD_PATH = CWD + "/../../build/pyoxidizer"
    
    Config(
        application_name="hg_pyO2_041",
        python_distribution=default_python_distribution(),
        embedded_python_config=EmbeddedPythonConfig(
            legacy_windows_fs_encoding=True,
            legacy_windows_stdio=True,
         sys_frozen=True,
    #        sys_paths=["$ORIGIN/lib"],
    #        sys_paths=["$ORIGIN"],
    #        sys_paths=["C:/Users/Matt/hg"],
         unbuffered_stdio=True,
        ),
        python_run_mode=python_run_mode_eval("import hgdemandimport; hgdemandimport.enable(); import mercurial.dispatch; mercurial.dispatch.run()"),
        packaging_rules=[
            # Mercurial requires a fully featured Python because extensions may use
            # anything.
            StdlibExtensionsPolicy("all"),
            Stdlib(include_source=True),
            SetupPyInstall(
                package_path="C:/Users/Matt/hg",
                extra_global_arguments=["clean", "--all", "build"],
                # Our code doesn't yet work with the in-memory importer. So package
                # relative to the executable for now.
                #install_location="app-relative:lib",
            ),
            PipInstallSimple("hg-evolve"
            ),
        ]
    )
    
    
    # END OF COMMON USER-ADJUSTED SETTINGS.
    #
    # Everything below this is typically managed by PyOxidizer and doesn't need
    # to be updated by people.
    
    PYOXIDIZER_VERSION = "0.5.0"
    PYOXIDIZER_COMMIT = "c7d9c79690d3f6ad8187e2d409c9e5e14b14d916"
  
  I'm hacking on getting the disabled extensions to list in `hg help extensions`.  With this config, `hgext.__path__` is None.  If I uncomment the `sys_paths=C:/Users...` line, it prints the path as C:/Users/..., so I assume that counts as unbundled modules.
  
  It's not getting in my way or anything, so not a high priority.

REPOSITORY
  rHG Mercurial

CHANGES SINCE LAST ACTION
  https://phab.mercurial-scm.org/D7452/new/

REVISION DETAIL
  https://phab.mercurial-scm.org/D7452

To: mharbison72, #hg-reviewers, indygreg
Cc: indygreg, mercurial-devel

Patch

diff --git a/mercurial/debugcommands.py b/mercurial/debugcommands.py
--- a/mercurial/debugcommands.py
+++ b/mercurial/debugcommands.py
@@ -1029,7 +1029,12 @@ 
     fm = ui.formatter(b'debugextensions', opts)
     for extname, extmod in sorted(exts, key=operator.itemgetter(0)):
         isinternal = extensions.ismoduleinternal(extmod)
-        extsource = pycompat.fsencode(extmod.__file__)
+        extsource = None
+
+        if util.safehasattr(extmod, '__file__'):
+            extsource = pycompat.fsencode(extmod.__file__)
+        elif getattr(sys, 'oxidized', False):
+            extsource = pycompat.sysexecutable
         if isinternal:
             exttestedwith = []  # never expose magic string to users
         else: