Submitter | via Mercurial-devel |
---|---|
Date | Nov. 3, 2020, 4:11 p.m. |
Message ID | <CAHcr6HYu50aLdjnxXMLwfzYZ0mvuiQ5Fe5qtgVzG_Yi8Ai76mg@mail.gmail.com> |
Download | mbox | patch |
Permalink | /patch/47542/ |
State | New |
Headers | show |
Comments
Spent a little more time with the latest pyoxidizer, but started getting weird results: complaining from pyoxidizer that it needed a .to_string() in the run_command: Some("my string here") value, and if I patch around that or comment out run_command I get a panic inside starlark on an optional, which I don't think makes sense either. I guess I need some guidance on how to debug why hgdemandimport thinks it can't be loaded from the single-file executable? Any insight? On Tue, Nov 3, 2020 at 11:11 AM Augie Fackler <augie@google.com> wrote: > Okay, patching up for main, I get this: > > error[PYOXIDIZER_BUILD]: adding PythonModuleSource<name=hgdemandimport> > > Caused by: > 0: adding PythonModuleSource<hgdemandimport> > 1: resource collector does not allow resources in filesystem-relative > --> ./pyoxidizer.bzl:61:5 > | > 61 | exe.add_python_resources(exe.pip_install(["--verbose", ROOT])) > | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > add_python_resources() > > > error: adding PythonModuleSource<name=hgdemandimport> > > Caused by: > 0: adding PythonModuleSource<hgdemandimport> > 1: resource collector does not allow resources in filesystem-relative > > so it's sad about hgdemandimport, but I'm not sure what to do with this > information. For the record, my patch to pyoxidizer.bzl is this: > > diff --git a/rust/hgcli/pyoxidizer.bzl b/rust/hgcli/pyoxidizer.bzl > --- a/rust/hgcli/pyoxidizer.bzl > +++ b/rust/hgcli/pyoxidizer.bzl > @@ -40,12 +40,11 @@ def make_exe(dist): > # extensions. > packaging_policy.extension_module_filter = "all" > packaging_policy.resources_location = "in-memory" > - packaging_policy.resources_location_fallback = > "filesystem-relative:lib" > packaging_policy.register_resource_callback(resource_callback) > > config = dist.make_python_interpreter_config() > config.raw_allocator = "system" > - config.run_mode = "eval:%s" % RUN_CODE > + config.run_command = RUN_CODE > # We want to let the user load extensions from the file system > config.filesystem_importer = True > # We need this to make resourceutil happy, since it looks for > sys.frozen. > > > On Tue, Nov 3, 2020 at 9:54 AM Augie Fackler <augie@google.com> wrote: > >> Sigh, you changed the config file again in an incompatible way >> (config.run_mode isn't a thing anymore) so I'll use the v0.9.0 tag. It >> still fails, but still not helpfully? >> >> Successfully installed mercurial-5.6-4-8711dc13474c-20201103 >> Removed build tracker: >> '/private/var/folders/cy/2vk7k97j21qd6yr76gp6x5m8001p1v/T/pip-req-tracker-z19m5oxc' >> adding <PythonModuleSource> >> error[PYOXIDIZER_BUILD]: resource collector does not allow resources in >> filesystem-relative >> --> ./pyoxidizer.bzl:63:9 >> | >> 63 | exe.add_python_resource(r) >> | ^^^^^^^^^^^^^^^^^^^^^^^^^^ add_python_resource >> >> error: resource collector does not allow resources in filesystem-relative >> >> I'll see if I can update the pyoxidizer.bzl in a bit here to work on >> main, I guess? >> >> On Mon, Nov 2, 2020 at 11:06 PM Gregory Szorc <gregory.szorc@gmail.com> >> wrote: >> >>> I'm unsure what's going on here. >>> >>> For this to occur, `.resources_location_fallback = None` must hold on >>> the policy instance and some resource being added must have its >>> `.add_location = "filesystem-relative:..."`. >>> >>> This error message is horrible. And I just committed a change to the >>> `main` branch to print more context about which exact resource is failing. >>> If you don't want to run the latest commit from `main`, you can change this >>> to something like: >>> >>> for resource in exe.pip_install["--verbose", ROOT]): >>> print("adding %s" % resource) >>> exe.add_python_resource(resource) >>> >>> Or you could look for references to "filesystem-relative" in the >>> configuration file. There's likely something setting that as the >>> `.add_location =` value. >>> >>> FWIW I _think_ Martin did all the necessary work to enable Mercurial to >>> be shipped as a standalone binary? I left it out of the in-tree config >>> because files mode was backwards compatible and safer. But a single file >>> binary is something we could implement next release. >>> >>> On Mon, Nov 2, 2020 at 11:06 AM Augie Fackler <augie@google.com> wrote: >>> >>>> Hi Greg! >>>> >>>> I'm working on updating hg and trying to switch us to pyoxidizer for >>>> Windows, but a speedbump on the way is that our Mac builds have broken: >>>> we'd been relying on the one-file build mode of the binary, and now that >>>> appears to be impossible? Am I missing something? I tried editing the >>>> pyoxidizer.bzl to make all resources be "in-memory" but then I get a >>>> failure trying to add the hg packages, eg: >>>> >>>> error[PYOXIDIZER_BUILD]: resource collector does not allow resources in >>>> filesystem-relative >>>> --> ./pyoxidizer.bzl:64:5 >>>> | >>>> 64 | exe.add_python_resources(exe.pip_install(["--verbose", ROOT])) >>>> | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>>> add_python_resources() >>>> >>>> which left me stumped. Am I missing something obvious in the docs or >>>> configuration that would let us get back to one-file executables for our >>>> macOS users? >>>> >>>> Thanks, >>>> Augie >>>> >>>
Closing the loop on this: Greg helped me out a bunch and we got this working again. I'll send some patches next week to make pyoxidized `hg` be one-file on Linux and macOS. :) On Tue, Nov 3, 2020 at 6:41 PM Augie Fackler <augie@google.com> wrote: > Spent a little more time with the latest pyoxidizer, but started getting > weird results: complaining from pyoxidizer that it needed a .to_string() in > the run_command: Some("my string here") value, and if I patch around that > or comment out run_command I get a panic inside starlark on an optional, > which I don't think makes sense either. > > I guess I need some guidance on how to debug why hgdemandimport thinks it > can't be loaded from the single-file executable? Any insight? > > On Tue, Nov 3, 2020 at 11:11 AM Augie Fackler <augie@google.com> wrote: > >> Okay, patching up for main, I get this: >> >> error[PYOXIDIZER_BUILD]: adding PythonModuleSource<name=hgdemandimport> >> >> Caused by: >> 0: adding PythonModuleSource<hgdemandimport> >> 1: resource collector does not allow resources in filesystem-relative >> --> ./pyoxidizer.bzl:61:5 >> | >> 61 | exe.add_python_resources(exe.pip_install(["--verbose", ROOT])) >> | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >> add_python_resources() >> >> >> error: adding PythonModuleSource<name=hgdemandimport> >> >> Caused by: >> 0: adding PythonModuleSource<hgdemandimport> >> 1: resource collector does not allow resources in filesystem-relative >> >> so it's sad about hgdemandimport, but I'm not sure what to do with this >> information. For the record, my patch to pyoxidizer.bzl is this: >> >> diff --git a/rust/hgcli/pyoxidizer.bzl b/rust/hgcli/pyoxidizer.bzl >> --- a/rust/hgcli/pyoxidizer.bzl >> +++ b/rust/hgcli/pyoxidizer.bzl >> @@ -40,12 +40,11 @@ def make_exe(dist): >> # extensions. >> packaging_policy.extension_module_filter = "all" >> packaging_policy.resources_location = "in-memory" >> - packaging_policy.resources_location_fallback = >> "filesystem-relative:lib" >> packaging_policy.register_resource_callback(resource_callback) >> >> config = dist.make_python_interpreter_config() >> config.raw_allocator = "system" >> - config.run_mode = "eval:%s" % RUN_CODE >> + config.run_command = RUN_CODE >> # We want to let the user load extensions from the file system >> config.filesystem_importer = True >> # We need this to make resourceutil happy, since it looks for >> sys.frozen. >> >> >> On Tue, Nov 3, 2020 at 9:54 AM Augie Fackler <augie@google.com> wrote: >> >>> Sigh, you changed the config file again in an incompatible way >>> (config.run_mode isn't a thing anymore) so I'll use the v0.9.0 tag. It >>> still fails, but still not helpfully? >>> >>> Successfully installed mercurial-5.6-4-8711dc13474c-20201103 >>> Removed build tracker: >>> '/private/var/folders/cy/2vk7k97j21qd6yr76gp6x5m8001p1v/T/pip-req-tracker-z19m5oxc' >>> adding <PythonModuleSource> >>> error[PYOXIDIZER_BUILD]: resource collector does not allow resources in >>> filesystem-relative >>> --> ./pyoxidizer.bzl:63:9 >>> | >>> 63 | exe.add_python_resource(r) >>> | ^^^^^^^^^^^^^^^^^^^^^^^^^^ add_python_resource >>> >>> error: resource collector does not allow resources in filesystem-relative >>> >>> I'll see if I can update the pyoxidizer.bzl in a bit here to work on >>> main, I guess? >>> >>> On Mon, Nov 2, 2020 at 11:06 PM Gregory Szorc <gregory.szorc@gmail.com> >>> wrote: >>> >>>> I'm unsure what's going on here. >>>> >>>> For this to occur, `.resources_location_fallback = None` must hold on >>>> the policy instance and some resource being added must have its >>>> `.add_location = "filesystem-relative:..."`. >>>> >>>> This error message is horrible. And I just committed a change to the >>>> `main` branch to print more context about which exact resource is failing. >>>> If you don't want to run the latest commit from `main`, you can change this >>>> to something like: >>>> >>>> for resource in exe.pip_install["--verbose", ROOT]): >>>> print("adding %s" % resource) >>>> exe.add_python_resource(resource) >>>> >>>> Or you could look for references to "filesystem-relative" in the >>>> configuration file. There's likely something setting that as the >>>> `.add_location =` value. >>>> >>>> FWIW I _think_ Martin did all the necessary work to enable Mercurial to >>>> be shipped as a standalone binary? I left it out of the in-tree config >>>> because files mode was backwards compatible and safer. But a single file >>>> binary is something we could implement next release. >>>> >>>> On Mon, Nov 2, 2020 at 11:06 AM Augie Fackler <augie@google.com> wrote: >>>> >>>>> Hi Greg! >>>>> >>>>> I'm working on updating hg and trying to switch us to pyoxidizer for >>>>> Windows, but a speedbump on the way is that our Mac builds have broken: >>>>> we'd been relying on the one-file build mode of the binary, and now that >>>>> appears to be impossible? Am I missing something? I tried editing the >>>>> pyoxidizer.bzl to make all resources be "in-memory" but then I get a >>>>> failure trying to add the hg packages, eg: >>>>> >>>>> error[PYOXIDIZER_BUILD]: resource collector does not allow resources >>>>> in filesystem-relative >>>>> --> ./pyoxidizer.bzl:64:5 >>>>> | >>>>> 64 | exe.add_python_resources(exe.pip_install(["--verbose", ROOT])) >>>>> | >>>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ >>>>> add_python_resources() >>>>> >>>>> which left me stumped. Am I missing something obvious in the docs or >>>>> configuration that would let us get back to one-file executables for our >>>>> macOS users? >>>>> >>>>> Thanks, >>>>> Augie >>>>> >>>>
Patch
diff --git a/rust/hgcli/pyoxidizer.bzl b/rust/hgcli/pyoxidizer.bzl --- a/rust/hgcli/pyoxidizer.bzl +++ b/rust/hgcli/pyoxidizer.bzl @@ -40,12 +40,11 @@ def make_exe(dist): # extensions. packaging_policy.extension_module_filter = "all" packaging_policy.resources_location = "in-memory" - packaging_policy.resources_location_fallback = "filesystem-relative:lib" packaging_policy.register_resource_callback(resource_callback) config = dist.make_python_interpreter_config() config.raw_allocator = "system" - config.run_mode = "eval:%s" % RUN_CODE + config.run_command = RUN_CODE # We want to let the user load extensions from the file system config.filesystem_importer = True