Patchwork D10683: pyoxidizer: support producing MSI installers

login
register
mail settings
Submitter phabricator
Date May 7, 2021, 12:01 a.m.
Message ID <differential-rev-PHID-DREV-uoezkpy7phwnvptyssac-req@mercurial-scm.org>
Download mbox | patch
Permalink /patch/49006/
State Superseded
Headers show

Comments

phabricator - May 7, 2021, 12:01 a.m.
indygreg created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REVISION SUMMARY
  Newer versions of PyOxidizer have support for building WiX MSI
  installers "natively." Essentially, you can script the definition
  of your WiX installer via Starlark and PyOxidizer can invoke
  WiX tools to produce the installer.
  
  This commit teaches our PyOxidizer config file to produce MSI
  installers similarly to how
  `contrib/packaging/packging.py wix` would do it.
  
  We had to make a very minor change to `mercurial.wxs` to reflect
  different paths depending on who builds. This is because when
  PyOxidizer builds WiX installers, it does so from an isolated
  directory, not Mercurial's source directory. We simply copy the
  files into the build environment so they are accessible.
  
  After this change, running `pyoxidizer build msi` produces a nearly
  identical install layout to what the previous method produces.
  When I applied this series on top of the 5.8 tag, here is the
  list of differences and explanations:
  
  - docs/*.html files are missing from the new installer because the Python build environment doesn't have docutils.
  - .pyd and .exe files differ, likely because I'm using a different Visual Studio toolchain on my local computer than the official build environment.
  - Various .dist-info/ directories have different names. This is because older versions of PyOxidizer had buggy behavior and weren't properly normalizing package names in .dist-info/ directories. e.g. we went from `cached-property-1.5.2.dist-info` to `cached_property-1.5.2.dist-info`.
  - Translations (.mo files) may be missing if gettext isn't in %Path%. This is because the packaging.py code installs gettext and ensures it can be found.
  - Some *.dist-info/RECORD files vary due to SHA-256 content digest divergence due to build environment differences. (This should be harmless.)
  - The new install layout ships a python3.dll because newer versions of PyOxidizer ship this file.
  - The new install layout has a different vcruntime140.dll and also a vcruntime140_1.dll because newer versions of PyOxidizer ship a newer version of the Visual C++ Redistributable Runtime.
  
  The new PyOxidizer functionality is not yet integrated with
  packaging.py. This will come in a subsequent commit. So for now, the
  new functionality introduced here is unused.

REPOSITORY
  rHG Mercurial

BRANCH
  default

REVISION DETAIL
  https://phab.mercurial-scm.org/D10683

AFFECTED FILES
  contrib/packaging/wix/mercurial.wxs
  rust/hgcli/pyoxidizer.bzl

CHANGE DETAILS




To: indygreg, #hg-reviewers
Cc: mercurial-patches, mercurial-devel

Patch

diff --git a/rust/hgcli/pyoxidizer.bzl b/rust/hgcli/pyoxidizer.bzl
--- a/rust/hgcli/pyoxidizer.bzl
+++ b/rust/hgcli/pyoxidizer.bzl
@@ -1,5 +1,20 @@ 
+# The following variables can be passed in as parameters:
+#
+# VERSION
+#   Version string of program being produced.
+#
+# MSI_NAME
+#   Root name of MSI installer.
+#
+# EXTRA_MSI_FEATURES
+#   ; delimited string of extra features to advertise in the built MSA.
+
 ROOT = CWD + "/../.."
 
+VERSION = VARS.get("VERSION", "5.8")
+MSI_NAME = VARS.get("MSI_NAME", "mercurial")
+EXTRA_MSI_FEATURES = VARS.get("EXTRA_MSI_FEATURES")
+
 IS_WINDOWS = "windows" in BUILD_TARGET_TRIPLE
 
 # Code to run in Python interpreter.
@@ -80,8 +95,144 @@ 
 
     return m
 
+
+# This adjusts the InstallManifest produced from exe generation to provide
+# additional files found in a Windows install layout.
+def make_windows_install_layout(manifest):
+    # Copy various files to new install locations. This can go away once
+    # we're using the importlib resource reader.
+    RECURSIVE_COPIES = {
+        "lib/mercurial/locale/": "locale/",
+        "lib/mercurial/templates/": "templates/",
+    }
+    for (search, replace) in RECURSIVE_COPIES.items():
+        for path in manifest.paths():
+            if path.startswith(search):
+                new_path = path.replace(search, replace)
+                print("copy %s to %s" % (path, new_path))
+                file = manifest.get_file(path)
+                manifest.add_file(file, path = new_path)
+
+    # Similar to above, but with filename pattern matching.
+    # lib/mercurial/helptext/**/*.txt -> helptext/
+    # lib/mercurial/defaultrc/*.rc -> defaultrc/
+    for path in manifest.paths():
+        if path.startswith("lib/mercurial/helptext/") and path.endswith(".txt"):
+            new_path = path[len("lib/mercurial/"):]
+        elif path.startswith("lib/mercurial/defaultrc/") and path.endswith(".rc"):
+            new_path = path[len("lib/mercurial/"):]
+        else:
+            continue
+
+        print("copying %s to %s" % (path, new_path))
+        manifest.add_file(manifest.get_file(path), path = new_path)
+
+    # We also install a handful of additional files.
+    EXTRA_CONTRIB_FILES = [
+        "bash_completion",
+        "hgweb.fcgi",
+        "hgweb.wsgi",
+        "logo-droplets.svg",
+        "mercurial.el",
+        "mq.el",
+        "tcsh_completion",
+        "tcsh_completion_build.sh",
+        "xml.rnc",
+        "zsh_completion",
+    ]
+
+    for f in EXTRA_CONTRIB_FILES:
+        manifest.add_file(FileContent(path = ROOT + "/contrib/" + f), directory = "contrib")
+
+    # Individual files with full source to destination path mapping.
+    EXTRA_FILES = {
+        "contrib/hgk": "contrib/hgk.tcl",
+        "contrib/win32/postinstall.txt": "ReleaseNotes.txt",
+        "contrib/win32/ReadMe.html": "ReadMe.html",
+        "doc/style.css": "doc/style.css",
+        "COPYING": "Copying.txt",
+    }
+
+    for source, dest in EXTRA_FILES.items():
+        print("adding extra file %s" % dest)
+        manifest.add_file(FileContent(path = ROOT + "/" + source), path = dest)
+
+    # And finally some wildcard matches.
+    manifest.add_manifest(glob(
+        include = [ROOT + "/contrib/vim/*"],
+        strip_prefix = ROOT + "/"
+    ))
+    manifest.add_manifest(glob(
+        include = [ROOT + "/doc/*.html"],
+        strip_prefix = ROOT + "/"
+    ))
+
+    # But we don't ship hg-ssh on Windows, so exclude its documentation.
+    manifest.remove("doc/hg-ssh.8.html")
+
+    return manifest
+
+
+def make_msi(manifest):
+    manifest = make_windows_install_layout(manifest)
+
+    if "x86_64" in BUILD_TARGET_TRIPLE:
+        platform = "x64"
+    else:
+        platform = "x86"
+
+    manifest.add_file(
+        FileContent(path = ROOT + "/contrib/packaging/wix/COPYING.rtf"),
+        path = "COPYING.rtf",
+    )
+    manifest.remove("Copying.txt")
+    manifest.add_file(
+        FileContent(path = ROOT + "/contrib/win32/mercurial.ini"),
+        path = "defaultrc/mercurial.rc",
+    )
+    manifest.add_file(
+        FileContent(filename = "editor.rc", content = "[ui]\neditor = notepad\n"),
+        path = "defaultrc/editor.rc",
+    )
+
+    wix = WiXInstaller("hg", "%s-%s.msi" % (MSI_NAME, VERSION))
+
+    # Materialize files in the manifest to the install layout.
+    wix.add_install_files(manifest)
+
+    # From mercurial.wxs.
+    wix.install_files_root_directory_id = "INSTALLDIR"
+
+    # Pull in our custom .wxs files.
+    defines = {
+        "PyOxidizer": "1",
+        "Platform": platform,
+        "Version": VERSION,
+        "Comments": "Installs Mercurial version %s" % VERSION,
+        "PythonVersion": "3",
+        "MercurialHasLib": "1",
+    }
+
+    if EXTRA_MSI_FEATURES:
+        defines["MercurialExtraFeatures"] = EXTRA_MSI_FEATURES
+
+    wix.add_wxs_file(
+        ROOT + "/contrib/packaging/wix/mercurial.wxs",
+        preprocessor_parameters=defines,
+    )
+
+    # Our .wxs references to other files. Pull those into the build environment.
+    for f in ("defines.wxi", "guids.wxi", "COPYING.rtf"):
+        wix.add_build_file(f, ROOT + "/contrib/packaging/wix/" + f)
+
+    wix.add_build_file("mercurial.ico", ROOT + "/contrib/win32/mercurial.ico")
+
+    return wix
+
+
 register_target("distribution", make_distribution)
 register_target("exe", make_exe, depends = ["distribution"])
 register_target("app", make_manifest, depends = ["distribution", "exe"], default = True)
+register_target("msi", make_msi, depends = ["app"])
 
 resolve_targets()
diff --git a/contrib/packaging/wix/mercurial.wxs b/contrib/packaging/wix/mercurial.wxs
--- a/contrib/packaging/wix/mercurial.wxs
+++ b/contrib/packaging/wix/mercurial.wxs
@@ -135,9 +135,13 @@ 
     <UIRef Id="WixUI_FeatureTree" />
     <UIRef Id="WixUI_ErrorProgressText" />
 
+    <?ifdef PyOxidizer?>
+    <WixVariable Id="WixUILicenseRtf" Value="COPYING.rtf" />
+    <Icon Id="hgIcon.ico" SourceFile="mercurial.ico" />
+    <?else?>
     <WixVariable Id="WixUILicenseRtf" Value="contrib\packaging\wix\COPYING.rtf" />
-
     <Icon Id="hgIcon.ico" SourceFile="contrib/win32/mercurial.ico" />
+    <?endif?>
 
     <Upgrade Id='$(var.ProductUpgradeCode)'>
       <UpgradeVersion