Patchwork D12384: rust-status: cap the number of concurrent threads to 16

login
register
mail settings
Submitter phabricator
Date March 18, 2022, 4:15 p.m.
Message ID <differential-rev-PHID-DREV-elvp6tii5tgjdrtn5r6g-req@mercurial-scm.org>
Download mbox | patch
Permalink /patch/50728/
State New
Headers show

Comments

phabricator - March 18, 2022, 4:15 p.m.
Alphare created this revision.
Herald added a reviewer: hg-reviewers.
Herald added a subscriber: mercurial-patches.

REVISION SUMMARY
  During benchmarking it was determined that the use of more threads is very
  advantageous... until we use more than 16. This is most likely due to some
  resource contention (thrashing, etc.). Until we have time to figure out and
  fix the underlying cause, let's just cap at 16 threads.

REPOSITORY
  rHG Mercurial

BRANCH
  stable

REVISION DETAIL
  https://phab.mercurial-scm.org/D12384

AFFECTED FILES
  rust/hg-core/src/dirstate_tree/status.rs

CHANGE DETAILS




To: Alphare, #hg-reviewers
Cc: mercurial-patches, mercurial-devel

Patch

diff --git a/rust/hg-core/src/dirstate_tree/status.rs b/rust/hg-core/src/dirstate_tree/status.rs
--- a/rust/hg-core/src/dirstate_tree/status.rs
+++ b/rust/hg-core/src/dirstate_tree/status.rs
@@ -47,6 +47,17 @@ 
     ignore_files: Vec<PathBuf>,
     options: StatusOptions,
 ) -> Result<(DirstateStatus<'on_disk>, Vec<PatternFileWarning>), StatusError> {
+    // Force the global rayon threadpool to not exceed 16 concurrent threads.
+    // This is a stop-gap measure until we figure out why using more than 16
+    // threads makes `status` slower for each additional thread.
+    // We use `ok()` in case the global threadpool has already been instantiated
+    // in `rhg` or some other caller.
+    // TODO find the underlying cause and fix it, then remove this.
+    rayon::ThreadPoolBuilder::new()
+        .num_threads(16)
+        .build_global()
+        .ok();
+
     let (ignore_fn, warnings, patterns_changed): (IgnoreFnType, _, _) =
         if options.list_ignored || options.list_unknown {
             let mut hasher = Sha1::new();