Patchwork [1,of,9] test: add a file dedicated to push race between clients

login
register
mail settings
Submitter Pierre-Yves David
Date May 28, 2017, 11:11 p.m.
Message ID <91964bf217343464079f.1496013083@nodosa.octopoid.net>
Download mbox | patch
Permalink /patch/21018/
State Accepted
Headers show

Comments

Pierre-Yves David - May 28, 2017, 11:11 p.m.
# HG changeset patch
# User Pierre-Yves David <pierre-yves.david@octobus.net>
# Date 1495916698 -7200
#      Sat May 27 22:24:58 2017 +0200
# Node ID 91964bf217343464079f0d62fda986ba865a4fe5
# Parent  b647b923486f38d83b92089eafa9faafaa79785d
# EXP-Topic pushrace
# Available At https://www.mercurial-scm.org/repo/users/marmoute/mercurial/
#              hg pull https://www.mercurial-scm.org/repo/users/marmoute/mercurial/ -r 91964bf21734
test: add a file dedicated to push race between clients

There are very few tests around the detection of push race. This file will be
dedicated to covering these cases more through fully. We start with a simple
case. More complex cases get added in later changesets.

My end goal here is to provide a way for server to accept concurrent push as
long as they are not touching the same heads. However, I want to buff the test
coverage of that code before touching anything.
Yuya Nishihara - June 2, 2017, 2:13 p.m.
On Mon, 29 May 2017 01:11:23 +0200, Pierre-Yves David wrote:
> # HG changeset patch
> # User Pierre-Yves David <pierre-yves.david@octobus.net>
> # Date 1495916698 -7200
> #      Sat May 27 22:24:58 2017 +0200
> # Node ID 91964bf217343464079f0d62fda986ba865a4fe5
> # Parent  b647b923486f38d83b92089eafa9faafaa79785d
> # EXP-Topic pushrace
> # Available At https://www.mercurial-scm.org/repo/users/marmoute/mercurial/
> #              hg pull https://www.mercurial-scm.org/repo/users/marmoute/mercurial/ -r 91964bf21734
> test: add a file dedicated to push race between clients

Queued these, thanks.

> +  >             def delete():
> +  >                 try:
> +  >                     os.unlink(watchpath)
> +  >                 except OSError, exc:

Changed to "OSError as exc".

Patch

diff --git a/tests/test-push-race.t b/tests/test-push-race.t
new file mode 100644
--- /dev/null
+++ b/tests/test-push-race.t
@@ -0,0 +1,191 @@ 
+============================================================================================
+Test cases where there are race condition between two clients pushing to the same repository
+============================================================================================
+
+This file tests cases where two clients push to a server at the same time. The
+"raced" client is done preparing it push bundle when the "racing" client
+perform its push. The "raced" client starts its actual push after the "racing"
+client push is fully complete.
+
+A set of extension and shell functions ensures this scheduling.
+
+  $ cat >> delaypush.py << EOF
+  > """small extension orchestrate push race
+  > 
+  > Client with the extensions will create a file when ready and get stuck until
+  > a file is created."""
+  > 
+  > import atexit
+  > import errno
+  > import os
+  > import time
+  > 
+  > from mercurial import (
+  >     exchange,
+  >     extensions,
+  > )
+  > 
+  > def delaypush(orig, pushop):
+  >     # notify we are done preparing
+  >     readypath = pushop.repo.ui.config('delaypush', 'ready-path', None)
+  >     if readypath is not None:
+  >         with open(readypath, 'w') as r:
+  >             r.write('foo')
+  >         pushop.repo.ui.status('wrote ready: %s\n' % readypath)
+  >     # now wait for the other process to be done
+  >     watchpath = pushop.repo.ui.config('delaypush', 'release-path', None)
+  >     if watchpath is not None:
+  >         pushop.repo.ui.status('waiting on: %s\n' % watchpath)
+  >         limit = 100
+  >         while 0 < limit and not os.path.exists(watchpath):
+  >             limit -= 1
+  >             time.sleep(0.1)
+  >         if limit <= 0:
+  >             repo.ui.warn('exiting without watchfile: %s' % watchpath)
+  >         else:
+  >             # delete the file at the end of the push
+  >             def delete():
+  >                 try:
+  >                     os.unlink(watchpath)
+  >                 except OSError, exc:
+  >                     if exc.errno != errno.ENOENT:
+  >                         raise
+  >             atexit.register(delete)
+  >     return orig(pushop)
+  > 
+  > def uisetup(ui):
+  >     extensions.wrapfunction(exchange, '_pushbundle2', delaypush)
+  > EOF
+
+  $ waiton () {
+  >     # wait for a file to be created (then delete it)
+  >     count=100
+  >     while [ ! -f $1 ] ;
+  >     do
+  >         sleep 0.1;
+  >         count=`expr $count - 1`;
+  >         if [ $count -lt 0 ];
+  >         then
+  >              break
+  >         fi;
+  >     done
+  >     [ -f $1 ] || echo "ready file still missing: $1"
+  >     rm -f $1
+  > }
+
+  $ release () {
+  >     # create a file and wait for it be deleted
+  >     count=100
+  >     touch $1
+  >     while [ -f $1 ] ;
+  >     do
+  >         sleep 0.1;
+  >         count=`expr $count - 1`;
+  >         if [ $count -lt 0 ];
+  >         then
+  >              break
+  >         fi;
+  >     done
+  >     [ ! -f $1 ] || echo "delay file still exist: $1"
+  > }
+
+  $ cat >> $HGRCPATH << EOF
+  > [ui]
+  > ssh = python "$TESTDIR/dummyssh"
+  > # simplify output
+  > logtemplate = {node|short} {desc} ({branch})
+  > [alias]
+  > graph = log -G --rev 'sort(all(), "topo")'
+  > EOF
+
+Setup
+-----
+
+create a repo with one root
+
+  $ hg init server
+  $ cd server
+  $ echo root > root
+  $ hg ci -Am "C-ROOT"
+  adding root
+  $ cd ..
+
+clone it in two clients
+
+  $ hg clone ssh://user@dummy/server client-racy
+  requesting all changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 1 changes to 1 files
+  updating to branch default
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ hg clone ssh://user@dummy/server client-other
+  requesting all changes
+  adding changesets
+  adding manifests
+  adding file changes
+  added 1 changesets with 1 changes to 1 files
+  updating to branch default
+  1 files updated, 0 files merged, 0 files removed, 0 files unresolved
+
+setup one to allow race on push
+
+  $ cat >> client-racy/.hg/hgrc << EOF
+  > [extensions]
+  > delaypush = $TESTTMP/delaypush.py
+  > [delaypush]
+  > ready-path = $TESTTMP/readyfile
+  > release-path = $TESTTMP/watchfile
+  > EOF
+
+Simple race, both try to push to the server at the same time
+------------------------------------------------------------
+
+Both try to replace the same head
+
+#  a
+#  | b
+#  |/
+#  *
+
+Creating changesets
+
+  $ echo b > client-other/a
+  $ hg -R client-other/ add client-other/a
+  $ hg -R client-other/ commit -m "C-A"
+  $ echo b > client-racy/b
+  $ hg -R client-racy/ add client-racy/b
+  $ hg -R client-racy/ commit -m "C-B"
+
+Pushing
+
+  $ hg -R client-racy push -r 'tip' > ./push-log 2>&1 &
+
+  $ waiton $TESTTMP/readyfile
+
+  $ hg -R client-other push -r 'tip'
+  pushing to ssh://user@dummy/server
+  searching for changes
+  remote: adding changesets
+  remote: adding manifests
+  remote: adding file changes
+  remote: added 1 changesets with 1 changes to 1 files
+
+  $ release $TESTTMP/watchfile
+
+Check the result of the push
+
+  $ cat ./push-log
+  pushing to ssh://user@dummy/server
+  searching for changes
+  wrote ready: $TESTTMP/readyfile
+  waiting on: $TESTTMP/watchfile
+  abort: push failed:
+  'repository changed while pushing - please try again'
+
+  $ hg -R server graph
+  o  98217d5a1659 C-A (default)
+  |
+  @  842e2fac6304 C-ROOT (default)
+