Patchwork crew-stable test failure in ubuntu 12.10

login
register
mail settings
Submitter Giorgos Keramidas
Date Jan. 26, 2013, 12:15 a.m.
Message ID <20130126001544.GA29876@saturn>
Download mbox | patch
Permalink /patch/740/
State Rejected
Headers show

Comments

Giorgos Keramidas - Jan. 26, 2013, 12:15 a.m.
I've recently pulled a crew-stable snapshot from rev ddbe689af784 and
tried to build it on Ubuntu 12.10:

    $ cat /etc/lsb-release
    DISTRIB_ID=Ubuntu
    DISTRIB_RELEASE=12.10
    DISTRIB_CODENAME=quantal
    DISTRIB_DESCRIPTION="Ubuntu 12.10"

    $ hg log -r crew-stable
    changeset:   18479:ddbe689af784
    branch:      stable
    bookmark:    crew-stable
    tag:         tip
    user:        FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
    date:        Thu Jan 24 00:21:22 2013 +0900
    summary:     doc: use "tag" revset predicate instead of "tagged" for example in help

but it fails to pass any inotify related tests, with errors like the
ones attached below. Is this something I managed to break on my side, by
doing something stupid with inotify?


ERROR: /home/gkeramidas/hg/mercurial/gker/tests/test-inotify.t output changed
Matt Mackall - Jan. 26, 2013, 12:53 a.m.
On Sat, 2013-01-26 at 01:15 +0100, Giorgos Keramidas wrote:
> I've recently pulled a crew-stable snapshot from rev ddbe689af784 and
> tried to build it on Ubuntu 12.10:
> 
>     $ cat /etc/lsb-release
>     DISTRIB_ID=Ubuntu
>     DISTRIB_RELEASE=12.10
>     DISTRIB_CODENAME=quantal
>     DISTRIB_DESCRIPTION="Ubuntu 12.10"
> 
>     $ hg log -r crew-stable
>     changeset:   18479:ddbe689af784
>     branch:      stable
>     bookmark:    crew-stable
>     tag:         tip
>     user:        FUJIWARA Katsunori <foozy@lares.dti.ne.jp>
>     date:        Thu Jan 24 00:21:22 2013 +0900
>     summary:     doc: use "tag" revset predicate instead of "tagged" for example in help
> 
> but it fails to pass any inotify related tests, with errors like the
> ones attached below. Is this something I managed to break on my side, by
> doing something stupid with inotify?
> 
> ===== start of error log =====
> 
> --- /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-debuginotify.t
> +++ /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-debuginotify.t.err
> @@ -7,7 +7,13 @@
>  inserve
>  
>    $ hg inserve -d --pid-file=hg.pid
> +  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x28e13f8>> ignored
> +  abort: inotify service not available: Too many open files

Sounds like you have too many inotify instances.

       EMFILE The user limit on the total number of inotify instances has been
       reached.

https://www.kernel.org/doc/man-pages/online/pages/man2/inotify_init.2.html

       /proc/sys/fs/inotify/max_user_instances
              This specifies an upper limit on the number of inotify instances that
              can be created per real user ID.

https://www.kernel.org/doc/man-pages/online/pages/man7/inotify.7.html

$ cat /proc/sys/fs/inotify/max_user_instances 
128
Giorgos Keramidas - Feb. 3, 2013, 8:51 p.m.
On 2013-02-03 14:40, Matt Mackall <mpm@selenic.com> wrote:
>On Sun, 2013-02-03 at 21:22 +0100, Giorgos Keramidas wrote:
>> The biggest consumer of inotify instances seems to be Chrome so far,
>> but this is a very quick and dirty way of crawling /proc:
>> 
>> : 0203 21:19 saturn:/proc$ find . -exec ls -ld {} + 2>/dev/null | fgrep notify | \
>> :   fgrep gkeramidas | sed -e 's@/fd/.*@@' -e 's@.*\./@@' | grep '^[0-9]'  | \
>> :   awk '{print $1}' | fgrep -v /exe | \
>> :   while read pp ; do \
>> :     cat $pp/cmdline | awk '{print $1}' ; \
>> :   done | \
>> :   sort | uniq -c | sort -nr
>> :      72 /opt/google/chrome/chrome
>> :       7 /usr/lib/gnome-settings-daemon/gnome-settings-daemon
>> :       6 kdeinit4:
>> :       5 update-notifier
>> :       4 xfce4-terminal--geometry=100x62
>> :       4 /usr/bin/pulseaudio--start--log-target=syslog
>> :       4 /usr/bin/gnome-screensaver--no-daemon
>> :       3 zeitgeist-datahub
>> :       3 /usr/lib/gvfs/gvfs-udisks2-volume-monitor
>> :       3 /usr/lib/gvfs/gvfsd-trash--spawner:1.7/org/gtk/gvfs/exec_spaw/0
>> :       3 /usr/bin/signon-ui
>> :       2 /bin/dbus-daemon--config-file=/etc/at-spi2/accessibility.conf--nofork--print-address3
>> :       2 //bin/dbus-daemon--fork--print-pid5--print-address7--session
>> : 0203 21:19 saturn:/proc$
>> 
>> I'll dig a bit more, and see if there's anything we can do from our side
>> about it, or if this is not something Mercurial should care about at all.
> 
> Looks like a flagrant Chrome bug to me. I assume you're not running 72
> copies of Chrome or even have 72 tabs open.

Of course not.  I'm looking at a window with 3 tabs right now, so this
is either some form of greedy pre-allocation of inotify instances when
Chrome tries to play 'smart' or a bug.

Patch

===== start of error log =====

--- /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-debuginotify.t
+++ /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-debuginotify.t.err
@@ -7,7 +7,13 @@ 
 inserve
 
   $ hg inserve -d --pid-file=hg.pid
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x28e13f8>> ignored
+  abort: inotify service not available: Too many open files
+  abort: child process failed to start
+  [255]
   $ cat hg.pid >> "$DAEMON_PIDS"
+  cat: hg.pid: No such file or directory
+  [1]
 
 let the daemon finish its stuff
 
@@ -16,26 +22,125 @@ 
 empty
 
   $ hg debuginotify
-  directories being watched:
-    /
-    .hg/
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x17f03b0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
+  ** unknown exception encountered, please report by visiting
+  ** http://mercurial.selenic.com/wiki/BugTracker
+  ** Python 2.7.3 (default, Sep 26 2012, 21:51:14) [GCC 4.7.2]
+  ** Mercurial Distributed SCM (version 2.4.2+244-2.5-rc+15-aa2f6c4fb5d4+ddbe689af784+20130125)
+  ** Extensions loaded: inotify
+  Traceback (most recent call last):
+    File "/tmp/hgtests.s9UEKm/install/bin/hg", line 38, in <module>
+      mercurial.dispatch.run()
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 28, in run
+      sys.exit((dispatch(request(sys.argv[1:])) or 0) & 255)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 65, in dispatch
+      return _runcatch(req)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 88, in _runcatch
+      return _dispatch(req)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 743, in _dispatch
+      cmdpats, cmdoptions)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 514, in runcommand
+      ret = _runcommand(ui, options, cmd, d)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 833, in _runcommand
+      return checkargs()
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 804, in checkargs
+      return cmdfunc()
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 740, in <lambda>
+      d = lambda: util.checksignature(func)(ui, *args, **cmdoptions)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/util.py", line 475, in check
+      return func(*args, **kwargs)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/hgext/inotify/__init__.py", line 30, in debuginotify
+      response = cli.debugquery()
+    File "/tmp/hgtests.s9UEKm/install/lib/python/hgext/inotify/client.py", line 59, in decorated_function
+      raise QueryFailed('inotify query failed')
+  hgext.inotify.client.QueryFailed: inotify query failed
+  [1]
   $ mkdir a
   $ sleep 1
 
 only 'a
 
   $ hg debuginotify
-  directories being watched:
-    /
-    .hg/
-    a/
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x27b03b0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
+  ** unknown exception encountered, please report by visiting
+  ** http://mercurial.selenic.com/wiki/BugTracker
+  ** Python 2.7.3 (default, Sep 26 2012, 21:51:14) [GCC 4.7.2]
+  ** Mercurial Distributed SCM (version 2.4.2+244-2.5-rc+15-aa2f6c4fb5d4+ddbe689af784+20130125)
+  ** Extensions loaded: inotify
+  Traceback (most recent call last):
+    File "/tmp/hgtests.s9UEKm/install/bin/hg", line 38, in <module>
+      mercurial.dispatch.run()
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 28, in run
+      sys.exit((dispatch(request(sys.argv[1:])) or 0) & 255)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 65, in dispatch
+      return _runcatch(req)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 88, in _runcatch
+      return _dispatch(req)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 743, in _dispatch
+      cmdpats, cmdoptions)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 514, in runcommand
+      ret = _runcommand(ui, options, cmd, d)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 833, in _runcommand
+      return checkargs()
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 804, in checkargs
+      return cmdfunc()
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 740, in <lambda>
+      d = lambda: util.checksignature(func)(ui, *args, **cmdoptions)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/util.py", line 475, in check
+      return func(*args, **kwargs)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/hgext/inotify/__init__.py", line 30, in debuginotify
+      response = cli.debugquery()
+    File "/tmp/hgtests.s9UEKm/install/lib/python/hgext/inotify/client.py", line 59, in decorated_function
+      raise QueryFailed('inotify query failed')
+  hgext.inotify.client.QueryFailed: inotify query failed
+  [1]
   $ rmdir a
   $ sleep 1
 
 empty again
 
   $ hg debuginotify
-  directories being watched:
-    /
-    .hg/
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x14173b0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
+  ** unknown exception encountered, please report by visiting
+  ** http://mercurial.selenic.com/wiki/BugTracker
+  ** Python 2.7.3 (default, Sep 26 2012, 21:51:14) [GCC 4.7.2]
+  ** Mercurial Distributed SCM (version 2.4.2+244-2.5-rc+15-aa2f6c4fb5d4+ddbe689af784+20130125)
+  ** Extensions loaded: inotify
+  Traceback (most recent call last):
+    File "/tmp/hgtests.s9UEKm/install/bin/hg", line 38, in <module>
+      mercurial.dispatch.run()
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 28, in run
+      sys.exit((dispatch(request(sys.argv[1:])) or 0) & 255)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 65, in dispatch
+      return _runcatch(req)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 88, in _runcatch
+      return _dispatch(req)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 743, in _dispatch
+      cmdpats, cmdoptions)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 514, in runcommand
+      ret = _runcommand(ui, options, cmd, d)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 833, in _runcommand
+      return checkargs()
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 804, in checkargs
+      return cmdfunc()
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/dispatch.py", line 740, in <lambda>
+      d = lambda: util.checksignature(func)(ui, *args, **cmdoptions)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/mercurial/util.py", line 475, in check
+      return func(*args, **kwargs)
+    File "/tmp/hgtests.s9UEKm/install/lib/python/hgext/inotify/__init__.py", line 30, in debuginotify
+      response = cli.debugquery()
+    File "/tmp/hgtests.s9UEKm/install/lib/python/hgext/inotify/client.py", line 59, in decorated_function
+      raise QueryFailed('inotify query failed')
+  hgext.inotify.client.QueryFailed: inotify query failed
+  [1]
   $ kill `cat hg.pid`
+  cat: hg.pid: No such file or directory
+  /tmp/tmpyw76XYhg-tst: 30: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or
+  kill -l [exitstatus]
+  [2]

ERROR: /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-debuginotify.t output changed
!
--- /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-dirty-dirstate.t
+++ /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-dirty-dirstate.t.err
@@ -9,33 +9,62 @@ 
 
   $ hg init test; cd test
   $ hg inserve -d --pid-file=../hg.pid
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x1971680>> ignored
+  abort: inotify service not available: Too many open files
+  abort: child process failed to start
+  [255]
   $ cat ../hg.pid >> "$DAEMON_PIDS"
+  cat: ../hg.pid: No such file or directory
+  [1]
   $ echo foo > foo
   $ hg add
   adding foo
   $ hg ci -m foo
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x12f06c8>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ cd ..
   $ hg --config "inotify.pidfile=../hg2.pid" clone test test2
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x191b7e8>> ignored
+  abort: inotify service not available: Too many open files
   updating to branch default
+  inotify-client: could not start inotify server: child process failed to start
   1 files updated, 0 files merged, 0 files removed, 0 files unresolved
   $ cat ../hg2.pid >> "$DAEMON_PIDS"
+  cat: ../hg2.pid: No such file or directory
+  [1]
   $ cd test2
   $ echo bar > bar
   $ hg add
   adding bar
   $ hg ci -m bar
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x1517878>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ cd ../test
   $ echo spam > spam
   $ hg add
   adding spam
   $ hg ci -m spam
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0xe636c8>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ cd ../test2
   $ hg st
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0xfc4878>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
 
 abort, outstanding changes
 
   $ hg fetch -q
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0xfad878>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ hg st
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x2347878>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ cd ..
 
 issue1719: inotify and mq
@@ -47,26 +76,56 @@ 
 inserve
 
   $ hg inserve -d --pid-file=../hg-test-1719.pid
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x1c81998>> ignored
+  abort: inotify service not available: Too many open files
+  abort: child process failed to start
+  [255]
   $ cat ../hg-test-1719.pid >> "$DAEMON_PIDS"
+  cat: ../hg-test-1719.pid: No such file or directory
+  [1]
   $ echo content > file
   $ hg add file
   $ hg qnew -f test.patch
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0xffd998>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ hg status
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x1bc9998>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ hg qpop
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x28d1998>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   popping test.patch
   patch queue now empty
 
 st should not output anything
 
   $ hg status
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x15e1998>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ hg qpush
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x29ec998>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   applying test.patch
   now at: test.patch
 
 st should not output anything
 
   $ hg status
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x1bf4998>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ hg qrefresh
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0xc0d998>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ hg status
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x107f998>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
 
   $ cd ..

ERROR: /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-dirty-dirstate.t output changed
!
--- /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-issue1371.t
+++ /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-issue1371.t.err
@@ -8,15 +8,23 @@ 
 inserve
 
   $ hg inserve -d --pid-file=hg.pid 2>&1
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x217e3f8>> ignored
+  abort: inotify service not available: Too many open files
+  abort: child process failed to start
+  [255]
   $ cat hg.pid >> "$DAEMON_PIDS"
+  cat: hg.pid: No such file or directory
+  [1]
   $ hg ci -Am m
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x2b49440>> ignored
+  abort: inotify service not available: Too many open files
   adding a
   adding b
   adding c
   adding d
   adding e
   adding f
-  adding hg.pid
+  inotify-client: could not start inotify server: child process failed to start
 
 let the daemon finish its stuff
 
@@ -25,12 +33,18 @@ 
 eed to test all file operations
 
   $ hg rm a
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x2197440>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ rm b
   $ echo c >> c
   $ touch g
   $ hg add g
   $ hg mv e h
   $ hg status
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x270f440>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   M c
   A g
   A h
@@ -42,3 +56,7 @@ 
 Are we able to kill the service? if not, the service died on some error
 
   $ kill `cat hg.pid`
+  cat: hg.pid: No such file or directory
+  /tmp/tmpHqiYjkhg-tst: 36: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or
+  kill -l [exitstatus]
+  [2]

ERROR: /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-issue1371.t output changed
!
--- /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-issue1542.t
+++ /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-issue1542.t.err
@@ -12,12 +12,20 @@ 
 inserve
 
   $ hg inserve -d --pid-file=hg.pid 2>&1
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x247e488>> ignored
+  abort: inotify service not available: Too many open files
+  abort: child process failed to start
+  [255]
   $ cat hg.pid >> "$DAEMON_PIDS"
+  cat: hg.pid: No such file or directory
+  [1]
   $ hg st
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x1a40440>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   A dir/c
   ? a
   ? dir/b
-  ? hg.pid
 
 moving dir out
 
@@ -26,11 +34,17 @@ 
 status
 
   $ hg st
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x20a0440>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   ! dir/c
   ? a
-  ? hg.pid
   $ sleep 1
 
 Are we able to kill the service? if not, the service died on some error
 
   $ kill `cat hg.pid`
+  cat: hg.pid: No such file or directory
+  /tmp/tmpm7Y01jhg-tst: 32: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or
+  kill -l [exitstatus]
+  [2]

ERROR: /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-issue1542.t output changed
!
--- /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-issue1556.t
+++ /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-issue1556.t.err
@@ -16,16 +16,28 @@ 
 inserve
 
   $ hg inserve -d --pid-file=hg.pid 2>&1
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x2ae6488>> ignored
+  abort: inotify service not available: Too many open files
+  abort: child process failed to start
+  [255]
   $ cat hg.pid >> "$DAEMON_PIDS"
+  cat: hg.pid: No such file or directory
+  [1]
 
 status
 
   $ hg st
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x1f78440>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   A a
   ! b
-  ? hg.pid
   $ sleep 1
 
 Are we able to kill the service? if not, the service died on some error
 
   $ kill `cat hg.pid`
+  cat: hg.pid: No such file or directory
+  /tmp/tmpqJMQmdhg-tst: 26: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or
+  kill -l [exitstatus]
+  [2]

ERROR: /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-issue1556.t output changed
!
--- /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-lookup.t
+++ /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-lookup.t.err
@@ -4,11 +4,31 @@ 
   $ echo "[extensions]" > .hg/hgrc
   $ echo "inotify=" >> .hg/hgrc
   $ hg inserve -d --pid-file .hg/inotify.pid
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x29ad368>> ignored
+  abort: inotify service not available: Too many open files
+  abort: child process failed to start
+  [255]
   $ echo a > a
   $ hg ci -Aqm0
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x29623f8>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ hg co -q null
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0xc1f3f8>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ hg co -q
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x1d083f8>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ hg st
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x1cbd3f8>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ cat a
   a
   $ kill `cat .hg/inotify.pid`
+  cat: .hg/inotify.pid: No such file or directory
+  /tmp/tmpVSTqYohg-tst: 24: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or
+  kill -l [exitstatus]
+  [2]

ERROR: /home/gkeramidas/hg/mercurial/gker/tests/test-inotify-lookup.t output changed
!
--- /home/gkeramidas/hg/mercurial/gker/tests/test-inotify.t
+++ /home/gkeramidas/hg/mercurial/gker/tests/test-inotify.t.err
@@ -28,17 +28,30 @@ 
 and make sure that inotify.pidfile works
 
   $ hg --config "inotify.pidfile=../hg2.pid" status
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x21d96c8>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   M a
 
 make sure that pidfile worked. Output should be silent.
 
   $ kill `cat ../hg2.pid`
+  cat: ../hg2.pid: No such file or directory
+  /tmp/tmpEHSOHahg-tst: 32: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or
+  kill -l [exitstatus]
+  [2]
   $ cd ../repo1
 
 inserve
 
   $ hg inserve -d --pid-file=hg.pid
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x1943518>> ignored
+  abort: inotify service not available: Too many open files
+  abort: child process failed to start
+  [255]
   $ cat hg.pid >> "$DAEMON_PIDS"
+  cat: hg.pid: No such file or directory
+  [1]
 
 let the daemon finish its stuff
 
@@ -47,17 +60,23 @@ 
 cannot start, already bound
 
   $ hg inserve
-  abort: inotify-server: cannot start: socket is already bound
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x22df518>> ignored
+  abort: inotify service not available: Too many open files
   [255]
 
 issue907
 
   $ hg status
-  ? hg.pid
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x23004d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
 
 clean
 
   $ hg status -c
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x21504d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   C a
   C b
   C c
@@ -70,7 +89,6 @@ 
 all
 
   $ hg status -A
-  ? hg.pid
   C a
   C b
   C c
@@ -84,12 +102,20 @@ 
 
   $ echo x > dir/x
   $ hg status .
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0xfb54d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   M dir/x
-  ? hg.pid
   $ hg status dir
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x1f514d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   M dir/x
   $ cd dir
   $ hg status .
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x29574d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   M x
   $ cd ..
 
@@ -100,36 +126,77 @@ 
   $ mkdir h
   $ echo h > h/h
   $ hg ci -Am t
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x23c74d0>> ignored
+  abort: inotify service not available: Too many open files
   adding h/h
-  adding hg.pid
+  inotify-client: could not start inotify server: child process failed to start
   $ hg rm h
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0xce44d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   removing h/h
   $ echo h >h
   $ hg add h
   $ hg status
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x24a54d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   A h
   R h/h
   $ hg ci -m0
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x1de84d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
 
 Test for issue1735: inotify watches files in .hg/merge
 
   $ hg st
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x27184d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ echo a > a
   $ hg ci -Am a
-  $ hg st
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x280c4d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
+  $ hg st
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x160e4d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ echo b >> a
   $ hg ci -m ab
-  $ hg st
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x14054d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
+  $ hg st
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x235a4d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ echo c >> a
   $ hg st
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x1cb74d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   M a
   $ HGMERGE=internal:local hg up 0
-  1 files updated, 1 files merged, 2 files removed, 0 files unresolved
-  $ hg st
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x230c4d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
+  1 files updated, 1 files merged, 1 files removed, 0 files unresolved
+  $ hg st
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x12984d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   M a
   $ HGMERGE=internal:local hg up
-  3 files updated, 1 files merged, 0 files removed, 0 files unresolved
-  $ hg st
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x11ab4d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
+  2 files updated, 1 files merged, 0 files removed, 0 files unresolved
+  $ hg st
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x1e5c4d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   M a
 
 Test for 1844: "hg ci folder" will not commit all changes beneath "folder"
@@ -139,14 +206,23 @@ 
   $ hg add 1844
   adding 1844/foo
   $ hg ci -m 'working'
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x18ee4d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ echo b >> 1844/foo
   $ hg ci 1844 -m 'broken'
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x16664d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
 
 Test for issue884: "Build products not ignored until .hgignore is touched"
 
   $ echo '^build$' > .hgignore
   $ hg add .hgignore
   $ hg ci .hgignore -m 'ignorelist'
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x29274d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
 
 Now, lets add some build products...
 
@@ -157,7 +233,14 @@ 
 build/x & build/y shouldn't appear in "hg st"
 
   $ hg st
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0xd904d0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ kill `cat hg.pid`
+  cat: hg.pid: No such file or directory
+  /tmp/tmpEHSOHahg-tst: 130: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or
+  kill -l [exitstatus]
+  [2]
 
   $ cd ..
 
@@ -170,13 +253,21 @@ 
   $ cd 60_456789_70_456789_80_456789_90_456789_100_56789_
 
   $ hg --config inotify.pidfile=hg3.pid clone -q ../../repo1
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0x26715f0>> ignored
+  abort: inotify service not available: Too many open files
+  inotify-client: could not start inotify server: child process failed to start
   $ readlink repo1/.hg/inotify.sock
-  */inotify.sock (glob)
+  [1]
 
 Trying to start the server a second time should fail as usual.
 
   $ hg --cwd repo1 inserve
-  abort: inotify-server: cannot start: socket is already bound
+  Exception AttributeError: 'fd' in <bound method watcher.__del__ of <hgext.inotify.linux.watcher.watcher object at 0xe68680>> ignored
+  abort: inotify service not available: Too many open files
   [255]
 
   $ kill `cat hg3.pid`
+  cat: hg3.pid: No such file or directory
+  /tmp/tmpEHSOHahg-tst: 148: kill: Usage: kill [-s sigspec | -signum | -sigspec] [pid | job]... or
+  kill -l [exitstatus]
+  [2]