Patchwork [2,of,2,V2] dirstate: track otherparent files same as nonnormal

login
register
mail settings
Submitter Durham Goode
Date March 6, 2017, 12:51 a.m.
Message ID <a465bc91a3c1a04c9c75.1488761461@dev111.prn1.facebook.com>
Download mbox | patch
Permalink /patch/18914/
State Accepted
Headers show

Comments

Durham Goode - March 6, 2017, 12:51 a.m.
# HG changeset patch
# User Durham Goode <durham@fb.com>
# Date 1488761157 28800
#      Sun Mar 05 16:45:57 2017 -0800
# Node ID a465bc91a3c1a04c9c75a3cbd13fd5b777487de9
# Parent  4a751a57ef0682dc3d7e46bdfa67a26b13cd9031
dirstate: track otherparent files same as nonnormal

Calling dirstate.setparents() is expensive in a large repo because it iterates
over every file in the dirstate. It does so to undo any merge state or
otherparent state files. Merge state files are already covered by
dirstate._nonnormalset, so we just need to track otherparent files in a similar
manner to avoid the full iteration here.

Fixing this shaves 20-25% off histedit in large repos.

I tested this by adding temporary debug logic to verify that the old files
processed in the loop matched the new files processed in the loop and running
the test suite.
Augie Fackler - March 7, 2017, 5:29 p.m.
On Sun, Mar 05, 2017 at 04:51:01PM -0800, Durham Goode wrote:
> # HG changeset patch
> # User Durham Goode <durham@fb.com>
> # Date 1488761157 28800
> #      Sun Mar 05 16:45:57 2017 -0800
> # Node ID a465bc91a3c1a04c9c75a3cbd13fd5b777487de9
> # Parent  4a751a57ef0682dc3d7e46bdfa67a26b13cd9031
> dirstate: track otherparent files same as nonnormal

Very nice. Queued.
Pierre-Yves David - March 9, 2017, 12:10 a.m.
On 03/07/2017 09:29 AM, Augie Fackler wrote:
> On Sun, Mar 05, 2017 at 04:51:01PM -0800, Durham Goode wrote:
>> # HG changeset patch
>> # User Durham Goode <durham@fb.com>
>> # Date 1488761157 28800
>> #      Sun Mar 05 16:45:57 2017 -0800
>> # Node ID a465bc91a3c1a04c9c75a3cbd13fd5b777487de9
>> # Parent  4a751a57ef0682dc3d7e46bdfa67a26b13cd9031
>> dirstate: track otherparent files same as nonnormal
>
> Very nice. Queued.

That changesets changes the signature of a function in compiled module. 
This force a `hg make local` every time the boundary created by this 
revision is crossed.

In the past we have tried hard to not create such situation, should we 
do something to smooth things in this specific instance?

Cheers,
via Mercurial-devel - March 9, 2017, 12:14 a.m.
On Wed, Mar 8, 2017 at 4:10 PM, Pierre-Yves David
<pierre-yves.david@ens-lyon.org> wrote:
>
>
> On 03/07/2017 09:29 AM, Augie Fackler wrote:
>>
>> On Sun, Mar 05, 2017 at 04:51:01PM -0800, Durham Goode wrote:
>>>
>>> # HG changeset patch
>>> # User Durham Goode <durham@fb.com>
>>> # Date 1488761157 28800
>>> #      Sun Mar 05 16:45:57 2017 -0800
>>> # Node ID a465bc91a3c1a04c9c75a3cbd13fd5b777487de9
>>> # Parent  4a751a57ef0682dc3d7e46bdfa67a26b13cd9031
>>> dirstate: track otherparent files same as nonnormal
>>
>>
>> Very nice. Queued.
>
>
> That changesets changes the signature of a function in compiled module. This
> force a `hg make local` every time the boundary created by this revision is
> crossed.
>
> In the past we have tried hard to not create such situation, should we do
> something to smooth things in this specific instance?

Good catch. This is perhaps why some tests failed for me for much of
the afternoon.

>
> Cheers,
>
> --
> Pierre-Yves David
>
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel@mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel
Durham Goode - March 9, 2017, 12:16 a.m.
On 3/8/17 4:14 PM, Martin von Zweigbergk wrote:
> On Wed, Mar 8, 2017 at 4:10 PM, Pierre-Yves David
> <pierre-yves.david@ens-lyon.org> wrote:
>>
>>
>> On 03/07/2017 09:29 AM, Augie Fackler wrote:
>>>
>>> On Sun, Mar 05, 2017 at 04:51:01PM -0800, Durham Goode wrote:
>>>>
>>>> # HG changeset patch
>>>> # User Durham Goode <durham@fb.com>
>>>> # Date 1488761157 28800
>>>> #      Sun Mar 05 16:45:57 2017 -0800
>>>> # Node ID a465bc91a3c1a04c9c75a3cbd13fd5b777487de9
>>>> # Parent  4a751a57ef0682dc3d7e46bdfa67a26b13cd9031
>>>> dirstate: track otherparent files same as nonnormal
>>>
>>>
>>> Very nice. Queued.
>>
>>
>> That changesets changes the signature of a function in compiled module. This
>> force a `hg make local` every time the boundary created by this revision is
>> crossed.
>>
>> In the past we have tried hard to not create such situation, should we do
>> something to smooth things in this specific instance?
>
> Good catch. This is perhaps why some tests failed for me for much of
> the afternoon.

Good point.  Feel free to drop #2 from the queue and I can resend with a 
backwards compatible version.
Jun Wu - March 9, 2017, 12:20 a.m.
Excerpts from Pierre-Yves David's message of 2017-03-08 16:10:04 -0800:
> 
> On 03/07/2017 09:29 AM, Augie Fackler wrote:
> > On Sun, Mar 05, 2017 at 04:51:01PM -0800, Durham Goode wrote:
> >> # HG changeset patch
> >> # User Durham Goode <durham@fb.com>
> >> # Date 1488761157 28800
> >> #      Sun Mar 05 16:45:57 2017 -0800
> >> # Node ID a465bc91a3c1a04c9c75a3cbd13fd5b777487de9
> >> # Parent  4a751a57ef0682dc3d7e46bdfa67a26b13cd9031
> >> dirstate: track otherparent files same as nonnormal
> >
> > Very nice. Queued.
> 
> That changesets changes the signature of a function in compiled module. 
> This force a `hg make local` every time the boundary created by this 
> revision is crossed.
> 
> In the past we have tried hard to not create such situation, should we 
> do something to smooth things in this specific instance?

If we cannot change C signature, then some improvements of chg couldn't be
done. I would prefer code cleanness and performance over C compatibility.

> 
> Cheers,
>
Augie Fackler - March 9, 2017, 12:29 a.m.
> On Mar 8, 2017, at 19:20, Jun Wu <quark@fb.com> wrote:
> 
> Excerpts from Pierre-Yves David's message of 2017-03-08 16:10:04 -0800:
>> 
>> On 03/07/2017 09:29 AM, Augie Fackler wrote:
>>> On Sun, Mar 05, 2017 at 04:51:01PM -0800, Durham Goode wrote:
>>>> # HG changeset patch
>>>> # User Durham Goode <durham@fb.com>
>>>> # Date 1488761157 28800
>>>> #      Sun Mar 05 16:45:57 2017 -0800
>>>> # Node ID a465bc91a3c1a04c9c75a3cbd13fd5b777487de9
>>>> # Parent  4a751a57ef0682dc3d7e46bdfa67a26b13cd9031
>>>> dirstate: track otherparent files same as nonnormal
>>> 
>>> Very nice. Queued.
>> 
>> That changesets changes the signature of a function in compiled module. 
>> This force a `hg make local` every time the boundary created by this 
>> revision is crossed.
>> 
>> In the past we have tried hard to not create such situation, should we 
>> do something to smooth things in this specific instance?
> 
> If we cannot change C signature, then some improvements of chg couldn't be
> done. I would prefer code cleanness and performance over C compatibility.

You can always introduce a new function with the newly desired signature. We've done that a fair amount in the past.

I'll drop this presently. Durham, you may want to `hg touch` this locally to avoid losing it.

> 
>> 
>> Cheers,
via Mercurial-devel - March 9, 2017, 12:31 a.m.
On Wed, Mar 8, 2017 at 4:16 PM, Durham Goode <durham@fb.com> wrote:
>
>
> On 3/8/17 4:14 PM, Martin von Zweigbergk wrote:
>>
>> On Wed, Mar 8, 2017 at 4:10 PM, Pierre-Yves David
>> <pierre-yves.david@ens-lyon.org> wrote:
>>>
>>>
>>>
>>> On 03/07/2017 09:29 AM, Augie Fackler wrote:
>>>>
>>>>
>>>> On Sun, Mar 05, 2017 at 04:51:01PM -0800, Durham Goode wrote:
>>>>>
>>>>>
>>>>> # HG changeset patch
>>>>> # User Durham Goode <durham@fb.com>
>>>>> # Date 1488761157 28800
>>>>> #      Sun Mar 05 16:45:57 2017 -0800
>>>>> # Node ID a465bc91a3c1a04c9c75a3cbd13fd5b777487de9
>>>>> # Parent  4a751a57ef0682dc3d7e46bdfa67a26b13cd9031
>>>>> dirstate: track otherparent files same as nonnormal
>>>>
>>>>
>>>>
>>>> Very nice. Queued.
>>>
>>>
>>>
>>> That changesets changes the signature of a function in compiled module.
>>> This
>>> force a `hg make local` every time the boundary created by this revision
>>> is
>>> crossed.
>>>
>>> In the past we have tried hard to not create such situation, should we do
>>> something to smooth things in this specific instance?
>>
>>
>> Good catch. This is perhaps why some tests failed for me for much of
>> the afternoon.
>
>
> Good point.  Feel free to drop #2 from the queue and I can resend with a
> backwards compatible version.

Done. Although I probably did it incorrectly, so it wouldn't surprise
me if your old commit is there too.
Augie Fackler - March 9, 2017, 12:34 a.m.
> On Mar 8, 2017, at 19:31, Martin von Zweigbergk <martinvonz@google.com> wrote:
> 
> On Wed, Mar 8, 2017 at 4:16 PM, Durham Goode <durham@fb.com> wrote:
>> 
>> 
>> On 3/8/17 4:14 PM, Martin von Zweigbergk wrote:
>>> 
>>> On Wed, Mar 8, 2017 at 4:10 PM, Pierre-Yves David
>>> <pierre-yves.david@ens-lyon.org> wrote:
>>>> 
>>>> 
>>>> 
>>>> On 03/07/2017 09:29 AM, Augie Fackler wrote:
>>>>> 
>>>>> 
>>>>> On Sun, Mar 05, 2017 at 04:51:01PM -0800, Durham Goode wrote:
>>>>>> 
>>>>>> 
>>>>>> # HG changeset patch
>>>>>> # User Durham Goode <durham@fb.com>
>>>>>> # Date 1488761157 28800
>>>>>> #      Sun Mar 05 16:45:57 2017 -0800
>>>>>> # Node ID a465bc91a3c1a04c9c75a3cbd13fd5b777487de9
>>>>>> # Parent  4a751a57ef0682dc3d7e46bdfa67a26b13cd9031
>>>>>> dirstate: track otherparent files same as nonnormal
>>>>> 
>>>>> 
>>>>> 
>>>>> Very nice. Queued.
>>>> 
>>>> 
>>>> 
>>>> That changesets changes the signature of a function in compiled module.
>>>> This
>>>> force a `hg make local` every time the boundary created by this revision
>>>> is
>>>> crossed.
>>>> 
>>>> In the past we have tried hard to not create such situation, should we do
>>>> something to smooth things in this specific instance?
>>> 
>>> 
>>> Good catch. This is perhaps why some tests failed for me for much of
>>> the afternoon.
>> 
>> 
>> Good point.  Feel free to drop #2 from the queue and I can resend with a
>> backwards compatible version.
> 
> Done. Although I probably did it incorrectly, so it wouldn't surprise
> me if your old commit is there too.

Ah, thanks. I won't do the drop then. :)
via Mercurial-devel - March 9, 2017, 12:36 a.m.
On Wed, Mar 8, 2017 at 4:34 PM, Augie Fackler <raf@durin42.com> wrote:
>
>> On Mar 8, 2017, at 19:31, Martin von Zweigbergk <martinvonz@google.com> wrote:
>>
>> On Wed, Mar 8, 2017 at 4:16 PM, Durham Goode <durham@fb.com> wrote:
>>>
>>>
>>> On 3/8/17 4:14 PM, Martin von Zweigbergk wrote:
>>>>
>>>> On Wed, Mar 8, 2017 at 4:10 PM, Pierre-Yves David
>>>> <pierre-yves.david@ens-lyon.org> wrote:
>>>>>
>>>>>
>>>>>
>>>>> On 03/07/2017 09:29 AM, Augie Fackler wrote:
>>>>>>
>>>>>>
>>>>>> On Sun, Mar 05, 2017 at 04:51:01PM -0800, Durham Goode wrote:
>>>>>>>
>>>>>>>
>>>>>>> # HG changeset patch
>>>>>>> # User Durham Goode <durham@fb.com>
>>>>>>> # Date 1488761157 28800
>>>>>>> #      Sun Mar 05 16:45:57 2017 -0800
>>>>>>> # Node ID a465bc91a3c1a04c9c75a3cbd13fd5b777487de9
>>>>>>> # Parent  4a751a57ef0682dc3d7e46bdfa67a26b13cd9031
>>>>>>> dirstate: track otherparent files same as nonnormal
>>>>>>
>>>>>>
>>>>>>
>>>>>> Very nice. Queued.
>>>>>
>>>>>
>>>>>
>>>>> That changesets changes the signature of a function in compiled module.
>>>>> This
>>>>> force a `hg make local` every time the boundary created by this revision
>>>>> is
>>>>> crossed.
>>>>>
>>>>> In the past we have tried hard to not create such situation, should we do
>>>>> something to smooth things in this specific instance?
>>>>
>>>>
>>>> Good catch. This is perhaps why some tests failed for me for much of
>>>> the afternoon.
>>>
>>>
>>> Good point.  Feel free to drop #2 from the queue and I can resend with a
>>> backwards compatible version.
>>
>> Done. Although I probably did it incorrectly, so it wouldn't surprise
>> me if your old commit is there too.
>
> Ah, thanks. I won't do the drop then. :)

I'd still appreciate it you or Pierre-Yves could check that the
obsmarker was transferred. I suspect it wasn't, but I don't know how
to fix.
Augie Fackler - March 9, 2017, 12:37 a.m.
> On Mar 8, 2017, at 19:36, Martin von Zweigbergk <martinvonz@google.com> wrote:
> 
> On Wed, Mar 8, 2017 at 4:34 PM, Augie Fackler <raf@durin42.com> wrote:
>> 
>>> On Mar 8, 2017, at 19:31, Martin von Zweigbergk <martinvonz@google.com> wrote:
>>> 
>>> On Wed, Mar 8, 2017 at 4:16 PM, Durham Goode <durham@fb.com> wrote:
>>>> 
>>>> 
>>>> On 3/8/17 4:14 PM, Martin von Zweigbergk wrote:
>>>>> 
>>>>> On Wed, Mar 8, 2017 at 4:10 PM, Pierre-Yves David
>>>>> <pierre-yves.david@ens-lyon.org> wrote:
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On 03/07/2017 09:29 AM, Augie Fackler wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> On Sun, Mar 05, 2017 at 04:51:01PM -0800, Durham Goode wrote:
>>>>>>>> 
>>>>>>>> 
>>>>>>>> # HG changeset patch
>>>>>>>> # User Durham Goode <durham@fb.com>
>>>>>>>> # Date 1488761157 28800
>>>>>>>> #      Sun Mar 05 16:45:57 2017 -0800
>>>>>>>> # Node ID a465bc91a3c1a04c9c75a3cbd13fd5b777487de9
>>>>>>>> # Parent  4a751a57ef0682dc3d7e46bdfa67a26b13cd9031
>>>>>>>> dirstate: track otherparent files same as nonnormal
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> Very nice. Queued.
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> That changesets changes the signature of a function in compiled module.
>>>>>> This
>>>>>> force a `hg make local` every time the boundary created by this revision
>>>>>> is
>>>>>> crossed.
>>>>>> 
>>>>>> In the past we have tried hard to not create such situation, should we do
>>>>>> something to smooth things in this specific instance?
>>>>> 
>>>>> 
>>>>> Good catch. This is perhaps why some tests failed for me for much of
>>>>> the afternoon.
>>>> 
>>>> 
>>>> Good point.  Feel free to drop #2 from the queue and I can resend with a
>>>> backwards compatible version.
>>> 
>>> Done. Although I probably did it incorrectly, so it wouldn't surprise
>>> me if your old commit is there too.
>> 
>> Ah, thanks. I won't do the drop then. :)
> 
> I'd still appreciate it you or Pierre-Yves could check that the
> obsmarker was transferred. I suspect it wasn't, but I don't know how
> to fix.

Everything looked fine to me at a quick glance.
Pierre-Yves David - March 9, 2017, 12:58 a.m.
I also confirms that I see a prune marker for this changeset

On 03/08/2017 04:37 PM, Augie Fackler wrote:
>
>> On Mar 8, 2017, at 19:36, Martin von Zweigbergk <martinvonz@google.com> wrote:
>>
>> On Wed, Mar 8, 2017 at 4:34 PM, Augie Fackler <raf@durin42.com> wrote:
>>>
>>>> On Mar 8, 2017, at 19:31, Martin von Zweigbergk <martinvonz@google.com> wrote:
>>>>
>>>> On Wed, Mar 8, 2017 at 4:16 PM, Durham Goode <durham@fb.com> wrote:
>>>>>
>>>>>
>>>>> On 3/8/17 4:14 PM, Martin von Zweigbergk wrote:
>>>>>>
>>>>>> On Wed, Mar 8, 2017 at 4:10 PM, Pierre-Yves David
>>>>>> <pierre-yves.david@ens-lyon.org> wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 03/07/2017 09:29 AM, Augie Fackler wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sun, Mar 05, 2017 at 04:51:01PM -0800, Durham Goode wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> # HG changeset patch
>>>>>>>>> # User Durham Goode <durham@fb.com>
>>>>>>>>> # Date 1488761157 28800
>>>>>>>>> #      Sun Mar 05 16:45:57 2017 -0800
>>>>>>>>> # Node ID a465bc91a3c1a04c9c75a3cbd13fd5b777487de9
>>>>>>>>> # Parent  4a751a57ef0682dc3d7e46bdfa67a26b13cd9031
>>>>>>>>> dirstate: track otherparent files same as nonnormal
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Very nice. Queued.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> That changesets changes the signature of a function in compiled module.
>>>>>>> This
>>>>>>> force a `hg make local` every time the boundary created by this revision
>>>>>>> is
>>>>>>> crossed.
>>>>>>>
>>>>>>> In the past we have tried hard to not create such situation, should we do
>>>>>>> something to smooth things in this specific instance?
>>>>>>
>>>>>>
>>>>>> Good catch. This is perhaps why some tests failed for me for much of
>>>>>> the afternoon.
>>>>>
>>>>>
>>>>> Good point.  Feel free to drop #2 from the queue and I can resend with a
>>>>> backwards compatible version.
>>>>
>>>> Done. Although I probably did it incorrectly, so it wouldn't surprise
>>>> me if your old commit is there too.
>>>
>>> Ah, thanks. I won't do the drop then. :)
>>
>> I'd still appreciate it you or Pierre-Yves could check that the
>> obsmarker was transferred. I suspect it wasn't, but I don't know how
>> to fix.
>
> Everything looked fine to me at a quick glance.
>

Patch

diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -57,8 +57,14 @@  def nonnormalentries(dmap):
     try:
         return parsers.nonnormalentries(dmap)
     except AttributeError:
-        return set(fname for fname, e in dmap.iteritems()
-                   if e[0] != 'n' or e[3] == -1)
+        nonnorm = set()
+        otherparent = set()
+        for fname, e in dmap.iteritems():
+            if e[0] != 'n' or e[3] == -1:
+                nonnorm.add(fname)
+            if e[0] == 'n' and e[2] == -2:
+                otherparent.add(fname)
+        return nonnorm, otherparent
 
 class dirstate(object):
 
@@ -131,7 +137,15 @@  class dirstate(object):
 
     @propertycache
     def _nonnormalset(self):
-        return nonnormalentries(self._map)
+        nonnorm, otherparents = nonnormalentries(self._map)
+        self._otherparentset = otherparents
+        return nonnorm
+
+    @propertycache
+    def _otherparentset(self):
+        nonnorm, otherparents = nonnormalentries(self._map)
+        self._nonnormalset = nonnorm
+        return otherparents
 
     @propertycache
     def _filefoldmap(self):
@@ -341,7 +355,12 @@  class dirstate(object):
         self._pl = p1, p2
         copies = {}
         if oldp2 != nullid and p2 == nullid:
-            for f, s in self._map.iteritems():
+            candidatefiles = self._nonnormalset.union(self._otherparentset)
+            for f in candidatefiles:
+                s = self._map.get(f)
+                if s is None:
+                    continue
+
                 # Discard 'm' markers when moving away from a merge state
                 if s[0] == 'm':
                     if f in self._copymap:
@@ -427,7 +446,8 @@  class dirstate(object):
 
     def invalidate(self):
         for a in ("_map", "_copymap", "_filefoldmap", "_dirfoldmap", "_branch",
-                  "_pl", "_dirs", "_ignore", "_nonnormalset"):
+                  "_pl", "_dirs", "_ignore", "_nonnormalset",
+                  "_otherparentset"):
             if a in self.__dict__:
                 delattr(self, a)
         self._lastnormaltime = 0
@@ -486,6 +506,8 @@  class dirstate(object):
         self._map[f] = dirstatetuple(state, mode, size, mtime)
         if state != 'n' or mtime == -1:
             self._nonnormalset.add(f)
+        if size == -2:
+            self._otherparentset.add(f)
 
     def normal(self, f):
         '''Mark a file normal and clean.'''
@@ -560,6 +582,7 @@  class dirstate(object):
                 size = -1
             elif entry[0] == 'n' and entry[2] == -2: # other parent
                 size = -2
+                self._otherparentset.add(f)
         self._map[f] = dirstatetuple('r', 0, size, 0)
         self._nonnormalset.add(f)
         if size == 0 and f in self._copymap:
@@ -758,7 +781,7 @@  class dirstate(object):
                     break
 
         st.write(parsers.pack_dirstate(self._map, self._copymap, self._pl, now))
-        self._nonnormalset = nonnormalentries(self._map)
+        self._nonnormalset, self._otherparentset = nonnormalentries(self._map)
         st.close()
         self._lastnormaltime = 0
         self._dirty = self._dirtypl = False
diff --git a/mercurial/parsers.c b/mercurial/parsers.c
--- a/mercurial/parsers.c
+++ b/mercurial/parsers.c
@@ -564,7 +564,8 @@  quit:
 */
 static PyObject *nonnormalentries(PyObject *self, PyObject *args)
 {
-	PyObject *dmap, *nonnset = NULL, *fname, *v;
+	PyObject *dmap, *fname, *v;
+	PyObject *nonnset = NULL, *otherpset = NULL, *result = NULL;
 	Py_ssize_t pos;
 
 	if (!PyArg_ParseTuple(args, "O!:nonnormalentries",
@@ -575,6 +576,10 @@  static PyObject *nonnormalentries(PyObje
 	if (nonnset == NULL)
 		goto bail;
 
+	otherpset = PySet_New(NULL);
+	if (otherpset == NULL)
+		goto bail;
+
 	pos = 0;
 	while (PyDict_Next(dmap, &pos, &fname, &v)) {
 		dirstateTupleObject *t;
@@ -585,15 +590,26 @@  static PyObject *nonnormalentries(PyObje
 		}
 		t = (dirstateTupleObject *)v;
 
+		if (t->state == 'n' && t->size == -2) {
+			if (PySet_Add(otherpset, fname) == -1) {
+				goto bail;
+			}
+		}
+
 		if (t->state == 'n' && t->mtime != -1)
 			continue;
 		if (PySet_Add(nonnset, fname) == -1)
 			goto bail;
 	}
 
-	return nonnset;
+	result = Py_BuildValue("(OO)", nonnset, otherpset);
+	if (result == NULL)
+		goto bail;
+	return result;
 bail:
 	Py_XDECREF(nonnset);
+	Py_XDECREF(otherpset);
+	Py_XDECREF(result);
 	return NULL;
 }