Patchwork [v2] convert: if getting a file from Perforce fails try to get it one more time

login
register
mail settings
Submitter Eugene Baranov
Date July 12, 2015, 1:44 p.m.
Message ID <f346506462419f4b1311.1436708650@waste.org>
Download mbox | patch
Permalink /patch/9952/
State Accepted
Headers show

Comments

Eugene Baranov - July 12, 2015, 1:44 p.m.
# HG changeset patch
# User Eugene Baranov <eug.baranov@gmail.com>
# Date 1436375500 -3600
#      Wed Jul 08 18:11:40 2015 +0100
# Node ID f346506462419f4b1311c9be731954e253815c70
# Parent  026105c442d7f20d239c1e5e5fac1cfa080323da
convert: if getting a file from Perforce fails try to get it one more time

When converting a particularly large Perforce changelist (especially with  some
big files), it is very likely to run into an intermittent network issue (e.g.
WSAECONNRESET or WSAETIMEDOUT) getting one of the files, which will result in
the entire changelist converting being aborted. Which can be quite unfortunate
since you might have waited hours getting all other files. To mitigate this
let's attempt to get the file one more time, escalating original exception
if that attempt fails.
Matt Mackall - July 13, 2015, 7:48 p.m.
On Sun, 2015-07-12 at 08:44 -0500, Eugene Baranov wrote:
> # HG changeset patch
> # User Eugene Baranov <eug.baranov@gmail.com>
> # Date 1436375500 -3600
> #      Wed Jul 08 18:11:40 2015 +0100
> # Node ID f346506462419f4b1311c9be731954e253815c70
> # Parent  026105c442d7f20d239c1e5e5fac1cfa080323da
> convert: if getting a file from Perforce fails try to get it one more time
> 
> When converting a particularly large Perforce changelist (especially with  some
> big files), it is very likely to run into an intermittent network issue (e.g.
> WSAECONNRESET or WSAETIMEDOUT) getting one of the files, which will result in
> the entire changelist converting being aborted. Which can be quite unfortunate
> since you might have waited hours getting all other files. To mitigate this
> let's attempt to get the file one more time, escalating original exception
> if that attempt fails.

Horrors. Queued for default, thanks.

Patch

diff --git a/hgext/convert/p4.py b/hgext/convert/p4.py
--- a/hgext/convert/p4.py
+++ b/hgext/convert/p4.py
@@ -196,38 +196,54 @@ 
     def getfile(self, name, rev):
         cmd = 'p4 -G print %s' \
             % util.shellquote("%s#%s" % (self.depotname[name], rev))
-        stdout = util.popen(cmd, mode='rb')
 
-        mode = None
-        contents = ""
-        keywords = None
+        lasterror = None
+        while True:
+            stdout = util.popen(cmd, mode='rb')
 
-        for d in loaditer(stdout):
-            code = d["code"]
-            data = d.get("data")
+            mode = None
+            contents = ""
+            keywords = None
 
-            if code == "error":
-                raise IOError(d["generic"], data)
+            for d in loaditer(stdout):
+                code = d["code"]
+                data = d.get("data")
 
-            elif code == "stat":
-                action = d.get("action")
-                if action in ["purge", "delete", "move/delete"]:
-                    return None, None
-                p4type = self.re_type.match(d["type"])
-                if p4type:
-                    mode = ""
-                    flags = (p4type.group(1) or "") + (p4type.group(3) or "")
-                    if "x" in flags:
-                        mode = "x"
-                    if p4type.group(2) == "symlink":
-                        mode = "l"
-                    if "ko" in flags:
-                        keywords = self.re_keywords_old
-                    elif "k" in flags:
-                        keywords = self.re_keywords
+                if code == "error":
+                    # if this is the first time error happened
+                    # re-attempt getting the file
+                    if not lasterror:
+                        lasterror = IOError(d["generic"], data)
+                        # this will exit inner-most for-loop
+                        break
+                    else:
+                        raise lasterror
 
-            elif code == "text" or code == "binary":
-                contents += data
+                elif code == "stat":
+                    action = d.get("action")
+                    if action in ["purge", "delete", "move/delete"]:
+                        return None, None
+                    p4type = self.re_type.match(d["type"])
+                    if p4type:
+                        mode = ""
+                        flags = ((p4type.group(1) or "")
+                               + (p4type.group(3) or ""))
+                        if "x" in flags:
+                            mode = "x"
+                        if p4type.group(2) == "symlink":
+                            mode = "l"
+                        if "ko" in flags:
+                            keywords = self.re_keywords_old
+                        elif "k" in flags:
+                            keywords = self.re_keywords
+
+                elif code == "text" or code == "binary":
+                    contents += data
+
+                lasterror = None
+
+            if not lasterror:
+                break
 
         if mode is None:
             return None, None