builder: mozilla-central_win7-ix_test-chromez-pgo slave: t-w732-ix-176 starttime: 1447264676.86 results: exception (4) buildid: 20151111043010 builduid: ff41f637a18a42f2940cbbdf80245a27 revision: 84a7cf29f4f14c9b359db2f7f19c0abd6a8e178e ========= Started set props: master (results: 0, elapsed: 0 secs) (at 2015-11-11 09:57:56.856239) ========= master: http://buildbot-master111.bb.releng.scl3.mozilla.com:8201/ ========= Finished set props: master (results: 0, elapsed: 0 secs) (at 2015-11-11 09:57:56.856668) ========= ========= Started set props: basedir (results: 0, elapsed: 0 secs) (at 2015-11-11 09:57:56.856985) ========= 'cd' in dir C:\\slave\test-pgo\. (timeout 1200 secs) watching logfiles {} argv: ['cd'] environment: ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld\AppData\Roaming COMMONPROGRAMFILES=C:\Program Files\Common Files COMPUTERNAME=T-W732-IX-176 COMSPEC=C:\windows\system32\cmd.exe CYGWINBASE=C:\cygwin DCLOCATION=SCL3 DNSSUFFIX=wintest.releng.scl3.mozilla.com FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\cltbld KTS_HOME=C:\Program Files\KTS KTS_VERSION=1.19c LOCALAPPDATA=C:\Users\cltbld\AppData\Local LOGONSERVER=\\T-W732-IX-176 MONDIR=C:\Monitor_config\ MOZBUILDDIR=C:\mozilla-build\ MOZ_CRASHREPORTER_NO_REPORT=1 MOZ_NO_REMOTE=1 NO_EM_RESTART=1 NUMBER_OF_PROCESSORS=8 OS=Windows_NT OURDRIVE=C: PATH=C:\Python24;C:\Python24\Scripts;C:\cygwin\bin;C:\WINDOWS\System32;C:\program files\gnuwin32\bin;C:\WINDOWS;C:\mozilla-build\msys\bin;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\info-zip;C:\mozilla-build\wget;C:\Program Files (x86)\GnuWin32\bin; PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_IDENTIFIER=x86 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files PROMPT=$P$G PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=C:\slave\test-pgo RUNLOGFILE=C:\slave\\runslave.log SLAVEDIR=C:\slave\ SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:\Users\cltbld\AppData\Local\Temp TEST1=testie TMP=C:\Users\cltbld\AppData\Local\Temp USERDOMAIN=T-W732-IX-176 USERNAME=cltbld USERPROFILE=C:\Users\cltbld WINDIR=C:\windows WINDOWS_TRACING_FLAGS=3 WINDOWS_TRACING_LOGFILE=C:\BVTBin\Tests\installpackage\csilogfile.log XPCOM_DEBUG_BREAK=warn using PTY: False C:\slave\test-pgo program finished with exit code 0 elapsedTime=0.575000 basedir: 'C:\\slave\\test-pgo' ========= master_lag: 0.08 ========= ========= Finished set props: basedir (results: 0, elapsed: 0 secs) (at 2015-11-11 09:57:57.513946) ========= ========= Started downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2015-11-11 09:57:57.514296) ========= ========= Finished downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2015-11-11 09:57:57.661234) ========= ========= Started 'rm -rf ...' (results: 0, elapsed: 0 secs) (at 2015-11-11 09:57:57.661603) ========= 'rm' '-rf' 'properties' in dir C:\\slave\test-pgo\. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-rf', 'properties'] environment: ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld\AppData\Roaming COMMONPROGRAMFILES=C:\Program Files\Common Files COMPUTERNAME=T-W732-IX-176 COMSPEC=C:\windows\system32\cmd.exe CYGWINBASE=C:\cygwin DCLOCATION=SCL3 DNSSUFFIX=wintest.releng.scl3.mozilla.com FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\cltbld KTS_HOME=C:\Program Files\KTS KTS_VERSION=1.19c LOCALAPPDATA=C:\Users\cltbld\AppData\Local LOGONSERVER=\\T-W732-IX-176 MONDIR=C:\Monitor_config\ MOZBUILDDIR=C:\mozilla-build\ MOZ_CRASHREPORTER_NO_REPORT=1 MOZ_NO_REMOTE=1 NO_EM_RESTART=1 NUMBER_OF_PROCESSORS=8 OS=Windows_NT OURDRIVE=C: PATH=C:\Python24;C:\Python24\Scripts;C:\cygwin\bin;C:\WINDOWS\System32;C:\program files\gnuwin32\bin;C:\WINDOWS;C:\mozilla-build\msys\bin;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\info-zip;C:\mozilla-build\wget;C:\Program Files (x86)\GnuWin32\bin; PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_IDENTIFIER=x86 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files PROMPT=$P$G PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=C:\slave\test-pgo RUNLOGFILE=C:\slave\\runslave.log SLAVEDIR=C:\slave\ SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:\Users\cltbld\AppData\Local\Temp TEST1=testie TMP=C:\Users\cltbld\AppData\Local\Temp USERDOMAIN=T-W732-IX-176 USERNAME=cltbld USERPROFILE=C:\Users\cltbld WINDIR=C:\windows WINDOWS_TRACING_FLAGS=3 WINDOWS_TRACING_LOGFILE=C:\BVTBin\Tests\installpackage\csilogfile.log XPCOM_DEBUG_BREAK=warn using PTY: False program finished with exit code 0 elapsedTime=0.501000 ========= master_lag: 0.02 ========= ========= Finished 'rm -rf ...' (results: 0, elapsed: 0 secs) (at 2015-11-11 09:57:58.179208) ========= ========= Started set props: script_repo_url (results: 0, elapsed: 0 secs) (at 2015-11-11 09:57:58.179527) ========= script_repo_url: https://hg.mozilla.org/build/mozharness ========= Finished set props: script_repo_url (results: 0, elapsed: 0 secs) (at 2015-11-11 09:57:58.179881) ========= ========= Started 'bash -c ...' (results: 0, elapsed: 0 secs) (at 2015-11-11 09:57:58.180149) ========= 'bash' '-c' 'wget -Oarchiver_client.py --no-check-certificate --tries=10 --waitretry=3 https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py' in dir C:\\slave\test-pgo\. (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'wget -Oarchiver_client.py --no-check-certificate --tries=10 --waitretry=3 https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py'] environment: ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld\AppData\Roaming COMMONPROGRAMFILES=C:\Program Files\Common Files COMPUTERNAME=T-W732-IX-176 COMSPEC=C:\windows\system32\cmd.exe CYGWINBASE=C:\cygwin DCLOCATION=SCL3 DNSSUFFIX=wintest.releng.scl3.mozilla.com FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\cltbld KTS_HOME=C:\Program Files\KTS KTS_VERSION=1.19c LOCALAPPDATA=C:\Users\cltbld\AppData\Local LOGONSERVER=\\T-W732-IX-176 MONDIR=C:\Monitor_config\ MOZBUILDDIR=C:\mozilla-build\ MOZ_CRASHREPORTER_NO_REPORT=1 MOZ_NO_REMOTE=1 NO_EM_RESTART=1 NUMBER_OF_PROCESSORS=8 OS=Windows_NT OURDRIVE=C: PATH=C:\Python24;C:\Python24\Scripts;C:\cygwin\bin;C:\WINDOWS\System32;C:\program files\gnuwin32\bin;C:\WINDOWS;C:\mozilla-build\msys\bin;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\info-zip;C:\mozilla-build\wget;C:\Program Files (x86)\GnuWin32\bin; PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_IDENTIFIER=x86 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files PROMPT=$P$G PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=C:\slave\test-pgo RUNLOGFILE=C:\slave\\runslave.log SLAVEDIR=C:\slave\ SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:\Users\cltbld\AppData\Local\Temp TEST1=testie TMP=C:\Users\cltbld\AppData\Local\Temp USERDOMAIN=T-W732-IX-176 USERNAME=cltbld USERPROFILE=C:\Users\cltbld WINDIR=C:\windows WINDOWS_TRACING_FLAGS=3 WINDOWS_TRACING_LOGFILE=C:\BVTBin\Tests\installpackage\csilogfile.log XPCOM_DEBUG_BREAK=warn using PTY: False --09:57:58-- https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py => `archiver_client.py' Resolving hg.mozilla.org... 63.245.215.102, 63.245.215.25 Connecting to hg.mozilla.org|63.245.215.102|:443... connected. HTTP request sent, awaiting response... 200 Script output follows Length: 12,141 (12K) [text/x-python] 0K .......... . 100% 1.96 GB/s 09:57:58 (1.96 GB/s) - `archiver_client.py' saved [12141/12141] program finished with exit code 0 elapsedTime=0.663000 ========= master_lag: 0.02 ========= ========= Finished 'bash -c ...' (results: 0, elapsed: 0 secs) (at 2015-11-11 09:57:58.859586) ========= ========= Started 'rm -rf ...' (results: 0, elapsed: 11 secs) (at 2015-11-11 09:57:58.859895) ========= 'rm' '-rf' 'scripts' in dir C:\\slave\test-pgo\. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-rf', 'scripts'] environment: ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld\AppData\Roaming COMMONPROGRAMFILES=C:\Program Files\Common Files COMPUTERNAME=T-W732-IX-176 COMSPEC=C:\windows\system32\cmd.exe CYGWINBASE=C:\cygwin DCLOCATION=SCL3 DNSSUFFIX=wintest.releng.scl3.mozilla.com FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\cltbld KTS_HOME=C:\Program Files\KTS KTS_VERSION=1.19c LOCALAPPDATA=C:\Users\cltbld\AppData\Local LOGONSERVER=\\T-W732-IX-176 MONDIR=C:\Monitor_config\ MOZBUILDDIR=C:\mozilla-build\ MOZ_CRASHREPORTER_NO_REPORT=1 MOZ_NO_REMOTE=1 NO_EM_RESTART=1 NUMBER_OF_PROCESSORS=8 OS=Windows_NT OURDRIVE=C: PATH=C:\Python24;C:\Python24\Scripts;C:\cygwin\bin;C:\WINDOWS\System32;C:\program files\gnuwin32\bin;C:\WINDOWS;C:\mozilla-build\msys\bin;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\info-zip;C:\mozilla-build\wget;C:\Program Files (x86)\GnuWin32\bin; PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_IDENTIFIER=x86 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files PROMPT=$P$G PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=C:\slave\test-pgo RUNLOGFILE=C:\slave\\runslave.log SLAVEDIR=C:\slave\ SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:\Users\cltbld\AppData\Local\Temp TEST1=testie TMP=C:\Users\cltbld\AppData\Local\Temp USERDOMAIN=T-W732-IX-176 USERNAME=cltbld USERPROFILE=C:\Users\cltbld WINDIR=C:\windows WINDOWS_TRACING_FLAGS=3 WINDOWS_TRACING_LOGFILE=C:\BVTBin\Tests\installpackage\csilogfile.log XPCOM_DEBUG_BREAK=warn using PTY: False program finished with exit code 0 elapsedTime=11.201000 ========= master_lag: 0.02 ========= ========= Finished 'rm -rf ...' (results: 0, elapsed: 11 secs) (at 2015-11-11 09:58:10.078362) ========= ========= Started 'bash -c ...' (results: 0, elapsed: 1 secs) (at 2015-11-11 09:58:10.078726) ========= 'bash' '-c' u'python archiver_client.py mozharness --repo mozilla-central --rev 84a7cf29f4f14c9b359db2f7f19c0abd6a8e178e --destination scripts --debug' in dir C:\\slave\test-pgo\. (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', u'python archiver_client.py mozharness --repo mozilla-central --rev 84a7cf29f4f14c9b359db2f7f19c0abd6a8e178e --destination scripts --debug'] environment: ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld\AppData\Roaming COMMONPROGRAMFILES=C:\Program Files\Common Files COMPUTERNAME=T-W732-IX-176 COMSPEC=C:\windows\system32\cmd.exe CYGWINBASE=C:\cygwin DCLOCATION=SCL3 DNSSUFFIX=wintest.releng.scl3.mozilla.com FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\cltbld KTS_HOME=C:\Program Files\KTS KTS_VERSION=1.19c LOCALAPPDATA=C:\Users\cltbld\AppData\Local LOGONSERVER=\\T-W732-IX-176 MONDIR=C:\Monitor_config\ MOZBUILDDIR=C:\mozilla-build\ MOZ_CRASHREPORTER_NO_REPORT=1 MOZ_NO_REMOTE=1 NO_EM_RESTART=1 NUMBER_OF_PROCESSORS=8 OS=Windows_NT OURDRIVE=C: PATH=C:\Python24;C:\Python24\Scripts;C:\cygwin\bin;C:\WINDOWS\System32;C:\program files\gnuwin32\bin;C:\WINDOWS;C:\mozilla-build\msys\bin;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\info-zip;C:\mozilla-build\wget;C:\Program Files (x86)\GnuWin32\bin; PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_IDENTIFIER=x86 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files PROMPT=$P$G PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=C:\slave\test-pgo RUNLOGFILE=C:\slave\\runslave.log SLAVEDIR=C:\slave\ SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:\Users\cltbld\AppData\Local\Temp TEST1=testie TMP=C:\Users\cltbld\AppData\Local\Temp USERDOMAIN=T-W732-IX-176 USERNAME=cltbld USERPROFILE=C:\Users\cltbld WINDIR=C:\windows WINDOWS_TRACING_FLAGS=3 WINDOWS_TRACING_LOGFILE=C:\BVTBin\Tests\installpackage\csilogfile.log XPCOM_DEBUG_BREAK=warn using PTY: False 2015-11-11 09:58:10,658 truncating revision to first 12 chars 2015-11-11 09:58:10,658 Setting DEBUG logging. 2015-11-11 09:58:10,658 attempt 1/10 2015-11-11 09:58:10,658 Getting archive location from https://api.pub.build.mozilla.org/archiver/hgmo/mozilla-central/84a7cf29f4f1?&preferred_region=us-west-2&suffix=tar.gz&subdir=testing/mozharness 2015-11-11 09:58:11,098 unpacking tar archive at: mozilla-central-84a7cf29f4f1/testing/mozharness/ program finished with exit code 0 elapsedTime=1.380000 ========= master_lag: 0.04 ========= ========= Finished 'bash -c ...' (results: 0, elapsed: 1 secs) (at 2015-11-11 09:58:11.496314) ========= ========= Started downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2015-11-11 09:58:11.496649) ========= ========= Finished downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2015-11-11 09:58:11.511459) ========= ========= Started tinderboxprint_script_revlink (results: 0, elapsed: 0 secs) (at 2015-11-11 09:58:11.511827) ========= TinderboxPrint: script_revlink: https://hg.mozilla.org/build/mozharness/rev/production ========= Finished tinderboxprint_script_revlink (results: 0, elapsed: 0 secs) (at 2015-11-11 09:58:11.512295) ========= ========= Started 'c:/mozilla-build/python27/python -u ...' exception (results: 4, elapsed: 34 mins, 17 secs) (at 2015-11-11 09:58:11.512630) ========= 'c:/mozilla-build/python27/python' '-u' 'scripts/scripts/talos_script.py' '--suite' 'chromez' '--add-option' '--webServer,localhost' '--branch-name' 'Firefox' '--system-bits' '32' '--cfg' 'talos/windows_config.py' '--download-symbols' 'ondemand' '--use-talos-json' '--blob-upload-branch' 'Firefox' in dir C:\\slave\test-pgo\. (timeout 3600 secs) (maxTime 7200 secs) watching logfiles {} argv: ['c:/mozilla-build/python27/python', '-u', 'scripts/scripts/talos_script.py', '--suite', 'chromez', '--add-option', '--webServer,localhost', '--branch-name', 'Firefox', '--system-bits', '32', '--cfg', 'talos/windows_config.py', '--download-symbols', 'ondemand', '--use-talos-json', '--blob-upload-branch', 'Firefox'] environment: ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld\AppData\Roaming COMMONPROGRAMFILES=C:\Program Files\Common Files COMPUTERNAME=T-W732-IX-176 COMSPEC=C:\windows\system32\cmd.exe CYGWINBASE=C:\cygwin DCLOCATION=SCL3 DNSSUFFIX=wintest.releng.scl3.mozilla.com FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\cltbld KTS_HOME=C:\Program Files\KTS KTS_VERSION=1.19c LOCALAPPDATA=C:\Users\cltbld\AppData\Local LOGONSERVER=\\T-W732-IX-176 MONDIR=C:\Monitor_config\ MOZBUILDDIR=C:\mozilla-build\ MOZ_CRASHREPORTER_NO_REPORT=1 MOZ_NO_REMOTE=1 NO_EM_RESTART=1 NUMBER_OF_PROCESSORS=8 OS=Windows_NT OURDRIVE=C: PATH=C:\Python24;C:\Python24\Scripts;C:\cygwin\bin;C:\WINDOWS\System32;C:\program files\gnuwin32\bin;C:\WINDOWS;C:\mozilla-build\msys\bin;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\info-zip;C:\mozilla-build\wget;C:\Program Files (x86)\GnuWin32\bin; PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_IDENTIFIER=x86 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files PROMPT=$P$G PROPERTIES_FILE=C:\slave\test-pgo/buildprops.json PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=C:\slave\test-pgo RUNLOGFILE=C:\slave\\runslave.log SLAVEDIR=C:\slave\ SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:\Users\cltbld\AppData\Local\Temp TEST1=testie TMP=C:\Users\cltbld\AppData\Local\Temp USERDOMAIN=T-W732-IX-176 USERNAME=cltbld USERPROFILE=C:\Users\cltbld WINDIR=C:\windows WINDOWS_TRACING_FLAGS=3 WINDOWS_TRACING_LOGFILE=C:\BVTBin\Tests\installpackage\csilogfile.log XPCOM_DEBUG_BREAK=warn using PTY: False 09:58:12 INFO - MultiFileLogger online at 20151111 09:58:12 in C:\slave\test-pgo 09:58:12 INFO - Run as scripts/scripts/talos_script.py --suite chromez --add-option --webServer,localhost --branch-name Firefox --system-bits 32 --cfg talos/windows_config.py --download-symbols ondemand --use-talos-json --blob-upload-branch Firefox 09:58:12 INFO - Dumping config to C:\slave\test-pgo\logs\localconfig.json. 09:58:12 INFO - {'append_to_log': False, 09:58:12 INFO - 'base_work_dir': 'C:\\slave\\test-pgo', 09:58:12 INFO - 'blob_upload_branch': 'Firefox', 09:58:12 INFO - 'blob_uploader_auth_file': 'C:\\slave\\test-pgo\\oauth.txt', 09:58:12 INFO - 'branch': 'Firefox', 09:58:12 INFO - 'buildbot_json_path': 'buildprops.json', 09:58:12 INFO - 'config_files': ('talos/windows_config.py',), 09:58:12 INFO - 'default_actions': ('clobber', 09:58:12 INFO - 'read-buildbot-config', 09:58:12 INFO - 'download-and-extract', 09:58:12 INFO - 'populate-webroot', 09:58:12 INFO - 'create-virtualenv', 09:58:12 INFO - 'install', 09:58:12 INFO - 'run-tests'), 09:58:12 INFO - 'default_blob_upload_servers': ('https://blobupload.elasticbeanstalk.com',), 09:58:12 INFO - 'download_minidump_stackwalk': True, 09:58:12 INFO - 'download_symbols': 'ondemand', 09:58:12 INFO - 'exes': {'easy_install': ('C:\\slave\\test-pgo\\build/venv/scripts/python', 09:58:12 INFO - 'C:\\slave\\test-pgo\\build/venv/scripts/easy_install-2.7-script.py'), 09:58:12 INFO - 'hg': 'c:/mozilla-build/hg/hg', 09:58:12 INFO - 'mozinstall': ('C:\\slave\\test-pgo\\build/venv/scripts/python', 09:58:12 INFO - 'C:\\slave\\test-pgo\\build/venv/scripts/mozinstall-script.py'), 09:58:12 INFO - 'python': 'c:/mozilla-build/python27/python.exe', 09:58:12 INFO - 'tooltool.py': ('c:/mozilla-build/python27/python.exe', 09:58:12 INFO - 'C:/mozilla-build/tooltool.py'), 09:58:12 INFO - 'virtualenv': ('c:/mozilla-build/python27/python.exe', 09:58:12 INFO - 'c:/mozilla-build/buildbotve/virtualenv.py')}, 09:58:12 INFO - 'find_links': ('http://pypi.pvt.build.mozilla.org/pub', 09:58:12 INFO - 'http://pypi.pub.build.mozilla.org/pub'), 09:58:12 INFO - 'installer_path': 'installer.exe', 09:58:12 INFO - 'log_level': 'info', 09:58:12 INFO - 'log_name': 'talos', 09:58:12 INFO - 'log_to_console': True, 09:58:12 INFO - 'metro_harness_path_frmt': '%(metro_base_path)s/metro/metrotestharness.exe', 09:58:12 INFO - 'minidump_stackwalk_path': 'win32-minidump_stackwalk.exe', 09:58:12 INFO - 'minidump_tooltool_manifest_path': 'config/tooltool-manifests/win32/releng.manifest', 09:58:12 INFO - 'opt_config_files': (), 09:58:12 INFO - 'pip_index': False, 09:58:12 INFO - 'sps_profile': False, 09:58:12 INFO - 'sps_profile_interval': 0, 09:58:12 INFO - 'suite': 'chromez', 09:58:12 INFO - 'system_bits': '32', 09:58:12 INFO - 'talos_extra_options': ('--webServer', 'localhost'), 09:58:12 INFO - 'title': 'T-W732-IX-176', 09:58:12 INFO - 'use_talos_json': True, 09:58:12 INFO - 'virtualenv_modules': ('pywin32', 'talos', 'mozinstall'), 09:58:12 INFO - 'virtualenv_path': 'C:\\slave\\test-pgo\\build/venv', 09:58:12 INFO - 'virtualenv_python_dll': 'c:/mozilla-build/python27/python27.dll', 09:58:12 INFO - 'volatile_config': {'actions': None, 'add_actions': None, 'no_actions': None}, 09:58:12 INFO - 'work_dir': 'build'} 09:58:12 INFO - ##### 09:58:12 INFO - ##### Running clobber step. 09:58:12 INFO - ##### 09:58:12 INFO - Running pre-action listener: _resource_record_pre_action 09:58:12 INFO - Running main action method: clobber 09:58:12 INFO - rmtree: C:\slave\test-pgo\build 09:58:12 INFO - Using _rmtree_windows ... 09:58:12 INFO - retry: Calling _rmtree_windows with args: ('C:\\slave\\test-pgo\\build',), kwargs: {}, attempt #1 10:01:15 INFO - Running post-action listener: _resource_record_post_action 10:01:15 INFO - ##### 10:01:15 INFO - ##### Running read-buildbot-config step. 10:01:15 INFO - ##### 10:01:15 INFO - Running pre-action listener: _resource_record_pre_action 10:01:15 INFO - Running main action method: read_buildbot_config 10:01:15 INFO - Using buildbot properties: 10:01:15 INFO - { 10:01:15 INFO - "properties": { 10:01:15 INFO - "buildnumber": 18, 10:01:15 INFO - "product": "firefox", 10:01:15 INFO - "script_repo_revision": "production", 10:01:15 INFO - "builddir": "mozilla-central_win7-ix_test-chromez", 10:01:15 INFO - "repository": "", 10:01:15 INFO - "buildername": "Windows 7 32-bit mozilla-central pgo talos chromez", 10:01:15 INFO - "buildid": "20151111043010", 10:01:15 INFO - "slavename": "t-w732-ix-176", 10:01:15 INFO - "pgo_build": "True", 10:01:15 INFO - "basedir": "C:\\slave\\test-pgo", 10:01:15 INFO - "project": "", 10:01:15 INFO - "platform": "win7-ix", 10:01:15 INFO - "master": "http://buildbot-master111.bb.releng.scl3.mozilla.com:8201/", 10:01:15 INFO - "slavebuilddir": "test", 10:01:15 INFO - "scheduler": "tests-mozilla-central-win32-pgo-talos", 10:01:15 INFO - "branch": "mozilla-central", 10:01:15 INFO - "repo_path": "mozilla-central", 10:01:15 INFO - "stage_platform": "win32-pgo", 10:01:15 INFO - "builduid": "ff41f637a18a42f2940cbbdf80245a27", 10:01:15 INFO - "revision": "84a7cf29f4f14c9b359db2f7f19c0abd6a8e178e" 10:01:15 INFO - }, 10:01:15 INFO - "sourcestamp": { 10:01:15 INFO - "repository": "", 10:01:15 INFO - "hasPatch": false, 10:01:15 INFO - "project": "", 10:01:15 INFO - "branch": "mozilla-central-win32-pgo-talos", 10:01:15 INFO - "changes": [ 10:01:15 INFO - { 10:01:15 INFO - "category": null, 10:01:15 INFO - "files": [ 10:01:15 INFO - { 10:01:15 INFO - "url": null, 10:01:15 INFO - "name": "https://queue.taskcluster.net/v1/task/okY6L-EUTauJ6xYY9PCTpw/artifacts/public/build/firefox-45.0a1.en-US.win32.zip" 10:01:15 INFO - } 10:01:15 INFO - ], 10:01:15 INFO - "repository": "", 10:01:15 INFO - "rev": "84a7cf29f4f14c9b359db2f7f19c0abd6a8e178e", 10:01:15 INFO - "who": "sendchange", 10:01:15 INFO - "when": 1447264413, 10:01:15 INFO - "number": 6649488, 10:01:15 INFO - "comments": "", 10:01:15 INFO - "project": "", 10:01:15 INFO - "at": "Wed 11 Nov 2015 09:53:33", 10:01:15 INFO - "branch": "mozilla-central-win32-pgo-talos", 10:01:15 INFO - "revlink": "", 10:01:15 INFO - "properties": [ 10:01:15 INFO - [ 10:01:15 INFO - "buildid", 10:01:15 INFO - "20151111043010", 10:01:15 INFO - "Change" 10:01:15 INFO - ], 10:01:15 INFO - [ 10:01:15 INFO - "builduid", 10:01:15 INFO - "ff41f637a18a42f2940cbbdf80245a27", 10:01:15 INFO - "Change" 10:01:15 INFO - ], 10:01:15 INFO - [ 10:01:15 INFO - "pgo_build", 10:01:15 INFO - "True", 10:01:15 INFO - "Change" 10:01:15 INFO - ] 10:01:15 INFO - ], 10:01:15 INFO - "revision": "84a7cf29f4f14c9b359db2f7f19c0abd6a8e178e" 10:01:15 INFO - } 10:01:15 INFO - ], 10:01:15 INFO - "revision": "84a7cf29f4f14c9b359db2f7f19c0abd6a8e178e" 10:01:15 INFO - } 10:01:15 INFO - } 10:01:15 INFO - Found installer url https://queue.taskcluster.net/v1/task/okY6L-EUTauJ6xYY9PCTpw/artifacts/public/build/firefox-45.0a1.en-US.win32.zip. 10:01:15 INFO - Running post-action listener: _resource_record_post_action 10:01:15 INFO - ##### 10:01:15 INFO - ##### Running download-and-extract step. 10:01:15 INFO - ##### 10:01:15 INFO - Running pre-action listener: _resource_record_pre_action 10:01:15 INFO - Running main action method: download_and_extract 10:01:15 INFO - mkdir: C:\slave\test-pgo\build\tests 10:01:15 INFO - proxxy config: {'regions': ['.use1.', '.usw2.', '.scl3'], 'instances': ['proxxy1.srv.releng.use1.mozilla.com', 'proxxy1.srv.releng.usw2.mozilla.com', 'proxxy1.srv.releng.scl3.mozilla.com'], 'urls': [('http://ftp.mozilla.org', 'ftp.mozilla.org'), ('https://ftp.mozilla.org', 'ftp.mozilla.org'), ('https://ftp-ssl.mozilla.org', 'ftp.mozilla.org'), ('http://pvtbuilds.pvt.build.mozilla.org', 'pvtbuilds.mozilla.org'), ('http://pypi.pvt.build.mozilla.org', 'pypi.pvt.build.mozilla.org'), ('http://pypi.pub.build.mozilla.org', 'pypi.pub.build.mozilla.org'), ('https://queue.taskcluster.net', 'queue.taskcluster.net')]} 10:01:15 INFO - https://queue.taskcluster.net/v1/task/okY6L-EUTauJ6xYY9PCTpw/artifacts/public/build/test_packages.json matches https://queue.taskcluster.net 10:01:15 INFO - trying https://queue.taskcluster.net/v1/task/okY6L-EUTauJ6xYY9PCTpw/artifacts/public/build/test_packages.json 10:01:15 INFO - Downloading https://queue.taskcluster.net/v1/task/okY6L-EUTauJ6xYY9PCTpw/artifacts/public/build/test_packages.json to C:\slave\test-pgo\build\test_packages.json 10:01:15 INFO - retry: Calling _download_file with args: (), kwargs: {'url': 'https://queue.taskcluster.net/v1/task/okY6L-EUTauJ6xYY9PCTpw/artifacts/public/build/test_packages.json', 'file_name': 'C:\\slave\\test-pgo\\build\\test_packages.json'}, attempt #1 10:01:16 INFO - Downloaded 1221 bytes. 10:01:16 INFO - Reading from file C:\slave\test-pgo\build\test_packages.json 10:01:16 INFO - Using the following test package requirements: 10:01:16 INFO - {u'common': [u'firefox-45.0a1.en-US.win32.common.tests.zip'], 10:01:16 INFO - u'cppunittest': [u'firefox-45.0a1.en-US.win32.common.tests.zip', 10:01:16 INFO - u'firefox-45.0a1.en-US.win32.cppunittest.tests.zip'], 10:01:16 INFO - u'jittest': [u'firefox-45.0a1.en-US.win32.common.tests.zip', 10:01:16 INFO - u'jsshell-win32.zip'], 10:01:16 INFO - u'mochitest': [u'firefox-45.0a1.en-US.win32.common.tests.zip', 10:01:16 INFO - u'firefox-45.0a1.en-US.win32.mochitest.tests.zip'], 10:01:16 INFO - u'mozbase': [u'firefox-45.0a1.en-US.win32.common.tests.zip'], 10:01:16 INFO - u'reftest': [u'firefox-45.0a1.en-US.win32.common.tests.zip', 10:01:16 INFO - u'firefox-45.0a1.en-US.win32.reftest.tests.zip'], 10:01:16 INFO - u'talos': [u'firefox-45.0a1.en-US.win32.common.tests.zip', 10:01:16 INFO - u'firefox-45.0a1.en-US.win32.talos.tests.zip'], 10:01:16 INFO - u'web-platform': [u'firefox-45.0a1.en-US.win32.common.tests.zip', 10:01:16 INFO - u'firefox-45.0a1.en-US.win32.web-platform.tests.zip'], 10:01:16 INFO - u'webapprt': [u'firefox-45.0a1.en-US.win32.common.tests.zip'], 10:01:16 INFO - u'xpcshell': [u'firefox-45.0a1.en-US.win32.common.tests.zip', 10:01:16 INFO - u'firefox-45.0a1.en-US.win32.xpcshell.tests.zip']} 10:01:16 INFO - Downloading packages: [u'firefox-45.0a1.en-US.win32.common.tests.zip'] for test suite category: common 10:01:16 INFO - proxxy config: {'regions': ['.use1.', '.usw2.', '.scl3'], 'instances': ['proxxy1.srv.releng.use1.mozilla.com', 'proxxy1.srv.releng.usw2.mozilla.com', 'proxxy1.srv.releng.scl3.mozilla.com'], 'urls': [('http://ftp.mozilla.org', 'ftp.mozilla.org'), ('https://ftp.mozilla.org', 'ftp.mozilla.org'), ('https://ftp-ssl.mozilla.org', 'ftp.mozilla.org'), ('http://pvtbuilds.pvt.build.mozilla.org', 'pvtbuilds.mozilla.org'), ('http://pypi.pvt.build.mozilla.org', 'pypi.pvt.build.mozilla.org'), ('http://pypi.pub.build.mozilla.org', 'pypi.pub.build.mozilla.org'), ('https://queue.taskcluster.net', 'queue.taskcluster.net')]} 10:01:16 INFO - https://queue.taskcluster.net/v1/task/okY6L-EUTauJ6xYY9PCTpw/artifacts/public/build/firefox-45.0a1.en-US.win32.common.tests.zip matches https://queue.taskcluster.net 10:01:16 INFO - trying https://queue.taskcluster.net/v1/task/okY6L-EUTauJ6xYY9PCTpw/artifacts/public/build/firefox-45.0a1.en-US.win32.common.tests.zip 10:01:16 INFO - Downloading https://queue.taskcluster.net/v1/task/okY6L-EUTauJ6xYY9PCTpw/artifacts/public/build/firefox-45.0a1.en-US.win32.common.tests.zip to C:\slave\test-pgo\build\firefox-45.0a1.en-US.win32.common.tests.zip 10:01:16 INFO - retry: Calling _download_file with args: (), kwargs: {'url': u'https://queue.taskcluster.net/v1/task/okY6L-EUTauJ6xYY9PCTpw/artifacts/public/build/firefox-45.0a1.en-US.win32.common.tests.zip', 'file_name': u'C:\\slave\\test-pgo\\build\\firefox-45.0a1.en-US.win32.common.tests.zip'}, attempt #1 10:08:36 WARNING - Socket error when accessing https://queue.taskcluster.net/v1/task/okY6L-EUTauJ6xYY9PCTpw/artifacts/public/build/firefox-45.0a1.en-US.win32.common.tests.zip: The read operation timed out 10:08:36 INFO - retry: attempt #1 caught exception: The read operation timed out 10:08:36 INFO - retry: Failed, sleeping 30 seconds before retrying 10:09:06 INFO - retry: Calling _download_file with args: (), kwargs: {'url': u'https://queue.taskcluster.net/v1/task/okY6L-EUTauJ6xYY9PCTpw/artifacts/public/build/firefox-45.0a1.en-US.win32.common.tests.zip', 'file_name': u'C:\\slave\\test-pgo\\build\\firefox-45.0a1.en-US.win32.common.tests.zip'}, attempt #2 10:28:56 WARNING - Socket error when accessing https://queue.taskcluster.net/v1/task/okY6L-EUTauJ6xYY9PCTpw/artifacts/public/build/firefox-45.0a1.en-US.win32.common.tests.zip: The read operation timed out 10:28:56 INFO - retry: attempt #2 caught exception: The read operation timed out 10:28:56 INFO - retry: Failed, sleeping 60 seconds before retrying 10:29:56 INFO - retry: Calling _download_file with args: (), kwargs: {'url': u'https://queue.taskcluster.net/v1/task/okY6L-EUTauJ6xYY9PCTpw/artifacts/public/build/firefox-45.0a1.en-US.win32.common.tests.zip', 'file_name': u'C:\\slave\\test-pgo\\build\\firefox-45.0a1.en-US.win32.common.tests.zip'}, attempt #3 10:32:29 WARNING - Socket error when accessing https://queue.taskcluster.net/v1/task/okY6L-EUTauJ6xYY9PCTpw/artifacts/public/build/firefox-45.0a1.en-US.win32.common.tests.zip: The read operation timed out 10:32:29 INFO - retry: attempt #3 caught exception: The read operation timed out 10:32:29 INFO - Can't download from https://queue.taskcluster.net/v1/task/okY6L-EUTauJ6xYY9PCTpw/artifacts/public/build/firefox-45.0a1.en-US.win32.common.tests.zip to C:\slave\test-pgo\build\firefox-45.0a1.en-US.win32.common.tests.zip! 10:32:29 INFO - Caught exception: The read operation timed out 10:32:29 INFO - Caught exception: The read operation timed out 10:32:29 INFO - Caught exception: The read operation timed out 10:32:29 FATAL - Failed to download from all available URLs, aborting 10:32:29 FATAL - Running post_fatal callback... 10:32:29 FATAL - Exiting 3 10:32:29 INFO - Running post-action listener: _resource_record_post_action 10:32:29 INFO - Running post-action listener: set_extra_try_arguments 10:32:29 INFO - Running post-run listener: _resource_record_post_run 10:32:29 INFO - Running post-run listener: _upload_blobber_files 10:32:29 INFO - Blob upload gear active. 10:32:29 WARNING - Blob upload directory does not exist! program finished with exit code 3 elapsedTime=2057.718000 ========= master_lag: 0.14 ========= ========= Finished 'c:/mozilla-build/python27/python -u ...' exception (results: 4, elapsed: 34 mins, 17 secs) (at 2015-11-11 10:32:29.369675) ========= ========= Started no change (results: 0, elapsed: 0 secs) (at 2015-11-11 10:32:29.370362) ========= 'cmd' '/C' 'for' '%f' 'in' '(*)' 'do' '@type' '%f' in dir C:\\slave\test-pgo\properties (timeout 1200 secs) watching logfiles {} argv: ['cmd', '/C', 'for', '%f', 'in', '(*)', 'do', '@type', '%f'] environment: ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld\AppData\Roaming COMMONPROGRAMFILES=C:\Program Files\Common Files COMPUTERNAME=T-W732-IX-176 COMSPEC=C:\windows\system32\cmd.exe CYGWINBASE=C:\cygwin DCLOCATION=SCL3 DNSSUFFIX=wintest.releng.scl3.mozilla.com FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\cltbld KTS_HOME=C:\Program Files\KTS KTS_VERSION=1.19c LOCALAPPDATA=C:\Users\cltbld\AppData\Local LOGONSERVER=\\T-W732-IX-176 MONDIR=C:\Monitor_config\ MOZBUILDDIR=C:\mozilla-build\ MOZ_CRASHREPORTER_NO_REPORT=1 MOZ_NO_REMOTE=1 NO_EM_RESTART=1 NUMBER_OF_PROCESSORS=8 OS=Windows_NT OURDRIVE=C: PATH=C:\Python24;C:\Python24\Scripts;C:\cygwin\bin;C:\WINDOWS\System32;C:\program files\gnuwin32\bin;C:\WINDOWS;C:\mozilla-build\msys\bin;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\info-zip;C:\mozilla-build\wget;C:\Program Files (x86)\GnuWin32\bin; PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_IDENTIFIER=x86 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files PROMPT=$P$G PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=C:\slave\test-pgo\properties RUNLOGFILE=C:\slave\\runslave.log SLAVEDIR=C:\slave\ SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:\Users\cltbld\AppData\Local\Temp TEST1=testie TMP=C:\Users\cltbld\AppData\Local\Temp USERDOMAIN=T-W732-IX-176 USERNAME=cltbld USERPROFILE=C:\Users\cltbld WINDIR=C:\windows WINDOWS_TRACING_FLAGS=3 WINDOWS_TRACING_LOGFILE=C:\BVTBin\Tests\installpackage\csilogfile.log XPCOM_DEBUG_BREAK=warn using PTY: False program finished with exit code 0 elapsedTime=0.101000 ========= master_lag: 0.02 ========= ========= Finished no change (results: 0, elapsed: 0 secs) (at 2015-11-11 10:32:29.487314) ========= ========= Started 'rm -f ...' (results: 0, elapsed: 0 secs) (at 2015-11-11 10:32:29.487673) ========= 'rm' '-f' 'oauth.txt' in dir C:\\slave\test-pgo\. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-f', 'oauth.txt'] environment: ALLUSERSPROFILE=C:\ProgramData APPDATA=C:\Users\cltbld\AppData\Roaming COMMONPROGRAMFILES=C:\Program Files\Common Files COMPUTERNAME=T-W732-IX-176 COMSPEC=C:\windows\system32\cmd.exe CYGWINBASE=C:\cygwin DCLOCATION=SCL3 DNSSUFFIX=wintest.releng.scl3.mozilla.com FP_NO_HOST_CHECK=NO HOMEDRIVE=C: HOMEPATH=\Users\cltbld KTS_HOME=C:\Program Files\KTS KTS_VERSION=1.19c LOCALAPPDATA=C:\Users\cltbld\AppData\Local LOGONSERVER=\\T-W732-IX-176 MONDIR=C:\Monitor_config\ MOZBUILDDIR=C:\mozilla-build\ MOZ_CRASHREPORTER_NO_REPORT=1 MOZ_NO_REMOTE=1 NO_EM_RESTART=1 NUMBER_OF_PROCESSORS=8 OS=Windows_NT OURDRIVE=C: PATH=C:\Python24;C:\Python24\Scripts;C:\cygwin\bin;C:\WINDOWS\System32;C:\program files\gnuwin32\bin;C:\WINDOWS;C:\mozilla-build\msys\bin;C:\mozilla-build\python27;C:\mozilla-build\python27\Scripts;C:\mozilla-build\info-zip;C:\mozilla-build\wget;C:\Program Files (x86)\GnuWin32\bin; PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC PROCESSOR_ARCHITECTURE=x86 PROCESSOR_IDENTIFIER=x86 Family 6 Model 30 Stepping 5, GenuineIntel PROCESSOR_LEVEL=6 PROCESSOR_REVISION=1e05 PROGRAMDATA=C:\ProgramData PROGRAMFILES=C:\Program Files PROMPT=$P$G PSMODULEPATH=C:\windows\system32\WindowsPowerShell\v1.0\Modules\ PUBLIC=C:\Users\Public PWD=C:\slave\test-pgo RUNLOGFILE=C:\slave\\runslave.log SLAVEDIR=C:\slave\ SYSTEMDRIVE=C: SYSTEMROOT=C:\windows TEMP=C:\Users\cltbld\AppData\Local\Temp TEST1=testie TMP=C:\Users\cltbld\AppData\Local\Temp USERDOMAIN=T-W732-IX-176 USERNAME=cltbld USERPROFILE=C:\Users\cltbld WINDIR=C:\windows WINDOWS_TRACING_FLAGS=3 WINDOWS_TRACING_LOGFILE=C:\BVTBin\Tests\installpackage\csilogfile.log XPCOM_DEBUG_BREAK=warn using PTY: False program finished with exit code 0 elapsedTime=0.101000 ========= master_lag: 0.01 ========= ========= Finished 'rm -f ...' (results: 0, elapsed: 0 secs) (at 2015-11-11 10:32:29.602473) ========= ========= Started reboot slave lost (results: 0, elapsed: 0 secs) (at 2015-11-11 10:32:29.602776) ========= ========= Finished reboot slave lost (results: 0, elapsed: 0 secs) (at 2015-11-11 10:32:30.182884) ========= ========= Total master_lag: 0.34 =========