builder: release-birch-firefox_uptake_monitoring slave: bld-linux64-spot-306 starttime: 1517517854.08 results: failure (2) revision: f3b38b0e099bd2939f1a915ab2c655b8ec4b5954 ========= Started set props: master (results: 0, elapsed: 0 secs) (at 2018-02-01 12:44:14.077042) ========= master: http://buildbot-master73.bb.releng.usw2.mozilla.com:8001/ ========= Finished set props: master (results: 0, elapsed: 0 secs) (at 2018-02-01 12:44:14.077628) ========= ========= Started downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2018-02-01 12:44:14.077923) ========= ========= Finished downloading to buildprops.json (results: 0, elapsed: 0 secs) (at 2018-02-01 12:44:14.135177) ========= ========= Started 'bash -c ...' (results: 0, elapsed: 0 secs) (at 2018-02-01 12:44:14.135446) ========= bash -c 'wget -Oarchiver_client.py --no-check-certificate --tries=10 --waitretry=3 https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py' in dir /builds/slave/rel-bir-fx_uptk_mntr-000000000/. (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'wget -Oarchiver_client.py --no-check-certificate --tries=10 --waitretry=3 https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-306.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-bir-fx_uptk_mntr-000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False --2018-02-01 12:44:14-- https://hg.mozilla.org/build/tools/raw-file/default/buildfarm/utils/archiver_client.py Resolving hg.mozilla.org... 63.245.215.102, 63.245.215.25 Connecting to hg.mozilla.org|63.245.215.102|:443... connected. HTTP request sent, awaiting response... 200 Script output follows Length: 12179 (12K) [text/x-python] Saving to: ‘archiver_client.py’ 0K .......... . 100% 144M=0s 2018-02-01 12:44:14 (144 MB/s) - ‘archiver_client.py’ saved [12179/12179] program finished with exit code 0 elapsedTime=0.212643 ========= master_lag: 0.01 ========= ========= Finished 'bash -c ...' (results: 0, elapsed: 0 secs) (at 2018-02-01 12:44:14.358784) ========= ========= Started 'rm -rf ...' (results: 0, elapsed: 0 secs) (at 2018-02-01 12:44:14.359069) ========= rm -rf scripts properties in dir /builds/slave/rel-bir-fx_uptk_mntr-000000000/. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-rf', 'scripts', 'properties'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-306.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-bir-fx_uptk_mntr-000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.002521 ========= master_lag: 0.05 ========= ========= Finished 'rm -rf ...' (results: 0, elapsed: 0 secs) (at 2018-02-01 12:44:14.406718) ========= ========= Started 'bash -c ...' (results: 0, elapsed: 0 secs) (at 2018-02-01 12:44:14.407035) ========= bash -c 'python archiver_client.py mozharness --repo projects/birch --rev f3b38b0e099bd2939f1a915ab2c655b8ec4b5954 --destination scripts --debug' in dir /builds/slave/rel-bir-fx_uptk_mntr-000000000/. (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', u'python archiver_client.py mozharness --repo projects/birch --rev f3b38b0e099bd2939f1a915ab2c655b8ec4b5954 --destination scripts --debug'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-306.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-bir-fx_uptk_mntr-000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False 2018-02-01 12:44:14,454 truncating revision to first 12 chars 2018-02-01 12:44:14,454 Setting DEBUG logging. 2018-02-01 12:44:14,454 attempt 1/10 2018-02-01 12:44:14,454 Getting archive location from https://api.pub.build.mozilla.org/archiver/hgmo/projects/birch/f3b38b0e099b?&preferred_region=us-west-2&suffix=tar.gz&subdir=testing/mozharness 2018-02-01 12:44:15,090 unpacking tar archive at: birch-f3b38b0e099b/testing/mozharness/ program finished with exit code 0 elapsedTime=0.835091 ========= master_lag: 0.01 ========= ========= Finished 'bash -c ...' (results: 0, elapsed: 0 secs) (at 2018-02-01 12:44:15.251475) ========= ========= Started set props: script_repo_revision (results: 0, elapsed: 0 secs) (at 2018-02-01 12:44:15.251819) ========= script_repo_revision: f3b38b0e099bd2939f1a915ab2c655b8ec4b5954 ========= Finished set props: script_repo_revision (results: 0, elapsed: 0 secs) (at 2018-02-01 12:44:15.252182) ========= ========= Started downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2018-02-01 12:44:15.252454) ========= ========= Finished downloading to oauth.txt (results: 0, elapsed: 0 secs) (at 2018-02-01 12:44:15.262286) ========= ========= Started '/tools/buildbot/bin/python scripts/scripts/release/uptake_monitoring.py ...' failed (results: 2, elapsed: 8 mins, 23 secs) (at 2018-02-01 12:44:15.262554) ========= /tools/buildbot/bin/python scripts/scripts/release/uptake_monitoring.py -c releases/bouncer_firefox_release.py in dir /builds/slave/rel-bir-fx_uptk_mntr-000000000/. (timeout 3600 secs) (maxTime 14400 secs) watching logfiles {} argv: ['/tools/buildbot/bin/python', 'scripts/scripts/release/uptake_monitoring.py', '-c', 'releases/bouncer_firefox_release.py'] environment: CCACHE_COMPRESS=1 CCACHE_DIR=/builds/ccache CCACHE_HASHDIR= CCACHE_UMASK=002 DISPLAY=:2 G_BROKEN_FILENAMES=1 HG_SHARE_BASE_DIR=/builds/hg-shared HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-306.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LC_ALL=C LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld MOZ_AUTOMATION=1 MOZ_CRASHREPORTER_NO_REPORT=1 MOZ_OBJDIR=obj-firefox PATH=/tools/buildbot/bin:/usr/local/bin:/usr/lib64/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin PROPERTIES_FILE=/builds/slave/rel-bir-fx_uptk_mntr-000000000/buildprops.json PWD=/builds/slave/rel-bir-fx_uptk_mntr-000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TINDERBOX_OUTPUT=1 TMOUT=86400 TOOLTOOL_CACHE=/builds/tooltool_cache TOOLTOOL_HOME=/builds USER=cltbld _=/tools/buildbot/bin/python using PTY: False 12:44:15 INFO - MultiFileLogger online at 20180201 12:44:15 in /builds/slave/rel-bir-fx_uptk_mntr-000000000 12:44:15 INFO - Using buildbot properties: 12:44:15 INFO - { 12:44:15 INFO - "platforms": "linux, linux64, win32, win64, macosx64", 12:44:15 INFO - "partial_versions": "57.0build4", 12:44:15 INFO - "buildnumber": 0, 12:44:15 INFO - "build_number": 12, 12:44:15 INFO - "platform": null, 12:44:15 INFO - "balrog_api_root": "https://balrog-admin.stage.mozaws.net/api", 12:44:15 INFO - "basedir": "/builds/slave/rel-bir-fx_uptk_mntr-000000000", 12:44:15 INFO - "appVersion": "60.0.2", 12:44:15 INFO - "slavename": "bld-linux64-spot-306", 12:44:15 INFO - "version": "60.0.2", 12:44:15 INFO - "master": "http://buildbot-master73.bb.releng.usw2.mozilla.com:8001/", 12:44:15 INFO - "branch": "birch", 12:44:15 INFO - "revision": "f3b38b0e099bd2939f1a915ab2c655b8ec4b5954", 12:44:15 INFO - "tuxedo_server_url": "http://localhost/api", 12:44:15 INFO - "product": "firefox", 12:44:15 INFO - "repository": "https://hg.mozilla.org/projects/birch", 12:44:15 INFO - "buildername": "release-birch-firefox_uptake_monitoring", 12:44:15 INFO - "taskId": "OZJCCIvgS5e_5KeQqo7Dhg", 12:44:15 INFO - "next_version": "60.0.3", 12:44:15 INFO - "script_repo_revision": "f3b38b0e099bd2939f1a915ab2c655b8ec4b5954", 12:44:15 INFO - "project": "", 12:44:15 INFO - "release_promotion": true, 12:44:15 INFO - "repo_path": "projects/birch" 12:44:15 INFO - } 12:44:15 INFO - Overriding tuxedo_server_url with http://localhost/api 12:44:15 INFO - Overriding version with 60.0.2 12:44:15 INFO - Run as scripts/scripts/release/uptake_monitoring.py -c releases/bouncer_firefox_release.py 12:44:15 INFO - Dumping config to /builds/slave/rel-bir-fx_uptk_mntr-000000000/logs/localconfig.json. 12:44:15 INFO - {'append_to_log': False, 12:44:15 INFO - 'base_work_dir': '/builds/slave/rel-bir-fx_uptk_mntr-000000000', 12:44:15 INFO - 'buildbot_json_path': 'buildprops.json', 12:44:15 INFO - 'config_files': ('releases/bouncer_firefox_release.py',), 12:44:15 INFO - 'credentials_file': 'oauth.txt', 12:44:15 INFO - 'log_level': 'info', 12:44:15 INFO - 'log_to_console': True, 12:44:15 INFO - 'min_uptake': 10000, 12:44:15 INFO - 'opt_config_files': (), 12:44:15 INFO - 'partial_versions': (u'57.0',), 12:44:15 INFO - 'partials': {'candidates-dir': {'add-locales': True, 12:44:15 INFO - 'check_uptake': False, 12:44:15 INFO - 'paths': {'linux': {'bouncer-platform': 'linux', 12:44:15 INFO - 'path': '/firefox/candidates/%(version)s-candidates/build%(build_number)s/update/linux-i686/:lang/firefox-%(prev_version)s-%(version)s.partial.mar'}, 12:44:15 INFO - 'linux64': {'bouncer-platform': 'linux64', 12:44:15 INFO - 'path': '/firefox/candidates/%(version)s-candidates/build%(build_number)s/update/linux-x86_64/:lang/firefox-%(prev_version)s-%(version)s.partial.mar'}, 12:44:15 INFO - 'macosx64': {'bouncer-platform': 'osx', 12:44:15 INFO - 'path': '/firefox/candidates/%(version)s-candidates/build%(build_number)s/update/mac/:lang/firefox-%(prev_version)s-%(version)s.partial.mar'}, 12:44:15 INFO - 'win32': {'bouncer-platform': 'win', 12:44:15 INFO - 'path': '/firefox/candidates/%(version)s-candidates/build%(build_number)s/update/win32/:lang/firefox-%(prev_version)s-%(version)s.partial.mar'}, 12:44:15 INFO - 'win64': {'bouncer-platform': 'win64', 12:44:15 INFO - 'path': '/firefox/candidates/%(version)s-candidates/build%(build_number)s/update/win64/:lang/firefox-%(prev_version)s-%(version)s.partial.mar'}}, 12:44:15 INFO - 'product-name': 'Firefox-%(version)sbuild%(build_number)s-Partial-%(prev_version)sbuild%(prev_build_number)s', 12:44:15 INFO - 'ssl-only': False}, 12:44:15 INFO - 'releases-dir': {'add-locales': True, 12:44:15 INFO - 'check_uptake': True, 12:44:15 INFO - 'paths': {'linux': {'bouncer-platform': 'linux', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/update/linux-i686/:lang/firefox-%(prev_version)s-%(version)s.partial.mar'}, 12:44:15 INFO - 'linux64': {'bouncer-platform': 'linux64', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/update/linux-x86_64/:lang/firefox-%(prev_version)s-%(version)s.partial.mar'}, 12:44:15 INFO - 'macosx64': {'bouncer-platform': 'osx', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/update/mac/:lang/firefox-%(prev_version)s-%(version)s.partial.mar'}, 12:44:15 INFO - 'win32': {'bouncer-platform': 'win', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/update/win32/:lang/firefox-%(prev_version)s-%(version)s.partial.mar'}, 12:44:15 INFO - 'win64': {'bouncer-platform': 'win64', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/update/win64/:lang/firefox-%(prev_version)s-%(version)s.partial.mar'}}, 12:44:15 INFO - 'product-name': 'Firefox-%(version)s-Partial-%(prev_version)s', 12:44:15 INFO - 'ssl-only': False}}, 12:44:15 INFO - 'pip_index': True, 12:44:15 INFO - 'platforms': (u'linux', u'linux64', u'win32', u'win64', u'macosx64'), 12:44:15 INFO - 'poll_interval': 60, 12:44:15 INFO - 'poll_timeout': 1200, 12:44:15 INFO - 'products': {'complete-mar': {'add-locales': True, 12:44:15 INFO - 'check_uptake': True, 12:44:15 INFO - 'paths': {'linux': {'bouncer-platform': 'linux', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/update/linux-i686/:lang/firefox-%(version)s.complete.mar'}, 12:44:15 INFO - 'linux64': {'bouncer-platform': 'linux64', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/update/linux-x86_64/:lang/firefox-%(version)s.complete.mar'}, 12:44:15 INFO - 'macosx64': {'bouncer-platform': 'osx', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/update/mac/:lang/firefox-%(version)s.complete.mar'}, 12:44:15 INFO - 'win32': {'bouncer-platform': 'win', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/update/win32/:lang/firefox-%(version)s.complete.mar'}, 12:44:15 INFO - 'win64': {'bouncer-platform': 'win64', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/update/win64/:lang/firefox-%(version)s.complete.mar'}}, 12:44:15 INFO - 'product-name': 'Firefox-%(version)s-Complete', 12:44:15 INFO - 'ssl-only': False}, 12:44:15 INFO - 'complete-mar-candidates': {'add-locales': True, 12:44:15 INFO - 'check_uptake': False, 12:44:15 INFO - 'paths': {'linux': {'bouncer-platform': 'linux', 12:44:15 INFO - 'path': '/firefox/candidates/%(version)s-candidates/build%(build_number)s/update/linux-i686/:lang/firefox-%(version)s.complete.mar'}, 12:44:15 INFO - 'linux64': {'bouncer-platform': 'linux64', 12:44:15 INFO - 'path': '/firefox/candidates/%(version)s-candidates/build%(build_number)s/update/linux-x86_64/:lang/firefox-%(version)s.complete.mar'}, 12:44:15 INFO - 'macosx64': {'bouncer-platform': 'osx', 12:44:15 INFO - 'path': '/firefox/candidates/%(version)s-candidates/build%(build_number)s/update/mac/:lang/firefox-%(version)s.complete.mar'}, 12:44:15 INFO - 'win32': {'bouncer-platform': 'win', 12:44:15 INFO - 'path': '/firefox/candidates/%(version)s-candidates/build%(build_number)s/update/win32/:lang/firefox-%(version)s.complete.mar'}, 12:44:15 INFO - 'win64': {'bouncer-platform': 'win64', 12:44:15 INFO - 'path': '/firefox/candidates/%(version)s-candidates/build%(build_number)s/update/win64/:lang/firefox-%(version)s.complete.mar'}}, 12:44:15 INFO - 'product-name': 'Firefox-%(version)sbuild%(build_number)s-Complete', 12:44:15 INFO - 'ssl-only': False}, 12:44:15 INFO - 'installer': {'add-locales': True, 12:44:15 INFO - 'alias': 'firefox-latest', 12:44:15 INFO - 'check_uptake': True, 12:44:15 INFO - 'paths': {'linux': {'bouncer-platform': 'linux', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/linux-i686/:lang/firefox-%(version)s.tar.bz2'}, 12:44:15 INFO - 'linux64': {'bouncer-platform': 'linux64', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/linux-x86_64/:lang/firefox-%(version)s.tar.bz2'}, 12:44:15 INFO - 'macosx64': {'bouncer-platform': 'osx', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/mac/:lang/Firefox%%20%(version)s.dmg'}, 12:44:15 INFO - 'win32': {'bouncer-platform': 'win', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/win32/:lang/Firefox%%20Setup%%20%(version)s.exe'}, 12:44:15 INFO - 'win64': {'bouncer-platform': 'win64', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/win64/:lang/Firefox%%20Setup%%20%(version)s.exe'}}, 12:44:15 INFO - 'product-name': 'Firefox-%(version)s', 12:44:15 INFO - 'ssl-only': False}, 12:44:15 INFO - 'installer-ssl': {'add-locales': True, 12:44:15 INFO - 'alias': 'firefox-latest-ssl', 12:44:15 INFO - 'check_uptake': True, 12:44:15 INFO - 'paths': {'linux': {'bouncer-platform': 'linux', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/linux-i686/:lang/firefox-%(version)s.tar.bz2'}, 12:44:15 INFO - 'linux64': {'bouncer-platform': 'linux64', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/linux-x86_64/:lang/firefox-%(version)s.tar.bz2'}, 12:44:15 INFO - 'macosx64': {'bouncer-platform': 'osx', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/mac/:lang/Firefox%%20%(version)s.dmg'}, 12:44:15 INFO - 'win32': {'bouncer-platform': 'win', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/win32/:lang/Firefox%%20Setup%%20%(version)s.exe'}, 12:44:15 INFO - 'win64': {'bouncer-platform': 'win64', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/win64/:lang/Firefox%%20Setup%%20%(version)s.exe'}}, 12:44:15 INFO - 'product-name': 'Firefox-%(version)s-SSL', 12:44:15 INFO - 'ssl-only': True}, 12:44:15 INFO - 'stub-installer': {'add-locales': True, 12:44:15 INFO - 'alias': 'firefox-stub', 12:44:15 INFO - 'check_uptake': True, 12:44:15 INFO - 'paths': {'win32': {'bouncer-platform': 'win', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/win32/:lang/Firefox%%20Installer.exe'}, 12:44:15 INFO - 'win64': {'bouncer-platform': 'win64', 12:44:15 INFO - 'path': '/firefox/releases/%(version)s/win32/:lang/Firefox%%20Installer.exe'}}, 12:44:15 INFO - 'product-name': 'Firefox-%(version)s-stub', 12:44:15 INFO - 'ssl-only': True}}, 12:44:15 INFO - 'shipped-locales-url': 'https://hg.mozilla.org/%(repo)s/raw-file/%(revision)s/browser/locales/shipped-locales', 12:44:15 INFO - 'tuxedo_server_url': u'http://localhost/api', 12:44:15 INFO - 'version': u'60.0.2', 12:44:15 INFO - 'virtualenv_modules': ('redo', 'requests'), 12:44:15 INFO - 'virtualenv_path': 'venv', 12:44:15 INFO - 'volatile_config': {'actions': None, 'add_actions': None, 'no_actions': None}, 12:44:15 INFO - 'work_dir': 'build'} 12:44:15 INFO - [mozharness: 2018-02-01 20:44:15.648657Z] Running create-virtualenv step. 12:44:15 INFO - Running main action method: create_virtualenv 12:44:15 INFO - Creating virtualenv /builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv 12:44:15 INFO - mkdir: /builds/slave/rel-bir-fx_uptk_mntr-000000000/build 12:44:15 INFO - Running command: ['/tools/buildbot/bin/python', '/builds/slave/rel-bir-fx_uptk_mntr-000000000/scripts/external_tools/virtualenv/virtualenv.py', '--always-copy', '/builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv'] in /builds/slave/rel-bir-fx_uptk_mntr-000000000/build 12:44:15 INFO - Copy/paste: /tools/buildbot/bin/python /builds/slave/rel-bir-fx_uptk_mntr-000000000/scripts/external_tools/virtualenv/virtualenv.py --always-copy /builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv 12:44:15 INFO - Using partial env: {'VIRTUALENV_NO_DOWNLOAD': '1'} 12:44:17 INFO - Using real prefix '/tools/python27' 12:44:17 INFO - Cannot find file lib (bad symlink) 12:44:17 INFO - New python executable in /builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv/bin/python 12:44:17 INFO - Overwriting /builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv/lib/python2.7/distutils/__init__.py with new content 12:44:18 INFO - Installing setuptools, pip, wheel...done. 12:44:18 INFO - Return code: 0 12:44:18 INFO - Installing redo into virtualenv /builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv 12:44:18 INFO - retry: Calling run_command with args: [['/builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv/bin/pip', 'install', '--timeout', '120', 'redo']], kwargs: {'error_level': 'warning', 'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x24120d0>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x1ecfd50>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x23101f0>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x22e0920>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x1fa88a0>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x1f15f08>, 'level': 'critical'}], 'cwd': '/builds/slave/rel-bir-fx_uptk_mntr-000000000/build', 'env': {'MOZ_AUTOMATION': '1', 'LESSOPEN': '|/usr/bin/lesspipe.sh %s', 'TMOUT': '86400', 'LOGNAME': 'cltbld', 'USER': 'cltbld', 'MOZ_OBJDIR': 'obj-firefox', 'PATH': '/tools/buildbot/bin:/usr/local/bin:/usr/lib64/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin', 'DISPLAY': ':2', 'CCACHE_UMASK': '002', 'LANG': 'en_US.UTF-8', 'CCACHE_HASHDIR': '', 'TERM': 'linux', 'SHELL': '/bin/bash', 'SHLVL': '1', 'G_BROKEN_FILENAMES': '1', 'HISTSIZE': '1000', 'HOME': '/home/cltbld', 'HG_SHARE_BASE_DIR': '/builds/hg-shared', 'MOZ_CRASHREPORTER_NO_REPORT': '1', 'TINDERBOX_OUTPUT': '1', 'MAIL': '/var/spool/mail/cltbld', 'LC_ALL': 'C', 'TOOLTOOL_HOME': '/builds', '_': '/tools/buildbot/bin/python', 'HOSTNAME': 'bld-linux64-spot-306.build.releng.usw2.mozilla.com', 'HISTCONTROL': 'ignoredups', 'PWD': '/builds/slave/rel-bir-fx_uptk_mntr-000000000', 'PROPERTIES_FILE': '/builds/slave/rel-bir-fx_uptk_mntr-000000000/buildprops.json', 'CCACHE_DIR': '/builds/ccache', 'CCACHE_COMPRESS': '1', 'TOOLTOOL_CACHE': '/builds/tooltool_cache'}}, attempt #1 12:44:18 INFO - Running command: ['/builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv/bin/pip', 'install', '--timeout', '120', 'redo'] in /builds/slave/rel-bir-fx_uptk_mntr-000000000/build 12:44:18 INFO - Copy/paste: /builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv/bin/pip install --timeout 120 redo 12:44:18 INFO - Using env: {'CCACHE_COMPRESS': '1', 12:44:18 INFO - 'CCACHE_DIR': '/builds/ccache', 12:44:18 INFO - 'CCACHE_HASHDIR': '', 12:44:18 INFO - 'CCACHE_UMASK': '002', 12:44:18 INFO - 'DISPLAY': ':2', 12:44:18 INFO - 'G_BROKEN_FILENAMES': '1', 12:44:18 INFO - 'HG_SHARE_BASE_DIR': '/builds/hg-shared', 12:44:18 INFO - 'HISTCONTROL': 'ignoredups', 12:44:18 INFO - 'HISTSIZE': '1000', 12:44:18 INFO - 'HOME': '/home/cltbld', 12:44:18 INFO - 'HOSTNAME': 'bld-linux64-spot-306.build.releng.usw2.mozilla.com', 12:44:18 INFO - 'LANG': 'en_US.UTF-8', 12:44:18 INFO - 'LC_ALL': 'C', 12:44:18 INFO - 'LESSOPEN': '|/usr/bin/lesspipe.sh %s', 12:44:18 INFO - 'LOGNAME': 'cltbld', 12:44:18 INFO - 'MAIL': '/var/spool/mail/cltbld', 12:44:18 INFO - 'MOZ_AUTOMATION': '1', 12:44:18 INFO - 'MOZ_CRASHREPORTER_NO_REPORT': '1', 12:44:18 INFO - 'MOZ_OBJDIR': 'obj-firefox', 12:44:18 INFO - 'PATH': '/tools/buildbot/bin:/usr/local/bin:/usr/lib64/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin', 12:44:18 INFO - 'PROPERTIES_FILE': '/builds/slave/rel-bir-fx_uptk_mntr-000000000/buildprops.json', 12:44:18 INFO - 'PWD': '/builds/slave/rel-bir-fx_uptk_mntr-000000000', 12:44:18 INFO - 'SHELL': '/bin/bash', 12:44:18 INFO - 'SHLVL': '1', 12:44:18 INFO - 'TERM': 'linux', 12:44:18 INFO - 'TINDERBOX_OUTPUT': '1', 12:44:18 INFO - 'TMOUT': '86400', 12:44:18 INFO - 'TOOLTOOL_CACHE': '/builds/tooltool_cache', 12:44:18 INFO - 'TOOLTOOL_HOME': '/builds', 12:44:18 INFO - 'USER': 'cltbld', 12:44:18 INFO - '_': '/tools/buildbot/bin/python'} 12:44:19 INFO - Ignoring indexes: https://pypi.python.org/simple 12:44:19 INFO - Collecting redo 12:44:20 INFO - Downloading http://pypi.pvt.build.mozilla.org/pub/redo-1.5-py2.py3-none-any.whl 12:44:20 INFO - Installing collected packages: redo 12:44:20 INFO - Successfully installed redo-1.5 12:44:20 INFO - Return code: 0 12:44:20 INFO - Installing requests into virtualenv /builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv 12:44:20 INFO - retry: Calling run_command with args: [['/builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv/bin/pip', 'install', '--timeout', '120', 'requests']], kwargs: {'error_level': 'warning', 'error_list': [{'substr': 'not found or a compiler error:', 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x24120d0>, 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x1ecfd50>, 'level': 'warning'}, {'regex': <_sre.SRE_Pattern object at 0x23101f0>, 'level': 'debug'}, {'substr': 'command not found', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x22e0920>, 'level': 'warning'}, {'substr': 'Traceback (most recent call last)', 'level': 'error'}, {'substr': 'SyntaxError: ', 'level': 'error'}, {'substr': 'TypeError: ', 'level': 'error'}, {'substr': 'NameError: ', 'level': 'error'}, {'substr': 'ZeroDivisionError: ', 'level': 'error'}, {'regex': <_sre.SRE_Pattern object at 0x1fa88a0>, 'level': 'critical'}, {'regex': <_sre.SRE_Pattern object at 0x1f15f08>, 'level': 'critical'}], 'cwd': '/builds/slave/rel-bir-fx_uptk_mntr-000000000/build', 'env': {'MOZ_AUTOMATION': '1', 'LESSOPEN': '|/usr/bin/lesspipe.sh %s', 'TMOUT': '86400', 'LOGNAME': 'cltbld', 'USER': 'cltbld', 'MOZ_OBJDIR': 'obj-firefox', 'PATH': '/tools/buildbot/bin:/usr/local/bin:/usr/lib64/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin', 'DISPLAY': ':2', 'CCACHE_UMASK': '002', 'LANG': 'en_US.UTF-8', 'CCACHE_HASHDIR': '', 'TERM': 'linux', 'SHELL': '/bin/bash', 'SHLVL': '1', 'G_BROKEN_FILENAMES': '1', 'HISTSIZE': '1000', 'HOME': '/home/cltbld', 'HG_SHARE_BASE_DIR': '/builds/hg-shared', 'MOZ_CRASHREPORTER_NO_REPORT': '1', 'TINDERBOX_OUTPUT': '1', 'MAIL': '/var/spool/mail/cltbld', 'LC_ALL': 'C', 'TOOLTOOL_HOME': '/builds', '_': '/tools/buildbot/bin/python', 'HOSTNAME': 'bld-linux64-spot-306.build.releng.usw2.mozilla.com', 'HISTCONTROL': 'ignoredups', 'PWD': '/builds/slave/rel-bir-fx_uptk_mntr-000000000', 'PROPERTIES_FILE': '/builds/slave/rel-bir-fx_uptk_mntr-000000000/buildprops.json', 'CCACHE_DIR': '/builds/ccache', 'CCACHE_COMPRESS': '1', 'TOOLTOOL_CACHE': '/builds/tooltool_cache'}}, attempt #1 12:44:20 INFO - Running command: ['/builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv/bin/pip', 'install', '--timeout', '120', 'requests'] in /builds/slave/rel-bir-fx_uptk_mntr-000000000/build 12:44:20 INFO - Copy/paste: /builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv/bin/pip install --timeout 120 requests 12:44:20 INFO - Using env: {'CCACHE_COMPRESS': '1', 12:44:20 INFO - 'CCACHE_DIR': '/builds/ccache', 12:44:20 INFO - 'CCACHE_HASHDIR': '', 12:44:20 INFO - 'CCACHE_UMASK': '002', 12:44:20 INFO - 'DISPLAY': ':2', 12:44:20 INFO - 'G_BROKEN_FILENAMES': '1', 12:44:20 INFO - 'HG_SHARE_BASE_DIR': '/builds/hg-shared', 12:44:20 INFO - 'HISTCONTROL': 'ignoredups', 12:44:20 INFO - 'HISTSIZE': '1000', 12:44:20 INFO - 'HOME': '/home/cltbld', 12:44:20 INFO - 'HOSTNAME': 'bld-linux64-spot-306.build.releng.usw2.mozilla.com', 12:44:20 INFO - 'LANG': 'en_US.UTF-8', 12:44:20 INFO - 'LC_ALL': 'C', 12:44:20 INFO - 'LESSOPEN': '|/usr/bin/lesspipe.sh %s', 12:44:20 INFO - 'LOGNAME': 'cltbld', 12:44:20 INFO - 'MAIL': '/var/spool/mail/cltbld', 12:44:20 INFO - 'MOZ_AUTOMATION': '1', 12:44:20 INFO - 'MOZ_CRASHREPORTER_NO_REPORT': '1', 12:44:20 INFO - 'MOZ_OBJDIR': 'obj-firefox', 12:44:20 INFO - 'PATH': '/tools/buildbot/bin:/usr/local/bin:/usr/lib64/ccache:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/tools/git/bin:/tools/python27/bin:/tools/python27-mercurial/bin:/home/cltbld/bin', 12:44:20 INFO - 'PROPERTIES_FILE': '/builds/slave/rel-bir-fx_uptk_mntr-000000000/buildprops.json', 12:44:20 INFO - 'PWD': '/builds/slave/rel-bir-fx_uptk_mntr-000000000', 12:44:20 INFO - 'SHELL': '/bin/bash', 12:44:20 INFO - 'SHLVL': '1', 12:44:20 INFO - 'TERM': 'linux', 12:44:20 INFO - 'TINDERBOX_OUTPUT': '1', 12:44:20 INFO - 'TMOUT': '86400', 12:44:20 INFO - 'TOOLTOOL_CACHE': '/builds/tooltool_cache', 12:44:20 INFO - 'TOOLTOOL_HOME': '/builds', 12:44:20 INFO - 'USER': 'cltbld', 12:44:20 INFO - '_': '/tools/buildbot/bin/python'} 12:44:21 INFO - Ignoring indexes: https://pypi.python.org/simple 12:44:21 INFO - Collecting requests 12:44:22 INFO - Downloading http://pypi.pvt.build.mozilla.org/pub/requests-2.18.3-py2.py3-none-any.whl (88kB) 12:44:22 INFO - Collecting certifi>=2017.4.17 (from requests) 12:44:24 INFO - Downloading http://pypi.pvt.build.mozilla.org/pub/certifi-2017.7.27.1-py2.py3-none-any.whl (349kB) 12:44:24 INFO - Collecting chardet<3.1.0,>=3.0.2 (from requests) 12:44:26 INFO - Downloading http://pypi.pvt.build.mozilla.org/pub/chardet-3.0.4-py2.py3-none-any.whl (133kB) 12:44:26 INFO - Collecting urllib3<1.23,>=1.21.1 (from requests) 12:44:27 INFO - Downloading http://pypi.pvt.build.mozilla.org/pub/urllib3-1.22-py2.py3-none-any.whl (132kB) 12:44:27 INFO - Collecting idna<2.6,>=2.5 (from requests) 12:44:29 INFO - Downloading http://pypi.pvt.build.mozilla.org/pub/idna-2.5-py2.py3-none-any.whl (55kB) 12:44:29 INFO - Installing collected packages: certifi, chardet, urllib3, idna, requests 12:44:29 INFO - Successfully installed certifi-2017.7.27.1 chardet-3.0.4 idna-2.5 requests-2.18.3 urllib3-1.22 12:44:29 INFO - Return code: 0 12:44:29 INFO - Done creating virtualenv /builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv. 12:44:29 INFO - Getting output from command: ['/builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv/bin/pip', 'freeze'] 12:44:29 INFO - Copy/paste: /builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv/bin/pip freeze 12:44:30 INFO - Reading from file tmpfile_stdout 12:44:30 INFO - Reading from file tmpfile_stderr 12:44:30 INFO - Current package versions: 12:44:30 INFO - certifi == 2017.7.27.1 12:44:30 INFO - chardet == 3.0.4 12:44:30 INFO - idna == 2.5 12:44:30 INFO - redo == 1.5 12:44:30 INFO - requests == 2.18.3 12:44:30 INFO - urllib3 == 1.22 12:44:30 INFO - [mozharness: 2018-02-01 20:44:30.451096Z] Finished create-virtualenv step (success) 12:44:30 INFO - [mozharness: 2018-02-01 20:44:30.451294Z] Running activate-virtualenv step. 12:44:30 INFO - Running main action method: activate_virtualenv 12:44:30 INFO - [mozharness: 2018-02-01 20:44:30.452485Z] Finished activate-virtualenv step (success) 12:44:30 INFO - [mozharness: 2018-02-01 20:44:30.452699Z] Running monitor-uptake step. 12:44:30 INFO - Running main action method: monitor_uptake 12:44:30 INFO - Starting the loop to determine the uptake monitoring ... 12:44:30 INFO - Requesting http://localhost/api/uptake/?product=Firefox-60.0.2&os=win from tuxedo 12:45:31 INFO - retry: Calling with args: (), kwargs: {}, attempt #2 12:47:01 INFO - retry: Calling with args: (), kwargs: {}, attempt #3 12:49:15 INFO - retry: Calling with args: (), kwargs: {}, attempt #4 12:52:38 INFO - retry: Calling with args: (), kwargs: {}, attempt #5 12:52:38 INFO - retry: Giving up on 12:52:38 INFO - [mozharness: 2018-02-01 20:52:38.373300Z] Finished monitor-uptake step (failed) 12:52:38 FATAL - Uncaught exception: Traceback (most recent call last): 12:52:38 FATAL - File "/builds/slave/rel-bir-fx_uptk_mntr-000000000/scripts/mozharness/base/script.py", line 2059, in run 12:52:38 FATAL - self.run_action(action) 12:52:38 FATAL - File "/builds/slave/rel-bir-fx_uptk_mntr-000000000/scripts/mozharness/base/script.py", line 1998, in run_action 12:52:38 FATAL - self._possibly_run_method(method_name, error_if_missing=True) 12:52:38 FATAL - File "/builds/slave/rel-bir-fx_uptk_mntr-000000000/scripts/mozharness/base/script.py", line 1938, in _possibly_run_method 12:52:38 FATAL - return getattr(self, method_name)() 12:52:38 FATAL - File "scripts/scripts/release/uptake_monitoring.py", line 176, in monitor_uptake 12:52:38 FATAL - uptake = self._get_release_uptake(auth) 12:52:38 FATAL - File "scripts/scripts/release/uptake_monitoring.py", line 138, in _get_release_uptake 12:52:38 FATAL - related_product, bouncer_platform)) 12:52:38 FATAL - File "scripts/scripts/release/uptake_monitoring.py", line 113, in _get_product_uptake 12:52:38 FATAL - page = retry(get_tuxedo_page) 12:52:38 FATAL - File "/builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv/lib/python2.7/site-packages/redo/__init__.py", line 152, in retry 12:52:38 FATAL - return action(*args, **kwargs) 12:52:38 FATAL - File "scripts/scripts/release/uptake_monitoring.py", line 96, in get_tuxedo_page 12:52:38 FATAL - verify=False, timeout=60) 12:52:38 FATAL - File "/builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv/lib/python2.7/site-packages/requests/api.py", line 72, in get 12:52:38 FATAL - return request('get', url, params=params, **kwargs) 12:52:38 FATAL - File "/builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv/lib/python2.7/site-packages/requests/api.py", line 58, in request 12:52:38 FATAL - return session.request(method=method, url=url, **kwargs) 12:52:38 FATAL - File "/builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv/lib/python2.7/site-packages/requests/sessions.py", line 508, in request 12:52:38 FATAL - resp = self.send(prep, **send_kwargs) 12:52:38 FATAL - File "/builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv/lib/python2.7/site-packages/requests/sessions.py", line 618, in send 12:52:38 FATAL - r = adapter.send(request, **kwargs) 12:52:38 FATAL - File "/builds/slave/rel-bir-fx_uptk_mntr-000000000/build/venv/lib/python2.7/site-packages/requests/adapters.py", line 508, in send 12:52:38 FATAL - raise ConnectionError(e, request=request) 12:52:38 FATAL - ConnectionError: HTTPConnectionPool(host='localhost', port=80): Max retries exceeded with url: /api/uptake/?product=Firefox-60.0.2&os=win (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 111] Connection refused',)) 12:52:38 FATAL - Running post_fatal callback... 12:52:38 FATAL - Exiting -1 12:52:38 INFO - Running post-run listener: copy_logs_to_upload_dir 12:52:38 INFO - Copying logs to upload dir... 12:52:38 INFO - mkdir: /builds/slave/rel-bir-fx_uptk_mntr-000000000/build/upload/logs program finished with exit code 255 elapsedTime=503.127325 ========= master_lag: 0.01 ========= ========= Finished '/tools/buildbot/bin/python scripts/scripts/release/uptake_monitoring.py ...' failed (results: 2, elapsed: 8 mins, 23 secs) (at 2018-02-01 12:52:38.398578) ========= ========= Started no change (results: 0, elapsed: 0 secs) (at 2018-02-01 12:52:38.400715) ========= bash -c 'for file in `ls -1`; do cat $file; done' in dir /builds/slave/rel-bir-fx_uptk_mntr-000000000/properties (timeout 1200 secs) watching logfiles {} argv: ['bash', '-c', 'for file in `ls -1`; do cat $file; done'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-306.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-bir-fx_uptk_mntr-000000000/properties SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.005280 ========= master_lag: 0.04 ========= ========= Finished no change (results: 0, elapsed: 0 secs) (at 2018-02-01 12:52:38.449195) ========= ========= Started 'rm -f ...' (results: 0, elapsed: 0 secs) (at 2018-02-01 12:52:38.449494) ========= rm -f oauth.txt in dir /builds/slave/rel-bir-fx_uptk_mntr-000000000/. (timeout 1200 secs) watching logfiles {} argv: ['rm', '-f', 'oauth.txt'] environment: CCACHE_HASHDIR= G_BROKEN_FILENAMES=1 HISTCONTROL=ignoredups HISTSIZE=1000 HOME=/home/cltbld HOSTNAME=bld-linux64-spot-306.build.releng.usw2.mozilla.com LANG=en_US.UTF-8 LESSOPEN=|/usr/bin/lesspipe.sh %s LOGNAME=cltbld MAIL=/var/spool/mail/cltbld PATH=/usr/local/bin:/usr/lib64/ccache:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/cltbld/bin PWD=/builds/slave/rel-bir-fx_uptk_mntr-000000000 SHELL=/bin/bash SHLVL=1 TERM=linux TMOUT=86400 USER=cltbld _=/tools/buildbot/bin/python using PTY: False program finished with exit code 0 elapsedTime=0.002579 ========= master_lag: 0.05 ========= ========= Finished 'rm -f ...' (results: 0, elapsed: 0 secs) (at 2018-02-01 12:52:38.498391) ========= ========= Started reboot skipped (results: 3, elapsed: 0 secs) (at 2018-02-01 12:52:38.498687) ========= ========= Finished reboot skipped (results: 3, elapsed: 0 secs) (at 2018-02-01 12:52:38.498929) ========= ========= Total master_lag: 0.16 =========