View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0015518 | CentOS-7 | python-s3transfer | public | 2018-12-03 21:19 | 2019-01-02 19:36 |
Reporter | andrewn | ||||
Priority | normal | Severity | minor | Reproducibility | always |
Status | assigned | Resolution | open | ||
Product Version | |||||
Target Version | Fixed in Version | ||||
Summary | 0015518: awscli errors out on new python-s3transfer build | ||||
Description | awscli can't find botocore because python-s3transfer-0.1.13-1.el7.noarch puts its bundled deps in /usr/lib64/fence-agents/bundled but aws adds /usr/lib/fence-agents/bundled to sys.path. | ||||
Steps To Reproduce | $ rpm -q awscli python-s3transfer package awscli is not installed package python-s3transfer is not installed $ sudo yum install -y -q -e 1 awscli $ rpm -q awscli python-s3transfer awscli-1.14.28-5.el7_5.1.noarch python-s3transfer-0.1.13-1.el7.noarch $ aws Traceback (most recent call last): File "/usr/bin/aws", line 19, in <module> import awscli.clidriver File "/usr/lib/python2.7/site-packages/awscli/clidriver.py", line 18, in <module> import botocore.session ImportError: No module named botocore.session | ||||
Additional Information | $ rpm -qf /usr/lib64/fence-agents/bundled/botocore/session.py python-s3transfer-0.1.13-1.el7.noarch $ grep -F sys.path /usr/lib/python2.7/site-packages/awscli/clidriver.py sys.path.insert(0, '/usr/lib/fence-agents/bundled') | ||||
Tags | aws, python | ||||
abrt_hash | |||||
URL | |||||
For extra points, this was taken from an 'older' working host: $ rpm -q --whatprovides /usr/lib/fence-agents/bundled python-s3transfer-0.1.10-8.el7.noarch Work around hack for anyone encountering this issue who needs to get things working ASAP: sed -i -e 's/\/lib\//\/lib64\//' /usr/lib/python2.7/site-packages/awscli/clidriver.py And, the hack for everyone playing the puppet game: exec {'patch-clidriver': command => "sed -i -e 's/\/lib\//\/lib64\//' /usr/lib/python2.7/site-packages/awscli/clidriver.py", onlyif => '[ $(grep -c lib64 /usr/lib/python2.7/site-packages/awscli/clidriver.py) -eq 0 ]', path => ['/usr/bin', '/usr/sbin'], require => Package['awscli'] } |
|
Same issue here, after applying the latest system updates via "yum update", the aws cli no longer works, can't backup our files. # awscli-1.14.28-5.el7_5.1.noarch # python-s3transfer-0.1.13-1.el7.noarch # Traceback (most recent call last): File "/usr/bin/aws", line 19, in <module> import awscli.clidriver File "/usr/lib/python2.7/site-packages/awscli/clidriver.py", line 18, in <module> import botocore.session ImportError: No module named botocore.session |
|
This has also been killing our amazon auto scaling group bootstraps: ``` cloud-init: File "/usr/bin/aws", line 19, in <module> cloud-init: import awscli.clidriver cloud-init: File "/usr/lib/python2.7/site-packages/awscli/clidriver.py", line 18, in <module> cloud-init: import botocore.session cloud-init: ImportError: No module named botocore.session cloud-init: + EC2_ASG= cloud-init: ++ aws elb describe-load-balancers --region us-east-1 cloud-init: ++ jq -r '.LoadBalancerDescriptions[] | select(.Instances[].InstanceId == "i-XXX") | .LoadBalancerName ' cloud-init: Traceback (most recent call last): cloud-init: File "/usr/bin/aws", line 19, in <module> cloud-init: import awscli.clidriver cloud-init: File "/usr/lib/python2.7/site-packages/awscli/clidriver.py", line 18, in <module> ``` Not the first time this year python/aws-cli has killed hosts :-( |
|
Attached patch is a result of the sed command in comment #1.
awscli-1.14.28-5-lib64fix.patch (477 bytes)
--- /usr/lib/python2.7/site-packages/awscli/clidriver.py.orig 2018-06-27 14:15:42.000000000 +0100 +++ /usr/lib/python2.7/site-packages/awscli/clidriver.py 2018-12-20 22:10:18.878851451 +0000 @@ -14,7 +14,7 @@ import signal import logging -sys.path.insert(0, '/usr/lib/fence-agents/bundled') +sys.path.insert(0, '/usr/lib64/fence-agents/bundled') import botocore.session from botocore import __version__ as botocore_version from botocore.hooks import HierarchicalEmitter |
|
I suspect that the patch to change this single occurrence of /usr/lib/ to /usr/lib64/ is not the correct fix. There's a big patch in the awscli package that adds those lines into the python source and all of those added lines are /usr/lib/. I suspect it happens to work by patching one file as that is adding the /usr/lib64 path to the PATH so just happens to fix it for all those others at the same time. I'm wondering if the problem package is actually python-s3transfer which is installing the bundled botocore stuff under /usr/lib64 on CentOS and I wonder if that is the same on RHEL or if RHEL uses /usr/lib/ instead. | |
Confirmed. Problem is actually the python-s3transfer package in CentOS installs under /usr/lib64/fence-agents/bundled but under /usr/lib/fence-agents/bundled on RHEL. The patch to awscli to make it use the bundled version hardcodes the RHEL path into all the .py files so they do not find it on CentOS. It'll require a rebuild of python-s3transfer on CentOS 7.6 to fix this. | |
Category changed to python-s3transfer. | |
This is fixed in python-s3transfer-0.1.13-1.el7.0.1.noarch.rpm , whihc is syncing to the master server now. Should be in updates in about an hour on mirror.centos.org. | |
Please note that users who have applied the bypass mentioned in the first comment of this bug report will have to reverse that change in order to work with the fixed package. | |
On a fresh install of 7.6.1810 `yum install awscli` works as expected for me. (it did indeed install `python-s3transfer.noarch 0:0.1.13-1.el7.0.1`) Awesome! Thanks. |
|
I can confirm that the latest build of python-s3transfer resolves the issue for me. Thank you! |
|
Date Modified | Username | Field | Change |
---|---|---|---|
2018-12-03 21:19 | andrewn | New Issue | |
2018-12-04 02:52 | pingram | Note Added: 0033200 | |
2018-12-09 20:00 | thedot33 | Note Added: 0033283 | |
2018-12-09 20:00 | thedot33 | Tag Attached: aws | |
2018-12-09 20:03 | thedot33 | Tag Attached: python | |
2018-12-11 06:03 | alexh | Note Added: 0033307 | |
2018-12-20 22:15 | TrevorH | File Added: awscli-1.14.28-5-lib64fix.patch | |
2018-12-20 22:15 | TrevorH | Note Added: 0033407 | |
2018-12-20 23:14 | TrevorH | Note Added: 0033409 | |
2018-12-21 16:47 | TrevorH | Note Added: 0033425 | |
2018-12-21 17:32 | toracat | Status | new => confirmed |
2018-12-21 17:32 | toracat | Category | awscli => python-s3transfer |
2018-12-21 17:32 | toracat | Note Added: 0033426 | |
2018-12-21 18:21 | JohnnyHughes | Note Added: 0033427 | |
2018-12-21 18:22 | JohnnyHughes | Status | confirmed => feedback |
2018-12-21 18:23 | TrevorH | Note Added: 0033428 | |
2018-12-31 01:02 | alexh | Note Added: 0033483 | |
2019-01-02 19:36 | andrewn | Note Added: 0033499 | |
2019-01-02 19:36 | andrewn | Status | feedback => assigned |