View Issue Details

IDProjectCategoryView StatusLast Update
0015518CentOS-7python-s3transferpublic2019-01-02 19:36
Reporterandrewn 
PrioritynormalSeverityminorReproducibilityalways
Status assignedResolutionopen 
Product Version 
Target VersionFixed in Version 
Summary0015518: awscli errors out on new python-s3transfer build
Descriptionawscli can't find botocore because python-s3transfer-0.1.13-1.el7.noarch puts its bundled deps in /usr/lib64/fence-agents/bundled but aws adds /usr/lib/fence-agents/bundled to sys.path.
Steps To Reproduce$ rpm -q awscli python-s3transfer
package awscli is not installed
package python-s3transfer is not installed
$ sudo yum install -y -q -e 1 awscli
$ rpm -q awscli python-s3transfer
awscli-1.14.28-5.el7_5.1.noarch
python-s3transfer-0.1.13-1.el7.noarch
$ aws
Traceback (most recent call last):
  File "/usr/bin/aws", line 19, in <module>
    import awscli.clidriver
  File "/usr/lib/python2.7/site-packages/awscli/clidriver.py", line 18, in <module>
    import botocore.session
ImportError: No module named botocore.session
Additional Information$ rpm -qf /usr/lib64/fence-agents/bundled/botocore/session.py
python-s3transfer-0.1.13-1.el7.noarch
$ grep -F sys.path /usr/lib/python2.7/site-packages/awscli/clidriver.py
sys.path.insert(0, '/usr/lib/fence-agents/bundled')
Tagsaws, python
abrt_hash
URL

Activities

pingram

pingram

2018-12-04 02:52

reporter   ~0033200

For extra points, this was taken from an 'older' working host:
$ rpm -q --whatprovides /usr/lib/fence-agents/bundled
python-s3transfer-0.1.10-8.el7.noarch

Work around hack for anyone encountering this issue who needs to get things working ASAP:

sed -i -e 's/\/lib\//\/lib64\//' /usr/lib/python2.7/site-packages/awscli/clidriver.py

And, the hack for everyone playing the puppet game:
exec {'patch-clidriver':
  command => "sed -i -e 's/\/lib\//\/lib64\//' /usr/lib/python2.7/site-packages/awscli/clidriver.py",
  onlyif => '[ $(grep -c lib64 /usr/lib/python2.7/site-packages/awscli/clidriver.py) -eq 0 ]',
  path => ['/usr/bin', '/usr/sbin'],
  require => Package['awscli']
}
thedot33

thedot33

2018-12-09 20:00

reporter   ~0033283

Same issue here, after applying the latest system updates via "yum update", the aws cli no longer works, can't backup our files.



# awscli-1.14.28-5.el7_5.1.noarch
# python-s3transfer-0.1.13-1.el7.noarch
 
# Traceback (most recent call last):
  File "/usr/bin/aws", line 19, in <module>
    import awscli.clidriver
  File "/usr/lib/python2.7/site-packages/awscli/clidriver.py", line 18, in <module>
    import botocore.session
ImportError: No module named botocore.session
alexh

alexh

2018-12-11 06:03

reporter   ~0033307

This has also been killing our amazon auto scaling group bootstraps:
```
cloud-init: File "/usr/bin/aws", line 19, in <module>
cloud-init: import awscli.clidriver
cloud-init: File "/usr/lib/python2.7/site-packages/awscli/clidriver.py", line 18, in <module>
cloud-init: import botocore.session
cloud-init: ImportError: No module named botocore.session
cloud-init: + EC2_ASG=
cloud-init: ++ aws elb describe-load-balancers --region us-east-1
cloud-init: ++ jq -r '.LoadBalancerDescriptions[] | select(.Instances[].InstanceId == "i-XXX") | .LoadBalancerName '
cloud-init: Traceback (most recent call last):
cloud-init: File "/usr/bin/aws", line 19, in <module>
cloud-init: import awscli.clidriver
cloud-init: File "/usr/lib/python2.7/site-packages/awscli/clidriver.py", line 18, in <module>
```

Not the first time this year python/aws-cli has killed hosts :-(
TrevorH

TrevorH

2018-12-20 22:15

manager   ~0033407

Attached patch is a result of the sed command in comment #1.

awscli-1.14.28-5-lib64fix.patch (477 bytes)
--- /usr/lib/python2.7/site-packages/awscli/clidriver.py.orig	2018-06-27 14:15:42.000000000 +0100
+++ /usr/lib/python2.7/site-packages/awscli/clidriver.py	2018-12-20 22:10:18.878851451 +0000
@@ -14,7 +14,7 @@
 import signal
 import logging
 
-sys.path.insert(0, '/usr/lib/fence-agents/bundled')
+sys.path.insert(0, '/usr/lib64/fence-agents/bundled')
 import botocore.session
 from botocore import __version__ as botocore_version
 from botocore.hooks import HierarchicalEmitter
TrevorH

TrevorH

2018-12-20 23:14

manager   ~0033409

I suspect that the patch to change this single occurrence of /usr/lib/ to /usr/lib64/ is not the correct fix. There's a big patch in the awscli package that adds those lines into the python source and all of those added lines are /usr/lib/. I suspect it happens to work by patching one file as that is adding the /usr/lib64 path to the PATH so just happens to fix it for all those others at the same time. I'm wondering if the problem package is actually python-s3transfer which is installing the bundled botocore stuff under /usr/lib64 on CentOS and I wonder if that is the same on RHEL or if RHEL uses /usr/lib/ instead.
TrevorH

TrevorH

2018-12-21 16:47

manager   ~0033425

Confirmed. Problem is actually the python-s3transfer package in CentOS installs under /usr/lib64/fence-agents/bundled but under /usr/lib/fence-agents/bundled on RHEL. The patch to awscli to make it use the bundled version hardcodes the RHEL path into all the .py files so they do not find it on CentOS. It'll require a rebuild of python-s3transfer on CentOS 7.6 to fix this.
toracat

toracat

2018-12-21 17:32

manager   ~0033426

Category changed to python-s3transfer.
JohnnyHughes

JohnnyHughes

2018-12-21 18:21

administrator   ~0033427

This is fixed in python-s3transfer-0.1.13-1.el7.0.1.noarch.rpm , whihc is syncing to the master server now. Should be in updates in about an hour on mirror.centos.org.
TrevorH

TrevorH

2018-12-21 18:23

manager   ~0033428

Please note that users who have applied the bypass mentioned in the first comment of this bug report will have to reverse that change in order to work with the fixed package.
alexh

alexh

2018-12-31 01:02

reporter   ~0033483

On a fresh install of 7.6.1810 `yum install awscli` works as expected for me. (it did indeed install `python-s3transfer.noarch 0:0.1.13-1.el7.0.1`)

Awesome! Thanks.
andrewn

andrewn

2019-01-02 19:36

reporter   ~0033499

I can confirm that the latest build of python-s3transfer resolves the issue for me.

Thank you!

Issue History

Date Modified Username Field Change
2018-12-03 21:19 andrewn New Issue
2018-12-04 02:52 pingram Note Added: 0033200
2018-12-09 20:00 thedot33 Note Added: 0033283
2018-12-09 20:00 thedot33 Tag Attached: aws
2018-12-09 20:03 thedot33 Tag Attached: python
2018-12-11 06:03 alexh Note Added: 0033307
2018-12-20 22:15 TrevorH File Added: awscli-1.14.28-5-lib64fix.patch
2018-12-20 22:15 TrevorH Note Added: 0033407
2018-12-20 23:14 TrevorH Note Added: 0033409
2018-12-21 16:47 TrevorH Note Added: 0033425
2018-12-21 17:32 toracat Status new => confirmed
2018-12-21 17:32 toracat Category awscli => python-s3transfer
2018-12-21 17:32 toracat Note Added: 0033426
2018-12-21 18:21 JohnnyHughes Note Added: 0033427
2018-12-21 18:22 JohnnyHughes Status confirmed => feedback
2018-12-21 18:23 TrevorH Note Added: 0033428
2018-12-31 01:02 alexh Note Added: 0033483
2019-01-02 19:36 andrewn Note Added: 0033499
2019-01-02 19:36 andrewn Status feedback => assigned