View Issue Details
ID | Project | Category | View Status | Date Submitted | Last Update |
---|---|---|---|---|---|
0018066 | CentOS-7 | sudo | public | 2021-02-13 11:25 | 2021-02-13 12:09 |
Reporter | houdinifxtd | Assigned To | |||
Priority | high | Severity | major | Reproducibility | always |
Status | new | Resolution | open | ||
Product Version | 7.7-1908 | ||||
Summary | 0018066: Sudo has 25 second delay when used in user data. Default network config is incorrect on boot for ami-0b2045146eb00b617 | ||||
Description | When I try to use the Centos 7 AMI in ap-southeast-2 for AWS it has problems without any modification. I can see 25 second delays every time a script may run that uses sudo in user data which slows down the ability to scale up a spot fleet. 2021-02-13 11:03:06 test sudo delay 2021-02-13 11:03:06 no sudo sudo 2021-02-13 11:03:31 sudo 2021-02-13 11:03:31 no sudo sudo 2021-02-13 11:03:56 sudo 2021-02-13 11:03:56 no sudo This may be related, I also see network interfaces which do not exist also have remnant configuration files from the AMI build. deleting this file and rebuilding the ami has no effect. ifcfg-eth0 should not exist, there is no interface matching this name. [root@ip-10-4-101-166 network-scripts]# ls ifcfg-ens5 ifdown ifdown-ippp ifdown-post ifdown-sit ifdown-tunnel ifup-bnep ifup-ipv6 ifup-plusb ifup-routes ifup-TeamPort init.ipv6-global ifcfg-eth0 ifdown-bnep ifdown-ipv6 ifdown-ppp ifdown-Team ifup ifup-eth ifup-isdn ifup-post ifup-sit ifup-tunnel network-functions ifcfg-lo ifdown-eth ifdown-isdn ifdown-routes ifdown-TeamPort ifup-aliases ifup-ippp ifup-plip ifup-ppp ifup-Team ifup-wireless network-functions-ipv6 [root@ip-10-4-101-166 network-scripts]# ifconfig ens5: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 9001 inet 10.4.101.166 netmask 255.255.255.0 broadcast 10.4.101.255 inet6 fe80::60:d6ff:fee7:5d2 prefixlen 64 scopeid 0x20<link> ether 02:60:d6:e7:05:d2 txqueuelen 1000 (Ethernet) RX packets 9264 bytes 13208806 (12.5 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 3070 bytes 222673 (217.4 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 ::1 prefixlen 128 scopeid 0x10<host> loop txqueuelen 1000 (Local Loopback) RX packets 108 bytes 9740 (9.5 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 108 bytes 9740 (9.5 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 I have read elsewhere that /etc/hosts must be configured properly and I believe this is valid from my user data: [root@ip-10-4-101-166 ~]# cat /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 127.0.0.1 ip-10-4-101-166.ap-southeast-2.compute.internal ip-10-4-101-166 | ||||
Steps To Reproduce | Start the ami with this user data: #!/bin/bash set -e # Send the log output from this script to user-data.log, syslog, and the console # From: https://alestic.com/2010/12/ec2-user-data-output/ exec > >(tee /var/log/user-data.log|logger -t user-data -s 2>/dev/console) 2>&1 # Log the given message. All logs are written to stderr with a timestamp. function log { local -r message="$1" local -r timestamp=$(date +"%Y-%m-%d %H:%M:%S") >&2 echo -e "$timestamp $message" } function has_yum { [[ -n "$(command -v yum)" ]] } if $(has_yum); then hostname=$(hostname -s) # in centos, failed dns lookup can cause commands to slowdown echo "127.0.0.1 $hostname.${aws_domain} $hostname" | tee -a /etc/hosts fi log "hostname: $(hostname)" log "hostname: $(hostname -f) $(hostname -s)" log "test sudo delay" log "no sudo" sudo echo "sudo" log "sudo" log "no sudo" sudo echo "sudo" log "sudo" log "no sudo" | ||||
Additional Information | I am using terraform to attach an EIP, which I can provide a snippet for: resource "aws_eip" "bastionip" { count = var.create_vpc ? 1 : 0 vpc = true instance = aws_instance.bastion[count.index].id } resource "aws_instance" "bastion" { count = var.create_vpc ? 1 : 0 ami = var.bastion_ami_id instance_type = var.instance_type key_name = var.aws_key_name subnet_id = tolist(var.public_subnet_ids)[0] vpc_security_group_ids = local.vpc_security_group_ids root_block_device { delete_on_termination = true } user_data = data.template_file.user_data_auth_client.rendered } data "template_file" "user_data_auth_client" { template = file("${path.module}/user-data-broken.sh") } | ||||
Tags | No tags attached. | ||||
abrt_hash | |||||
URL | |||||
I forgot to add that the dig command can resolve the hostname correctly so if DNS is the cause of slowing down sudo (as I see when I google the issue), it doesn't seem to help here. [root@ip-10-4-101-166 network-scripts]# dig $(hostname) ; <<>> DiG 9.11.4-P2-RedHat-9.11.4-26.P2.el7_9.3 <<>> ip-10-4-101-166.ap-southeast-2.compute.internal ;; global options: +cmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 8755 ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;ip-10-4-101-166.ap-southeast-2.compute.internal. IN A ;; ANSWER SECTION: ip-10-4-101-166.ap-southeast-2.compute.internal. 60 IN A 10.4.101.166 ;; Query time: 0 msec ;; SERVER: 10.4.0.2#53(10.4.0.2) ;; WHEN: Sat Feb 13 11:29:45 UTC 2021 ;; MSG SIZE rcvd: 81 |
|
Date Modified | Username | Field | Change |
---|---|---|---|
2021-02-13 11:25 | houdinifxtd | New Issue | |
2021-02-13 12:09 | houdinifxtd | Note Added: 0038244 |