View Issue Details

IDProjectCategoryView StatusLast Update
0016408mirrorunsortedpublic2019-09-20 10:48
Reportergenin_d 
PrioritynormalSeveritycrashReproducibilityalways
Status newResolutionopen 
PlatformVagrantOSCentOSOS Version7.6.1810
Summary0016408: Vagrant cant install box with centos7 due to missing file on the mirrir
DescriptionAfter some moment cant start centos7 box in Vagrant. Its messaging what some file miss on the mirror:
http://vault.centos.org/7.6.1810/os/x86_64/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found

I was checked link manually from internet browser and file is really missing there.

Steps To Reproduce1. select in vagrantfile box "centos/7"
2. vagrant up

Additional Information$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'centos/7'...
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'centos/7' version '1905.1' is up to date...
==> default: Setting the name of the VM: vagrant-configs_default_1568752865153_72095
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default:
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default:
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
[default] No Virtualbox Guest Additions installation found.
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.corbina.net
 * extras: centos-mirror.rbc.ru
 * updates: centos-mirror.rbc.ru
Resolving Dependencies
--> Running transaction check
---> Package centos-release.x86_64 0:7-6.1810.2.el7.centos will be updated
---> Package centos-release.x86_64 0:7-7.1908.0.el7.centos will be an update
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package Arch Version Repository Size
================================================================================
Updating:
 centos-release x86_64 7-7.1908.0.el7.centos base 26 k

Transaction Summary
================================================================================
Upgrade 1 Package

Total download size: 26 k
Downloading packages:
No Presto metadata available for base
Public key for centos-release-7-7.1908.0.el7.centos.x86_64.rpm is not installed
warning: /var/cache/yum/x86_64/7/base/packages/centos-release-7-7.1908.0.el7.centos.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID f4a80eb5: NOKEY
Retrieving key from file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Importing GPG key 0xF4A80EB5:
 Userid : "CentOS-7 Key (CentOS 7 Official Signing Key) <security@centos.org>"
 Fingerprint: 6341 ab27 53d7 8a78 a7c2 7bb1 24c6 a8a7 f4a8 0eb5
 Package : centos-release-7-6.1810.2.el7.centos.x86_64 (@anaconda)
 From : /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating : centos-release-7-7.1908.0.el7.centos.x86_64 1/2
  Cleanup : centos-release-7-6.1810.2.el7.centos.x86_64 2/2
  Verifying : centos-release-7-7.1908.0.el7.centos.x86_64 1/2
  Verifying : centos-release-7-6.1810.2.el7.centos.x86_64 2/2

Updated:
  centos-release.x86_64 0:7-7.1908.0.el7.centos

Complete!
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
 * base: mirror.corbina.net
 * extras: centos-mirror.rbc.ru
 * updates: centos-mirror.rbc.ru
http://vault.centos.org/7.6.1810/os/x86_64/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.
To address this issue please refer to the below wiki article

https://wiki.centos.org/yum-errors

If above article doesn't help to resolve this issue please use https://bugs.centos.org/.
Tags7.6.1810, centos7, missing dependency

Activities

tcooper

tcooper

2019-09-17 22:19

reporter   ~0035111

This, along with already closed Issue (0016407), indicate what may be a common problem with automated build tools expecting to find files in repositories and they are missing around the time the new update is released.

Our build system encountered this when yum update rev'd our OS version unexpectedly.

It is clear from previous bug reports and notes that the observed behavior is expected.

I have been manually watching the 7.7 update progress fairly closely and missed the change which appeared to start at 2019-09-15 01:00 when the target directory was created.

We had deployed test nodes from the updates in the CR repository and, finding unresolved issues with 3rd party builds of kernel modules, would NOT have elected to change our AMI builds had we know the update had landed or was landing. Now we need to roll back our builds to 7.6.1810 and the RPMs have not yet landed in vault.centos.org. It is a problem of our own creation but I'm wondering what I can be watching to prevent this in the future.

What I'm wondering is where the mirror change was announced and when was it announced.

Neither centos-mirror or centos-mirror-announce lists have any record that I can see.

Thanks
SevernE86

SevernE86

2019-09-18 07:40

reporter   ~0035112

According to the responses for issue https://bugs.centos.org/view.php?id=16407, this is due to the newest "old release" not getting copied back to the vault immediately after a new release is pushed and should be resolved in a week or two. As a side note, I'm a little disappointed at how flippantly that issue was treated given how - as seen here - it breaks Vagrant provisioning for the CentOS-published box. A workaround I've found is to use the generic/centos7 boxes for the time being - at least those will go through the default provisioning process without blowing up.
stefanpeturs

stefanpeturs

2019-09-19 09:32

reporter   ~0035135

I am experiencing the same issue.
Is there any news related to solving this?
pemosi

pemosi

2019-09-19 10:31

reporter   ~0035137

Same here. Using Vagrant and settings up a few "centos/7" boxes. Trying to learn how to configure everything using Ansible.
When this stuff just breaks all of a sudden it's quite annoying.
I'm also curious about the reply in bug 16407, as noted above.
https://bugs.centos.org/view.php?id=16407#c35109
"Yes, it's normal. The old release gets copied to vault after a week or two. It's not automatically there as soon as the next one comes out. Never has been, never will be."
Why??
Is it technically impossible to copy the current release to the vault before releasing the new one so that all VM boxes that rely on the now older version won't all break instantly?
I guess once the new Vagrant box version is out it will use the current version (if I understand the problem/workflow correctly), but still, is there anything preventing an early copy of the current version to the vault in anticipation of the release of the new version?
I assume it would prevent these hickups.

@stefanpeturs As mentioned above, switching to "generic/centos7" meanwhile solves the issue, but keep in mind the firewalld is running by default there, took me a while to figure out why lots of things stopped working when I switched... Also, the virtualbox guest additions plugin have a version mismatch with the pre-installed guest additions on "generic/centos7", so there's some more headache.
SevernE86

SevernE86

2019-09-19 10:39

reporter   ~0035138

@pemosi
I tried downgrading the CentOS box to v1902.01, and it still failed with the same issue. The problem isn't that the Vagrant box is "obsolete" - it's that the lag in updating the new legacy repositories breaks the default workflow for presumably any of the Vagrant boxes that CentOS publishes. Also, what issues are you having with the guest additions with the generic box? I haven't had any.
pemosi

pemosi

2019-09-19 10:49

reporter   ~0035139

@SevernE86 I'm not exactly sure how this works, but if you look at https://app.vagrantup.com/centos/boxes/7 and check the releases, you'll see that v1905.1, v1902.1, ..., all the way back to v1811.01 say "based on CentOS 7.6.1811".
So I assume all those versions will fail because they rely on 7.6.1811.
Hopefully, the next Vagrant box release will be based on CentOS 7.7.1911 and it will work again (or we have to wait until 7.6.1811 makes it to the vault).
Well, either that or I have misunderstood the whole thing :)
Please correct me if I'm wrong.
stefanpeturs

stefanpeturs

2019-09-19 11:40

reporter   ~0035141

Thanks for the update @pemosi - I'll try to make it work with the generic version.
I do agree with you that it seems obscure that it should take weeks to copy the old release to a vault - as it is actually breaking the construction of the machine.
Fingers crossed we won't have to wait for too long.
marcelo.altmann

marcelo.altmann

2019-09-19 16:36

reporter   ~0035143

I'm seeing the same issue with vagrant. It's a bit of surprise that this is considered "expected" since it's breaking vagrant to boot-up properly.
danls

danls

2019-09-19 19:26

reporter   ~0035146

I lost the entire day yesterday chasing this. In my case, the culprit was the vbguest plugin which tries to update the CentOS repos (see vagrant.d/gems/2.4.6/gems/vagrant-vbguest-0.19.0/lib/vagrant-vbguest/installers/centos.rb). I just switched to the "generic/centos7" box and the issue was fixed. I also added a comment on this issue to the maintainers of vbguest (https://github.com/dotless-de/vagrant-vbguest/issues/325) because it seems likely that this will happen again for every CentOS update going forward.
danls

danls

2019-09-19 19:27

reporter   ~0035147

Sorry, the correct issue is: https://github.com/dotless-de/vagrant-vbguest/issues/320
pemosi

pemosi

2019-09-20 10:43

reporter   ~0035157

I saw http://vault.centos.org/7.6.1810/os/x86_64/repodata/repomd.xml was now acessible so I tried switching back to "centos/7" and everything works again.
I'm just knowing that next release this thing will probably happen again.
SevernE86

SevernE86

2019-09-20 10:48

reporter   ~0035158

Kind of confirms what everybody's been saying that this was due to CentOS's workflow when upgrading versions. The question is, why does there have to be a gap between releasing a new version of CentOS and setting up the legacy repository? With all of the options that are out there for automated deployment, it's hard to think of a reason why this should still be occurring besides inertia for the status quo.

Issue History

Date Modified Username Field Change
2019-09-17 21:12 genin_d New Issue
2019-09-17 21:12 genin_d Tag Attached: 7.6.1810
2019-09-17 21:12 genin_d Tag Attached: centos7
2019-09-17 21:12 genin_d Tag Attached: missing dependency
2019-09-17 22:19 tcooper Note Added: 0035111
2019-09-18 07:40 SevernE86 Note Added: 0035112
2019-09-19 09:32 stefanpeturs Note Added: 0035135
2019-09-19 10:31 pemosi Note Added: 0035137
2019-09-19 10:39 SevernE86 Note Added: 0035138
2019-09-19 10:49 pemosi Note Added: 0035139
2019-09-19 11:40 stefanpeturs Note Added: 0035141
2019-09-19 16:36 marcelo.altmann Note Added: 0035143
2019-09-19 19:26 danls Note Added: 0035146
2019-09-19 19:27 danls Note Added: 0035147
2019-09-20 10:43 pemosi Note Added: 0035157
2019-09-20 10:48 SevernE86 Note Added: 0035158