CR  - or: Updating CentOS, forever :)

Yesterday the CentOS project released the Continuous Release (CR) repo which allows much faster tracking for Security updates that are added to RHEL.

It works as a yum repo that gets constant additions, but does not directly replace CentOS 5.7 which is planned to be released in the next 2 weeks or so.

To me, the release info seemed a little rushed and just to be on the safe side I'll start doing my updates right away.

I wrote down the steps and added some notes that might be helpful if you're updating more than just 1 or 2 boxes.

Impatient people with few servers and no care for experience, just look at the steps 1, 5 and 7. 

People who already know how to restore a system broken during a rushed update and don't want to do that today might want to read the other steps, too.

Step 1:

Download the repository for your cpu arch (i686  & x86_64 only)

32bit users do:

cd /tmp && wget

64bit users do:

cd /tmp && wget

Step 2:

Check you're already on CentOS 5.6 right now.

It's possible, but not fun, to jump more than 1 or 2 minor releases during a single upgrade.
If an RPM does any "fixes" in it's %pre or %post section it may fail badly.
So 5.5 to 5.6 or 5.7 is still quite ok, whereas 5.3 to 5.6 is already asking for major troble.

A almost current CentOS 5.6 would look like this:

# cat /etc/redhat-release ; uname -r
CentOS release 5.6 (Final)

The most current 5.6 kernel is version 238.19.1

Step 3 (optional):

Test everything is really up-to-date
Run yum update.

On my box there was the following updates:

  kernel-xen.x86_64 0:2.6.18-238.19.1.el5

  curl.x86_64 0:7.15.5-9.el5_6.3
  device-mapper-multipath.x86_64 0:0.4.7-42.el5_6.3
  glibc.i686 0:2.5-58.el5_6.4
  glibc.x86_64 0:2.5-58.el5_6.4
  glibc-common.x86_64 0:2.5-58.el5_6.4
  kpartx.x86_64 0:0.4.7-42.el5_6.3
  krb5-libs.i386 0:1.6.1-55.el5_6.2
  krb5-libs.x86_64 0:1.6.1-55.el5_6.2
  krb5-workstation.x86_64 0:1.6.1-55.el5_6.2
  ksh.x86_64 0:20100202-1.el5_6.6
  nscd.x86_64 0:2.5-58.el5_6.4
  perl.x86_64 4:5.8.8-32.el5_6.3


Something that is also helpful is to verify the system is shape:
Run rpm -Va to get a list of files that are updated / changed since their installation.
Keep an eye on those that are belonging to the base OS mostly.

# rpm -Va
.......T  c /etc/mime.types
prelink: /usr/lib/ at least one of file's dependencies has changed since prelinking
S.?.....    /usr/lib/
.......T  c /etc/httpd/conf.d/webalizer.conf

On a test box with ISPconfig this ran for around 2 minutes and returned no more than 50 files, much less if you don't count config files and man page cache.

Step 4 (recommended):

We're adding two yum plugins now:
yum-security plugin - this plugin is supposed to identify if you're missing any security updates.

Also we'll add the yum-download only plugin that allows to pre-fetch updates before installing them.

yum -y install yum-security yum-downloadonly

Step 5:

Installing the release rpm for CR will place the correct entry in /etc/yum/repos.d for yum to find.

32bit users:
rpm -i /tmp/centos-release-cr-5-6.el5.centos.1.i386.rpm
64bit users:
rpm -i /tmp/centos-release-cr-5-6.el5.centos.1.x86_64.rpm

Download the package list

yum makecache

Step 6 (optional): 

Predownload the updates to your local cache directory.
This will be helpful if you want to run a backup before the update, and also if you want the update to be as fast as possible. Using the downloadonly plugin we'll just fetch the updates to yum's cache directory.
It'll be able to grab them from there and also you'll be able to distribute them to other system's cache directories.

yum update -y --downloadonly

Don't forget to clear the cache later on.

Step 7:

This is now the simple update.

yum -y update

In my case, on one of the test boxes the / filesystem would have run out of space.
Reason: I only provision around 500MB of space for / and during the first yum update I already had downloaded yet another kernel. Together these two bloaty kernels used up all available space as you can see above.
As you can see I took the lazy and fast way of this situation.

Error Summary
Disk Requirements:
  At least 9MB needed on the / filesystem.
# cd /
# du -kx | sort -rn | head
369132  .
218222  ./lib
195754  ./lib/modules
97530   ./lib/modules/2.6.18-238.19.1.el5xen
97530   ./lib/modules/2.6.18-238.12.1.el5xen
# lvextend -L 1024 /dev/vg00/lvol00
  Extending logical volume lvol00 to 1.00 GB
  Logical volume lvol00 successfully resized
# /sbin/resize2fs  /dev/vg00/lvol00
resize2fs 1.39 (29-May-2006)
Filesystem at /dev/vg00/lvol00 is mounted on /; on-line resizing required
Performing an on-line resize of /dev/vg00/lvol00 to 1048576 (1k) blocks.
The filesystem on /dev/vg00/lvol00 is now 1048576 blocks long.

Here you can see the skipped waste of timedownload:

Transaction Summary
Install       1 Package(s)
Upgrade      57 Package(s)

Total size: 87 M
Downloading Packages:
Running rpm_check_debug

That's it your update is done!

About yum security

About plugin: It used to be a joke on CentOS but could now make a lot more sense once you get continous updates. But not just now. It's still unable to identify which updates are security related - see the related bugzilla at* *

There was a comment by KB Singh about this needing a lot of infrastructure to be put in place. From the manpage it looks like it requires to do the same patch management just like RH does, opening a BugZilla when there is a security issue and referencing it in the patch that's fixing it.

The main "problem" is that if the data is not correct the plugin cannot work correctly. For instance "--bz 123" will not fix BZ 123 if a package is updated to fix that BZ without referencing that it does so in the updateinfo.xml.

I'd put it in place anyway, hoping there will be a fix. -Otherwise, just another nail in the coffin for CentOS. -Maybe there is some licensing issue, maybe the info about the security updates in the updates.xml file is not GPLed and cannot be re-used. I wanna be clear about this: If you need quick security updates, need to be able to see which ones are missing, then CentOS is not the way to go at this time.

If they fix it, you can run:

yum --security check-update