Setup Yum Repository with Security Metadata

Define repository

Prepare the repo file on the server, so clients can download it.

cd /var/www/html/yum
cat <<'EOF' > hosting.repo
[hosting]
name=Hosting Delivery
baseurl=http://yum5.ipa.example.com/yum/hosting/
enabled=0
gpgcheck=0
EOF

Make or update repository

Use createrepo tool to make the repository. A wrapper script for creating or updating the existing repository is shown here.

tf=/usr/local/bin/updaterepo.sh
cat <<'EOF' > "${tf}"
#!/bin/sh
# reference:
#    https://gitlab.com/bgstack15/mirror/blob/master/usr/share/mirror/examples/rpm/update-smith122rpm.sh

# Prepare directory and files
test -z "${UR_REPODIR}" && UR_REPODIR=/var/www/html/yum/hosting
test -z "${UR_BASEURL}" && UR_BASEURL=http://yum5.ipa.example.com/yum/hosting
test -z "${UR_OWNERSHIP}" && UR_OWNERSHIP="root.root"
test -z "${UR_FILETYPES}" && UR_FILETYPES="rpm"

find "${UR_REPODIR}" -exec chown "${UR_OWNERSHIP}" {} + 1>/dev/null 2>&1
find "${UR_REPODIR}" -type f -exec chmod "0664" {} + 1>/dev/null 2>&1
find "${UR_REPODIR}" -type d -exec chmod "0775" {} + 1>/dev/null 2>&1
chmod 0754 "$0"
restorecon -RF "${UR_REPODIR}"

# Prepare repo for rpm
cd "${UR_REPODIR}"
createrepo -v -u "${UR_BASEURL}" --basedir "${UR_REPODIR}" --simple-md-filenames --no-database --update --pretty .
EOF

Run this script.

/usr/local/bin/updaterepo.sh

Manually make the security metadata

The security metadata that yum interprets is stored in updateinfo.xml.gz. To make this file and include it in repomd.xml, you need to prepare it and learn some information about it.

This is a trim example of updateinfo.xml. Please see the epel metadata for a full example. I do not have an automatic process for generating this file yet.

tf=updateinfo.xml
cat <<'EOF' > "${tf}"
<?xml version="1.0" encoding="UTF-8"?>
<updates>
  <update status="final" type="security" version="1" from="bgstack15@gmail.com">
    <id>HELP-210217</id>
    <title>bgscripts-core update</title>
    <release>Enterprise Linux 7</release>
    <issued date="2018-04-02"/>
    <rights>CC-BY-SA 4.0</rights>
    <description>bgscripts-core
[1.3-8]
- latest version from upstream
</description>
    <solution>This update is internal to the company.</solution>
    <references>
      <reference href="https://gitlab.com/bgstack15/bgscripts" type="self" title="bgscripts-core" />
    </references>
    <pkglist>
      <collection short="bgscripts">
        <name>bgscripts suite</name>
        <package name="bgscripts-core" version="1.3-8" release="" epoch="0" arch="noarch">
          <filename>bgscripts-core-1.3-8.noarch.rpm</filename>
          <sum type="md5">eaa20075720bf12d6e837a4f546241ab</sum>
        </package>
     </collection>
    </pkglist>
  </update>
</updates>
EOF

Update the repo metadata to include updateinfo.xml

A yum repository includes metadata of the package metadata, and stores this meta-metadata in repomd.xml. Insert the metadata for this new file, updateinfo.xml in the repomd file.
This script is an update version of updaterepo.sh, which was listed earlier in this document.

tf=/usr/local/bin/updaterepo.sh
        cat <<'EOF' > "${tf}"
#!/bin/sh
# reference:
#    https://gitlab.com/bgstack15/mirror/blob/master/usr/share/mirror/examples/rpm/update-smith122rpm.sh

# Prepare directory and files
test -z "${UR_REPODIR}" && UR_REPODIR=/var/www/html/yum/hosting
test -z "${UR_BASEURL}" && UR_BASEURL=http://yum5.ipa.example.com/yum/hosting
test -z "${UR_OWNERSHIP}" && UR_OWNERSHIP="root.root"
test -z "${UR_FILETYPES}" && UR_FILETYPES="rpm"
test -z "${UR_UPDATEINFO_INPUT}" && UR_UPDATEINFO_INPUT=/var/www/html/yum/build-hosting-repo/updateinfo.xml

find "${UR_REPODIR}" -exec chown "${UR_OWNERSHIP}" {} + 1>/dev/null 2>&1
find "${UR_REPODIR}" -type f -exec chmod "0664" {} + 1>/dev/null 2>&1
find "${UR_REPODIR}" -type d -exec chmod "0775" {} + 1>/dev/null 2>&1
chmod 0754 "$0"
restorecon -RF "${UR_REPODIR}"

# Prepare basic repo
cd "${UR_REPODIR}"
createrepo -v -u "${UR_BASEURL}" --basedir "${UR_REPODIR}" --simple-md-filenames --no-database --update --pretty .

# Inject custom updateinfo
# this task assumes the repomd file does not include node <data type="updateinfo"> yet.
UR_repomd="${UR_REPODIR}/repodata/repomd.xml"
UR_updateinfo_gz_short="repodata/updateinfo.xml.gz"
UR_updateinfo_gz="${UR_REPODIR}/${UR_updateinfo_gz_short}"

if ! test -e "${UR_UPDATEINFO_INPUT}" ;
then
   # file is absent, so decide how to fail.
   :
else
   # file exists, so continue with custom injection

   # learn open-size and open-checksum
   UR_updateinfo_opensize="$( /usr/bin/stat -c "%s" "${UR_UPDATEINFO_INPUT}" )"
   UR_updateinfo_openchecksum="$( /usr/bin/sha256sum "${UR_UPDATEINFO_INPUT}" | awk '{print $1}' )"

   # compress file and learn size and checksum
   /usr/bin/gzip < "${UR_UPDATEINFO_INPUT}" > "${UR_updateinfo_gz}"
   UR_updateinfo_size="$( /usr/bin/stat -c "%s" "${UR_updateinfo_gz}" )"
   UR_updateinfo_checksum="$( /usr/bin/sha256sum "${UR_updateinfo_gz}" | awk '{print $1}' )"
   UR_updateinfo_timestamp="$( /usr/bin/stat -c "%Y" "${UR_updateinfo_gz}" )"

   # insert information into repomd
   this_string="<data type=\"updateinfo\">
  <checksum type=\"sha256\">${UR_updateinfo_checksum}</checksum>
  <open-checksum type=\"sha256\">${UR_updateinfo_openchecksum}</open-checksum>
  <location xml:base=\"${UR_BASEURL}\" href=\"${UR_updateinfo_gz_short}\"/>
  <timestamp>${UR_updateinfo_timestamp}</timestamp>
  <size>${UR_updateinfo_size}</size>
  <open-size>${UR_updateinfo_opensize}</open-size>
</data>"

   {
      sed -r -e '/<\/repomd>/d' "${UR_repomd}"
      printf "%s\n%s\n" "${this_string}" "</repomd>"
   } > "${UR_repomd}.$$"
   /bin/touch --reference "${UR_repomd}" "${UR_repomd}.$$"
   /bin/mv -f "${UR_repomd}.$$" "${UR_repomd}"
fi
EOF

Summary

Using bash to modify xml files is obviously not ideal. However, this xml file is simple enough so this ugly mechanism suffices. For teams that know how to manage custom yum repositories and also want to just use yum update –security, this process should be a good basis or even complete solution!

Appendices

Appendix A: http proxy

If you use an http proxy for your yum traffic, the proxy might cache old versions of the metadata or package files. A quick and dirty way to clean up a squid proxy of the metadata file follows.

time squidclient -h localhost -r -p 3128 -m PURGE http://yum5.ipa.example.com/yum/hosting/repodata/updateinfo.xml.gz

Squid unfortunately does not allow recursive purging, so you will have to loop over all the metadata files and any package files you want to ensure get cleared.

References

Local file /var/cache/yum/x86_64/7Server/epel/69b82df00108c0ac8ac82fafbd0f3b89cc98d8dfe4fa350af7a23331a878fea2-updateinfo.xml.bz2

Advertisements

Find which package provides a file – Yum and Apt

On Fedora-like systems, the default package manager is yum (or dnf for Fedora 22+). The main way you use it is to install packages and auto-resolve the dependencies.
yum install irfan
Similarly, apt-get on the Debian family of GNU/Linux installs packages with this command.
apt-get install irfan
Now, let’s say you are looking for which package installs a particular file. The command for yum is:
yum provides */filename
On Ubuntu, part of the Debian family, you need to install the package apt-file before you can do similar lookup commands.
apt-get install apt-file
Now you can search with:
apt-file search */filename

Update just one repository on Linux

So if you build your own packages (rpm/dpkg/other), you might have your own repository of packages for your systems. And if you want to tell your hosts to update just the one repository, you might be looking for a solution.

A user on Ask Ubuntu had a fantastic answer for Ubuntu/debian flavor: http://askubuntu.com/questions/65245/apt-get-update-only-for-a-specific-repository/197532#197532.

For those of you who don’t want to click through, here is a summary of the entries, collapsed down to just one file (which might not be the best solution, but it works).
Add these lines to ~/.bashrc:
update-repo() {
for source in "$@"; do
sudo apt-get update -o Dir::Etc::sourcelist="sources.list.d/${source}" \
-o Dir::Etc::sourceparts="-" -o APT::Get::List-Cleanup="0"
done
}

_ppa_lists(){
local cur
_init_completion || return

COMPREPLY=( $( find /etc/apt/sources.list.d/ -name "*$cur*.list" \
-exec basename {} \; 2> /dev/null ) )
return 0
} &&
complete -F _ppa_lists update-repo

With rpm-based systems like RHEL/CentOS/Fedora, you are using yum or dnf. Here’s my dnf implementation:

_command_dnf=yum
which dnf 1>/dev/null 2>&1 && _command_dnf="$( which dnf 2>/dev/null )"

# update-repo command for dnf update just one repository
update-repo() {
case "${_command_dnf}" in
*dnf)
for source in "$@"; do
sudo "${_command_dnf}" check-update -q --refresh --disablerepo=* --enablerepo="${source}"
done
;;
*yum)
for source in "$@"; do
sudo "${_command_dnf}" clean metadata -q --disablerepo=* --enablerepo="${source}" -q; yum check-update -q --disablerepo=* --enablerepo="${source}"
done
;;
esac
}

# autocomplete for update-repo
_repo_lists() {
local cur
_init_completion || return
COMPREPLY=( $( grep -hoiE -- "^\[.*\]" /etc/yum.repos.d/* | tr -d '[]' | grep -E "^${2:-.*}" ) )
return 0
} &&
complete -F _repo_lists -o filenames update-repo

The functions whose names start with an underscore are the auto-complete commands. Who doesn’t like a good tab auto-completion? Now that I learned how to do my own bash-completion, I’ll be doing it a lot more!

Update yum repo with an easy script

Similar to how you can Build an apt repository on CentOS and update it with new packages with a simple script, you can do the same with a yum rpm repository.

cat <<'EOFUPDATE' > ./update-yumrepo.sh
#!/bin/sh

# working directory
repodir=/mnt/mirror/bgscripts/
cd ${repodir}
chmod 0644 *rpm 1>/dev/null 2>&1

# create the package index
createrepo .
EOFUPDATE
chmod u+x ./update-yumrepo.sh