Apt-get show one package per line

tl;dr

Use -V.

sudo apt-get -V upgrade

The story

As a part of my transition from rpm-based distros (Fedora and CentOS) to dpkg-based ones (Devuan), one of my pet peeves is the output of the package manager. Apt-get itself is actually quite decent. My biggest problem with it is the wall of text upon upgrading:

$ sudo apt-get dist-upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
  clearlooks-phenix-darkpurpy-theme darkpurpy-icon-theme eject fp-compiler-3.0.4 fp-docs-3.0.4 fp-ide-3.0.4
  fp-units-base-3.0.4 fp-units-db-3.0.4 fp-units-fcl-3.0.4 fp-units-fv-3.0.4 fp-units-gfx-3.0.4 fp-units-math-3.0.4
  fp-units-misc-3.0.4 fp-units-multimedia-3.0.4 fp-units-net-3.0.4 fp-units-rtl-3.0.4 fp-utils-3.0.4
  fpc-source-3.0.4 g++-6 g++-8 gdisk gir1.2-appindicator-0.1 gnome-icon-theme-extras gnome-mime-data guile-2.0-libs
  irqbalance jellyfin-ffmpeg liba52-0.7.4-dev libappindicator1 libart-2.0-2 libass5 libatasmart4 libavformat57
  libavresample3 libbind9-140 libbluray1 libbonobo2-0 libbonobo2-common libbonoboui2-0 libbonoboui2-common
  libboost-atomic1.62.0 libboost-chrono1.62.0 libboost-date-time1.62.0 libboost-serialization1.62.0
  libboost-system1.62.0 libboost-thread1.62.0 libcaca-dev libcdt5 libcgraph6 libck-connector0 libcrystalhd3:i386
  libcupsfilters1 libcupsimage2 libdca-dev libdevel-globaldestruction-perl libdns162 libdouble-conversion1
  libdts-dev libdvdread4 libebook-contacts-1.2-2 libecal-1.2-19 libedataserver-1.2-23 libegl1-mesa libevent-2.0-5
  libevent-2.1-6 libevent-core-2.1-6 libevent-extra-2.1-6 libevent-openssl-2.1-6 libevent-pthreads-2.1-6 libexo-1-0
  libflac-dev libfluidsynth1 libforms-dev libforms2 libgcr-3-common libgd-dev libgit2-27 libgles2-mesa libglew2.0
  libgnome-2-0 libgnome-desktop-3-17 libgnome-keyring-common libgnome-keyring0 libgnome2-common libgnomecanvas2-0
  libgnomecanvas2-common libgnomeui-0 libgnomeui-common libgnomevfs2-0 libgnomevfs2-common libgnomevfs2-extra
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
  libpcre2-posix0
The following NEW packages will be installed:
  glib-networking:i386 gstreamer1.0-plugins-good:i386 gstreamer1.0-x:i386 libaa1:i386 libavc1394-0:i386
  libbrotli1:i386 libbz2-1.0:i386 libcaca0:i386 libcairo-gobject2:i386 libcrypt1 libcrypt1:i386 libcrypt1-dev
  libdv4:i386 libfluidsynth2 libgudev-1.0-0:i386 libiec61883-0:i386 libisl22 libjim0.79 libncursesw6:i386
  libpcre2-posix2 libprocps7 libproxy1v5:i386 libpsl5:i386 libraw1394-11:i386 libshout3:i386 libslang2:i386
The following packages have been kept back:
  elogind libblkid1:i386 libelogind0 libmount1:i386 libpam-elogind libuuid1:i386
The following packages will be upgraded:
  base-passwd cpp-8 cpp-9 g++-8 g++-9 galera-3 gcc-8 gcc-8-base gcc-8-base:i386 gcc-9 gcc-9-base gcc-9-base:i386 git
  git-man gstreamer1.0-plugins-bad i965-va-driver i965-va-driver:i386 lib32gcc1 lib32stdc++6 libasan5 libatomic1
  libatomic1:i386 libc-dev-bin libc6 libc6:i386 libc6-dbg libc6-dev libc6-i386 libcc1-0 libebml4v5 libgcc-8-dev
  libgcc-9-dev libgcc1 libgcc1:i386 libgomp1 libgomp1:i386 libgstreamer-plugins-bad1.0-0 libitm1 liblsan0 libmpx2
  libobjc-8-dev libobjc4 libpci3 libpcre2-16-0 libpcre2-32-0 libpcre2-8-0 libpcre2-8-0:i386 libpcre2-dev libpst4
  libquadmath0 libsdl-mixer1.2 libsdl-mixer1.2-dev libsmbclient libssh-gcrypt-4 libstd-rust-dev libstdc++-8-dev
79 upgraded, 35 newly installed, 1 to remove and 6 not upgraded.
Need to get 401 MB of archives.
After this operation, 358 MB of additional disk space will be used.
Do you want to continue? [Y/n] n
Abort.

The wall of text for each section (no longer required, REMOVED, NEW, upgraded) is hard to parse.

The helpful folks over at the Devuan forum answered my plea for help. The answer is so simple I was very embarrassed I hadn’t thought of it. RTFM, indeed.

$ sudo apt-get -V dist-upgrade
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages were automatically installed and are no longer required:
   clearlooks-phenix-darkpurpy-theme (7.0.2-1+devuan2.0)
   darkpurpy-icon-theme (1.0)
   eject (2.1.5+deb1+cvs20081104-14+b1)
   fp-compiler-3.0.4 (3.0.4+dfsg-23)
   fp-docs-3.0.4 (3.0.4+dfsg-23)
   fp-ide-3.0.4 (3.0.4+dfsg-23)
   fp-units-base-3.0.4 (3.0.4+dfsg-23)
   fp-units-db-3.0.4 (3.0.4+dfsg-23)
   fp-units-fcl-3.0.4 (3.0.4+dfsg-23)
   fp-units-fv-3.0.4 (3.0.4+dfsg-23)
[...truncated...]
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
   libpcre2-posix0 (10.34-1)
The following NEW packages will be installed:
   glib-networking:i386 (2.62.1-1)
   gstreamer1.0-plugins-good:i386 (1.16.2-2)
   gstreamer1.0-x:i386 (1.16.2-2)
   libaa1:i386 (1.4p5-46+b1)
   libavc1394-0:i386 (0.5.4-5)
   libbrotli1:i386 (1.0.7-5+b1)
   libbz2-1.0:i386 (1.0.8-2)
[...truncated...]
The following packages have been kept back:
   elogind (239.3+20190131-1 => 241.3-3)
   libblkid1:i386 (2.32.1-0.1+devuan2.1 => 2.33.1-0.1+devuan1)
   libelogind0 (239.3+20190131-1 => 241.3-3)
   libmount1:i386 (2.32.1-0.1+devuan2.1 => 2.33.1-0.1+devuan1)
   libpam-elogind (239.3+20190131-1 => 241.3-3)
   libuuid1:i386 (2.32.1-0.1+devuan2.1 => 2.33.1-0.1+devuan1)
[...truncated...]
The following packages will be upgraded:
   base-passwd (3.5.46 => 3.5.47)
   cpp-8 (8.3.0-25 => 8.3.0-26)
   cpp-9 (9.2.1-20 => 9.2.1-21)
   g++-8 (8.3.0-25 => 8.3.0-26)
   g++-9 (9.2.1-20 => 9.2.1-21)
   galera-3 (25.3.28-1 => 25.3.28-2)
[...truncated...]
79 upgraded, 35 newly installed, 1 to remove and 6 not upgraded.
Need to get 401 MB of archives.
After this operation, 358 MB of additional disk space will be used.
Do you want to continue? [Y/n] n
Abort.

Tada.

How I host a Devuan ceres mirror for myself

Building on what I learned a long time ago for building an apt repo on CentOS, and some guidance from the Devuan forum, I have now validated my local Devuan mirror.

I chose to use debmirror because it can be limited to just certain suites (that’s Debian-speak for “version”) and package repositories. I exclusively use Devuan ceres (testing) because I need the FreeIPA packages. And also CentOS had an rpm of debmirror.

I run my script on Sunday and weekdays.
Cron job:

0	9	*	*	0-5	root	/etc/installed/debmirror-devuan.sh 1>/dev/null 2>&1

And my script:

#!/usr/bin/env sh
# File: /etc/installed/debmirror-devuan.sh
# Startdate: 2019-10-02 21:08
# Usage: by cron
{ debmirror --keyring /var/storage1/shares/public/Support/Platforms/devuan/keyrings/devuan-archive-keyring.gpg --verbose --config-file=/etc/debmirror.conf.devuan-ceres ; } 2>&1 | tee -a /var/storage1/shares/public/Support/Systems/storage1/var/log/debmirror/debmirror.devuan.$( date "+%F" ).log

Pretty basic, right? I had to prep the keyrings directory with the contents of a devuan install’s /usr/share/keyrings, but calling the debmirror was pretty easy. I pipe it to some logging as well.

Of course the debmirror.conf for this is extremely relevant.

# File: debmirror.conf.devuan-ceres
# Location: storage1:/etc/
# Author: bgstack15
# Startdate: 2019-09-27 22:43
# Title: Debmirror config for Devuan Ceres i386 and amd64 only
# Purpose: To provide a local mirror of devuan unstable so my clients can traverse only the local network
# History:
# Usage:
#    { time sudo debmirror --keyring /var/storage1/shares/public/Support/Platforms/devuan/keyrings/devuan-archive-keyring.gpg --verbose --config-file=/etc/debmirror.conf.devuan-ceres ; } 2>&1 | tee -a /var/storage1/shares/public/Support/Systems/storage1/var/log/debmirror/debmirror.devuan.$( date "+%F" ).log
# Reference:
#    https://dev1galaxy.org/viewtopic.php?id=1152
#    https://dev1galaxy.org/viewtopic.php?id=1571
#    https://duckduckgo.com/?q=devuan+debmirror
#    http://pkgmaster.devuan.org/devuan_mirror_walkthrough.txt
#    http://ftp.debian.org/debian/dists/sid/main/
#    http://pkgmaster.devuan.org/devuan/dists/ceres/
#    http://pkgmaster.devuan.org/merged/dists/ceres/
#    how to get working keyring https://ilostmynotes.blogspot.com/2008/05/debmirror-of-ubuntu-archive-validating.html
# Improve:
# Documentation:
#    To get the gpg keyring, copy d2-02a:/usr/share/keyrings/ to /mnt/public/Support/Platforms/devuan/keyrings.
#    This setup will copy all the debian+devuan packages into one devuan/ directory. No amprolla or apache rewrites are necessary.
# Location of the local mirror (use with care)
$mirrordir="/mnt/narcissus/mirror/devuan";

# Output options
$verbose=0;
$progress=0;
$debug=0;

# Download options
$host="packages.devuan.org";
$user="anonymous";
$passwd="anonymous@";
$remoteroot="merged";
$download_method="http";
@dists="ceres";
@sections="main,main/debian-installer,contrib,non-free";
@arches="i386,amd64";
# @ignores="";
# @excludes="";
# @includes="";
# @excludes_deb_section="";
# @limit_priority="";
$omit_suite_symlinks=0;
$skippackages=0;
# @rsync_extra="doc,tools";
$i18n=0;
$getcontents=1;
$do_source=1;
$max_batch=0;
# @di_dists="dists";
# @di_archs="arches";

# Save mirror state between runs; value sets validity of cache in days
$state_cache_days=0;

# Security/Sanity options
$ignore_release_gpg=0;
$ignore_release=0;
$check_md5sums=0;
$ignore_small_errors=0;

# Cleanup
$cleanup=0;
$post_cleanup=1;

# Locking options
$timeout=300;

# Rsync options
$rsync_batch=200;
$rsync_options="-aIL --partial";

# FTP/HTTP options
$passive=0;
# $proxy="http://proxy:port/";

# Dry run
$dry_run=0;

# Don't keep diff files but use them
$diff_mode="use";

# The config file must return true or perl complains.
# Always copy this.
1;

References

Weblinks

  1. How to mirror ascii? / Other Issues / Dev1 Galaxy Forum
  2. Make a mirror of devuan package repository with a modified debmirror / Documentation / Dev1 Galaxy Forum
  3. http://pkgmaster.devuan.org/devuan_mirror_walkthrough.txt
  4. http://ftp.debian.org/debian/dists/sid/main/
  5. http://pkgmaster.devuan.org/devuan/dists/ceres/
  6. http://pkgmaster.devuan.org/merged/dists/ceres/
  7. how to get working keyring: Technical notes, my online memory: Debmirror of ubuntu archive, with valid gpg keys

Web searches

  1. devuan debmirror

Fixing problem Repository ceres InRelease changed its Label value from Master to Devuan

tl;dr

rm /var/lib/apt/lists/*

The fix

If you encounter an error that resembles the following, on Devuan GNU/Linux, there is a fix for it!

# sudo apt-get update
Reading package lists... Done
E: Repository 'http://packages.roundr.devuan.org/merged ceres InRelease' changed its 'Label' value from 'Master' to 'Devuan'
N: This must be accepted explicitly before updates for this repository can be applied. See apt-secure(8) manpage for details.

There’s a new label in use, it seems. Big deal, except for the fact you can’t really get around it. The apt-secure(8) page does not seem to provide any answers.

To view the current labels for the enabled repos:

# apt policy
Package files:
 100 /var/lib/dpkg/status
     release a=now
 500 http://packages.devuan.org/merged ceres/non-free i386 Packages
     release v=1.0.0,o=Devuan,a=unstable,n=ceres,l=Master,c=non-free,b=i386
     origin packages.devuan.org
 500 http://packages.devuan.org/merged ceres/contrib i386 Packages
     release v=1.0.0,o=Devuan,a=unstable,n=ceres,l=Master,c=contrib,b=i386
     origin packages.devuan.org
 500 http://packages.devuan.org/merged ceres/main i386 Packages
     release v=1.0.0,o=Devuan,a=unstable,n=ceres,l=Master,c=main,b=i386
     origin packages.devuan.org
Pinned packages:

The fix is to remove the cached lists for the repositories and fetch it all again.

rm /var/lib/apt/lists/*

That’s all there is to it! Then run apt-get update again, and you’re back on your way.

References

Weblinks

  1. man page apt_preferences(5)

Local resources

  1. bash autocomplete for apt and apt-get

Unattended software updates on Devuan

updated 2019-06-02

Devuan, as a fork of debian that uses sysvinit (or another– your choice), still uses debian-based utilities. I come from the Fedora/Red Hat/CentOS rpm-based family of distributions, and I struggle with the dpkg-based package management on occasion.

I really dislike how the software upgrades will sometimes pause in the middle, to display the changelog. If I wanted a changelog, I’d go read it! When I issue a command to update packages, I want to walk away, and come back, and it be done, not get stuck at 20% because openssh changed some defaults and wants to tell me. It emails me anyway! I find the defaults of apt-get to be not sane.

Here is how to configure apt-get to run without pausing to display duplicate information or ask you questions.

export DEBIAN_FRONTEND=noninteractive
apt-get -q -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" upgrade

References

Weblinks

  1. https://obviate.io/2008/12/04/truly-non-interactive-unattended-apt-get-install/
  2. http://www.microhowto.info/howto/perform_an_unattended_installation_of_a_debian_package.html

Find which package provides a file – Yum and Apt

Last updated 2019-06-26

Yum-based distros

On Fedora-like systems, the default package manager is yum (or dnf for Fedora 22+). The main way you use it is to install packages and auto-resolve the dependencies.

yum install irfan

Similarly, apt-get on the Debian family of GNU/Linux installs packages with this command.

apt-get install irfan

Now, let’s say you are looking for which package installs a particular file. The command for yum is:

yum provides */filename

If you want to use rpm, that is available too.

rpm -qf /etc/sudoers

Apt-based distros

On apt-based distros (Debian, Devuan, Ubuntu, etc.), you need to install the package apt-file before you can do similar lookup commands.

apt-get install apt-file

Now you can search with:

apt-file search */filename

You can use the lower-level package manager:

dpkg-query -S /usr/share/filename

Building an apt repository on CentOS

Apt is a dpkg management tool used by Debian and its offpsring, particularly Ubuntu and Linux Mint.
CentOS is from the RHEL/Fedora side of the Linux family tree and uses yum (and dnf nowadays).
Making a simple, signed apt repository on centos (or manually, on any system really) is possible. This is how to do it.

Building an apt repository

So you have packages you want to make available for your LAN or wherever. This document will show you how to make a directory with all the right parts for an apt repository that is gpg-signed (to stave off that annoying “Do you trust the source?” question).

Preparing gpg keys

Note: generating new keys can require some time orand entropy generation.

# as root; no sudo!
gpg --gen-key

The first time you run gpg –gen-key, break it after it has generated some directories and files.
Add the SHA256 requirement to the gpg conf.

cat <<'EOF' >> ~/.gnupg/gpg.conf
cert-digest-algo SHA256
digest-algo SHA256
EOF

Reference: Weblink 3
Run gpg again and this time follow the prompts to generate a key.

gpg --gen-key

If you need to generate extra entropy, consider running some mundane tasks in another terminal.

while true; do dd if=/dev/sda of=/dev/zero; find / | xargs file >/dev/null 2>&1; done

Just break it off when you get the gpg keys you need.
Export the keys as needed with these commands.

gpg --list-keys
# take the key name shown and do this:
gpg --output debian-repo-public.gpg --armor --export 123456AB
gpg --output debian-repo-private.gpg --armor --export-secret-key 123456AB

So the end state of this section is to have the public key as a file, preferably in the repository directory.

Installing required packages

Install epel-release which wil lget you the dpkg-dev and tar packages you need (just in case tar isn’t on your system).

yum –y install epel-release
yum –y install dpkg-dev tar

Building the repository building script

Make a script that automates building the Release and Package files.

updatescript=/mnt/mirror/ubuntu/example-debian/update-repo.sh
cat <<'EOFSH' >${updatescript}
#!/bin/sh

# working directory
repodir=/mnt/mirror/ubuntu/example-debian/
cd ${repodir}

# create the package index
dpkg-scanpackages -m . > Packages
cat Packages | gzip -9c > Packages.gz

# create the Release file
PKGS=$(wc -c Packages)
PKGS_GZ=$(wc -c Packages.gz)
cat <<EOF > Release
Architectures: all
Date: \$(date -R)
MD5Sum:
 $(md5sum Packages  | cut -d" " -f1) $PKGS
 $(md5sum Packages.gz  | cut -d" " -f1) $PKGS_GZ
SHA1:
 $(sha1sum Packages  | cut -d" " -f1) $PKGS
 $(sha1sum Packages.gz  | cut -d" " -f1) $PKGS_GZ
SHA256:
 $(sha256sum Packages | cut -d" " -f1) $PKGS
 $(sha256sum Packages.gz | cut -d" " -f1) $PKGS_GZ
EOF
gpg -abs -o Release.gpg Release
EOFSH
chmod 755 ${updatescript}

It might be useful to modify the script to chmod 444 *.deb or something similar.
When running the script, make sure you use the correct key to sign the release file. Note that this script calls gpg, which will interactively ask the user to enter the passphrase for the key.

Managing the repository

The repository is ready to receive files and be updated.
The example location is /mnt/mirror/ubuntu/example-debian/.

Adding packages to the repo

Move any .deb packages you want to the repo directory.
Run the update-repo script form root (because the gpg keys were generated as root).

./update-repo.sh

Provide the passphrase.

Configuring a client

For each system you want to add the repository to, you need to follow these steps.
Import the public key into apt and add the repo to the sources.

sudo wget --quiet http://mirror.example.com/ubuntu/example-debian/example-debian.gpg -O /root/example-debian.gpg
sudo apt-key add /root/example-debian.gpg	
sudo wget --quiet http://mirror.example.com/ubuntu/example-debian/example-debian.list -O /etc/apt/sources.list.d/example-debian.list

Update the available package list.

apt-get update

The system is now ready to install packages from your repository.

Addenda

Last modified 2020-02-28

You can store your plaintext gpg passphrase (use at your own risk!) and pass a few parameters to gpg for the whole task to be automatic.

gpg --batch --yes --passphrase-file /root/.gnupg/passphrasefile -abs -o Release.gpg Release

Weblinks

  1. Main layout of entire document https://www.sidorenko.io/blog/2015/05/19/easy-creation-of-a-simple-apt-repo/
  2. Manipulating gpg keys https://www.debuntu.org/how-to-importexport-gpg-key-pair/
  3. Using SHA256 for apt http://askubuntu.com/questions/760796/how-to-fix-apt-signature-by-key-uses-weak-digest-algorithm-sha1-after-install/776599#776599
  4. Extra information about debian repos https://wiki.debian.org/RepositoryFormat
  5. Discussion of various debian repo utilities https://wiki.debian.org/HowToSetupADebianRepository
  6. Alternate method for making a repo http://hyperlogos.org/page/Simple-recipe-custom-UbuntuDebian-repositories-apt-ftparchive
  7. How to make a super simple, unsigned repo https://help.ubuntu.com/community/Repositories/Personal