Build dpkg with Jenkins project

Here is how I build dpkgs in Jenkins. I have not yet tried the jenkinsfile syntax, but I’m sure that’s a better way to do it.

Prerequisites

Install the Debian Package Builder plugin. It adds a nice option for a build step which simplifies the debuild process.
Now, I use a single repository to store my rpm specs and dpkg debian/ directories. It’s not ideal, I realize, but it was based on the architecture I understood at the time, as well as it was modeled after a few upstream places I rip off follow.
So I had to set up the Gitlab plugins:

My debuild workflow in Jenkins

Make a new item, of type Freestyle project.
screenshot of Jenkins wui where the user is about to make a new project named "veracrypt."
To load my repository with its specs and debian/ directories, I pointed to my public gitlab repo which is configured elsewhere in Jenkins.
Screenshot of jenkins project showing General settings, Gitlab connection
I am pulling down the updates branch, of my main git repo, and saving to a local subdirectory.
Screenshot of project configuration showing SCM, and additional behavior of "Check out to a sub-directory"
I’m behind a NAT, so I don’t expect to easily set up a webhook. But I really wish I would bother to get it hooked up so any repo changes would do this. I’ll have to check that out in the future.
For the build steps, I have some shell running before and after the main “Build debian package” section.
Screenshot of build steps from jenkins project, showing parts of the shell statments and Build debian package step.
I had to revamp the debian/watch file in my veracrypt sources, because I wasn’t smart enough to get it to reliably download from sourceforge (oh how the might have fallen). But uscan is a wonderful tool that downloads the source tarball for a package.
My first full shell step:

uscan -v -ddd --destdir ../../ --symlink work/veracrypt ; mkdir -p dpkg ; tar -zx -C dpkg --strip-components 1 -f $( find . -maxdepth 1 -iregex '.*\/veracrypt_[0-9]+.*orig.*tar.*z.?' | head -n1 ) ; cp -pr work/veracrypt/debian dpkg/

The syntax got a little weird when I was switching between the bzip2 and the gzip file. Also, I don’t always know the filename, ergo the find command.
And then, the Build debian package step. The source tarball was extracted to the dpkg/ location, and the debian dir copied there. I really appreciate the ability to specify where the debian/ directory is.
And the final shell step deploys the files to my nfs share for final processing by hand. I hand-curate what’s in my own repo. I don’t have a proper pool/ setup like the real Devuan repos, so I curate it myself, plus my volume is low enough I can do it all myself.

mkdir -p /mnt/public/Public/${JOB_NAME} ;
cp -p *.build *.buildinfo *.changes *.deb *.debian.tar* *.dsc /mnt/public/Public/${JOB_NAME}/

And that’s it! For now I will trigger these builds manually.

Mirror an OBS deb repository locally

Story

I run an OBS repository for all my packages, and it is available at the main site: https://build.opensuse.org/project/show/home:bgstack15.

But I wanted to mirror this for myself, so I don’t have to configure all my systems to point outward to get updates. I already host a Devuan ceres mirror for myself, and so mirroring this Open Build System repository is the last step to be self-hosting entirely for all systems except the mirror server.

I first dabbled with debmirror, but it kept wanting to try rsync despite my best configuration, plus it really insists on using the dists/ directory which isn’t used in the OBS deb repo design. So, I researched scraping down a whole site, and I found httrack which exists to serve a local copy of an Internet site. Bingo!

After a few hours of work, here is my solution for mirroring an OBS deb repo locally.

Solution

Create a user who will own the files and execute the httrack command, because httrack didn’t want to be run as root. Also, this new user can’t munge other data.

useradd obsmirror

Configure a script (available at gitlab)

#!/bin/sh
# File: /etc/installed/obsmirror.sh
# License: CC-BY-SA 4.0
# Author: bgstack15
# Startdate: 2020-01-05 18:01
# Title: Script that scrapes down OBS site to serve a copy to intranet
# Purpose: save down my OBS site so I can serve it locally
# History:
# Usage:
#    in a cron job: /etc/cron.d/mirror.cron
#       50	12	*	*	*	root	/etc/installed/obsmirror.sh 1>/dev/null 2>&1
# Reference:
#    https://unix.stackexchange.com/questions/114044/how-to-make-wget-download-recursive-combining-accept-with-exclude-directorie?rq=1
#    man 1 httrack
#    https://software.opensuse.org//download.html?project=home%3Abgstack15&package=freefilesync
# Improve:
#    use some text file as a list of recently-synced URLs, and if today's URL matches a recent one, then run the httrack with the --update flag. Probably keep a running list forever.
# Documentation:
#    Download the release key and trust it.
#       curl -s http://repo.example.com/mirror/obs/Release.key | apt-key add -
#    Use a sources.list.d/ file with contents:
#       deb https://repo.example.com/mirror/obs/ /
# Dependencies:
#    binaries: curl httrack grep head tr sed awk chmod chown find rm ln
#    user: obsmirror

logfile="/var/log/obsmirror/obsmirror.$( date "+%FT%H%M%S" ).log"
{
   test "${DEBUG:-NONE}" = "FULL" && set -x
   inurl="http://download.opensuse.org/repositories/home:/bgstack15/Debian_Unstable"
   workdir=/tmp/obs-stage
   outdir=/var/www/mirror/obs
   thisuser=obsmirror
   echo "logfile=${logfile}"

   mkdir -p "${workdir}" ; chmod "0711" "${workdir}" ; chown "${thisuser}:$( id -Gn obsmirror )" "${workdir}" 
   cd "${workdir}"
   # get page contents
   step1="$( curl -s -L "${inurl}/all" )"
   # get first listed package
   step2="$( echo "${step1}" | grep --color=always -oE 'href="[a-zA-Z0-9_.+\-]+\.deb"' | head -n1 | grep -oE '".*"' | tr -d '"' )"
   # get full url to a package
   step3="$( curl -s -I "${inurl}/all/${step2}" | awk '/Location:/ {print $2}' )"
   # get directory of the mirror to save down
   step4="$( echo "${step3}" | sed -r -e "s/all\/${step2}//;" -e 's/\s*$//;' )"
   # get domain of full url
   domainname="$( echo "${step3}" | grep -oE '(ht|f)tps?:\/\/[^\/]+\/' | cut -d'/' -f3 )"
   echo "TARGET URL: ${step4}"
   test -z "${DRYRUN}" && {
      # clean workdir of specific domain name in use right now.
      echo su "${thisuser}" -c "rm -rf \"${workdir:-SOMETHING}/${domainname:-SOMETHING}\""
      su "${thisuser}" -c "rm -rf \"${workdir:-SOMETHING}/${domainname:-SOMETHING}\"*"
      # have to skip the orig.tar.gz files because they are large and slow down the sync process significantly.
      echo su "${thisuser}" -c "httrack \"${step4}\" -*.orig.t* -v --mirror --update -s0 -r3 -%e0 \"${workdir}\""
      time su "${thisuser}" -c "httrack ${step4} -*.orig.t* -v --mirror --update -s0 -r3 -%e0 ${workdir}"
   }
   # -s0 ignore robots.txt
   # -r3 only go down 3 links
   # -%e0 follow 0 links to external sites

   # find most recent directory of that level
   levelcount="$(( $( printf "%s" "${inurl}" | tr -dc '/' | wc -c ) - 1 ))"
   subdir="$( find "${workdir}" -mindepth "${levelcount}" -maxdepth "${levelcount}" -type d -name 'Debian_Unstable' -printf '%T@ %p\n' | sort -n -k1 | head -n1 | awk '{print $2}' )"

   # if the work directory actually synced
   if test -n "${subdir}" ;
   then

      printf "%s " "DIRECTORY SIZE:"
      du -sxBM "${subdir:-.}"
      mkdir -p "$( dirname "${outdir}" )"
      # get current target of symlink
      current_target="$( find "${outdir}" -maxdepth 0 -type l -printf '%l\n' )"

      # if the current link is pointing to a different directory than this subdir
      if test "${current_target}" != "${subdir}" ;
      then
         # then replace it with a link to this one
         test -L "${outdir}" && unlink "${outdir}"
         echo ln -sf "${subdir}" "${outdir}"
         ln -sf "${subdir}" "${outdir}"
      fi

   else
      echo "ERROR: No subdir found, so cannot update the symlink."
   fi

   # disable the index.html with all the httrack comments and original site links
   find "${workdir}" -iname '*index.html' -exec rm {} +
} 2>&1 | tee -a "${logfile}"

And place this in cron!

#       50	12	*	*	*	root	/etc/installed/obsmirror.sh 1>/dev/null 2>&1

Explanation of script

So the logic is a little convoluted, because the OBS front page actually redirects downloads to various mirrors where the files are kept. So I needed to learn what the actual site is, and then pull down that whole site.
I couldn’t just use httrack –getfiles because it makes just a flat directory, which breaks the Packages contents’ accuracy to the paths of the package files. But I didn’t want the whole complex directory structure, just the repository structure. So I make a symlink to it in my actual web contents location.

Package for devuan: chicago95-theme-all

Because self-promotion (erm, I mean… learning!) is the purpose of this blog, here is a package I have assembled from a nifty upstream: Chicago95!

You can go get it from the OBS repository now.

screenshot of XFCE with chicago95 theme in use
Screenshot from https://github.com/grassmunk/Chicago95 project

chicago95-theme-all is a metapackage that pulls in all the elements needed to configure your Devuan GNU+Linux system to look like a classic non-free OS from 1995!

$ apt-cache search chicago
chicago95-theme-all - XFCE Windows 95 Total Conversion
chicago95-theme-cursors - Mouse cursor themes for Chicago95
chicago95-theme-doc - Documentation for Chicago95
chicago95-theme-fonts - Fonts for Chicago95
chicago95-theme-greeter - Lightdm webkit greeter for Chicago95
chicago95-theme-gtk - GTK and WM themes for Chicago95
chicago95-theme-icons - Icon themes for Chicago95
chicago95-theme-login-sound - Login sound for Chicago95
chicago95-theme-plymouth - Plymouth theme for Chicago95
$ apt-cache policy chicago95-theme-all
chicago95-theme-all:
  Installed: (none)
  Candidate: 0.0.1-1+devuan
  Version table:
     0.0.1-1+devuan 500
        500 http://download.opensuse.org/repositories/home:/bgstack15/Debian_Unstable  Packages

Of course, like any other theme, you need to manually change your settings to use the theme. But these packages make it easy to install it so you can control the files with the package manager.

Backstory

I was helping a family member set up a system to look like a particular non-free OS, and an old one was acceptable, and even preferred. After some research, I discovered a few good places to look for themes:

There were multiple options for the type of theme I wanted, but Chicago95 was the most cohesive and cleanly-installed set.

Devuan generate new ssh keys for freeipa host

If a Devuan system is a freeipa client, but you cannot ssh -o GSSAPIAuthentication=yes to it, even though all the regular troubleshooting steps work, and the logs don’t show you anything, the host ssh keys might be wrong in freeipa.

Generate new ssh keys for freeipa host

All the steps can be taken on the host in question.
As root, make sure you can kinit -k to get a kerberos key with the host keystore. If this step doesn’t work, you need to go fix that, which is beyond the scope of this post.

kinit -k
# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: host/d2-03a.ipa.example.com@IPA.EXAMPLE.COM

Valid starting       Expires              Service principal
12/31/2019 07:25:47  01/01/2020 07:25:47  krbtgt/IPA.EXAMPLE.COM@IPA.EXAMPLE.CO

Now, generate new ssh keys. Apparently on Devuan systems, restarting the daemon is not good enough. On CentOS, if you delete the ssh host keys, restarting the daemon will just generate new ones which can cause some interesting effects when connecting to a host that did so. However, on Devuan you have to run:

rm -rf /etc/ssh/ssh_host_*_key*
dpkg-reconfigure openssh-server
service ssh restart

And then, with the fresh keytab from the kinit -k earlier, it’s a piece of cake to modify this host in freeipa to use a new set of ssh public keys!

LC_ALL="" LC_CTYPE="C.UTF-8" ipa host-mod --sshpubkey="$( cat /etc/ssh/ssh_host_rsa_key.pub )" --sshpubkey="$( cat /etc/ssh/ssh_host_ecdsa_key.pub )" --sshpubkey="$( cat /etc/ssh/ssh_host_ed25519_key.pub )" $( hostname -s )
----------------------
Modified host "d2-03a"
----------------------
  Host name: d2-03a.ipa.example.com
  Principal name: host/d2-03a.ipa.example.com@IPA.EXAMPLE.COM
  Principal alias: host/d2-03a.ipa.example.com@IPA.EXAMPLE.COM
  SSH public key: ssh-rsa
                  AAAAB3NzaC1yc4EAAAADAQABAAABg[truncated]
                  root@d2-03a, ecdsa-sha2-nistp256
                  AAAAE@VjZHNhLXNoYTItbmlzdHAyNTYAAAAI[truncated]
                  root@d2-03a, ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBU/CbzrNnMivn5kAiHTU6WSadY/FWPG8qZ3sGleDbHr
                  root@d2-03a
  SSH public key fingerprint: SHA256:tMcJ2uFNmx6K+dF+Gm6WUBO4AvBmGVj9247mvg5LxU4 root@d2-03a (ssh-rsa),
                              SHA256:uJeRc0dkao/DmnQm2hyQUSfeC0HgIZppB2NVyA+BoTA root@d2-03a (ecdsa-sha2-nistp256),
                              SHA256:j+trvcJAQx5PeaJbUJ8xImBDgCJ2U/nW3h5D3m2kTj4 root@d2-03a (ssh-ed25519)
  Password: False
  Keytab: True
  Managed by: d2-03a.ipa.example.com

My ipa command kept complaining about all these language problems. Maybe I failed to set them correctly in my preseed. Whatever.

References

Internet searches

freeipa new ssh host key

Weblinks

6.8. Managing Public SSH Keys for Hosts
How To: Ubuntu / Debian Linux Regenerate OpenSSH Host Keys – nixCraft

Man pages

ipa help host-mod

fluxbox call scrot -s

This is an obscure one, but it was useful for me and since I’m the only author on this blog, I get to choose what goes on here. If you don’t like it, contact me and tell me that you have topics to share here.
I already had fluxbox calling scrot for full-screen captures and window captures.

107      :Exec scrot   -z -e 'mv $f ~/Pictures'
Mod1 107 :Exec scrot -ubz -e 'mv $f ~/Pictures'

But I wanted to add an option to “select range” with scrot, i.e., to draw a rectangle around the area to capture.
So the syntax for that is easy.

scrot -s -f -l 'style=dash,width=1' -z -e 'mv $f ~/Pictures'

But if you put that in an :Exec statement for fluxbox in ~/.fluxbox/keys, it fails. I placed it in a shell script and sent the output to an existing tty.

{
   scrot -s -f -l 'style=dash,width=1' -z -e 'mv $f ~/Pictures'
} 1>/dev/pts/3 2>&1

And running that with my hotkey in Fluxbox prints “couldn’t grab keyboard” which was enough to search.
Some guys in i3 inserted a delay, and then it works.
So my final answer is pretty simple.

Ctrl 107 :Exec sleep 0.08 ; scrot -s -f -l 'style=dash,width=1' -z -e 'mv $f ~/Pictures'

Obviously my use of the floating decimal value to sleep depends on GNU sleep. I might be young, but I do remember when sleep only worked with integers. And about my 0.08, scrot did work with an even smaller delay in a shell script called by fluxbox, but when run by fluxbox itself, 0.05 didn’t work and I needed the 0.08 delay. Haha, so that means there was the overhead of calling bash.

References

Weblinks

scrot/giblib: couldn’t grab keyboard:Resource temporarily unavailable / Multimedia and Games / Arch Linux Forums

Libreoffice Calc hide page breaks

tl;dr

Tools menu -> Options… -> Libreoffice Calc / View tab -> Visual Aids section -> “Page breaks” checkbox
LibreOffice Options window open to LibreOffice Calc/View tab

The story

I run in to this on occasion, and here the solution. Sometimes upon opening a file, and certainly after printing, my spreadsheet will retain the additional dashed lines that show how many cells will fit on a printed page. These additional lines annoy me, so I wanted to know how to turn them off. After some Internet searching, I found the answer.

spreadsheet showing a random dashed line interrupting some data

References

Internet searches

libreoffice calc hide page breaks

Weblinks

Ripped from How to hide default print range/page break lines after clearing a print range? – Ask LibreOffice

Fix xscreensaver prompt with no words

On a Devuan minimal install where you add all the components yourself, you might experience xscreensaver without its selected fonts installed. This leads to an amusing problem as seen in the following image.
xscreensaver unlock prompt but without any words

Thankfully it at least shows the username and number of asterisks for the length of your password. You just have to know which button is the OK button (it’s the one on the right).

I asked for help on the forum. Over time, as I have been installing new instances of Devuan, I keep coming across this problem and refining the answer.

Solution

apt-get install xfonts-100dpi xfonts-100dpi-transcoded freetype2-doc gsfonts libfreetype6 libfontembed1 fonts-noto-mono fonts-liberation fonts-droid-fallback

For some reason the xfonts-100dpi and similar fonts are not included when I install xscreensaver. Ah, well. It’s easy enough to fix.

Side discussion

Man, the Debian/Devuan ecosystem is nice. There’s a quick solution to everything. In the CentOS world, everything is stable and secure (I do miss SELinux) and understandable and configurable. But it doesn’t have the breadth of prebuilt packages and overall general compatibility to everybody’s homegrown packages in the wild (because most people use dpkg). While I would never want to use Devuan in an enterprise, I really, really love it for home.