Install build dependencies from source files, dpkg and rpm

For a dpkg

cd $packagedir ;
mk-build-deps
sudo apt-get install ./${package}-build-deps*.deb

Source: ubuntu – Given a debian source package – How do I install the build-deps? – Server Fault

For an rpm

yum-builddep my-package.spec

Or

dnf builddep my-package.spec

fedora – Automatically install build dependencies prior to building an RPM package – Stack Overflow/13228992#13228992

Build rpm with Jenkins project

Here is how I build rpm files in Jenkins.

Prerequisites

Add a Fedora node to the cluster. I am running Jenkins on Devuan, which obviously is not ideal for building rpms.
I added a few labels, which are space-delimited which I found unusual. But whatever. I used a service account, and set up an ssh key for passwordless authentication.
Configuring a Jenkins node, with name fc30x-01a and a remote directory and ssh setup.
Add a user and grant them some specific sudo permissions:

useradd jenkins

cat <<EOF >/etc/sudoers.d/70_jenkins
User_Alias JENKINS = jenkins
Defaults:JENKINS !requiretty
JENKINS fc30x-01a=(root)	NOPASSWD: /usr/bin/dnf -y builddep *
EOF

Install some build tools:

sudo dnf -y install rpm-build rpmdevtools

My rpmbuild workflow in Jenkins

Add a new project. Restrict it to run on label “fedora.”
Project configuration showing restrict where project is run, to label "fedora"
Like last time, I am checking out my git repo to a local subdirectory.
Screenshot of project configuration showing SCM, and additional behavior of "Check out to a sub-directory"
All the build steps for this project are shell commands.
The first command uses some tooling I learned about for this project: spectool. The dnf installs the build dependencies, and spectool downloads all the source files that are not already present in the directory.

pwd ; ls -altr ; mkdir -p rpmbuild ; cd rpmbuild ;
cp -p ../work/veracrypt/* . || :;
sudo dnf -y builddep *.spec ;
spectool -g *.spec ;

The actual build command happens in the second step. I am using a few macro definitions to keep everything happening in the present working directory.

cd rpmbuild ; rpmbuild --define "_topdir %(pwd)" --define "_builddir %{_topdir}" --define "_rpmdir %{_topdir}" --define "_sourcedir %{_topdir}" --define "_srcrpmdir %{_topdir}" --define "_rpmfilename %%{NAME}-%%{VERSION}-%%{RELEASE}.%%{ARCH}.rpm" -ba *.spec

And the final step deploys the files to my nfs share for manual curation.

mkdir -p /mnt/public/Public/${JOB_NAME} ;
cp -p *.rpm */*.rpm /mnt/public/Public/${JOB_NAME}/ || :

References

Weblinks

  1. fedora – Automatically install build dependencies prior to building an RPM package – Stack Overflow
  2. rpm – How do I get rpmbuild to download all of the sources for a particular .spec? – Stack Overflow
  3. Rpmdevtools – Fedora Project Wiki
  4. Build RPMs using Jenkins/Hudson

All packages in my copr are CentOS 8-ready

I have confirmed that all the packages in my COPR have compiled for CentOS 8! I personally only have a single headless vm for CentOS 8 so the applications might malfunction. Please report to me if any failures occur.

Backstory time

When CentOS 8 was finally released (September 24), I downloaded it right away and updated my kickstart process for it (future post).

Some packages built just fine right away, on the rhel8beta repo. Some failed because they use ImageMagick which wasn’t compiled yet, but has since been added. I use the “convert” utility like a lot of packagers do, for converting an svg to instantiated png files for icons. Anyway, I was considering rewriting my scripts to use GraphicsMagick but thankfully upstream added ImageMagick, which reduced my workload considerably.

Some packages failed because various libraries were unavailable, and were still unavailable. Using one of my preferred upstreams (city-fan) and rebuilding some Fedora packages, I added libraries necessary to compile the rest of my packages.

It is worth noting that CentOS 8 uses dnf modules or “streams.” I don’t fully understand the concept, and personally think it only causes problems. I had to compile curl-7.66.0 and host it myself, because of which repositories I had to enable and disable to get the right perl modules to compile it. Libcurl is required to compile FreeFileSync, and various unimportant perl modules are required to compile libcurl. However, due to the modules in dnf/yum, there is a conflict between certain latest perl packages in that stream, with other repositories, and I was unable to configure the COPR to take any old version of the perl modules.

How I use the COPR to build and host rpms for CentOS and Fedora

Introduction

For about a year now, I have been using the Fedora Project’s public Cool Other Packages Repository (copr) to build and host rpms for my Fedora and CentOS GNU/Linux installations.
This post documents the process I use to take an upstream package, build the packages for the different chroots, and host them for download.

The process

No local tools are required, other than the source control software, normally git.

Select the upstream package

My example will use FreeFileSync, which is a great program and it’s quick to compile.
Additionally, because the upstream provides only a zip file, I am using my collaborative Opensource Tracking repo for the tarball which dpkg seemed to require and I gave up investigating how to get it to use a zip file as a source.

Prepare the spec file and additional artifacts

My spec file, patches, and additional items for building FreeFileSync on Fedora are on my gitlab page. This topic today is not intended to show you how to use rpmbuild, which is a deep and useful topic.

Add a package to copr

With the rpm sources available on the Internet, we’re ready to work in the copr environment.

Create a project

A project can host a single package or many. Also useful to note, that a spec file can produce more than one rpm. So for the copr, a “package” can include either a single rpm artifact or multiple. For example, a libssl.spec will probably produce a libssl-devel, libssl, and libss-docs rpms. You would only have to set up the libssl.spec, and any produced rpms will just be handled automatically.
So, on the main page of COPR, select “New Project.”

Name the project and include any long-form text you care to share. The build options, farther down the page, are important. You can always change these options later, so don’t feel that you have to be extremely careful right now. Select the chroot arch environments you want to build the rpms for.

The external repositories section is really nice. If you need the packages from your favorite Internet yum repo, you can paste the baseurl values here.

Create a new package

In your shiny new project, you will want to add some packages!
Select the Packages tab, and then “New package.”

Plug in the relevant information.
For FreeFileSync, my rpmbuild input artifacts are at https://gitlab.com/bgstack15/stackrpms.git, with a committish (I love that term!) of “freefilesync-bump” which is the dev tree I use for testing the latest version of FreeFileSync.
The subdir for just the freefilesync package, in my entire git tree, is freefilesync/. I’ve got a lot of other spec files in there, but the copr can look in just one dir, which is pretty great.
Use the “rpkg” option, based on the git source. I don’t actually know how the other methods work, and the rpkg has always been good enough for git-hosted spec files for me.

If the package needs to be excluded from certain architectures, there’s a blacklist field you can use.
Save the package settings.

Trigger a build

On the package list, find the new package and select “Rebuild.”

You can choose which chroots to use, and the checkboxes are pre-populated with your defaults and blacklist. My freefilesync package has some unresolvable build dependencies on EL6, so I have excluded those.

Also, specifically, FreeFileSync needed some very custom dependencies co build on EL7– some higher versions of libs like curl and openssl, so it’s a complex dependency tree so either include copr://bgstack15/FreeFileSync in your project’s external repository list.

Notes for Yum repositories for CentOS and Fedora

This is my scratch page for third-party centos and fedora repositories that are useful. I’ve used these at least once in my personal experience. There’s obviously a lot more, and they’re only an internet search away.

EPEL
SCL
city-fan
remi
rpmfusion
negativo17
ius
nuxref

Lists of additional ones:
https://rpmfusion.org/FedoraThirdPartyRepos

Sites for “rpm find”:
https://www.rpmfind.net/
http://rpm.pbone.net/

FreeFileSync 10.9 on CentOS 7

Now, you can go install FreeFileSync 10.9 on CentOS 7 from an rpm built on CentOS 7! The upstream release follows some newer versions of libs (clearly compiled on a different platform than CentOS or even Fedora), and I have now compiled those versions based on the work of a genius fellow over at city-fan.

To get FreeFileSync on CentOS 7, you need to load up two of my coprs as seen in the following table.

COPR name repo file
bgstack15/stackrpms bgstack15-stackrpms-epel-7.repo
bgstack15/FreeFileSync bgstack15-FreeFileSync-epel-7.repo

The application depends on some newer libs, which are all available in a second copr.

======================================================================================
 Package                   Arch   Version                Repository              Size
======================================================================================
Installing:
 freefilesync              x86_64 10.9-2.el7             bgstack15-stackrpms    3.2 M
Installing for dependencies:
 libpsl                    x86_64 0.7.0-1.el7            bgstack15-FreeFileSync  45 k
 openssl-freefilesync-libs x86_64 1:1.1.0h-3.stack.el7   bgstack15-FreeFileSync 1.2 M
Updating for dependencies:
 curl                      x86_64 7.64.0-2.0.cf.rhel7    bgstack15-FreeFileSync 557 k
 libcurl                   x86_64 7.64.0-2.0.cf.rhel7    bgstack15-FreeFileSync 502 k
 libssh2                   x86_64 1.8.0-10.0.stack.rhel7 bgstack15-FreeFileSync 103 k

Transaction Summary
======================================================================================
Install  1 Package  (+2 Dependent packages)
Upgrade             ( 3 Dependent packages)

Total download size: 5.6 M
Is this ok [y/d/N]: 

The artifacts used to produce this build are in my gitlab.

rpm rebuild db

On rare occasions, rpm locks up. The issue is probably a broken database, so try cleaning it and rebuilding it.

Remove the rpm db files
Rebuild the database.

rm -f /var/lib/rpm/__db*
rpm --rebuilddb

Additionally, you can clean up the .lock files, after confirming no yum or rpm processes are running.

rm -f /var/lib/rpm/.*.lock

So many Internet references discuss rebuilding the rpm database, so so specific sources are provided.

Extract src.rpm files

Command is borrowed from How To: Extract an RPM Package Files Without Installing It (cyberciti.biz)

rpm2cpio myrpmfile.rpm | cpio -idmv

When you search for “rpm2cpio” it shows that page, but you have to click into it and scroll down to find the one command. Maybe this page will supplant that one in the Internet searches and make it easier to see the command to run from the lede in the search results.

Rpm spec macros and variables

A nice place to read about the macros in rpm specs is https://fedoraproject.org/wiki/Packaging:RPMMacros. You can also run a command similar to rpm -E ‘%{dist}’ to see a variable’s definition.

To use your own variable in a spec, you can use a %define.

%define myvar 3.1415926

To get even fancier, you can do shell commands in a definition.

%define version_num "%( echo %version | tr -d '\.' )"

This takes %{version} which could be 4.51 and sets a variable to “451” for which I had a use.