txt2man wrapper which adds man page headings support in text file

txt2man is great. It depends on only GNU awk, which I can certainly live with. I previously used go-md2man which is also fine, but I just don’t want to depend on golang.

txt2man has an area where I wanted to improve it, but I didn’t know how to contribute to upstream in a way that I expect they would want. So I wrote txt2man-wrapper! Txt2man adds the page titles, section number, etc., from parameters on the command line invocation, but I wanted to add the ability to store these values in the text file.

#!/bin/sh
# File: txt2man-wrapper
# Location: /usr/bin
# License: CC-BY-SA 4.0
# Author: bgstack15
# Startdate: 2020-04-15 17:35
# Title: Txt2man wrapper that adds title headings support
# Purpose: add title headings support to txt2man
# History:
# Usage:
# Custom layout for input text file, where first lines include "section 3", "title txt2man-wrapper", "project bgscripts", "volume Linux command reference" in no particular order, followed by "====="
# Reference:
# txt2man
# Improve:
# Dependencies:
# dep-raw: /usr/bin/awk, /usr/bin/txt2man
# rec-devuan: gawk | mawk, txt2man
# rec-fedora: coreutils, txt2man
test -z "${INFILE}" && test -n "${1}" && export INFILE="${1}"
if test -z "${INFILE}" || test "${INFILE}" = "-" ; then USE_STDIN=1 ; fi

if test "${USE_STDIN}" = "1" ;
then
input="$( cat )"
else
input="$( cat "${INFILE}" )"
fi

head="$( echo "${input}" | awk 'BEGIN{a=0} /^=====/{a=1} {if(a==0)print}' )"
short="$( echo "${input}" | awk '{if(a)print} /^=====/{a=1}' )"

ul="$( echo "${head}" | awk '$1=="title" {$1="";print}' | sed -r -e 's/^\s*//;' )"
uln="$( echo "${head}" | awk '$1=="section" {$1="";print}' | sed -r -e 's/^\s*//;' )"
dc="$( echo "${head}" | awk '$1=="date" {$1="";print}' | sed -r -e 's/^\s*//;' )"
ur="$( echo "${head}" | awk '$1=="project" {$1="";print}' | sed -r -e 's/^\s*//;' )"
uc="$( echo "${head}" | awk '$1=="volume" {$1="";print}' | sed -r -e 's/^\s*//;' )"
bolds1="$( echo '`__xxNONExx__`' "${short}" | tr -d '\r\n' | tr '`' '\n' | awk 'NR/2==int(NR/2){print}' | xargs -n1 printf '%s %s ' '-B' )"
bolds2="$( echo "*__xxNANExx__*" "${short}" | tr -d '\r\n' | tr '*' '\n' | awk 'NR/2==int(NR/2){print}' | xargs -n1 printf '%s %s ' '-B' )"
ital1="$( echo "${short}" "**__xxNBNExx__**" | tr -d '\r\n' | sed -r -e 's/\*\*/\n/g;' | awk 'NR/2==int(NR/2){print}' | xargs -n1 printf '%s %s ' '-I' )"
test -n "${DEBUG}" && echo "${bolds1}" "${bolds2}" "${ital1}" 1>&2
echo "${short}" | txt2man -t "${ul}" -r "${ur}" -s "${uln}" -v "${uc}" -d "${dc}" ${bolds1} ${bolds2} ${ital1}

Using my handy txt2man-wrapper above, or found in my bgscripts package, you can have a text file suitable for txt2man, with some extra header lines at the top.

title sizer
section 1
project bgscripts-core
volume General Commands Manual
date April 2020
=====
NAME
  sizer - summarize directory contents by file extension

And invoke txt2man-wrapper like so:

txt2man-wrapper < sizer.1.txt | gzip > sizer.1.gz

In a Makefile, you can use something similar to:

MAN_TXT:=$(wildcard usr/share/man/man*/*.txt)
MAN_GZ:= $(subst .txt,.gz,$(MAN_TXT))

build_man: $(MAN_GZ)

$(MAN_GZ): %.gz: %.txt
   txt2man-wrapper - < $< | ${gzipbin} > $@

Deplist improved, with suggested and recommended entries

STUB
make sure to document how to use `make DEPLIST DEPTYPE=sug DISTRO=devuan` and default of `make DEPLIST DEPTYPE=dep DISTRO=devuan` and make DEPLIST DEPTYPE=rec DISTRO=devuan

Here is my improved method for using target deplist.

In the Makefile:

deplist:
   @# deplist 2020-04-18 input must be comma separated
   @# DEPTYPE( dep , rec , sug ) for depends, recommends, or suggests
   @if test -z "${DISTRO}" ; then ${echobin} "Please run \`make deplist\` with DISTRO= one of: `make deplist_opts 2>&1 1>/dev/null | ${xargsbin}`. Aborted." 1>&2 ; exit 1 ; fi
   @if ! ${echobin} "${DEPTYPE}" | grep -qE "^(dep|rec|sug)$$" ; then ${echobin} "Please run \`make deplist\` with DEPTYPE= one of: dep, rec, sug. Undefined will use \`dep\`. Aborted." 1>&2 ; exit 1; fi
   @${grepbin} -h --exclude-dir='doc' -riIE "\<${DEPTYPE}-" ${SRCDIR} | ${awkbin} -v "domain=${DISTRO}" -v "deptype=${DEPTYPE}" 'tolower($$2) ~ deptype"-"domain {$$1="";$$2="";print}' | tr ',' '\n' | ${sortbin} | ${uniqbin} | ${sedbin} -r -e 's/^\s*//' -e "s/\s*\$$/${SEPARATOR}/" | ${xargsbin}

deplist_opts:
   @# deplist_opts 2020-04-18 find all available dependency domains
   @${grepbin} -h -o -riIE '\<(dep|rec|sug)-[^\ :]+:' ${SRCDIR} | ${sedbin} -r -e 's/(dep|rec|sug)-//;' -e 's/:$$//;' | ${sortbin} | ${uniqbin} 1>&2

Observe how these all depend on variables being defined earlier:

awkbin     :=$(shell which awk)
chmodbin   :=$(shell which chmod)
cpbin      :=$(shell which cp)
echobin    :=$(shell which echo)
falsebin   :=$(shell which false)
findbin    :=$(shell which find)
grepbin    :=$(shell which grep)
gzipbin    :=$(shell which gzip)
installbin :=$(shell which install)
rmbin      :=$(shell which rm)
rmdirbin   :=$(shell which rmdir)
sedbin     :=$(shell which sed)
sortbin    :=$(shell which sort)
truebin    :=$(shell which true)
uniqbin    :=$(shell which uniq)
xargsbin   :=$(shell which xargs)

In the source files in the project, you can use entries like these:

# dep-devuan: mawk | gawk, lightdm, upower
# dep-raw: awk, grep, sed, lightdm
# rec-devuan: lightdm-gtk-greeter
# sug-devuan: logout-manager
# dep-centos: lightdm, upower

Obviously this is a poor substitute for the make_shlibs from debhelper (which probably just wraps around ldd). But for my shell-driven packages, it’s pretty nice if you add the dependencies to each individual file.

Makefile trick: deplist

I whipped together a snippet for a Makefile I plan on using more in the future.

This target, deplist, searches the entire source tree for my “# Dependencies:” tags, by distribution name, and lists them on standard out.

deplist:
	@if test -z "$(DISTRO)" ; then echo "Please run \`make deplist\` with DISTRO= one of: `make deplist_opts 2>&1 1>/dev/null | xargs`. Aborted." ; exit 1 ; fi
	@grep -h --exclude='Makefile' --exclude-dir='doc' -A5 -riIE dependencies $(SRCDIR) | \
	   awk -v 'distro=$(DISTRO)' 'tolower($$0) ~ distro {$$1="";$$2="";print}' | \
	   awk 'BEGIN{cmd="xargs -n1"} $$0 !~ /\(/{print $$0 | cmd ; close(cmd);} $$0 ~ /\(/{print;}' | \
	   sort | uniq | sed -r -e 's/$$/$(SEPARATOR)/' | xargs

deplist_opts:
	@echo "el7" 1>&2
	@echo "devuan" 1>&2

And, of course, probably place these in your .PHONY list because these are not real files to be built.

.PHONY: clean install uninstall list deplist deplist_opts

Every file in the project that has an external dependency should have some comments in this format:

# Dependencies:
#    devuan: python3-tk python3-pil
#    el7: python36 python36-pil

To use this target in a debuild recipe, so that it will dynamically build the dpkg dependency list, use:

override_dh_gencontrol:
	printf "misc:Depends=" > debian/${APPNAME}.substvars
	make -C src deplist DISTRO=devuan SEPARATOR=',' | grep -vE 'make\[[0-9]' >> debian/${APPNAME}.substvars
	dh_gencontrol

References

Inspiration from rpm’s Dynamic Build Dependency feature.

Original research.

Sample makefile

Here is a quick little makefile I use in small projects. It compiles all *.cpp files, and then links all *.o files into a single binary. It also includes a nice “list” target, which lists all the available targets. I realize bash tab auto-complete usually is good enough, but for programmatic uses, make list is helpful.

# references:
#    BUILD_DIR and per-.cpp file https://spin.atomicobject.com/2016/08/26/makefile-c-projects/
awkbin     :=$(shell which awk)
cpbin      :=$(shell which cp)
echobin    :=$(shell which echo)
findbin    :=$(shell which find)
grepbin    :=$(shell which grep)
installbin :=$(shell which install)
rmbin      :=$(shell which rm)
sedbin     :=$(shell which sed)
sortbin    :=$(shell which sort)
truebin    :=$(shell which true)

CXX = g++

# to get full debug symbols, add to both FLAGS: -g
# to make all warnings act as errors: -Wall -Weffc++ -Wextra -Wsign-conversion -Werror
CXXFLAGS = -g -std=c++17 -Wall -Weffc++ -Wextra -Wsign-conversion -Werror

# to remove all debug symbols: -s
# to add full debug symbols: -g
LDFLAGS = -g

src = $(wildcard *.cpp)
obj = $(src:.cpp=.o)

BUILD_DIR ?= .

OUTEXE = mine

all: $(OUTEXE)

# compile, which is not actually used?
$(BUILD_DIR)%.o: %.cpp
	$(CXX) -c $(CXXFLAGS) $+

# link
$(OUTEXE): $(obj)
	$(CXX) -o $@ $^ $(LDFLAGS)

.PHONY: clean cleanall list

list:
	@$(MAKE) -pRrq -f $(lastword $(MAKEFILE_LIST)) : 2>/dev/null | ${awkbin} -v RS= -F: '/^# File/,/^# Finished Make data base/ {if ($$1 !~ "^[#.]") {print $$1}}' | ${sortbin} | ${grepbin} -E -v -e '^[^[:alnum:]]' -e '^$@$$' -e '\.(cp*|o)'

clean:
	rm -f $(obj)

cleanall:
	rm -f $(obj) $(OUTEXE)