logrotate ignores files with g+w permission

Another logrotate post!

If you have a logrotate definition, with permission g+w, logrotate will ignore it. You can see that if you run logrotate with -v flag.

-rw-------. 1 root root 349 Nov  2 15:02 nginx

If you want the nginx group to still be able to manage their own logrotate definition, use ACLs, which are already enabled by default on xfs and ext4

# setfacl -m 'group:nginx:rw-' nginx
# getfacl nginx
# file: nginx
# owner: root
# group: root
user::rw-
group::---
group:nginx:rw-                 #effective:---
mask::---
other::---

# ls -l nginx
-rw-------+ 1 root root 349 Nov  2 15:02 nginx
Advertisements

tar through ssh

When you want to copy a whole directory, without having to actually make a huge tarball, copy it over, and extract it, here is a way to do it.

You will want to ensure you have passwordless auth enabled, either with ssh keys or kerberos auth or something else.

tar -zcf - dirname | ssh bgstack15@destinationserver "tar -zxf -"

References

Weblinks

  1. How To Use tar Command Through Network Over SSH Session

rpm rebuild db

On rare occasions, rpm locks up. The issue is probably a broken database, so try cleaning it and rebuilding it.

Remove the rpm db files
Rebuild the database.

rm -f /var/lib/rpm/__db*
rpm --rebuilddb

So many Internet references discuss rebuilding the rpm database, so so specific sources are provided.

Apache use ssl virtual host to reverse proxy to http

Documenting partially for myself but also for anyone else who just wants to deal with the http virtual host, but have ssl as well.

You need some basic ssl configs, which I tend to place in a separate file so all virtal hosts can share the same settings.

tf=/etc/httpd/conf.d/all-ssl.cnf
touch "${tf}" ; chmod 0644 "${tf}" ; chown root.root "${tf}" ;
cat <<EOF 1> "${tf}"
SSLEngine on
SSLProtocol all -SSLv2 -SSLv3
SSLHonorCipherOrder on
SSLCipherSuite "EECDH+ECDSA+AESGCM EECDH+aRSA+AESGCM EECDH+ECDSA+SHA384 EECDH+ECDSA+SHA256 EECDH+aRSA+SHA384 EECDH+aRSA+SHA256 EECDH+aRSA+RC4 EECDH EDH+aRSA !RC4 !aNULL !eNULL !LOW !3DES !MD5 !EXP !PSK !SRP !DSS"

<Files ~ "\.(cgi|shtml|phtml|php3?)$">
        SSLOptions +StdEnvVars
</Files>
<Directory "/var/www/cgi-bin">
        SSLOptions +StdEnvVars
</Directory>

SSLCertificateFile /etc/pki/tls/certs/localhost.crt
SSLCertificateKeyFile /etc/pki/tls/private/localhost.key
SSLCertificateChainFile /etc/pki/tls/certs/chain-localhost.crt

SetEnvIf User-Agent ".*MSIE 4\.0b2.*"                 nokeepalive ssl-unclean-shutdown                 downgrade-1.0 force-response-1.0

LogLevel warn
ErrorLog logs/ssl_error_log
CustomLog logs/ssl_access_log combinedvhost

<Directory "/var/www/html/notfound/">
        AllowOverride None
        Order allow,deny
        Allow from all
</Directory>

# END OF FILE all-ssl.cnf
EOF

And the real config file:

tf=/etc/httpd/conf.d/repo.conf
touch "${tf}" ; chmod 0644 "${tf}" ; chown root.root "${tf}" ;
cat <<EOF 1> "${tf}" 
# reference:
# https://bgstack15.wordpress.com/2016/03/24/adding-adfs-integration-to-apache/
# ssl act as proxy: https://httpd.apache.org/docs/2.4/rewrite/proxy.html

#Listen 80 # not needed here in base C7 because this is provided in /etc/httpd/conf/httpd.conf
#Listen 443
<VirtualHost *:80>
ServerName repo1.int.example.com
ServerAlias *.int.example.com *
UseCanonicalName Off

DocumentRoot /var/www/html
Options +Indexes
IndexOptions IgnoreCase FancyIndexing FoldersFirst NameWidth=* DescriptionWidth=*
</VirtualHost>

<VirtualHost *:443>

ServerName repo1.int.example.com:443
ServerAlias *.int.example.com

Include conf.d/all-ssl.cnf
# try the <proxy> stuff from https://bgstack15.wordpress.com/2017/10/12/adding-reverse-proxy-for-plex-to-apache-vhost/

<Proxy *>
Order deny,allow
Allow from all
</Proxy>

SSLProxyEngine On
ProxyRequests Off
ProxyPreserveHost On
ProxyPass / http://repo1.int.example.com/
ProxyPassReverse / http://repo1.int.example.com/

</VirtualHost>

Add policykit rules for virt-manager

If a user wants to use virt-man to control virtual machines on localhost, he will be prompted to authenticate to policykit.

If you have a policykit rule established (as the libvirt package already provides) for the right group, it’s seamless. Just make sure the user is in group “libvirt” (from /usr/share/polkit-1/rules.d/50-libvirt.rules)
Or, to make a different group the (additional) accepted value:

tf=/etc/polkit-1/rules.d/80-libvirt.rules
touch "${tf}" ; chmod 0644 "${tf}"
cat <<EOF >"${tf}"
polkit.addRule(function(action, subject) {
  if (action.id == "org.libvirt.unix.manage" && subject.local && subject.active && subject.isInGroup("wheel")) {
      return polkit.Result.YES;
  }
});
EOF

LAPS for Linux

A famous non-free operating system vendor provides a Local Administrator Password Solution for its flagship OS.
Here is my attempt at a GNU/Linux client for the same thing. This tool can help store unique local admin passwords in ldap. Some Unix environments still suffer from using the same single root password for everything. Here is the fix for that.
Please note that the domain-side configuration actions are still required. This is only the client-side utility.

laps for linux

https://gitlab.com/bgstack15/laps

This is written in shell, and designed and tested on CentOS 7. An rpm spec is provided if you want to bundle it in that format. Contributions are welcome, of any sort.

There are two main ways to use my laps4linux.

laps is the Local Administrator Password Solution for GNU/Linux.
usage: laps.sh [-duV] [-c conffile] [-t|-a] [-f] [-r [-u ] [-h ]]
version 2018-10-22a
 -d debug   Show debugging info, including parsed variables.
 -- usage   Show this usage block.
 -V version Show script version number.
 -c conf    Read in this config file. Default is /etc/laps/laps.conf
 -f force   Skip the time check and just update the password regardless.
 --noforce  Do not force. Overrides environment variable LAPS_FORCE.
 -t test    Dry run only. Useful with debugging on.
 -a apply   Turn off dry run. Default.
 -r read    Read password; do not set it. Can only be used by a domain admin. Can only be used with -u.
 -u user    Connect with kerberos ticket for this user. Default is "machine" to use host keytab.
 -h   Read this hostname instead of $( hostname -s )
Debug levels:
 0 Silent
 9 displays sensitive info, specifically the generated password
10 function trace
Environment variables:
 See documentation at /usr/share/doc/laps/ for full explanation.

Additional notes

When implementing laps in your environment, you will want to ensure the ldap connectivity is working. I particularly had trouble with the kerberos auth to the domain controllers for ldaps.

To use for the first time for a Linux client, you will want to use the -f flag, because the undefined expirationTime ldap attribute will cause an error during the time comparison.

Maybe something like this could be implemented in FreeIPA. It’s just a simple schema extension, oh, and some ACLs. I don’t know if FreeIPA has ACLs on attributes, but it sounds like a logical thing to have.

Sysprep for Linux

Named after a famous non-free OS sysprep process, my GNU/Linux sysprep process helps my company when I need to clone existing vms. For some reason my apps teams are incapable of installing their applications again, so they just have me clone the system. I blank it out as much as possible, and then provision it like it’s new.

Deploy the sysprep.sh script to the original server and add a cronjob.

    th=originalserver350
    scp -p /etc/ansible/books/test/sysprep/files/sysprep.sh ${th}:~/
    ssh ${th}
    sudo su -
    /bin/cp -pf /home/*/sysprep.sh /root/sysprep.sh ; chmod 0755 /root/sysprep.sh ; chown root.root /root/sysprep.sh
    tf=/etc/cron.d/00_sysprep.cron
    touch "${tf}" ; chown root.root "${tf}" ; chmod 0600 "${tf}"
    cat <<-EOFCRON 1>"${tf}"
@reboot         root    test ! "\$( hostname -s )" = "$( hostname -s )" && test -e /root/sysprep.sh && sleep 20 && /bin/sh -x /root/sysprep.sh cron 1> /root/intermediate-clone.log 2>&1
EOFCRON
    exit # leave root shell
    exit # leave ssh to original server

Inject the regular server deployment scripts.
Modify the td and tv variables here as needed.

    td=/etc/ansible/books/stable/clone
    tv=RHEL7
    pushd "${td}"
    tar -zc -C /etc/ansible/roles/localization -f /tmp/localizations.tgz files
    scp -p /tmp/localizations.tgz "${th}":~
    popd
    pushd "${td}/files/${tv}"
    scp -p *.sh "${th}":~
    popd
    ssh "${th}"
    sudo su -
    /bin/mv -f /home/bgirton/*.sh /root/ ; chown root.root /root/*.sh ; chmod 0750 /root/*.sh ;
    tar -zx -C /etc -f /home/*/localizations.tgz ;
    test -e /etc/files && ! test -e /etc/templates && /bin/mv -f /etc/files /etc/templates ;
    chown -R root:root /etc/templates 2>/dev/null ;
    exit # leave root shell
    exit # leave ssh to original server

The cronjob to run the s1_init script will be provided by the intermediate template’s execution of the sysprep script.

Upon cloning the vm, the sysprep script will be executed! Here it is:

#!/bin/sh
# File: sysprep.sh
# Location: /etc/ansible/books/test/sysprep/files/
# Author: bgstack15
# Startdate: 2018-10-11 11:39
# Title: Script that Syspreps an Intermediate Server
# Purpose: Convert an intermediate clone of an existing machine to make it more suitable for use as a VM template
# History:
# Usage:
#    On the intermediate template, run this shell script and then power off the vm.
# Reference:
#    https://www.cyberciti.biz/tips/vmware-linux-lost-eth0-after-cloning-image.html
#    https://unix.stackexchange.com/questions/39370/how-to-reload-udev-rules-without-reboot
# Improve:
# Documentation:

echo "sysprep.sh version 2018-10-12b"

# make verbose
set -x

export PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin

# continue only if hostname indicates this is an intermediate template
hostname -s | grep -qiE 'tmp$' || exit 0

# clean up cronjob so this script is only called that very first boot of the intermediate template
/bin/rm -f /etc/cron.d/00_sysprep.cron

# prepare cronjob for s1_init for a normal clone deployment
# this is mostly for the eth0 renaming again
tf=/etc/cron.d/01_init.cron
touch "${tf}" ; chown root.root "${tf}" ; chmod 0600 "${tf}"
cat <<-EOFCRON 1>"${tf}"
@reboot         root    test ! "\$( hostname -s )" = "$( hostname -s )" && test -e /root/s1_init.sh && /bin/sh -x /root/s1_init.sh cron 1> /root/clone.log 2>&1
EOFCRON

# fix rootsh
this_rootsh="$( which rootsh 2>/dev/null )"
tf=/usr/local/bin/root
touch "${tf}" ; chmod 0755 "${tf}" ; chown root.root "${tf}"
cat < "${tf}"
if test "\$USER" = "root" ; then
  shift
  \$@
else
  /usr/bin/sudo ${this_rootsh:-/usr/local/bin/rootsh} \$@ --no-syslog
fi
EOF

# disable domain connectivity
/bin/rm -f /etc/krb5.keytab /etc/krb5.conf*
service sssd stop ; systemctl stop sssd.service ;
chkconfig sssd off ; systemctl disable sssd.service ;

# disable monitoring
for ts in ccmexecd scx-cimd ;
do
   service "${ts}" stop ; systemctl stop "${ts}.service"
   chkconfig "${ts}" off ; systemctl disable "${ts}.service"
done
# remove old ssl certificates
rm -f /etc/opt/microsoft/scx/ssl/*

# clean old log and config files
pushd /var/log
rm -f boot.log* btmp* cron* dmesg* dracut* maillog* messages* prepatch/* rootsh/* sa/* samba/* secure* spooler/* sssd/* syslog* vmware-* wtmp*
popd
pushd /root
rm -f core* *.ansible_backup KASetup*log log.*.txt VMwareTools*gz
rm -rf vmware-tools-distrib
popd
pushd /etc
rm -f *.cfsaved fstab.* fstab-* hosts.20*
popd
# clean ansible bupped files
rm -f *.20*@*~
# clean homedirs and bash histories
find /home -maxdepth 1 -mindepth 1 ! -name 'ansible' ! -name 'bgstack15' -exec rm -rf {} \;
for tf in /home/*/.bash_history ;
do
   cat /dev/null | tee "${tf}"
done

# fix networking, rhel6 on vmware
# from project clone RHEL6/s1_init.sh
nic_count="$( ip -o l | grep -cE 'eth[0-9]' )"
this_nic="$( ip -o l | grep -oE 'eth[0-9]' | sort | tail -n1 )"
out_nic=eth0
out_nic_file="/etc/sysconfig/network-scripts/ifcfg-${out_nic}"
this_nic_file="/etc/sysconfig/network-scripts/ifcfg-${this_nic}"
test "${nic_count}" -eq 1 && test "${this_nic}" != "${out_nic}" && {
   sed -i -r -e "s/${this_nic}/${out_nic}/g;" "${this_nic_file}" 2>/dev/null
   sed -i -r -e "s/${this_nic}/${out_nic}/g;" "${out_nic_file}" 2>/dev/null
   test -e "${this_nic_file}" && /bin/mv -f "${this_nic_file}" "${out_nic_file}"
}
/bin/rm -f /etc/udev/rules.d/70-persistent-net.rules
udevadm control --reload-rules && udevadm trigger
# the change will take effect upon next reboot

# fix iptables rule tied to eth0
sed -i -r -e '/dport 1270/s/-i eth[0-9]* //;' /etc/sysconfig/iptables

# generate new ssh host keys
# but this will also happen in the s1_init script
/bin/rm -rf /etc/ssh/*key*
# but do not actually generate them, because this is still just the intermediate template
#/sbin/service sshd stop ; /bin/sleep 5 ; /sbin/service sshd start ;

# clean up old sudoers backups
rm -f /etc/sudoers.20* /etc/sudoers.cf* /etc/sudoers.rpm* /etc/sudoers.d/disable_requiretty.*

# when this process is verified, the template should shut itself down
shutdown -h now ; systemctl poweroff ;

Netgear AC600 usb wnic drivers for Linux

The gist of it.

# download build tools on devuan
sudo apt-get install --reinstall linux-headers-$(uname -r) build-essential

! test "${USER}" = "root" && sudo su -
pushd /usr/src
git clone https://github.com/abperiasamy/rtl8812AU_8821AU_linux.git
cd rtl8812*
time make
time make install
modprobe rtl8812au
popd
test "${USER}" = "root" && exit

# list new nics
sudo iwconfig
# configure wicd to use new nic -- use gui

References

Weblinks

This post is a shameless rip-off of two other pages:

    1. https://ubuntuforums.org/showthread.php?t=2235778

      Hi, your device…
      Bus 001 Device 006: ID 0846:9052 NetGear, Inc.

      led me to.
      http://forum.ubuntu-fr.org/viewtopic.php?id=1572411
      and this led to..
      https://wikidevi.com/wiki/Netgear_A6100
      Which shows your device 0846:9052 NetGear
      has the rtl8812au chip
      and finally the chilli555 fix can be found,,
      Please first do..
      sudo apt-get install –reinstall linux-headers-$(uname -r) build-essential
      and then visit https://askubuntu.com/questions/368015/problem-with-building-compiling-a-driver-for-edimax-wireless-adapter-ew-7822uac

    2. https://askubuntu.com/questions/368015/problem-with-building-compiling-a-driver-for-edimax-wireless-adapter-ew-7822uac

And the source for the drivers:

  1. https://github.com/abperiasamy/rtl8812AU_8821AU_linux

Web searches

  1. netgear a6100 linux driver

Ansible find first accessible proxy and use it

If you need to find the first available http proxy and use it for a process, you can use a python snippet to discover it and use it.

https://gitlab.com/bgstack15/former-gists/tree/master/get_first_open_port.py

vars:
  http_proxies:
  - 192.168.1.5:3128
  - proxy5.internal.example.com:3128

tasks:

  - name: learn which proxy to use
    script: get_first_open_port.py {{ http_proxies | join( " " ) }}
    changed_when: false
    register: open_ports

  - set_fact:
      http_proxy: "{{ open_ports.stdout_lines[0] }}"
    when:
    - 'open_ports.stdout | length > 0'
    failed_when:
    - 'open_ports.stdout | length = 0'

  - name: use http_proxy environment variable
    script: script_needing_internet.sh -i {{ inputvar }}
    environment:
      http_proxy: "http://{{ http_proxy | default(omit) }}"

The sole output is the first hostname and port available.

#!/usr/bin/python2
# Filename: get_first_open_port.py
# Location: /etc/ansible/roles/install_sccm/files/
# Author: bgstack15
# Startdate: 2018-10-02 10:13
# Title: Script that Gets the First Open Port
# Purpose: Return to standard output the first valid host and port to use as a http proxy
# Project: projects derived from ansible role certreq
# History:
# Usage:
#    in ansible
# Reference:
#    https://stackoverflow.com/questions/19196105/python-how-to-check-if-a-network-port-is-open-on-linux
#    string split https://stackoverflow.com/questions/6670290/split-string-into-different-variables-instead-of-array-in-python
# Improve:
# Documentation:

import socket, sys
from contextlib import closing

GFOP_VERSION="2018-10-02a"

def check_socket(host, port):
   with closing(socket.socket(socket.AF_INET, socket.SOCK_STREAM)) as sock:
      sock.settimeout(2)
      if sock.connect_ex((host,port)) == 0:
         print host + ":" + str(port)
         return True
   return False


x = 0
for myarg in sys.argv:

   # show version
   if myarg == "--version" or myarg == "-V":
      print(sys.argv[0]+" "+GFOP_VERSION)
      sys.exit(0)

   x = x + 1
   # skip the script $0 name itself
   if x > 1:
      # split on the colon
      host, port = myarg.split(":",2)
      # short-circuit upon first successful one
      if check_socket(host,int(port)):
         sys.exit(0)

Quick script to purge URL from squid proxy

If you have the requisite permissions in squid.conf, you can just use a quick script to purge URLs. I use this for flushing local files I am developing and download to clients with http.

tf=/usr/local/bin/purge
touch "${tf}" ; chmod 0755 "${tf}"
echo <<EOF > "${tf}"
#!/bin/sh
for word in ${@} ;
do
   squidclient -h localhost -r -p 3128 -m PURGE "${word}"
done
EOF

For squid, I am using a few lines to allow purging:

acl PURGE method PURGE
http_access allow PURGE localhost

Then you can call it:

purge http://example.com/url1 http://example.com/url2 http://example.com/url3