Set word-read permissions on python libs

In an environment where the default umask is 0077 or similar, the pip utility for installing python libraries can set up new files that cannot be read by all users.

I dropped this script into my ~/bin dir so I can enforce other-read permissions easily on all the python libs. I wrote this script after hours and hours of troubleshooting python libs just to find out it’s an old-school permissions issue.


Python get Linux-compatible password hash

This snippet gets you a sha-512 ($6) password hash suitable for putting in /etc/shadow.

# Reference:
# python 2
import crypt, getpass, sys;
if len(sys.argv) >= 2: 
 thisraw=getpass.getpass(prompt='New password: ')

Debug the values passed to a function in python

Tested on python 2.

import inspect

def caller_args():
   frame = inspect.currentframe()
   outer_frames = inspect.getouterframes(frame)
   caller_frame = outer_frames[1][0]
   return inspect.getargvalues(caller_frame)

def updateval(infile,regex,result,verbose=False,apply=False,debug=0,stanza="",stanzaregex="",atbeginning=False):
   print caller_args()

It’s that simple!




Pretty print json in python

For python2

I wanted to show what variables are in use in a function, and I wanted to see it in a nicer format than a really long, single line.

import inspect, json
def function():
print json.dumps(locals(),indent=3,separators=(',',': '))


To view what parameters were passed in to a function, add these.

def caller_args():
   frame = inspect.currentframe()
   outer_frames = inspect.getouterframes(frame)
   caller_frame = outer_frames[1][0]
   return inspect.getargvalues(caller_frame)

def function():
print caller_args()


  2. compact encoding


python readlink function


def readlinkf(_inpath):
   if os.path.islink(infile):
      return os.path.join(os.path.dirname(_inpath),os.readlink(_inpath))
      return _inpath



In GNU, there’s a fantastic utility named readlink. The main way I use it is to get the full and true path of a file or symlink.

For example:

$ ls foo .config/qux
.config/qux  foo
$ readlink -f foo

In python, I wanted to achieve the same thing. it is possible with the os.path.islink and os.readlink functions. Here is my quick function for a readlink -f, since I didn’t find a specific implementation that outputs the full path.




Online reference


Search expressions

  1. python detect if file is symlink
  2. python readlink

Python script to update value in file


I do initial system configuration, for images and so on. All the time I have to update arbitrary values in config files. It’s all very repetitive. Sometimes though, instead of being able to use a whole templated config file, I need to modify the one that is in place.

Here today for you is my second-generation solution (the first was a crude shell script). It’s also a part of my bgscripts package available on github.

#!/usr/bin/env python3
# File: /usr/bgscripts/
# Author:
# Startdate: 2016-10-11 15:59
# Title: Python Script that Updates/Adds Value
# Purpose: Allows idempotent and programmatic modifications to config files
# Package: bgscripts
# History:
#    2016-07-27 wrote shell script 
#    2016-09-14 added the shell script version to bgscripts package
#    2016-10-12 added flags
# Usage:
# /etc/rc.conf "^ntpd_enable=.*" 'ntpd_enable="YES"' --apply
# Reference:
#    /usr/bgscripts/
#    re.sub from
#    shutil.copy2
#    keepalive (python script) from keepalive-1.0-5
# Improve:
#    idea: use argparse "nargs" optional input file to use stdin piping/redirection!

import re, shutil, os, argparse

# Parse parameters
parser = argparse.ArgumentParser(description="Idempotent value updater for a file",epilog="If searchstring is not found, deststring will be appended to infile")
parser.add_argument("-v","--verbose",help="displays output",      action="store_true",default=False)
parser.add_argument("-a","--apply",  help="perform substitution", action="store_true",default=False)
parser.add_argument("-L","--all",    help="replace all instances",action="store_true",default=False)
parser.add_argument("infile", help="file to use")
parser.add_argument("searchstring", help="regex string to search")
parser.add_argument("deststring", help="literal string that should be there")
parser.add_argument("-V","--version", action="version", version="%(prog)s " + updatevalversion)
args = parser.parse_args()

# Configure variables after parameters
verbose = args.verbose
doapply = args.apply
doall = args.all
infile = args.infile
searchstring = args.searchstring
destinationstring = args.deststring

wasfixed = False
outfile = infile + ".updateval-new"

# Make file if it does not exist
if not os.path.isfile(infile): open(infile, "w").close()

# If line exists, replace it
shutil.copy2(infile,outfile) # initialize duplicate file with same perms
with open(outfile, "w") as outf:
   for line in open(infile, "r"):
      p = re.compile(searchstring)
      if p.match(line) and ( not wasfixed or doall ):
         outline = re.sub(searchstring,destinationstring, line).rstrip('\n')
         wasfixed = True
         outline = line.rstrip('\n')
      if verbose: print(outline)
      #print(outline,file = outf)
      outf.write(outline + '\n')

# Append line if it has not been fixed yet
if not wasfixed:
   with open(outfile, "a") as outf:
      if verbose: print(destinationstring)
      outf.write(destinationstring + '\n')

# replace old file with new file
if doapply:

# Clean up outfile just in case
except Exception as e:

I’ll explain some of the code, just in case the comments don’t do a good job.
Lines 37-42 Whenever i use argparse, I always pull the information out of the argparse object into local variables. If I rename the object or parameter names, I don’t want to have to go update my code everywhere any of those variables are mentioned.

Line 45: This is an arbitrarily named, temporary export file. I wasn’t sure I could do an open() with read-write, so I just duplicate the file and then replace the original (if doapply=true).

Line 48: Basically, touch the file to make sure it exists.

Lines 51-62: This right here (specifically lines 55-56) is the reason I rewrote this in python. This was much harder in shell. Observe around line 61 how the print command was commented out in favor of the outf.write() object method call. The reason I had to give up the print command here was because when I was packaging my bgscripts set for rpm, it was compiling the python with the python2 interpreter, and it was easier to adjust my python code than which python interpreter rpmbuild uses. So a google search got me the output command. I don’t care how I get the task done; I want the task done.

My requirements included appending the destinationstring if the searchstring did not appear in the code. So lines 65-68 handle that.

Lines 70-78 are needed only because of the writing to the other file. If I could write to the original file, I wouldn’t need to do file manipulation.