Category: Bash

Jul 20

Bash Date Usage For Naming

I am recording some scripting I used to create backup classification/retention naming. It can be simplified into one function easily but I kept it like this so I can copy and paste easier which function I need. Script is pretty self explanatory but basically it takes today's date and name my eventual backup file name based on some logic.

# cat test_class.sh 
HOSTNAME=$(hostname -s)
BACKUP_CLASSIFICATION="UNCLASSIFIED"

function retention_date() {
  MM=`date -d ${1} +%m`
  DD=`date -d ${1} +%d`
  DAY=`date -d ${1} +%u`

  if [ $DD == 01 ]
  then
     if [ $MM == 01 ]
     then
       BACKUP_CLASSIFICATION="YEARLY"
     else
       BACKUP_CLASSIFICATION="MONTHLY"
     fi
  else
    if (($DAY == 7)); then
     BACKUP_CLASSIFICATION="WEEKLY"
    else
     BACKUP_CLASSIFICATION="DAILY"
    fi
  fi

}

function retention_today() {
  MM=`date '+%m'`
  DD=`date '+%d'`
  DAY=`date +%u`
  
  if [ $DD == 01 ]
  then
     if [ $MM == 01 ]
     then
       BACKUP_CLASSIFICATION="YEARLY"
     else
       BACKUP_CLASSIFICATION="MONTHLY"
     fi
  else
    if (($DAY == 7)); then
     BACKUP_CLASSIFICATION="WEEKLY"
    else
     BACKUP_CLASSIFICATION="DAILY"
    fi
  fi

}

echo "TEST TODAY"
DATE=`date +%Y-%m-%d`
retention_today
echo $HOSTNAME-$BACKUP_CLASSIFICATION-$DATE
  
echo 
echo "TEST SPECIFIC DATES"
testD=(
 '2018-01-01'
 '2018-02-02'
 '2018-03-01'
 '2018-02-06'
 '2018-07-14'
 '2018-07-15'
)

for D in "${testD[@]}"
do
  DATE=`date -d ${D} +%Y-%m-%d`
  retention_date $D
  echo $HOSTNAME-$BACKUP_CLASSIFICATION-$DATE
done

Run and output.

# ./test_class.sh 
TEST TODAY
oci04-DAILY-2018-07-20

TEST SPECIFIC DATES
oci04-YEARLY-2018-01-01
oci04-DAILY-2018-02-02
oci04-MONTHLY-2018-03-01
oci04-DAILY-2018-02-06
oci04-DAILY-2018-07-14
oci04-WEEKLY-2018-07-15

Comments Off on Bash Date Usage For Naming
comments

Posted in Backups, Bash

Jan 19

Expect and bash

Quick example of spawning a problem that have a password prompt and can't accept a documented parameter for a password. I used another bash script to simulate a password prompt but in my automation challenge it was an executable that prompted.

Main script just to take a password for passing to expect. Could also be hard coded.

$ cat expect_example_main.sh
#!/bin/bash
echo "Enter the password: "
read -s -e password
./expect_example.exp $password ;

Here is the expect script that will be interacting with the real program with the password prompt.

$ cat expect_example.exp
#!/usr/bin/expect -f

# Set variables
set password [lindex $argv 0]
set date [exec date +%F]

# Log results
log_file -a expect-$date.log

# Announce device & time
send_user "\n"
send_user ">>>>> Working @ [exec date] <<<<<\n" send_user "\n" spawn ./expect_example_prompt.sh expect "*assword:" {send "$password\r"} interact This is the simulated executable with the prompt. Expect will be spawning this one. $ cat expect_example_prompt.sh #!/bin/bash echo "Enter the password: " read -s -e pwd if [ $pwd == 'fool' ]; then echo "password correct" else echo "password NOT correct!" fi Showing run time with correct password. $ ./expect_example_main.sh Enter the password: >>>>> Working @ Thu Jan 19 14:54:00 CST 2017 <<<<< spawn ./expect_example_prompt.sh Enter the password: password correct Showing run time with incorrect password. $ ./expect_example_main.sh Enter the password: >>>>> Working @ Thu Jan 19 14:54:28 CST 2017 <<<<< spawn ./expect_example_prompt.sh Enter the password: password NOT correct!

Comments Off on Expect and bash
comments

Posted in Bash, Expect

Sep 09

Powerline for your terminal on Ubuntu

I noticed on Fedora it is really easy to enable powerline in your terminal. This article explains it well: http://fedoramagazine.org/add-power-terminal-powerline/

On Ubuntu 15.04 it looks like powerline is pretty easy to enable although it did not appear like the tmux or vim powerline plugins are available from regular repos.

Enable powerline in your terminal like this:

# apt-get install powerline fonts-powerline

~ cat .bashrc
[..]
if [ -f `which powerline-daemon` ]; then
  powerline-daemon -q
  POWERLINE_BASH_CONTINUATION=1
  POWERLINE_BASH_SELECT=1
  . /usr/share/powerline/bindings/bash/powerline.sh
fi

3
comments

Posted in Bash, Ubuntu

Dec 20

ZFSSA List Snapshots Script

Quick script to illustrate interacting with the ZFS Storage Appliance. In this example I am listing ZFSSA snapshots containing a search string.  Note I edited this for the article without re-testing it still works.

#!/bin/sh

Usage() {
 echo "$1 -u <Appliance user> -h <appliance> -j <project> -p <pool> -s <containsString>"
 exit 1
}

PROG=$0
while getopts u:h:s:j:p flag
do
  case "$flag" in
  p) pool="$OPTARG";;
  j) project="$OPTARG";;
  s) string="$OPTARG";;
  u) user="$OPTARG";;
  h) appliance="$OPTARG";;
  \?) Usage $PROG ;;
  esac
done

[ -z "$pool" -o -z "$project" -o -z "$appliance" -o -z "$user" ] && Usage $PROG

ssh -T $user@$appliance << EOF
script
var MyArguments = {
  pool: '$pool',
  project: '$project',
  string: '$string'
}

function ListSnapshotsbyS (Arg) {
  run('cd /');                          // Make sure we are at root child context level
  run('shares');
  try {
      run('set pool=' + Arg.pool);
  } catch (err) {
      printf ('ERROR: %s\n',err);
      return (err);
  }

  var allSnaps=[];
  try {
      run('select ' + Arg.project + ' snapshots');
      snapshots=list();
      for(i=0; i < snapshots.length; i++) {
          allSnaps.push(snapshots[i]);
      }
        run('done');
  } catch (err) {
      printf ('ERROR: %s\n',err);
            return(err);
  }

  for(i=0; i < allSnaps.length; i++) {
   if (Arg.string !="") {
    var idx=allSnaps[i].indexOf(Arg.string);
    if (idx>0) {
      printf('#%i: %s contained search string %s \n',i,allSnaps[i], Arg.string);
    }
   } else {
      printf('#%i: %s \n',i,allSnaps[i]);
   }
  }
  return(0);
}
ListSnapshotsbyS(MyArguments);
.
EOF

Comments Off on ZFSSA List Snapshots Script
comments

Posted in Bash, ZFS

Sep 23

Watch Process Id

Sometimes you want to keep tabs on a long running process and get notified by email when it is done. This is an example of just that.

#!/bin/bash
pid=$1
me="$(basename $0)($$):"
if [ -z "$pid" ]
then
    echo "$me a PID is required as an argument" >&2
    exit 2
fi

name=$(ps -p $pid -o comm=)
if [ $? -eq 0 ]
then
    echo "$me waiting for PID $pid to finish ($name)"
    while ps -p $pid > /dev/null; do sleep 1; done;
else
    echo "$me failed to find process with PID $pid" >&2
    exit 1
fi
## I used a python mailer but mostlikely this will be mail or mailx.
python pymail.py $pid

Comments Off on Watch Process Id
comments

Posted in Bash, Uncategorized

Jun 02

Multi-Array in Bash

Well kind of...  When you are used to working in Python or any real language then Bash arrays are pretty lame.  But it can help in a few circumstances when you have to use Bash.

# cat bash_array.sh
#!/bin/bash
# Array pretending to be a Pythonic dictionary
ARRAY_DETAILS=( "10.51.20.63:Host1:Solaris"
                "10.51.20.80:Host2:Linux"
                "10.20.50.11:Host3:Windows" )

for hostDetails in "${ARRAY_DETAILS[@]}" ; do
  arrIN=(${hostDetails//:/ })
  IP=${arrIN[0]}
  NAME=${arrIN[1]}
  TYPE=${arrIN[2]}
  printf "Hostname %s with IP %s and is type %s.\n" "$NAME" "$IP" "$TYPE"
done

And the result looks like this:

# ./bash_array.sh
Hostname Host1 with IP 10.51.20.63 and is type Solaris.
Hostname Host2 with IP 10.51.20.80 and is type Linux.
Hostname Host3 with IP 10.20.50.11 and is type Windows.

Comments Off on Multi-Array in Bash
comments

Posted in Bash

Feb 05

Bash And Exclusions in a List

Just a quick snippet on doing exclusions when you loop through a list.

DIRS=`ls -l --time-style="long-iso" $MYDIR | egrep '^d' | awk '{print $8}'`
EXCLUDELIST="mail Mail"

for EXCLUDE in $EXCLUDELIST
do
    DIRS=`echo $DIRS | sed "s/\b$EXCLUDE\b//g"`
done

for DIR in $DIRS
do
    echo  ${DIR} :
done

For some reason on Solaris sed had an issue with "\b" so I adjusted to sed "s/$EXCLUDE//g". Shown as follow:

#Linux:
$ echo "d1 d2 d3" | sed "s/\bd2\b//g"
d1  d3

# Solaris Fails:
# echo "d1 d2 d3" | sed "s/\bd2\b//g"
d1 d2 d3

# Solaris Works
# echo "d1 d2 d3" | sed "s/d2//g"
d1  d3

Comments Off on Bash And Exclusions in a List
comments

Posted in Bash

Jul 29

Disown and background a Unix process

Ever run a very large job and regretting not starting it in the excellent screen utility? If you don't have something like reptyr or retty, you can do the following.

Push the running job into the background using Control-Z and then background it. Then disown that job from the terminal. At least it will keep running. And if you want to kick off another job when the disowned process finish you can run a little script in a new terminal and checking for the disowned job to finish. Running the new script in screen first off course.

Background and disown process:

# rsync -av /zfsapp/u06/* /backup/u06/
sending incremental file list
temp01.dbf
^Z
[1]+  Stopped                 rsync -av /zfsapp/u06/* /backup/u06/

# bg
[1]+ rsync -av /zfsapp/u06/* /backup/u06/ &

# disown %1

# ps -ef | grep u06
    root 23903 23902   1 07:05:07 pts/5       0:01 rsync -av /zfsapp/u06/temp01.dbf
    root 23901  2656   1 07:05:07 pts/5       0:01 rsync -av /zfsapp/u06/temp01.dbf
    root 23902 23901   0 07:05:07 pts/5       0:00 rsync -av /zfsapp/u06/temp01.dbf

Check for a process id to finish before starting a new job:

# more cp_u06.sh
#!/bin/bash
while ps -p 23903 > /dev/null;
do
 printf "."
 sleep 60;
done
echo
echo "last rsync finished starting new"
rsync -av /zfsapp/u07/* /backup/u07/

Comments Off on Disown and background a Unix process
comments

Posted in Bash, Linux

Jul 10

Using Bash for root user on Solaris 10

By default Solaris 10 use "/" as root's home directory and plain sh as a shell.  If you want to change to using the /root directory as home and bash as a shell for more consistency with Solaris 11 you can do the following.

...
Oracle Corporation SunOS 5.10 Generic Patch January 2005

root@ldom2:~# grep root /etc/passwd
root:x:0:0:Super-User:/root:/usr/bin/bash

root@ldom2:~# mkdir /root
root@ldom2:~# pwd
/root

root@ldom2:~# cat .profile
#
# Simple profile places /usr/bin at front, followed by /usr/sbin.
#
# Use less(1) or more(1) as the default pager for the man(1) command.
#
export PATH=/usr/bin:/usr/sbin

if [ -f /usr/bin/less ]; then
 export PAGER="/usr/bin/less -ins"
elif [ -f /usr/bin/more ]; then
 export PAGER="/usr/bin/more -s"
fi

#
# Define default prompt to <username>@<hostname>:<path><"($|#) ">
# and print '#' for user "root" and '$' for normal users.
#
# Currently this is only done for bash/pfbash(1).
#

case ${SHELL} in
*bash)
 typeset +x PS1="\u@\h:\w\\$ "
 ;;
esac

root@ldom2:~# cat .bashrc
#
# Define default prompt to <username>@<hostname>:<path><"($|#) ">
# and print '#' for user "root" and '$' for normal users.
#

typeset +x PS1="\u@\h:\w\\$ "

Logout and back in and your shell should be bash and prompt fixed as well.

Comments Off on Using Bash for root user on Solaris 10
comments

Posted in Bash, Solaris

Nov 26

Unix Find and Remove Python Style

No doubt the Unix find and remove command comes in very useful when cleaning up large folders.  However find can quickly bump into the old "/usr/bin/find: Argument list too long" problem.

For reference here is a command that works well.  Except of course when too many files or directories involved.

find /usr/local/forms/*/output -name "*.html" -mtime +4 -exec rm {} \;

There is of course other ways to get this done with find, but I like python so I resorted to python as the example show below.

Here is an example that worked for me:


#!/usr/bin/env python
##  Adjust the humandays variable.  Set to 2000 days until we feel more comfortable.

import os
import glob
import time
import shutil

humandays = 2000
computerdays = 86400*humandays
now = time.time()
inputDirs = glob.glob('/usr/local/forms/*/input')

print "Script running on %s " % time.ctime(now)
print "using physical path /usr/local/forms/*/input and only removing directories older than %s days." % (humandays)

for inputDir in inputDirs:
  for r,d,f in os.walk(inputDir):
    for dir in d:
         timestamp = os.path.getmtime(os.path.join(r,dir))
         if now-computerdays > timestamp:
             try:
		  print "modified: %s" % time.ctime(timestamp),
		  removeDir=os.path.join(r,dir)
                  print "  remove ",removeDir,
                  ## Better be 100% sure before you uncomment the rmtree line!
                  ## shutil.rmtree(removeDir)
                  ## Better be 100% sure before you uncomment the rmtree line!
             except Exception,e:
                  print e
                  pass
             else:
                  print " -> success"

Comments Off on Unix Find and Remove Python Style
comments

Posted in Bash, Python

« Older Entries
Newer Entries »