Jan 14

Solaris 11.2 SRU updates

Previously I wrote about updating Solaris 11.1 without going full blown IPS. Oracle used to have incremental ISO's to download. I do not see those anymore for Solaris 11.2. Here is a previous article for reference:

Solaris 11.1 Update from ISO

In a data center I would maintain an IPS repo and incrementally update the repo when SRU's are released. Guests are then updated against the repo. If you don't want to maintain full blown IPS and just want to update a specific guest to a specific SRU version this article is focusing on that.

Some useful links you may want to look through first:
Create Local Repo:
http://www.oracle.com/technetwork/server-storage/solaris11/downloads/local-repository-2245081.html
http://www.oracle.com/technetwork/articles/servers-storage-admin/howto-set-up-repos-mirror-ips-2266101.html
http://appsdbaworkshop.blogspot.com/2014/09/configure-ips-repository-on-oracle.html

SRU Index:
Oracle Solaris 11.1 Support Repository Updates (SRU) Index (Doc ID 1501435.1)
Oracle Solaris 11.2 Support Repository Updates (SRU) Index (Doc ID 1672221.1)
https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=390313668034834&id=1672221.1&displayIndex=8&_afrWindowMode=0&_adf.ctrl-state=psan9sn51_234

We will focus on these steps:
Step 1. Stage SRU zip files and turn into a repo with install-repo.ksh.

Step 2. Update guest against repo.

Step 3. Optionally create an ISO.

Step 4: Update from custom ISO created in step 3.

I use Virtualbox for my testing so you can choose how to present folders or ISO's to the guest. Since I have the guest additions on Solaris 11.2 guest I chose to use that instead of one of the links shows screenshots to create a ISO in Windows and present to Virtualbox that way.

Step 1: Stage SRU zip files and turn into a repo with install-repo.ksh.

Download and stage SRU zip files:

# ls -l 11.2_SRU5.5_src/
total 5370490
-rwxrwx---   1 root     vboxsf   1539220019 Jan  7 09:43 p20131812_1100_Solaris86-64_1of2.zip
-rwxrwx---   1 root     vboxsf   1210469972 Jan  7 09:37 p20131812_1100_Solaris86-64_2of2.zip

Download install-repo.ksh script and md5sums.txt:
** Note I initially ignored md5sums.txt and that will cause a warning only.  Better off just downloading that as well.

# ls -l install-repo.ksh
-rwxrwx---   1 root     vboxsf      5594 Jan  9 08:20 install-repo.ksh

Prepare repo:

# pwd
/mnt/sf_DATA/isos/Solaris
# mkdir IPS_SRU_11_2_5_5_0
# pwd
/mnt/sf_DATA/isos/Solaris/11.2_SRU5.5

# ../install-repo.ksh -d ../IPS_SRU_11_2_5_5_0
/mnt/sf_DATA/isos/Solaris/11.2_SRU5.5_src/sol-*-repo-md5sums.txt: No such file or directory
Uncompressing p20131812_1100_Solaris86-64_1of2.zip...done.
Uncompressing p20131812_1100_Solaris86-64_2of2.zip...done.</pre>
<pre>Repository can be found in ../IPS_SRU_11_2_5_5_0. 

** When I used a Virtualbox guest additions  mount as a target for the repo the install will work but I had issues later updating the guest.
** Also keep in mind with this method of updating you still need the Oracle repo as well to solve any dependencies introduced in the SRU.

# pwd
/mnt/sf_DATA/isos/Solaris/11.2_SRU5.5

# pkg set-publisher -G '*' -g file:////mnt/sf_DATA/isos/Solaris/IPS_SRU_11_2_5_5_0/ solaris
pkg set-publisher: The origin URIs for 'solaris' do not appear to point to a valid pkg repository.
Please verify the repository's location and the client's network configuration.
Additional details:

Unable to contact valid package repository
Encountered the following error(s):
Unable to contact any configured publishers.
This is likely a network configuration problem.
file protocol error: code: 71 reason: The following catalog files have incorrect permissions:
	/mnt/sf_DATA/isos/Solaris/IPS_SRU_11_2_5_5_0/publisher/solaris/catalog/catalog.attrs: expected mode: 644, found mode: 770

** Redid the repo prep to local / file system. I had enough room there so it worked fine.

# pwd
/mnt/sf_DATA/isos/Solaris/11.2_SRU5.5
# ../install-repo.ksh -d /IPS_SRU_11_2_5_5_0
/mnt/sf_DATA/isos/Solaris/11.2_SRU5.5/sol-*-repo-md5sums.txt: No such file or directory
Uncompressing p20131812_1100_Solaris86-64_1of2.zip...done.
Uncompressing p20131812_1100_Solaris86-64_2of2.zip...done.
Repository can be found in /IPS_SRU_11_2_5_5_0.

Step 2: Update guest against repo.

# pkg set-publisher -g file:////IPS_SRU_11_2_5_5_0/ solaris

# pkg publisher
PUBLISHER                   TYPE     STATUS P LOCATION
solaris                     origin   online F file:///IPS_SRU_11_2_5_5_0/
solaris                     origin   online F http://pkg.oracle.com/solaris/release/

# pkg update -nv

# pkg update
            Packages to remove:   1
            Packages to update: 153
       Create boot environment: Yes
Create backup boot environment:  No
DOWNLOAD                                PKGS         FILES    XFER (MB)   SPEED
Completed                            154/154     9014/9014  345.9/345.9    0B/s

PHASE                                          ITEMS
Removing old actions                       1742/1742
Installing new actions                     2635/2635
Updating modified actions                  8361/8361
Updating package state database                 Done
Updating package cache                       154/154
Updating image state                            Done
Creating fast lookup database                   Done
Updating package cache                           1/1 

A clone of solaris-1 exists and has been updated and activated.
On the next boot the Boot Environment solaris-2 will be
mounted on '/'.  Reboot when ready to switch to this updated BE.

Updating package cache                           1/1 

---------------------------------------------------------------------------
NOTE: Please review release notes posted at:

http://www.oracle.com/pls/topic/lookup?ctx=solaris11&id=SERNS
---------------------------------------------------------------------------

# beadm list
BE                 Active Mountpoint Space  Policy Created
--                 ------ ---------- -----  ------ -------
solaris-1          N      /          29.69M static 2014-07-31 15:01
solaris-1-backup-1 -      -          183.0K static 2014-08-28 09:39
solaris-1-backup-2 -      -          191.0K static 2014-12-02 12:05
solaris-2          R      -          13.49G static 2015-01-09 10:21

Step 3: Optionally create an ISO.

# pwd
/mnt/sf_DATA/isos/Solaris/11.2_SRU5.5
# mkdir /IPS_SRU_11_2_5_5_0
# ./install-repo.ksh -d /IPS_SRU_11_2_5_5_0 -I -v
Uncompressing p20131812_1100_Solaris86-64_1of2.zip...done.
Uncompressing p20131812_1100_Solaris86-64_2of2.zip...done.
Repository can be found in /IPS_SRU_11_2_5_5_0.
Initiating repository verification.
Building ISO image...
...done.
ISO image and instructions for using the ISO image are at:
/mnt/sf_DATA/isos/Solaris/11.2_SRU5.5/sol-11_2-repo.iso
/mnt/sf_DATA/isos/Solaris/11.2_SRU5.5/README-repo-iso.txt

Step 4: Update from custom ISO created in step 3.
** Attach ISO to guest

root@solaris11:~# pkg list entire
NAME (PUBLISHER)                                  VERSION                    IFO
entire                                            0.5.11-0.175.2.0.0.42.0    i--

root@solaris11:~# pkg publisher
PUBLISHER                   TYPE     STATUS P LOCATION
solaris                     origin   online F http://pkg.oracle.com/solaris/release/

root@solaris11:~# pkg set-publisher -g file:///media/SOL-11_2_REPO/repo solaris

root@solaris11:~# pkg publisher
PUBLISHER                   TYPE     STATUS P LOCATION
solaris                     origin   online F file:///media/SOL-11_2_REPO/repo/
solaris                     origin   online F http://pkg.oracle.com/solaris/release/
</pre>
<pre>root@solaris11:~# pkg update
</pre>
<pre>root@solaris11:~# pkg list entire NAME (PUBLISHER) VERSION IFO entire 0.5.11-0.175.2.5.0.5.0 i-- 

Comments Off on Solaris 11.2 SRU updates
comments

Jan 04

Creating a javascript array with one to many type relationship

Sometimes I need to build an array of lists with a one to many type relationship.

For example this data:
1. item1 has a parent of key1.
2. item2 has a parent of key2.
3. item3 has a parent of key1.

Using PHP after packing the array I will end up with something like this:

<?php

$m = array();

$m['key1'][] = 'item1';
$m['key2'][] = 'item2';
$m['key1'][] = 'item3';

print_r($m);

?>

Result:

Array
(
    [key1] => Array
        (
            [0] => item1
            [1] => item3
        )

    [key2] => Array
        (
            [0] => item2
        )

)

In javascript there is some options to do something similar. I ended up with something like this:
packedArr['key1'] = "item1,item3"
packedArr['key2'] = "item2"

Not exactly array in array[key] but the comma separated list did what I needed.

Here is the javascript itself. To test it you can use online javascript testers or just make up a html page with a button and maybe have onclick display document.getElementById("demo").innerHTML = output;

<script>
function packArr() {
var pA = Array();

var inA = [ 
  ["child1","parent1"],
  ["child2","parent2"],
  ["child3","parent1"],
  ["child4","parent1"],
  ["child5","parent2"]
  ];

var inALen = inA.length;
for (var i = 0; i < inALen; i++) {
    //alert(inA[i]);
  if (pA[inA[i][1]] == null) {
     pA[inA[i][1]] = inA[i][0];
  } else {
    pA[inA[i][1]] = pA[inA[i][1]] + "," + inA[i][0];
  }
}

res="";
for ( var key in pA ) { 
  res = res + key + ": " + pA[key] + "<br>";
};
//res = myArray["parent1"]; 
document.getElementById("demo").innerHTML = res;
}
</script>

Comments Off on Creating a javascript array with one to many type relationship
comments

Jan 01

Ubuntu On a ZFS Root File System for Ubuntu 15.04

Start Update 03.18.15:
This is untested but I suspect if you are upgrading the kernel to 3.19.0 and you have issues you may need to change to the daily Vivid ppa. In my initial post I used stable and Utopic since Vivid was very new.
End Update 03.18.15:

This is what I did to make an Ubuntu 15.04 virtualbox guest (works for Ubuntu 14.10 also) boot with the ZFS file system.

Previous articles:

Ubuntu On a ZFS Root File System for Ubuntu 14.04

Booting Ubuntu on a ZFS Root File System

SYSTEM REQUIREMENTS
64-bit Ubuntu Live CD. (Not the alternate or 32-bit installer)
AMD64 or EM64T compatible computer. (ie: x86-64)
15GB disk, 2GB memory minimum, Virtualbox
Create new VM. I use bridged networking in case I want to use ssh during the setup.
Start the Ubuntu LiveCD and open a terminal at the desktop. I used the 15.04 64-bit alpha CD.
Control-F1 to first text terminal.

1. Setup repo.

$ sudo -i
# /etc/init.d/lightdm stop
# apt-add-repository --yes ppa:zfs-native/stable

** Change /etc/apt/sources.list.d/... down to trusty. Or utopic possibly. I did not test.

2. Install zfs.

# apt-get update
# apt-get install debootstrap ubuntu-zfs
# dmesg | grep ZFS:
[ 3900.114234] ZFS: Loaded module v0.6.3-4~trusty, ZFS pool version 5000, ZFS filesystem version 5

** takes a long time to compile initial module for 3.16.0-28-generic

3. Install ssh.

Using a ssh terminal makes it easier to copy and paste for both command execution and documentation. However with this bare bones environment at this point openssh might not install very clean. I played with it a little to get at least sshd to run.

# apt-get install ssh
# /etc/init.d/ssh start
# /usr/sbin/sshd

** check with ps if ssh process is running
** edit sshd_config and allow root login
** set root passwd

4. Setup disk partitions.

# fdisk -l
Disk /dev/loop0: 1 GiB, 1103351808 bytes, 2154984 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/sda: 15 GiB, 16106127360 bytes, 31457280 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xc2c7def9

Device Boot Start End Sectors Size Id Type
/dev/sda1 2048 411647 409600 200M be Solaris boot
/dev/sda2 411648 31457279 31045632 14.8G bf Solaris

5. Format partitions.

# mke2fs -m 0 -L /boot/grub -j /dev/disk/by-id/ata-VBOX_HARDDISK_VBf3c0d5ba-e6881c52-part1
# zpool create -o ashift=9 rpool /dev/disk/by-id/ata-VBOX_HARDDISK_VBf3c0d5ba-e6881c52-part2

6. ZFS Setup and Mountpoints.

# zpool create -o ashift=9 rpool /dev/disk/by-id/ata-VBOX_HARDDISK_VBf3c0d5ba-e6881c52-part2
# zfs create rpool/ROOT
# zfs create rpool/ROOT/ubuntu-1
# zfs umount -a
# zfs set mountpoint=/ rpool/ROOT/ubuntu-1
# zpool set bootfs=rpool/ROOT/ubuntu-1 rpool
# zpool export rpool
# zpool import -d /dev/disk/by-id -R /mnt rpool
# mkdir -p /mnt/boot/grub
# mount /dev/disk/by-id/ata-VBOX_HARDDISK_VBf3c0d5ba-e6881c52-part1 /mnt/boot/grub

7. Install Ubuntu 15.04 on /mnt

# debootstrap vivid /mnt
I: Retrieving Release
...
I: Base system installed successfully.

# cp /etc/hostname /mnt/etc/
# cp /etc/hosts /mnt/etc/
# vi /mnt/etc/fstab
# cat /mnt/etc/fstab
/dev/disk/by-id/ata-VBOX_HARDDISK_VBf3c0d5ba-e6881c52-part1 /boot/grub auto defaults 0 1

# cat /mnt/etc/network/interfaces
# interfaces(5) file used by ifup(8) and ifdown(8)
# Include files from /etc/network/interfaces.d:
source-directory /etc/network/interfaces.d
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet dhcp

8. Setup chroot to update and install ubuntu-minimal

# mount --bind /dev /mnt/dev
# mount --bind /proc /mnt/proc
# mount --bind /sys /mnt/sys
# chroot /mnt /bin/bash --login
# locale-gen en_US.UTF-8

# apt-get update
# apt-get install ubuntu-minimal software-properties-common

9. Setup ZOL repo

# apt-add-repository --yes ppa:zfs-native/stable

** leave grub repo off for now.

# cat /etc/apt/sources.list.d/zfs-native-ubuntu-stable-vivid.list
deb http://ppa.launchpad.net/zfs-native/stable/ubuntu trusty main
# deb-src http://ppa.launchpad.net/zfs-native/stable/ubuntu vivid main
# apt-get update
# apt-get install --no-install-recommends linux-image-generic linux-headers-generic

# apt-get install ubuntu-zfs

** skipped grub stuff for this pass

# apt-get install zfs-initramfs
# apt-get dist-upgrade

10. Make sure root has access

# passwd root

11. Test grub

# grub-probe /
bash: grub-probe: command not found

12. Use older patched grub from ZOL project

# apt-add-repository --yes ppa:zfs-native/grub
gpg: keyring `/tmp/tmp5urr4u7g/secring.gpg' created
gpg: keyring `/tmp/tmp5urr4u7g/pubring.gpg' created
gpg: requesting key F6B0FC61 from hkp server keyserver.ubuntu.com
gpg: /tmp/tmp5urr4u7g/trustdb.gpg: trustdb created
gpg: key F6B0FC61: public key "Launchpad PPA for Native ZFS for Linux" imported
gpg: Total number processed: 1
gpg: imported: 1 (RSA: 1)
OK

# cat /etc/apt/sources.list.d/zfs-native-ubuntu-grub-vivid.list
deb http://ppa.launchpad.net/zfs-native/grub/ubuntu raring main

# apt-get install grub2-common grub-pc
Installation finished. No error reported.
/usr/sbin/grub-probe: error: failed to get canonical path of `/dev/ata-VBOX_HARDDISK_VBf3c0d5ba-e6881c52-part2'.

** As you can see grub has issues with dev path.

# ln -s /dev/disk/by-id/ata-VBOX_HARDDISK_VBf3c0d5ba-e6881c52-part2 /dev/ata-VBOX_HARDDISK_VBf3c0d5ba-e6881c52-part2
# apt-get install grub2-common grub-pc

# grub-probe /
zfs
# ls /boot/grub/i386-pc/zfs*
/boot/grub/i386-pc/zfscrypt.mod /boot/grub/i386-pc/zfsinfo.mod /boot/grub/i386-pc/zfs.mod

** Note at the end I show a udev rule that can help work around this path issue.

# update-initramfs -c -k all

# grep "boot=zfs" /boot/grub/grub.cfg
linux /ROOT/ubuntu-1@/boot/vmlinuz-3.16.0-28-generic root=ZFS=rpool/ROOT/ubuntu-1 ro boot=zfs quiet splash $vt_handoff

# grep "boot=zfs" /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="quiet splash boot=zfs"

# update-grub
Generating grub configuration file ...
Warning: Setting GRUB_TIMEOUT to a non-zero value when GRUB_HIDDEN_TIMEOUT is set is no longer supported.
Found linux image: /boot/vmlinuz-3.16.0-28-generic
Found initrd image: /boot/initrd.img-3.16.0-28-generic
done

# grub-install $(readlink -f /dev/disk/by-id/ata-VBOX_HARDDISK_VBf3c0d5ba-e6881c52)
Installing for i386-pc platform.
Installation finished. No error reported.

# exit
logout

13. unmount chroot and shutdown

# umount /mnt/boot/grub
# umount /mnt/dev
# umount /mnt/proc
# umount /mnt/sys
# zfs umount -a
# zpool export rpool
# init 0

14. Cleanup and finish
** Create snapshot
** bootup
** install ssh and configure root to login in sshd_config, restart ssh

15. udev rule for grub bug

# cat /etc/udev/rules.d/70-zfs-grub-fix.rules
ENV{DEVTYPE}=="partition", IMPORT{parent}="ID_*", ENV{ID_FS_TYPE}=="zfs_member", SYMLINK+="$env{ID_BUS}-$env{ID_SERIAL} $env{ID_BUS}-$env{ID_SERIAL}-part%n"

# /etc/init.d/udev restart

16. install desktop software

# apt-get install ubuntu-desktop

Comments Off on Ubuntu On a ZFS Root File System for Ubuntu 15.04
comments

Dec 20

ZFSSA List Snapshots Script

Quick script to illustrate interacting with the ZFS Storage Appliance. In this example I am listing ZFSSA snapshots containing a search string.  Note I edited this for the article without re-testing it still works.

#!/bin/sh

Usage() {
 echo "$1 -u <Appliance user> -h <appliance> -j <project> -p <pool> -s <containsString>"
 exit 1
}

PROG=$0
while getopts u:h:s:j:p flag
do
  case "$flag" in
  p) pool="$OPTARG";;
  j) project="$OPTARG";;
  s) string="$OPTARG";;
  u) user="$OPTARG";;
  h) appliance="$OPTARG";;
  \?) Usage $PROG ;;
  esac
done

[ -z "$pool" -o -z "$project" -o -z "$appliance" -o -z "$user" ] && Usage $PROG

ssh -T $user@$appliance << EOF
script
var MyArguments = {
  pool: '$pool',
  project: '$project',
  string: '$string'
}

function ListSnapshotsbyS (Arg) {
  run('cd /');                          // Make sure we are at root child context level
  run('shares');
  try {
      run('set pool=' + Arg.pool);
  } catch (err) {
      printf ('ERROR: %s\n',err);
      return (err);
  }

  var allSnaps=[];
  try {
      run('select ' + Arg.project + ' snapshots');
      snapshots=list();
      for(i=0; i < snapshots.length; i++) {
          allSnaps.push(snapshots[i]);
      }
        run('done');
  } catch (err) {
      printf ('ERROR: %s\n',err);
            return(err);
  }

  for(i=0; i < allSnaps.length; i++) {
   if (Arg.string !="") {
    var idx=allSnaps[i].indexOf(Arg.string);
    if (idx>0) {
      printf('#%i: %s contained search string %s \n',i,allSnaps[i], Arg.string);
    }
   } else {
      printf('#%i: %s \n',i,allSnaps[i]);
   }
  }
  return(0);
}
ListSnapshotsbyS(MyArguments);
.
EOF

Comments Off on ZFSSA List Snapshots Script
comments

Dec 06

Live Migrate Oracle VM for SPARC Logical Domains

Short version of live migration between two T5-2's.  Using shared fiber LUN's.

Source Host:

Check running status:

# ldm ls | grep test10
test10           active     -n----  5002    8     8G       0.0%  0.0%  1m

Dry run first:

# ldm migrate-domain -n test10 10.2.10.12
Target Password:
Invalid shutdown-group: 0
Failed to recreate domain on target machine
Domain Migration of LDom test10 would fail if attempted

We have a couple issues.

# ldm set-domain shutdown-group=15 test10

# ldm migrate-domain -n test10 10.2.10.12
Target Password:
Failed to find required volume test10-disk0@primary-vds0 on target machine
Domain Migration of LDom test10 would fail if attempted

# ldm ls-bindings primary | grep test10
test10@primary-vcc          5002   on
vnet1@test10                00:14:4f:fa:17:4f 1913                      1500
test10-disk0                                   /dev/dsk/c0t600144F09D7311B5000054789ED30002d0s2
disk0@test10                test10-disk0

Target Host:
Fix the virtual device reference.

# ldm add-vdsdev /dev/dsk/c0t600144F09D7311B5000054789ED30002d0s2 test10-disk0@primary-vds0

# ldm migrate-domain -n test10 10.2.10.12

Check status on target.  I suggest also running a ping during the whole process.

# ldm ls | grep test10 test10           bound      -----t  5018    8     8G
# ldm ls | grep test10 test10           active     -n----  5018    8     8G       0.1%  0.1%  7m

Comments Off on Live Migrate Oracle VM for SPARC Logical Domains
comments

Nov 06

Sorting IP addresses in Excel

Sometimes you need to manipulate lists of IP addresses in Excel but of course it sorts on strings which is not ideal for IP addresses. A quick way to sort is adding another column with the following formula and sorting on that new column.

=((VALUE(LEFT(B6, FIND(".", B6)-1)))*256^3)+((VALUE(MID(B6, FIND(".", B6)+1, FIND(".", B6, FIND(".", B6)+1)-FIND(".", B6)-1)))*256^2)+((VALUE(MID(B6, FIND(".", B6, FIND(".", B6)+1)+1, FIND(".", B6, FIND(".", B6, FIND(".", B6)+1)+1)-FIND(".", B6, FIND(".", B6)+1)-1)))*256)+(VALUE(RIGHT(B6, LEN(B6)-FIND(".", B6, FIND(".", B6, FIND(".", B6)+1)+1))))

Comments Off on Sorting IP addresses in Excel
comments

Oct 29

KVM virsh console on CentOS 7

Several articles on using the console feature of KVM guests exists already. This is just a quick note on a test I did using specifically Centos 7 which is using systemd now.

Console allows you to A) see bootup messages and B) login on the console of the guest without SSH.

As far as I can tell you only need to do two things. Add console=ttyS0 to the kernel being booted and in the guest XML switch virtio to serial for the console. Keep in mind this is if you are using the RHEL 7 type when creating the VM. If you were using a RHEL 6 type when creating the guest the console probably still were configured to use serial.

This KVM hypervisor is on Fedora 20 and using libvirt version 1.1.3.6.

To send messages from the kernel to the serial console add console=ttyS0:

# grep console=ttyS0 /boot/grub2/grub.cfg
        linux16 /vmlinuz-3.10.0-123.el7.x86_64 root=UUID=ba0f2424-e66e-4862-90ff-7dccb63339c2 ro rd.lvm.lv=centos/swap vconsole.font=latarcyrheb-sun16 rd.lvm.lv=centos/root crashkernel=auto  vconsole.keymap=us LANG=en_US.UTF-8 console=ttyS0

# grep ttyS0 /etc/securetty
ttyS0

Change guest console type to serial:

<console type='pty' tty='/dev/pts/6'>
      <source path='/dev/pts/6'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>

# virsh console centos7
Connected to domain centos7
CentOS Linux 7 (Core)
Kernel 3.10.0-123.el7.x86_64 on an x86_64

localhost login: 

Above should do it but if you want to see even the grub2 menu options you can try the following:

# grep serial /etc/default/grub 
GRUB_TERMINAL=serial
GRUB_SERIAL_COMMAND="serial --speed=38400 --unit=0 --word=8 --parity=no --stop=1"

# grep LINUX_DEFAULT /etc/default/grub 
GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0"

# grub2-mkconfig -o /boot/grub2/grub.cfg

Of course if you are setting grub DEFAULT options you don't need to set anything for the kernel entries in /boot/grub2/grub.cfg like I did in step 1.

Note that when connecting to the console you might need to move your up or down arrows to see the actual menu entries. The terminal and curses might look a little flaky but it is there. I also set the grub TIMEOUT a little longer to wait at the menu entries.

Comments Off on KVM virsh console on CentOS 7
comments

Oct 07

Printing PCL to PDF

Anyone that ever worked on printing would know how much time can be wasted with root causing problems.  There are a lot of moving parts with different printers, drivers, conversions, operating systems, queue types and so forth.  Especially when special type of printing is required like barcodes.

One particularly painful area is printing and checking results.  A lot of time this is done to a printer not close by like on a manufacturing floor.  Or even across country.  And then depending on someone to check the printout, scan and email back to you introduce a cycle that can be very frustrating for everyone.

One good way to speed up the testing process is to print to PDF.  If you are just interested in generic printing to PDF and willing to tinker with CUPS you could look at CUPS-PDF which is fairly well documented.  In my case I had issues with the way PCL was being printed so I needed to do more than CUPS-PDF.  I might still circle back and see if I can get CUPS-PDF to do some kind pf PCL conversion but for now my solution that worked is documented below.

High level process is as follow:

1. Print from Solaris 11.1 to a remote queue using simple lp -d <queue> <file>

2. CUPS server identifies incoming file as text/plain mime type.  This is forced through adding *cupsfilter: "text/plain  0  -" to an Adobe distiller PPD. CUPS backend called "pdf" which converts to a file.  In my case the destination is /var/www/html/pdf since I want to share the pdf's through Apache.

3. CUPS backend "pdf" script use software called ghostpcl that is a subset of ghostscript to convert pcl to pdf.

4. Debugging Notes.

5. LPD vs IPP.

Details on above steps follow below.

Note copying any commands or scripts does not mean it is tested or working.  I provided below as guide but I tinker with the commands and scripts when editing the article.  I used Oracle Linux 6.5 in this instance so depending on your Linux variant files might have slightly different locations.

1. Printing from Solaris
Add the remote queue and print as follow. In my case I needed to match the printer name to the application so CUPS printer name is different from local queue.

# lpadmin -p testprinter -s lpd://10.200.55.43/pdfprinter
# lp -d testprinter /root/OUTPUT.TXT

2. Setup CUPS printer and accept incoming file as text/plain
You can also use other PPD files like the one that comes with the CUPS-PDF project but this one worked form me and available from Adobe.

# lpadmin -p pdfprinter -v pdf:/var/www/html/pdf -D "Generate PDF files" -E -P /usr/share/cups/model/distiller.ppd.gz
# chmod 777 /var/www/html/pdf

# pwd
/etc/cups/ppd
# ls -l
total 20
-rw-r--r--. 1 root root 17172 Oct 6 13:43 pdfprinter.ppd

# grep plain pdfprinter.ppd
*cupsfilter: "text/plain 0 -"

# grep plain mime.*
mime.convs:text/plain application/pdf 0 -
mime.types:text/plain txt printable(0,1024)

# /etc/init.d/cups restart

3. Backend script

# pwd
/usr/lib/cups/backend

# cat pdf
#!/bin/sh
# -------------------------------------------------------------------
# "/usr/lib/cups/backend/pdf":
# -------------------------------------------------------------------
#
set -x
FILENAME=
# filename of the PDF File
PRINTTIME=`date +%Y-%m-%d_%H.%M.%S`
# no argument, prints available URIs

# case of wrong number of arguments
if [ $# -ne 5 -a $# -ne 6 ]; then
echo "Usage: pdf job-id user title copies options [file]"
exit 1
fi
# get PDF directory from device URI, and check write status
PDFDIR=${DEVICE_URI#pdf:}
if [ ! -d "$PDFDIR" -o ! -w "$PDFDIR" ]; then
echo "ERROR: directory $PDFDIR not writable"
exit 1
fi
# generate output filename
OUTPUTFILENAME=
if [ "$3" = "" ]; then
OUTPUTFILENAME="$PDFDIR/unknown.pdf"
else
if [ "$2" != "" ]; then
OUTPUTFILENAME="$PDFDIR/$2-$PRINTTIME.pdf"
else
OUTPUTFILENAME="$PDFDIR/$PRINTTIME.pdf"
fi
fi

INPUTFILENAME=
if [ $# -eq 6 ]; then
INPUTFILENAME=$6
else
INPUTFILENAME=-
fi

# run ghostpcl
/usr/src/ghostpcl-9.15-linux-x86_64/pcl6-915-linux_x86_64 -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -dPDFSETTINGS="/ebook" -dAutoRotatePages=/All -dPDFFitPage -r300x300 -g2346x4100 -J"@PJL SET ORIENTATION = LANDSCAPE" -sOutputFile=$OUTPUTFILENAME $INPUTFILENAME

chmod 644 $OUTPUTFILENAME

## Normally the PDF file will be emailed to the creating user.  Add emailing code here if needed. Emailing to who?
exit 0
# EOF
# -------------------------------------------------------------------

4. Debugging Notes:

# grep debug cupsd.conf
LogLevel debug

# grep set pdf
set -x

The biggest pain in the whole process is to make sure CUPS applies only the filter for your backend and not run through any other filters.  Below is some of the lines in the log you want to pay attention to: "Auto-typing ...",  "File of type ..." ,  "Started backend ...". In my case if I see "Started filter ..."  it spells trouble.

# pwd
/var/log/cups

# grep "Job 134" error_log
I [07/Oct/2014:01:58:27 -0500] [Job 134] Adding start banner page "none".
I [07/Oct/2014:01:58:27 -0500] [Job 134] Queued on "pdfprinter" by "devuser".
D [07/Oct/2014:01:58:27 -0500] [Job 134] Auto-typing file...
D [07/Oct/2014:01:58:27 -0500] [Job 134] Request file type is text/plain.
I [07/Oct/2014:01:58:27 -0500] [Job 134] File of type text/plain queued by "devuser".
D [07/Oct/2014:01:58:27 -0500] [Job 134] job-sheets=none
D [07/Oct/2014:01:58:27 -0500] [Job 134] argv[0]="pdfprinter"
D [07/Oct/2014:01:58:27 -0500] [Job 134] argv[1]="134"
D [07/Oct/2014:01:58:27 -0500] [Job 134] argv[2]="devuser"
D [07/Oct/2014:01:58:27 -0500] [Job 134] argv[3]="standard input"
D [07/Oct/2014:01:58:27 -0500] [Job 134] argv[4]="1"
D [07/Oct/2014:01:58:27 -0500] [Job 134] argv[5]="document-name=/tmp/stdin-P3OQc job-originating-host-name=10.2.14.98 job-uuid=urn:uuid:0f507f16-0537-3307-6b15-b5134eda2bc5"
D [07/Oct/2014:01:58:27 -0500] [Job 134] argv[6]="/var/spool/cups/d00134-001"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[0]="CUPS_CACHEDIR=/var/cache/cups"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[1]="CUPS_DATADIR=/usr/share/cups"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[2]="CUPS_DOCROOT=/usr/share/cups/www"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[3]="CUPS_FONTPATH=/usr/share/cups/fonts"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[4]="CUPS_REQUESTROOT=/var/spool/cups"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[5]="CUPS_SERVERBIN=/usr/lib/cups"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[6]="CUPS_SERVERROOT=/etc/cups"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[7]="CUPS_STATEDIR=/var/run/cups"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[8]="HOME=/var/spool/cups/tmp"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[9]="PATH=/usr/lib/cups/filter:/usr/lib64/cups/filter:/usr/bin:/usr/sbin:/bin:/usr/bin"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[10]="SERVER_ADMIN=root@localhost"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[11]="SOFTWARE=CUPS/1.4.2"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[12]="TMPDIR=/var/spool/cups/tmp"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[13]="USER=root"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[14]="CUPS_SERVER=/var/run/cups/cups.sock"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[15]="CUPS_ENCRYPTION=IfRequested"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[16]="IPP_PORT=631"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[17]="CHARSET=utf-8"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[18]="LANG=en_US.UTF-8"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[19]="PPD=/etc/cups/ppd/pdfprinter.ppd"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[20]="RIP_MAX_CACHE=128m"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[21]="CONTENT_TYPE=text/plain"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[22]="DEVICE_URI=pdf:/var/www/html/pdf"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[23]="PRINTER_INFO=Generate PDF files"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[24]="PRINTER_LOCATION=VM 10.200.55.43"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[25]="PRINTER=pdfprinter"
D [07/Oct/2014:01:58:27 -0500] [Job 134] envp[26]="CUPS_FILETYPE=document"
I [07/Oct/2014:01:58:27 -0500] [Job 134] Started backend /usr/lib/cups/backend/pdf (PID 2170)
D [07/Oct/2014:01:58:27 -0500] [Job 134] + FILENAME=
D [07/Oct/2014:01:58:27 -0500] [Job 134] ++ date +%Y-%m-%d_%H.%M.%S
D [07/Oct/2014:01:58:27 -0500] [Job 134] + PRINTTIME=2014-10-07_01.58.27
D [07/Oct/2014:01:58:27 -0500] [Job 134] + '[' 6 -eq 0 ']'
D [07/Oct/2014:01:58:27 -0500] [Job 134] + '[' 6 -ne 5 -a 6 -ne 6 ']'
D [07/Oct/2014:01:58:27 -0500] [Job 134] + PDFDIR=/var/www/html/pdf
D [07/Oct/2014:01:58:27 -0500] [Job 134] + '[' '!' -d /var/www/html/pdf -o '!' -w /var/www/html/pdf ']'
D [07/Oct/2014:01:58:27 -0500] [Job 134] + OUTPUTFILENAME=
D [07/Oct/2014:01:58:27 -0500] [Job 134] + '[' 'standard input' = '' ']'
D [07/Oct/2014:01:58:27 -0500] [Job 134] + '[' devuser '!=' '' ']'
D [07/Oct/2014:01:58:27 -0500] [Job 134] + OUTPUTFILENAME=/var/www/html/pdf/devuser-2014-10-07_01.58.27.pdf
D [07/Oct/2014:01:58:27 -0500] [Job 134] + INPUTFILENAME=
D [07/Oct/2014:01:58:27 -0500] [Job 134] + '[' 6 -eq 6 ']'
D [07/Oct/2014:01:58:27 -0500] [Job 134] + INPUTFILENAME=/var/spool/cups/d00134-001
D [07/Oct/2014:01:58:27 -0500] [Job 134] + /usr/src/ghostpcl-9.15-linux-x86_64/pcl6-915-linux_x86_64 -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -dPDFSETTINGS=/ebook -dAutoRotatePages=/All -dPDFFitPage -r300x300 -g2346x4100 '-J@PJL SET ORIENTATION = LANDSCAPE' -sOutputFile=/var/www/html/pdf/devuser-2014-10-07_01.58.27.pdf /var/spool/cups/d00134-001
D [07/Oct/2014:01:58:27 -0500] [Job 134] + chmod 644 /var/www/html/pdf/devuser-2014-10-07_01.58.27.pdf
D [07/Oct/2014:01:58:27 -0500] [Job 134] + exit 0
I [07/Oct/2014:01:58:27 -0500] [Job 134] Job completed.
D [07/Oct/2014:01:58:28 -0500] [Job 134] Unloading...

5. IPP versus LPD:
Note that I am using cups-lpd to accept on the lpd port on the CUPS server but I think it might not be necessary. IPP would also have worked. I had an odd CUPS issue that printing local on the CUPS server using lp worked but sending the file from a remote Solaris server CUPS insisted on tacking on a CUPS-BANNER which caused issues and even triggering the gziptoany filter and possibly more filters for ps.  Sounds like a bug with CUPS and the fix for to make sure no banner is added was here: https://bugzilla.redhat.com/show_bug.cgi?id=414161

# grep server_args /etc/xinetd.d/cups-lpd
  server_args = -o job-sheets=none

Comments Off on Printing PCL to PDF
comments

Sep 23

Watch Process Id

Sometimes you want to keep tabs on a long running process and get notified by email when it is done. This is an example of just that.

#!/bin/bash
pid=$1
me="$(basename $0)($$):"
if [ -z "$pid" ]
then
    echo "$me a PID is required as an argument" >&2
    exit 2
fi

name=$(ps -p $pid -o comm=)
if [ $? -eq 0 ]
then
    echo "$me waiting for PID $pid to finish ($name)"
    while ps -p $pid > /dev/null; do sleep 1; done;
else
    echo "$me failed to find process with PID $pid" >&2
    exit 1
fi
## I used a python mailer but mostlikely this will be mail or mailx.
python pymail.py $pid

Comments Off on Watch Process Id
comments

Sep 23

Quick Python Mailer

import sys,smtplib
from email.MIMEMultipart import MIMEMultipart
from email.MIMEText import MIMEText

msg = MIMEMultipart()
msg['From'] = 'root@domain.ad'
msg['To'] = 'user@domain.com'
msg['Subject'] = 'Process finished -' + sys.argv[1]
message = 'here is the email'
msg.attach(MIMEText(message))

mailserver = smtplib.SMTP('mail.domain.com',25)
# identify ourselves to smtp gmail client
mailserver.ehlo()
# secure our email with tls encryption
mailserver.starttls()
# re-identify ourselves as an encrypted connection
mailserver.ehlo()

mailserver.sendmail('root@domain.ad','user@domain.com',msg.as_string())

mailserver.quit()

** This script need to be optimized. I am duplicating from and to fields

Comments Off on Quick Python Mailer
comments