Oracle VM SDK Create VM

I am playing with the java SOAP API to test creating a VM and since it took me a while to get it working I am jotting down my recipe for future refining. This is rough and needs a rewrite. Currently it is just a gutted version of the Sample that comes with the OVMM ISO.

Note I ran this on the OVMM server at first and using localhost for connecting. That worked but to run this code from a different server I copied java from the OVMM server. Newer java just gave me too much SSL issues.

Also I would prefer using REST/JSON but that is not working yet. REST/JSON gave me a NullPointerException which may provide me a clue as to why I could not get POST to work from my Python tests in a prior article. Which is actually the reason I am testing the Java code.

You are WARNED this code is rough. For example it will create duplicate VM’s with the same name. Also checkign the SERverPool is not safe it needs more logic. It will just grab the last one if you have more than one.

You will need jersey for this code to work. Also as I mentioned this needs a lot of cleanup. I had some dependency issues so I copied a lot of classes from the SDK into my tree $APPROOT/lib/com/oracle/ovm/mgr/ws/

rrosso@BL:~/src/ovm_java_api$ ls lib/
com  jersey-bundle-1.19.1.jar  OvmWsClient.jar

Little BASH script to show compile and run.

 
rrosso@BL:~/src/ovm_java_api$ cat buildrun.sh 
# Script to build and run OVM 3.3.3 specifc SDK API
# Run with two parms example: ./buildrun.sh vmname=vm1 server=ovs333
# JAVA ENV
#export JAVA_HOME=/home/rrosso/jdk1.8.0_121
export JAVA_HOME=/home/rrosso/java
export PATH=$JAVA_HOME/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/rrosso/bin
#java -version

APPROOT=/home/rrosso/src/ovm_java_api
BUILDROOT=com/oracle/ovm/mgr/ws/sample/
OVMLIB=$APPROOT/lib

# APP ROOT
cd $APPROOT

# BUILD
cd $BUILDROOT
rm WsDevClient*.class
javac -cp $OVMLIB/OvmWsClient.jar:. WsDevClient.java

# RUN
cd $APPROOT
#java -Djavax.net.debug=ssl -cp .:$OVMLIB com.oracle.ovm.mgr.ws.sample.WsDevClient
#echo "Trying to create $1 on $2"
java -cp .:$OVMLIB com.oracle.ovm.mgr.ws.sample.WsDevClient $1 $2

Example run working:

rrosso@BL:~/src/ovm_java_api$ ./buildrun.sh vmname=vm4.3 server=ovs333
Starting Ovm Api Test Using SOAP Implementation

Starting VM create of: vm4.3
create vm job id: Create Vm: vm4.3 on Server Pool: pool1(1487049718384) Uri: https://localhost:7002/ovm/core/wsapi/rest/Job/1487049718384

Client Source:

rrosso@BL:~/src/ovm_java_api$ cat com/oracle/ovm/mgr/ws/sample/WsDevClient.java
package com.oracle.ovm.mgr.ws.sample;
import java.security.SecureRandom;
import java.security.cert.CertificateException;
import java.security.cert.X509Certificate;
import java.util.List;
import java.util.Locale;
import java.util.MissingResourceException;
import java.util.ResourceBundle;
import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.KeyManager;
import javax.net.ssl.SSLContext;
import javax.net.ssl.SSLSession;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;
import com.oracle.ovm.mgr.ws.client.OvmWsClient;
import com.oracle.ovm.mgr.ws.client.OvmWsClientFactory;
import com.oracle.ovm.mgr.ws.model.BaseObject;
import com.oracle.ovm.mgr.ws.model.Id;
import com.oracle.ovm.mgr.ws.model.Job;
import com.oracle.ovm.mgr.ws.model.Job.JobRunState;
import com.oracle.ovm.mgr.ws.model.JobError;
import com.oracle.ovm.mgr.ws.model.Repository;
import com.oracle.ovm.mgr.ws.model.Server;
import com.oracle.ovm.mgr.ws.model.ServerPool;
//import com.oracle.ovm.mgr.ws.model.StorageElement;
import com.oracle.ovm.mgr.ws.model.Vm;
import com.oracle.ovm.mgr.ws.model.Vm.VmDomainType;
import com.oracle.ovm.mgr.ws.model.VmDiskMapping;
import com.oracle.ovm.mgr.ws.model.WsException;

/**
 * A simple web services create vm client.
 * .
 * BEFORE RUNNING THIS you should first override the WsDevClient_<username>.properties file with your own values. See that
 * file for details.
 */
public class WsDevClient
{
    private OvmWsClient                api;
    private Id<Vm>                     testVmId;
    private Id<ServerPool>             testPoolId;
    private Id<Server>                 testServerId;
    private Id<Repository>             testRepoId;
    private Id<Vm>                     importedAssemblyVmId;
    private Job                        repoPresentJob;

    public static void main(final String args[])
    {
        int index;
	String vmname="";
	String server="";
        for (index = 0; index < args.length; ++index)
        {
            //System.out.println("args[" + index + "]: " + args[index]);
       	    String[] splitted=args[index].split("=");
	    if ( splitted[0].equals("vmname") ) {
	      vmname=splitted[1];
              //System.out.println(vmname);
	
	    }
	    if ( splitted[0].equals("server") ) {
	      server=splitted[1];
              //System.out.println(server);
	    }
        }
       //String[] vmname=args[0].split("=");;
       //String[] server=args[1].split("=");;
        final WsDevClient wsClient = new WsDevClient();
       wsClient.run(vmname,server);
    }

    public void run(final String testVmName,final String testServerName)
    {
        try
        {
            // Configure the SSLContext with an insecure TrustManager
            // This is done to avoid the need to use valid certificates in the development environment.
            // This should not be done in a real / secure environment.
            final SSLContext ctx = SSLContext.getInstance("TLS");
            ctx.init(new KeyManager[0], new TrustManager[]
            {
                new InsecureTrustManager()
            }, new SecureRandom());
            SSLContext.setDefault(ctx);

            // Configure the HostnameVerifier to trust host names that do
            // not match the name on the server certificate.
            // This should not be done in real / secure environments.
            final HostnameVerifier hv = new HostnameVerifier()
            {
                public boolean verify(final String arg0, final SSLSession arg1)
                {
                    return true;
                }
            };
            HttpsURLConnection.setDefaultHostnameVerifier(hv);

            final ResourceBundle rootProps = ResourceBundle.getBundle("com.oracle.ovm.mgr.ws.sample.WsDevClient");
            ResourceBundle userProps = null;
            try
            {
                userProps = ResourceBundle.getBundle("com.oracle.ovm.mgr.ws.sample.WsDevClient_" + System.getProperty("user.name"));
            }
            catch (final MissingResourceException ex)
            {
                System.out.println(ex);
                System.out.println("Warning: No user-overridden properties found.  Using default values.  This is unlikely to work and is not recommended!");
            }
            // Load the properties. An exception will be thrown if any of these properties are not found.
            final String wsimpl = getProperty("wsimpl", rootProps, userProps);
            final String dialect = getProperty("dialect", rootProps, userProps);
            final String hostname = getProperty("hostname", rootProps, userProps);
            final String port = getProperty("port", rootProps, userProps);
            final String username = getProperty("username", rootProps, userProps);
            final String password = getProperty("password", rootProps, userProps);
            final String testRepo = getProperty("testRepo", rootProps, userProps);
            //final String testVmName = getProperty("testVmName", rootProps, userProps);
            final String debugStr = getProperty("debugHttpTraffic", rootProps, userProps);
            //final String testServerName = getProperty("testServerName", rootProps, userProps);
            final String testServerPoolName = getProperty("testServerPoolName", rootProps, userProps);
            final String assemblyUrl = getProperty("assemblyUrl", rootProps, userProps);
            final boolean debug = debugStr.equalsIgnoreCase("true");

            api = OvmWsClientFactory.getOvmWsClient(wsimpl);
            api.setDebug(debug);
            api.setDialect(dialect);
            api.initialize(hostname, port, true);
            // Authenticate with the OvmApi Service
            api.login(username, password, Locale.getDefault());
            System.out.println("Starting Ovm Api Test Using " + wsimpl + " Implementation\n");
	    final List<Repository> repos = api.repositoryGetAll();
            for (final Repository repo : repos)
            {
                //printRepository(repo);

                if (repo.getId().getValue().equals(testRepo))
                {
                    testRepoId = repo.getId();
                }
            }
	    // Get a list of all Server Pools
            final List<Id<ServerPool>> serverPoolIds = api.serverPoolGetIds();
            for (final Id<ServerPool> serverPoolId : serverPoolIds)
            {
                //printId("ServerPool", serverPoolId);
		// need better checking than just picking one
		testPoolId = serverPoolId;
            }
            // Run checks to make sure we can proceed.
            if (testRepoId == null)
            {
                throw new Exception("Repository for VM creation (with id " + testRepo + ") not found.  Exiting.");
            }
            System.out.println("Starting VM create of: " + testVmName);
            Vm testVm = new Vm();
            testVm.setVmDomainType(VmDomainType.XEN_HVM);
            testVm.setName(testVmName);
            testVm.setRepositoryId(testRepoId);
	    //final ServerPool serverPool = api.serverPoolGetById(testPoolId);
            final Job vmCreateJob = api.vmCreate(testVm, testPoolId);
            System.out.println("create vm job id: " + vmCreateJob.getId());
            testVmId = waitForJobComplete(api, vmCreateJob, Vm.class);
            // fetch vm create result object to fully populate the object (including the id and default values)
            // in preparation for the modify test
            testVm = api.vmGetById(testVmId);
        }
        catch (final Exception e)
        {
            e.printStackTrace();
            System.out.println("\nException occurred during processing.  Calling cleanup routine.");
        }
        finally
        {
                try
                {
                    api.logout();
                }
                catch (final WsException e)
                {
                    e.printStackTrace();
                }
        }
    }
    /**
     * @param string
     * @param rootProps
     * @param userProps
     * @return
     */
    private String getProperty(final String key, final ResourceBundle... bundles)
    {
        String value = null;
        MissingResourceException missingResourceEx = null;
        for (final ResourceBundle bundle : bundles)
        {
            if (bundle != null)
            {
                try
                {
                    value = bundle.getString(key);
                }
                catch (final MissingResourceException ex)
                {
                    missingResourceEx = ex;
                }
            }
        }
        if (value == null)
        {
            throw missingResourceEx;
        }
        return value;
    }
    /**
     * @param group
     */
    private void printJob(final Job job)
    {
        System.out.println("Job Id: " + job.getId());
        System.out.println("\tUser: " + job.getUser());
        System.out.println("\tStart Time: " + job.getStartTime());
        System.out.println("\tEnd Time: " + job.getEndTime());
        System.out.println("\tRun State: " + job.getJobRunState());
        System.out.println("\tGroup: " + job.isJobGroup());
        System.out.println("\tResult Id: " + job.getResultId());
        try
        {
            final String transcript = api.jobGetDebugTranscript(job.getId());
            System.out.println(transcript);
        }
        catch (final Exception e)
        {
            e.printStackTrace();
        }
        final List<Id<Job>> jobChildren = job.getChildJobIds();
        if (jobChildren != null)
        {
            for (final Id<Job> childJob : jobChildren)
            {
                System.out.println("\tChildJob: " + childJob);
            }
        }
        final JobError error = job.getError();
        if (error != null)
        {
            System.out.println("\tError Type: " + error.getType());
            System.out.println("\tError Message: " + error.getMessage());
        }

        System.out.println();
    }
    public void waitForJobComplete(final OvmWsClient api, final Job job) throws WsException
    {
        waitForJobComplete(api, job, null);
    }
    @SuppressWarnings("unchecked")
    public <T> Id<T> waitForJobComplete(final OvmWsClient api, Job job, final Class<T> type) throws WsException
    {
        while (job.isSummaryDone() == false)
        {
            try
            {
                Thread.sleep(1000);
            }
            catch (final Exception e)
            {
            }

            job = api.jobGetById(job.getId());

            if (job.getJobRunState().equals(JobRunState.FAILURE))
            {
                final JobError error = job.getError();
                if (error != null)
                {
                    System.out.println("    error type: " + error.getType());
                    System.out.println("    error message: " + error.getMessage());
                }
                System.out.println("    transcript: " + api.jobGetDebugTranscript(job.getId()));
            }
        }
        @SuppressWarnings("rawtypes")
        final Id resultId = job.getResultId();
        if (type == null)
        {
            return resultId;
        }
        else
        {
            final Id<T> typedResultId = resultId;
            return typedResultId;
        }
    }
    private static class InsecureTrustManager implements X509TrustManager
    {
        @Override
        public void checkClientTrusted(final X509Certificate[] arg0, final String arg1) throws CertificateException
        {
        }
        @Override
        public void checkServerTrusted(final X509Certificate[] arg0, final String arg1) throws CertificateException
        {
        }
        @Override
        public X509Certificate[] getAcceptedIssuers()
        {
            return null;
        }
    }
}

Properties file. I will work more of this into command line parms later.

rrosso@BL:~/src/ovm_java_api$ cat com//oracle/ovm/mgr/ws/sample/WsDevClient_rrosso.properties 
wsimpl=SOAP
dialect=XML
hostname=192.168.1.231
port=7002
username=admin
password=myAdminPassW
testRepo=0004fb00000300002e4e288cfa06e535
testServerPoolName=repo1
debugHttpTraffic=false
assemblyUrl=http://install-vi.us.oracle.com/tests/assemblies/virtual-appliance-simple/package.ovf
Posted in OVM

System Administration with Fabric and OVM

I recently tried fabric for a few automation tasks and could use it to interact with the Oracle VM CLI.  I am using the OVM Rest API whenever possible but in some cases this may work for you.  Also I was having issues with the REST API and POST type calls so needed another method.  I have expect also for CLI interaction but fabric is a lot more python like and powerful.

In addition not in this use case with OVM but if you use fabric to run against many hosts you could pretty easily get it to do parallel execution with a bit of python magic. CHeck here for great example: https://dmsimard.com/2013/11/29/capture-output-from-parallel-execution-with-fabric/

Example fabfile:

import os
from fabric.api import *
from fabric.colors import red, green
import re

def getprop(item,tag):
  item = item + ' '
  s=tag + ":(.*?) "
  ovmprop=re.findall(s, item)
  return ovmprop[0]


env.user = os.getenv('SSH_USER', 'admin')
env.password = os.getenv('SSH_PASSWORD', 'Welcome')

# http://docs.oracle.com/cd/E35328_01/E35336/E35336.pdf
def vm_list_raw():
  vms=run('list vm', shell=False)
  print(green(vms))

def vm_create(name):
  run('create Vm name=' + name + ' domainType=XEN_HVM repository=ovs2 on ServerPool name=ovsPool1', shell=False)

def vm_list():
  with settings(hide('running', 'commands', 'stdout', 'stderr')):
    vms=run('list vm', shell=False).split('\n\r')
    #print(green(vms))

    for line in vms:
      if 'id' in line:
        id=getprop(line,'id')
        name=getprop(line,'name')
        print '{:20} {:20}'.format(id,name)  

Examples:

$ fab -H 192.168.1.223:10000 vm_list
[192.168.1.223:10000] Executing task 'vm_list'
0004fb000006000048e25da9faac0ac6 VM1                 

Done.
Disconnecting from 192.168.1.223:10000... done.

$ fab -H 192.168.1.223:10000 vm_create:"name=VM6"
[192.168.1.223:10000] Executing task 'vm_create'
[192.168.1.223:10000] run: create Vm name=VM6 domainType=XEN_HVM repository=ovs2 on ServerPool name=ovsPool1
[192.168.1.223:10000] out: OVM> create Vm name=VM6 domainType=XEN_HVM repository=ovs2 on ServerPool name=ovsPool1
[192.168.1.223:10000] out: 
[192.168.1.223:10000] out: Command: create Vm name=VM6 domainType=XEN_HVM repository=ovs2 on ServerPool name=ovsPool1
[192.168.1.223:10000] out: 
[192.168.1.223:10000] out: Status: Success
[192.168.1.223:10000] out: 
[192.168.1.223:10000] out: Time: 2017-02-01 09:04:26,212 CST
[192.168.1.223:10000] out: 
[192.168.1.223:10000] out: JobId: 1485961465970
[192.168.1.223:10000] out: 
[192.168.1.223:10000] out: Data: 
[192.168.1.223:10000] out: 
[192.168.1.223:10000] out:   id:0004fb00000600007f77133a8d591c19  name:VM6
[192.168.1.223:10000] out: 
[192.168.1.223:10000] out: OVM> Connection closed.
[192.168.1.223:10000] out: 

Oracle VM (OVM) REST Api

############################################################################################
Update 20180331:
I tried a couple things from the comment section and can add the following:

@chaitanya: ovcmclient worked for me also with create and clone

@Dave: that still gave me the same errors and I am using documentation as its here https://docs.oracle.com/cd/E50245_01/E50253/html/vmprg-rest-example-vm-create-python.html

@Jeff: try chaitanya’s suggestion python-ovmclient it worked for me
############################################################################################

Previous article here:

Oracle OVM rest api example

I am adding a slightly updated script here even though I could not get POST (Create new VM) and PUT (Clone VM) to work yet.

Using a module for most functions.

import json
from time import sleep

def get_id_from_name(s,baseUri,obj,name):
  #sp_id=get_id_from_name(s,baseUri,'ServerPool','ovs-home')
  r=s.get(baseUri+'/'+obj)
  for i in r.json():
    if i['name'] == name:
      id = i['id']['value']
  return id

def wait_for_job(joburi,s):
        while True:
            time.sleep(1)
            r=s.get(joburi)
            job=r.json()
            if job['summaryDone']:
                print '{name}: {runState}'.format(name=job['name'], runState=job['jobRunState'])
                if job['jobRunState'].upper() == 'FAILURE':
                    raise Exception('Job failed: {error}'.format(error=job['error']))
                elif job['jobRunState'].upper() == 'SUCCESS':
                    if 'resultId' in job:
                        return job['resultId']
                    break
                else:
                    break     

def check_manager_state(baseUri,s):
  #https://hostname:port/ovm/core/wsapi/rest/Manager
  while True:
    r=s.get(baseUri+'/Manager')
    manager=r.json()
    if manager[0]['managerRunState'].upper() == 'RUNNING':
      break
      time.sleep(1)
      return;

def serverList(s,baseUri):
  print "\nServer List:"
  print "##############"
  r=s.get(baseUri+'/Server')
  for i in r.json():
    # do something with the content
    print '{:20} {:20}'.format(i['serverRunState'],i['name'])

def vmList(s,baseUri):
  print "\nVM List:"
  print "########"
  r=s.get(baseUri+'/Vm')
  for i in r.json():
    # print '{:20} {:20}'.format(i['vmRunState'],i['name'])
    print '{:20} {:35} {:30}'.format(i['vmRunState'],i['name'],i['id']['value']),
    for d in i['vmDiskMappingIds']:
      print d['value'],
    print
    # print '{name} '.format(name=i['name'])
    # print i

def showVm(s,baseUri,name):
  print "\nVM Show:"
  print "##########"
  r=s.get(baseUri+'/Vm')
  for i in r.json():
    if i['name'] == name:
      print '{:20} {:55}'.format(i['vmRunState'],i['name']), 
      for d in i['vmDiskMappingIds']:
	if 'CDROM' not in d['name']:
          print d['value'],
          disk=s.get(baseUri+'/VmDiskMapping/'+d['value'])
	  d = disk.json()
	  if "obiee" in d['virtualDiskId']['name']:
	    dName = d['virtualDiskId']['name'].replace('obiee_template',i['name'])
	    dName = dName.split('img')[0]+'img'
    	    print 'value: {} and name {} should be renamed -> {}'.format(d['virtualDiskId']['value'],d['virtualDiskId']['name'],dName),
	    print

def cmdVm(s,baseUri,cmd,name):
  print "\nVM " + cmd
  print "##########"
  vm_id=get_id_from_name(s,baseUri,'Vm',name)
  print vm_id
  r=s.put(baseUri+'/Vm/'+vm_id+'/'+cmd)
  job=r.json()
  # print job
  # wait for the job to complete
  vm_id=wait_for_job(job['id']['uri'],s)

def updateVm(s,baseUri,name):
  print "\nVM Update Vm Disk Names:"
  print "##########################"
  r=s.get(baseUri+'/Vm')
  for i in r.json():
    if i['name'] == name:
      #print i
      print '{:20} {:20} {:55}'.format(i['id']['value'],i['vmRunState'],i['name'])
      for disk in i['vmDiskMappingIds']:
        if 'CDROM' not in disk['name']:
          #print disk['value'],
          value=s.get(baseUri+'/VmDiskMapping/'+disk['value'])
          d = value.json()
	  oldName = d['virtualDiskId']['name']
          newName = d['virtualDiskId']['name'].replace('obiee_template',i['name'])
          newName = newName.split('img')[0]+'img'
	  d['virtualDiskId']['name']=newName
	  d['id']['name']=newName
	  d['name']=newName
          
          #print 'value: {:20} name: {:55} new name {}'.format(d['virtualDiskId']['value'],d['virtualDiskId']['name'],dName),
          print 'value: {:20} name: {:55} new name {}'.format(disk['value'],oldName,newName)
          #print d
          uri='{base}/VmDiskMapping/{id}'.format(base=baseUri,id=d['id']['value'])
	  #print uri
          r1=s.put(uri,data=json.dumps(d))
          job=r1.json()
    	  #print job
          # wait for the job to complete
          wait_for_job(job['id']['uri'],s)

      i['vmDiskMappingIds'][0]['name']='new.img'
      #print i
      uri='{base}/Vm/{id}'.format(base=baseUri,id=i['id']['value'])
      #print uri
      
      r=s.put(uri,data=json.dumps(i))
      job=r.json()
      #print job
      # wait for the job to complete
      wait_for_job(job['id']['uri'],s)

def updateVmMemory(s,baseUri,name,memory):
  print "\nVM Update Vm Memory for " + name
  print "########################################"
  vm_id=get_id_from_name(s,baseUri,'Vm',name)
  uri='{base}/Vm/{id}'.format(base=baseUri,id=vm_id)
  r=s.get(uri)
  d=r.json()
  #print d
  d['memory']='512'

  r=s.put(uri,data=json.dumps(d))
  job=r.json()
  #print job
  # wait for the job to complete
  wait_for_job(job['id']['uri'],s)

def updateVirtualDisk(s,baseUri,id,newDiskName):
  print "\nVM Update Vm Disk Mapping for " + id
  print "########################################"
  uri='{base}/VirtualDisk/{id}'.format(base=baseUri,id=id)

  r=s.get(uri)
  disk=r.json()
  #print disk

  #oldName = disk['virtualDiskId']['name']
  #newName = disk['virtualDiskId']['name'].replace('obiee_template',d['name'])
  #newName = newName.split('img')[0]+'img'
	  
  disk['name']=newDiskName

  #disk['name']='newname_system.img'
   
  r=s.put(uri,data=json.dumps(disk))
  job=r.json()
  # wait for the job to complete
  wait_for_job(job['id']['uri'],s)

def updateVmDiskNames(s,baseUri,name):
  print "\nVM Update Vm Disk Names for " + name
  print "########################################"
  vm_id=get_id_from_name(s,baseUri,'Vm',name)
  uri='{base}/Vm/{id}'.format(base=baseUri,id=vm_id)
  r=s.get(uri)
  vm=r.json()

  dNum=0  
  for disk in vm['vmDiskMappingIds']:
        if 'CDROM' not in disk['name']:
	  dNum = dNum +1
	  newDiskName=name + "_disk" + str(dNum)
	  #if "system" in disk['name']:
	  #  newDiskName=name + "_system.img"
 	  #if "data1" in disk['name']:
	  #  newDiskName=name + "_data1.img"

	  ## update VmDiskMapping as shown in Repository
	  dMapping=s.get(baseUri+'/VmDiskMapping/'+disk['value'])
	  dm=dMapping.json()
	  updateVirtualDisk(s,baseUri,dm['virtualDiskId']['value'],newDiskName)

Main program.

import requests
import json
from ovm_lib import *
#import logging

s=requests.Session()
s.auth=('admin','mypassword')
s.verify=False #disables SSL certificate verification
s.headers.update({'Accept': 'application/json', 'Content-Type': 'application/json'})

ovmmServer="192.168.1.223:7002"
print "Running against OVM Manager Server: " + ovmmServer
baseUri='https://'+ovmmServer+'/ovm/core/wsapi/rest'

#logging.basicConfig(level=logging.DEBUG)

## Create VM still failing
## GENERAL_JSON_PARSING_ERROR', u'message': u'GEN_000031:An error occurred parsing the JSON request'

def createVm(s,baseUri,repository,serverpool,vmName):
  #file:///home/rrosso/OvmSDK_3.4.2.1384/doc/api/webservices_r/resource_VmRs.html#path__Vm.html
  #repo_id=get_id_from_name(s,baseUri,'Repository','ovs1')
  #sp_id=get_id_from_name(s,baseUri,'ServerPool','ovs-home')
  repo_id=get_id_from_name(s,baseUri,'Repository',repository)
  sp_id=get_id_from_name(s,baseUri,'ServerPool',serverpool)
  #print 'repo_id {:20} ServerPool Id {:55}'.format(repo_id,sp_id)
  #OVM> create Vm name=MyVM repository=MyRepository domainType=XEN_HVM \ 
  #server=MyServer startPolicy=USE_POOL_POLICY on ServerPool name=MyServerPool

  data={ "name": vmName,
         "description": "A virtual machine created using the REST API",
         "vmDomainType": "XEN_PVM",
	 "repositoryId": repo_id,
         "serverPoolId": sp_id }

#  data={'serverPoolId':'0004fb00000200006aa35973e4d0e5af','repositoryId':'0004fb00000300000c6c2c52c5708b65'}
  print data
  uri='{base}/Vm'.format(base=baseUri)
  print uri
  #print json.dumps(data)
  r=s.post(uri,data=json.dumps(data))
  job=r.json()
  print job
  # wait for the job to complete
  vm_id=wait_for_job(job['id']['uri'],s)

## CloneVM failing
## The value for the argument "serverPoolId" was found to be null
def cloneVm(s,baseUri,templateVm,vmName):
  repo_id=get_id_from_name(s,baseUri,'Repository','ovs1')
  sp_id=get_id_from_name(s,baseUri,'ServerPool','ovs-home')
  template_id=get_id_from_name(s,baseUri,'Vm',templateVm)
  
  print 'clone {} into repo_id {:20} ServerPool Id {:55}'.format(template_id,repo_id,sp_id)  
  
  data={ "serverPoolId": sp_id,
  	 "repositoryId": repo_id,
  	 "createTemplate": False
        }

  uri='{base}/Vm/{vmId}/clone'.format(base=baseUri,vmId=template_id)
  r=s.put(uri,data=json.dumps(data))
  job=r.json()
  print job
  # wait for the job to complete
  vm_id=wait_for_job(job['id']['uri'],s)
  print "new vm id:" + vm_id

  ## change vm name here?

if __name__ == "__main__":
  print
  print

  #check_manager_state(baseUri,s)

  #createVm(s,baseUri,'VM3')
  createVm(s,baseUri,'ovs2','ovs-home','ovs2-VM3')
  #cloneVm(s,baseUri,'Ubuntu.0','VM4')
  
  serverList(s,baseUri)
  vmList(s,baseUri)

  #updateVmMemory(s,baseUri,'VM2','512')

  #updateVmDiskNames(s,baseUri,'VM2')
  #showVm(s,baseUri,'VM2')

  #cmdVm(s,baseUri,'start','VM2')
  #cmdVm(s,baseUri,'stop','VM2')

Oracle OVM rest api example

If you hate the Oracle OVM CLI and having to use expect scripts you may want to look into the web services API.  Note it looks like SOAP will be decommissioned soon so use REST.

$ cat ovm_using_rest.py 
import requests

s=requests.Session()
s.auth=('admin','pwd_removed')
s.verify=False #disables SSL certificate verification

s.headers.update({'Accept': 'application/json', 'Content-Type': 'application/json'})

baseUri='https://ovmm:7002/ovm/core/wsapi/rest'

print "\nServer List:"
print "##############"
r=s.get(baseUri+'/Server')
for i in r.json():
  # do something with the content
  print '{:20} {:20}'.format(i['serverRunState'],i['name'])

print "\nVM List:"
print "########"
r=s.get(baseUri+'/Vm')
for i in r.json():
  # do something with the content
  print '{:20} {:20}'.format(i['vmRunState'],i['name'])
  #print '{name} '.format(name=i['name'])

Output:

$ python ovm_using_rest.py 

Server List:
##############
/usr/lib/python2.7/dist-packages/urllib3/connectionpool.py:794: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.org/en/latest/security.html
  InsecureRequestWarning)
RUNNING              ovms2            
RUNNING              ovms3             
RUNNING              ovms1             

VM List:
########
TEMPLATE             EXALYTICS_BASE_LINUX_OEL5_GUEST_VM_TEMPLATE_2.2.0.0.0.el5
TEMPLATE             EXALYTICS_BASE_LINUX_OEL6_GUEST_VM_TEMPLATE_2.2.0.0.0.el6
RUNNING              OBIEXA3           
STOPPED              obiexa4           
[..]
STOPPED              EXALYTICS_BASE_LINUX_OEL5_GUEST_VM_TEMPLATE_2.0.1.4.0
TEMPLATE             OBIEE12C_TEMPLATE.0 
RUNNING              OBIEXA01         

Live Migrate Oracle VM for SPARC Logical Domains

Short version of live migration between two T5-2’s.  Using shared fiber LUN’s.

Source Host:

Check running status:

# ldm ls | grep test10
test10           active     -n----  5002    8     8G       0.0%  0.0%  1m

Dry run first:

# ldm migrate-domain -n test10 10.2.10.12
Target Password:
Invalid shutdown-group: 0
Failed to recreate domain on target machine
Domain Migration of LDom test10 would fail if attempted

We have a couple issues.

# ldm set-domain shutdown-group=15 test10

# ldm migrate-domain -n test10 10.2.10.12
Target Password:
Failed to find required volume test10-disk0@primary-vds0 on target machine
Domain Migration of LDom test10 would fail if attempted

# ldm ls-bindings primary | grep test10
test10@primary-vcc          5002   on
vnet1@test10                00:14:4f:fa:17:4f 1913                      1500
test10-disk0                                   /dev/dsk/c0t600144F09D7311B5000054789ED30002d0s2
disk0@test10                test10-disk0

Target Host:
Fix the virtual device reference.

# ldm add-vdsdev /dev/dsk/c0t600144F09D7311B5000054789ED30002d0s2 test10-disk0@primary-vds0

# ldm migrate-domain -n test10 10.2.10.12

Check status on target.  I suggest also running a ping during the whole process.

# ldm ls | grep test10 test10           bound      -----t  5018    8     8G
# ldm ls | grep test10 test10           active     -n----  5018    8     8G       0.1%  0.1%  7m

SUN Oracle ZFS Storage Simulator

Previously I wrote an article on getting the ZFS simulator to run on OVM.

Until recently I did not realize that I could upgrade the ZFS simulator on Virtualbox.  I kind of assumed the appliance is checking in the background and showing possible upgrades in the Available Updates page.  None of my simulators or real ZFS appliances was showing new Available updates either.  So here is what I did to update the simulator.  I assume it will work with the OVM ported version also.

If you go to this page https://wikis.oracle.com/display/fishworks/Software+Updates you can see what updates are available for your hardware or simulator.   Then updating is easy.  Just download the zip file you need.  Read the Release Notes.  Then uncompress the file where you are staging.  In the Maintenance > System screen click the plus sign next to Available Updates.  Find the .gz file in the folder structure and upload the image.  Follow the questions.

My simulator running under Virtualbox now shows the below version.  Note that the simulator was too far behind to skip to the latest version so I had to do an extra 2011.04.24 version also.

Sun ZFS Storage Appliance Simulator on OVM or KVM

For those familiar with the excellent ZFS file system and the Sun(now Oracle) storage products built on ZFS, the ZFS storage appliance interface is very easy to use; and definitely worth considering when looking at when purchasing a SAN.

Oracle has a simulator virtual machine to try out the interface.  Unfortunately it only runs on Virtualbox which is fine for those running Virtualbox on a desktop.  If you would like to run it on something more accessible by multiple users(KVM or OVM); the Solaris based image has some issues running.

I recently got the Virtualbox image to run on OVM and subsequently also got it to work on KVM.  This is a quick guide how to get the Virtualbox image to run as a qcow2 image on a KVM hypervisor.
Update: Changed to qed format. If you don’t have qed, qcow2 worked for me also.

As I understand there was also a vmware image but it disappeared from the Oracle website.  I am not sure why Oracle does not publish at least OVM images or make an effort to run the simulator on OVM.  Maybe there is a good reason and it’s possible that Oracle want to discourage it being used other than on Virtualbox.  Really not sure.

Stage the Image:
Download the simulator (link should be on this page somewhere): http://www.oracle.com/us/products/servers-storage/storage/nas/zfs-appliance-software/overview/index.html

From the vbox-2011.1.0.0.1.1.8 folder copy the Sun ZFS Storage 7000-disk1.vmdk file to the KVM host and convert to qcow2.

** Note my first attempt I used qcow and not qcow2 format and had issues starting the image so make sure and convert to qcow2.

# qemu-img convert "Sun ZFS Storage 7000-disk1.vmdk" -O qed SunZFSStorage7000-d1.qed
# image: SunZFSStorage7000-d1.qed
file format: qed
virtual size: 20G (21474836480 bytes)
disk size: 1.9G
cluster_size: 65536

Create Guest:
Create KVM guest. Use ide disk for SunZFSStorage7000-disk1.qed and specify qed format.

# virsh dumpxml ZfsApp

  ZfsApp
...

</pre>
<address> </address>
<pre>
...

</pre>
<address> </address>
<pre>
...

Boot new Virtual Machine from the sol-11_1-text-x86.iso. Choose language etc. Select shell when menu appears.

Update ZFS Image:

Now import and mount the ZFS file system.  Find the correct device name and update bootenv.rc:

In my case the disk device name for the boot disk is c7d0.  I use format to see the disk device name and then find the correct slice for the root partition.  You can use “par” and “pr” commands in format to see partitions.  In my case we are after /dev/dsk/c7d0s0 and we need to find the correct entry in the /device tree.

# format
Searching for disks...done

AVAILABLE DISK SELECTIONS:
0. c7d0
/pci@0,0/pci-ide@1,1/ide@0/cmdk@0,0
Specify disk (enter its number): ^C

# ls -l /dev/dsk/c7d0s0
lrwxrwxrwx 1 root root 50 May 26 10:35 /dev/dsk/c7d0s0 -> ../../devices/pci@0,0/pci-ide@1,1/ide@0/cmdk@0,0:a

From above I found the exact device name: /devices/pci@0,0/pci-ide@1,1/ide@0/cmdk@0,0:a

Lets go update the bootenv.rc now.

# zpool import -f system

# zpool list
NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT
system 19.9G 2.03G 17.8G 10% 1.00x ONLINE -

# zfs list | grep root
system/ak-nas-2011.04.24.1.0_1-1.8/root 1.26G 14.4G 1.25G legacy

# mkdir /a
# mount -F zfs system/ak-nas-2011.04.24.1.0_1-1.8/root /a

# zfs set readonly=off system/ak-nas-2011.04.24.1.0_1-1.8/root

# cp /etc/path_to_inst /a/etc
# vi /a/boot/solaris/bootenv.rc
...
setprop boot /devices/pci@0,0/pci-ide@1,1/ide@0/cmdk@0,0:a

# tail -1 /a/boot/solaris/bootenv.rc
setprop boot /devices/pci@0,0/pci-ide@1,1/ide@0/cmdk@0,0:a

# bootadm update-archive -R /a
updating /a/platform/i86pc/boot_archive
updating /a/platform/i86pc/amd64/boot_archive

# cd /
root@solaris:~/# umount /a
root@solaris:~/# zpool export system

# init 0

On next boot lets make sure Solaris detect the hardware correct.  When you see grub booting edit the kernel boot line and add “-arvs”.  Then continue booting.

Probably only need to add “-r” but I did “-arvs” to see more and also get into single user mode in case I needed to do more.

Once in single user mode with prompt just reboot.

For me at this point the image was booting into the zfs appliance setup and I could configure it.  Also on KVM adding SATA disks worked and the zfs appliance interface could use them for pools.

Update 04.23.14
I recently tried to update a simulator running on KVM but the update did not want to complete. Not sure if too many versions elapsed or something it does not like about being under KVM. Anyhow I tried moving an up to date (ak-2013.06.05.1.1) image on Virtualbox to KVM and that did work.

Migrate OVM Manager to a Different Server

For whatever reason you ever need to migrate an OVM Manager to a different host this would be helpful. I can’t imagine that this would need to happen that often but I recently did need to cut it over from a Virtualbox instance to an OVM Hypervisor. This is the OVM 3.2.1 local mysql(not Oracle XE) DB version.

Find the source UUID:

# more /u01/app/oracle/ovm-manager-3/.config
DBTYPE=MySQL
DBHOST=localhost
SID=ovs
LSNR=49500
OVSSCHEMA=ovs
APEX=8080
WLSADMIN=weblogic
OVSADMIN=admin
COREPORT=54321
UUID=0004fb0000010000434a8179cda0edbd
BUILDID=3.2.1.516

Copy recent backup and stage on new host. You can make a manual mysql backup or I just used the latest Auto backup:

# tar cpf a.tar AutoFullBackup-20130501_201957
# scp a.tar root@172.16.97.155:/root

Some pre-requisites the install would be pointing out:

# grep oracle /etc/group
dba:x:501:oracle

# tail -3 /etc/security/limits.conf
* soft nofile 16384
* hard nofile 65536
# End of file

[root@hovmmanager a]# ulimit -aH
...
open files                      (-n) 65536
...

Install Manager on new host using old UUID:

# pwd
/mnt/a

# ./runInstaller.sh --uuid 0004fb0000010000434a8179cda0edbd
Oracle VM Manager Release 3.2.1 Installer

Oracle VM Manager Installer log file:
/tmp/ovm-manager-3-install-2013-05-02-115510.log

Please select an installation type:
   1: Simple (includes database if necessary)
   2: Custom (using existing Oracle database)
   3: Uninstall
   4: Help

   Select Number (1-4): Please enter a valid value
   Select Number (1-4): 1

Starting production with local database installation ...

Verifying installation prerequisites ...
Unable to ping hostname 'hovmmanager.keste.com'.
*** WARNING: Recommended memory for the Oracle VM Manager server installation using Local MySql DB is 7680 MB RAM
Group 'dba' does not exist, create user 'oracle' with group 'dba' before installing
hardnofiles should be set to 8192 but was 4096

*** OK lets try again. Left above in to show the pre-requisites…

# ./runInstaller.sh --uuid 0004fb0000010000434a8179cda0edbd

Oracle VM Manager Release 3.2.1 Installer

Oracle VM Manager Installer log file:
/tmp/ovm-manager-3-install-2013-05-02-120741.log

Please select an installation type:
   1: Simple (includes database if necessary)
   2: Custom (using existing Oracle database)
   3: Uninstall
   4: Help

   Select Number (1-4): 1

Starting production with local database installation ...

Verifying installation prerequisites ...

One password is used for all users created and used during the installation.
Enter a password for all logins used during the installation:
Enter a password for all logins used during the installation (confirm):

Verifying configuration ...

Start installing the configured components:
   1: Continue
   2: Abort

   Select Number (1-2): 1

Step 1 of 9 : Database Software...
Installing Database Software...
Retrieving MySQL Database 5.5 ...
Unzipping MySQL RPM File ...
Installing MySQL 5.5 RPM package ...
Configuring MySQL Database 5.5 ...
Installing MySQL backup RPM package ...

Step 2 of 9 : Java ...
Installing Java ...

Step 3 of 9 : Database schema ...
Creating database 'ovs' ...
Creating user 'ovs' for database 'ovs'...

Step 4 of 9 : WebLogic ...
Retrieving Oracle WebLogic Server 11g ...
Installing Oracle WebLogic Server 11g ...

Step 5 of 9 : ADF ...
Retrieving Oracle Application Development Framework (ADF) ...
Unzipping Oracle ADF ...
Installing Oracle ADF ...
Installing Oracle ADF Patch...

Step 6 of 9 : Oracle VM  ...
Retrieving Oracle VM Manager Application ...
Extracting Oracle VM Manager Application ...
Installing Oracle VM Manager Core ...

Step 7 of 9 : Domain creation ...
Creating Oracle WebLogic Server domain ...
Starting Oracle WebLogic Server 11g ...
Configuring data source 'OVMDS' ...
Creating Oracle VM Manager user 'admin' ...

Step 8 of 9 : Deploy ...
Deploying Oracle VM Manager Core container ...
Deploying Oracle VM Manager UI Console ...
Deploying Oracle VM Manager Help ...
Granting ovm-admin role to user 'admin' ...
Set Log Rotation ...
Disabling HTTP and enabling HTTPS...
Configuring Https Identity and Trust...
Configuring Weblogic parameters...

Step 9 of 9 : Oracle VM Manager Shell ...
Retrieving Oracle VM Manager Shell & API ...
Extracting Oracle VM Manager Shell & API ...
Installing Oracle VM Manager Shell & API ...

Retrieving Oracle VM Manager Upgrade tool ...
Extracting Oracle VM Manager Upgrade tool ...
Installing Oracle VM Manager Upgrade tool ...

Retrieving Oracle VM Manager CLI tool ...
Extracting Oracle VM Manager CLI tool...
Installing Oracle VM Manager CLI tool ...
Copying Oracle VM Manager shell to '/usr/bin/ovm_shell.sh' ...
Installing ovm_admin.sh in '/u01/app/oracle/ovm-manager-3/bin' ...
Installing ovm_upgrade.sh in '/u01/app/oracle/ovm-manager-3/bin' ...
Enabling Oracle VM Manager service ...
Shutting down Oracle VM Manager instance ...
Restarting Oracle VM Manager instance ...
Waiting for the application to initialize ...
Oracle VM Manager is running ...
Oracle VM Manager installed.

Please wait while WebLogic configures the applications... This can take up to 5 minutes.

Installation Summary
--------------------
Database configuration:
  Database type               : MySQL
  Database host name          : localhost
  Database name               : ovs
  Database listener port      : 49500
  Database user               : ovs

Weblogic Server configuration:
  Administration username     : weblogic

Oracle VM Manager configuration:
  Username                    : admin
  Core management port        : 54321
  UUID                        : 0004fb0000010000434a8179cda0edbd

Passwords:
There are no default passwords for any users. The passwords to use for Oracle VM Manager, Database, and Oracle WebLogic Server have been set by you during this installation. In the case of a default install, all passwords are the same.

Oracle VM Manager UI:
  https://vmmanager.domain.com:7002/ovm/console
Log in with the user 'admin', and the password you set during the installation.

Please note that you need to install tightvnc-java on this computer to access a virtual machine's console.

For more information about Oracle Virtualization, please visit:
  http://www.oracle.com/virtualization/

Oracle VM Manager installation complete.

Please remove configuration file /tmp/ovm_configBltiZ5.

Restore DB:

# pwd
/u01/app/oracle/mysql/dbbackup

# mv /root/AutoFullBackup-20130501_201957/ ./

# /etc/init.d/ovmm stop
Stopping Oracle VM Manager                                 [  OK  ]
# /etc/init.d/ovmm_mysql stop
Shutting down OVMM MySQL..

# su - oracle
$

$ bash /u01/app/oracle/ovm-manager-3/ovm_shell/tools/RestoreDatabase.sh AutoFullBackup-20130501_201957
INFO: Expanding the backup image...
INFO: Applying logs to the backup snapshot...
INFO: Restoring the backup...
INFO: Success - Done!
INFO: Log of operations performed is available at: /u01/app/oracle/mysql/dbbackup/AutoFullBackup-20130501_201957/Restore.log

IMPORTANT:
      As 'root', please start the OVM Manager database and application using:
               service ovmm_mysql start; service ovmm start

$ logout

# /etc/init.d/ovmm_mysql start
Starting OVMM MySQL..                                      [  OK  ]
# /etc/init.d/ovmm start
Posted in OVM

Import Virtualbox Image Into Oracle VM

C:\Program Files\Oracle\VirtualBox>VBoxManage.exe showhdinfo c:\DATA\VirtualBox_VMs\fc18\fc18.vdi
UUID:                 3909d478-3ba5-4de5-bda1-65e802451aa0
Accessible:           yes
Logical size:         8192 MBytes
Current size on disk: 6192 MBytes
Type:                 normal (base)
Storage format:       VDI
Format variant:       dynamic default
In use by VMs:        fc18 (UUID: 5cc6b364-e935-4d00-a10a-da6aa3e1592f)
Location:             C:\DATA\VirtualBox_VMs\fc18\fc18.vdi

C:\DATA\VirtualBox_VMs\fc18>"c:\Program Files\Oracle\VirtualBox\VBoxManage.exe" clonehd fc18.vmdk fc18.img --format raw

– Run Filezilla FTP server local. Setup an user and share folder.
– Import Virtual disk in OVMM ftp://anonymous:rr@/fc18/system.img
– Rename appropriately something like vmname_system.img
– Create VM and attach disk image(s).