vCloud Director automation via Orchestrator, Automating Org,vDC,vApp Provisioning via CSV Part4 – Adding Approval and Email Notifications

This is the final blog of this series, in the previous 3 parts:

Par1: https://autodiscover.wordpress.com/2013/11/05/vcloud-orchestrator-automating-orgvdcvapp-provisioning-via-csv-part1/

Part2: https://autodiscover.wordpress.com/2013/11/11/vcloud-orchestrator-automating-orgvdcvapp-provisioning-via-csv-part2reading-csv-files/

Part3: https://autodiscover.wordpress.com/2013/12/23/vcloud-director-automation-via-orchestrator-automating-orgvdcvapp-provisioning-via-csv-part3-adding-nics-cpu-and-modify-memory/

we explored how to automate most of cloud provisioning elements including organizations, vDCs, vAPPs and Virtual machine and customizing their properties like adding vNICs, VHDs and memory/CPU.

In this final part, we will explore how we can add approval cycle to the above provisioning.

In our Scenarios, we will send an email notification to the administrator that include the CSV file used to generate the cloud as attachment, and include a hyperlink to approve/deny the request, let us see how we can do it.

Import the PowerShell Plugin:

We will use Powershell to send email notifications, I tried to use Javascripting but had no luck with attaching the CSV, Powershell comes to rescue here, so you need to import the powershell plugin to your Orchestrator through the Orchestrator configuration interface:

Once you import the powershell plugin, make sure to restart the VCO.

When you complete the restart, go to add a powershell host, you need to make sure that remote powershell is enabled on the server, once done kick of the add powershell host workflow:

if you are adding a kerberos host, make sure to type the username in UPN or email format otherwise you will get this weird error: Client not found in Kerberos database (6) (Dynamic Script Module name : addPowerShellHost#16)

Once added you are ready to go.

Building the Approval workflow:

Build a workflow that includes user interaction and decision as following:

The attributes are defined as following:

the scriptable task, sends email notification with attachments as we said, let us the Javascript portion of it:

//var scriptpart0 = "$file =c:\\customer.csv"

// URL that will open a page directly on the user interaction, so that user can enter the corresponding inputs
var urlAnswer = workflow.getAnswerUrl().url ;
var orcdir = "C:\\orchestrator\\" ;
var fileWriter = new FileWriter(orcdir + name+".html")
var code =     "<p>Click Here to <a href=\"" + urlAnswer + "\">Review it</a>";

fileWriter.open() ;

fileWriter.writeLine(code) ;
fileWriter.close() ;

var output;
var session;
try {
    session = host.openSession();
    var arg = name+".html";
    Server.log (arg);
    var script =  ‘& "’ + externalScript + ‘" ‘ + arg;
    output = System.getModule("com.vmware.library.powershell").invokeScript(host,script,session.getSessionId()) ;
} finally {
    if (session){
        host.closeSession(session.getSessionId());
    }
}

 

The script attaches the CSV file, then starts the powershell script from the host and attaching the HTML file (in the arguments, this HTML file contains a link to approve the reqeust, and it was built in the above Javascript), let us see the powershell script:

Param ($filename)
$file = "c:\orchestrator\customer.csv"
$htmlfile = "C:\orchestrator\" + $filename
$smtpServer = "vcenter.tsxen.com"

$att = new-object Net.Mail.Attachment($file)
$att1 = new-object Net.Mail.Attachment($htmlfile)

$msg = new-object Net.Mail.MailMessage

$smtp = new-object Net.Mail.SmtpClient($smtpServer)

$msg.From = "emailadmin@test.com"

$msg.To.Add("administrator@tsxen.com")

 

$msg.Subject = "New Cloud is requested"

$msg.Body = "A new cloud service is request, attached the generation file, you can approve the request using the below link"

$msg.Attachments.Add($att)
$msg.Attachments.Add($att1)

$smtp.Send($msg)

$att.Dispose()

 

Now you are ready to go, let see the outcome:

If you run the script successfully, you will receive the following email notification:

you can see the link to approve the request and the CSV file included in the email to approve it, if you click on the link, you can see the request and approve/deny it:

What is next:

You might think this is the end, however it is not, this blog series is the foundation of cloud automation and it is just a placeholder, cloud automation can go epic, here is some improvement suggestions for people who might want to take it further:

• Add error checking, the script is error checking free which might raise serious issues in production environment.
• Add More logging.
• Add automation to network provisioning and vShield operations.
• Automate application provisioning on top of provisioned VMs.

the above is a small list, we can spend years adding to this list, but those are the areas that I will be working on in the upcoming version of this script.

Till next.

vCloud Director automation via Orchestrator, Automating Org,vDC,vApp Provisioning via CSV Part3- Adding NICs, CPU and modify Memory

In part1: https://autodiscover.wordpress.com/2013/11/05/vcloud-orchestrator-automating-orgvdcvapp-provisioning-via-csv-part1/

and pat2: https://autodiscover.wordpress.com/2013/11/11/vcloud-orchestrator-automating-orgvdcvapp-provisioning-via-csv-part2reading-csv-files/

We explored the building blocks for automating vCloud Director cloud provisioning, we provisioned Orgs, vDCs and VMs by reading a CSV file and automating most of the provisioning process, in this blog post we will take a look on how we can add NICs, CPU and modify memory based on VM specs.

Remember, the CSV file structure is:

so we need to have VM name, Template name, No. of vCPUs, memory in GB, no. of HDDs and the size of each harddisk.

To start, you will need to clone add NIC to VM, Change Number of CPUs, Change memory capacity and Add a harddisk workflows.

Next, we will create what we will call “A caller” workflow, because each workflow requires certain parameters that only can be ready via query because some of them still not created yet like VM with that name or vDC for example.

We will create 4 callers to call each of the above work flows, here are their Java Scripts code:

VHD Caller Script:

var queryService = vapp.getHost().getQueryService();
var expression = new VclExpression(VclQueryVMField.NAME, vmname , VclExpressionType.EQUALS);
var filter = new VclFilter(expression);
var params = new VclQueryParams();
params.setFilter(filter);
var resultSet = queryService.queryRecords(VclQueryRecordType.ADMINVM, params);
    var records = resultSet.getRecords(new VclQueryResultAdminVMRecord());
Server.log (records.length);
for each (var record in records) { 
        if (record.vdc == vdc.getReference().href) {
Server.log("i am in the VMREF")
            var VMref = new VclReference(); 

        VMref.href = record.href; 
        VMref.name = record.name; 
        VMref.type = record.type; 
      myVMname = host.getEntityByReference(VclFinderType.VM, VMref);
}
}
Server.log (myVMname.name);
var addHDworkflowParameters = new Properties();
addHDworkflowParameters.put("vm",myVMname);
addHDworkflowParameters.put("size",hdsize);
addHDworkflowParameters.put("controller","SCSI Controller 0");
var myaddHDWorkflowToken = new WorkflowToken() ;
myaddHDWorkflowToken = MyAddVHDWF.execute(addHDworkflowParameters);
myaddHDworkflowTokens = new Array(); 
myaddHDworkflowTokens.push(myaddHDWorkflowToken);
System.getModule("com.vmware.library.workflow").waitAllWorkflowComplete(myaddHDworkflowTokens)

 

 

Memory Caller:

var ChangeVMMemParameters = new Properties();
var queryService = INPVMvApp.getHost().getQueryService();
var expression = new VclExpression(VclQueryVMField.NAME, INPVMNameString , VclExpressionType.EQUALS);
var filter = new VclFilter(expression);
var params = new VclQueryParams();
params.setFilter(filter);
var resultSet = queryService.queryRecords(VclQueryRecordType.ADMINVM, params);
    var records = resultSet.getRecords(new VclQueryResultAdminVMRecord());
Server.log (records.length);
for each (var record in records) { 

        if (record.vdc == vdc.getReference().href) {
Server.log("i am in the VMREF")
            var VMref = new VclReference(); 

        VMref.href = record.href; 
        VMref.name = record.name; 
        VMref.type = record.type;
 
      myVMname = host.getEntityByReference(VclFinderType.VM, VMref);
}
}
Server.log (myVMname.name);
ChangeVMMemParameters.put("vm",myVMname);
ChangeVMMemParameters.put("newSize",INPNewMem);

var ChangeVMmemWorkflowToken = new WorkflowToken() ;
ChangeVMmemWorkflowToken = AttChangeVMMemWF.execute(ChangeVMMemParameters);
ChangeVMmemWorkflowTokens = new Array(); 
ChangeVMmemWorkflowTokens.push(ChangeVMmemWorkflowToken);
System.getModule("com.vmware.library.workflow").waitAllWorkflowComplete(ChangeVMmemWorkflowTokens)

Change CPU Caller:

var ChangeVMCPUParameters = new Properties();
var queryService = INPVMvApp.getHost().getQueryService();
var expression = new VclExpression(VclQueryVMField.NAME, INPVMNameString , VclExpressionType.EQUALS);
var filter = new VclFilter(expression);
var params = new VclQueryParams();
params.setFilter(filter);
var resultSet = queryService.queryRecords(VclQueryRecordType.ADMINVM, params);
    var records = resultSet.getRecords(new VclQueryResultAdminVMRecord());
Server.log (records.length);
for each (var record in records) { 

        if (record.vdc == vdc.getReference().href) {
Server.log("i am in the VMREF")
            var VMref = new VclReference(); 

        VMref.href = record.href; 
        VMref.name = record.name; 
        VMref.type = record.type;
 
      myVMname = host.getEntityByReference(VclFinderType.VM, VMref);
}
}
Server.log (myVMname.name);
ChangeVMCPUParameters.put("vm",myVMname);
ChangeVMCPUParameters.put("cpuNumber",INPNewCPU);

var ChangeVMCPUWorkflowToken = new WorkflowToken() ;
ChangeVMmemWorkflowToken = AttChangeVMCPUWF.execute(ChangeVMCPUParameters);
ChangeVMCPUWorkflowTokens = new Array(); 
ChangeVMCPUWorkflowTokens.push(ChangeVMmemWorkflowToken);
System.getModule("com.vmware.library.workflow").waitAllWorkflowComplete(ChangeVMCPUWorkflowTokens)

NIC Caller:

var ChangeVMCPUParameters = new Properties();
var queryService = INPVMvApp.getHost().getQueryService();
var expression = new VclExpression(VclQueryVMField.NAME, INPVMNameString , VclExpressionType.EQUALS);
var filter = new VclFilter(expression);
var params = new VclQueryParams();
params.setFilter(filter);
var resultSet = queryService.queryRecords(VclQueryRecordType.ADMINVM, params);
    var records = resultSet.getRecords(new VclQueryResultAdminVMRecord());
Server.log (records.length);
for each (var record in records) { 

        if (record.vdc == vdc.getReference().href) {
Server.log("i am in the VMREF")
            var VMref = new VclReference(); 

        VMref.href = record.href; 
        VMref.name = record.name; 
        VMref.type = record.type;
 
      myVMname = host.getEntityByReference(VclFinderType.VM, VMref);
}
}
Server.log (myVMname.name);
ChangeVMCPUParameters.put("vm",myVMname);
ChangeVMCPUParameters.put("ipAddressingMode",ipAddressingMode);

var ChangeVMCPUWorkflowToken = new WorkflowToken() ;
ChangeVMmemWorkflowToken = AttChangeVMCPUWF.execute(ChangeVMCPUParameters);
ChangeVMCPUWorkflowTokens = new Array(); 
ChangeVMCPUWorkflowTokens.push(ChangeVMmemWorkflowToken);
System.getModule("com.vmware.library.workflow").waitAllWorkflowComplete(ChangeVMCPUWorkflowTokens)

 

Now we finished with our callers scripts, let us go back to the original code to see what is happening.

now to continue interpreting the CSV file, to do that, we need to modify the original script we used to read the CSV file to be like this (this is the final script code):

var i = 0;
var csvFileName = "c:\\Orchestrator\\customer.csv";
Server.log(csvFileName);

var fileReader = new FileReader(csvFileName);
fileReader.open();
var line=fileReader.readLine();
Server.log (line);

while (line != null){
Server.log ("loop Started");
var lineValues = line.split(",");
var linelength = lineValues.length
var vmname = lineValues[0];
var INPTemplateNameString = lineValues[1];
var MyNoCPU = lineValues[2];
var MyNoMemory = lineValues[3] * 1000;
var MyNoNICS = lineValues[4];
var addVMworkflowParameters = new Properties();
var ChangeVMmemworkflowParameters = new Properties();
var ChangeVMCPUworkflowParameters = new Properties();
var ChangeVMNICworkflowParameters = new Properties();

addVMworkflowParameters.put("INPdestinationVapp",MyVappName);
addVMworkflowParameters.put("INPTemplateNameString",INPTemplateNameString);
addVMworkflowParameters.put("INPVMNameString",vmname);
var myaddVMWorkflowToken = new WorkflowToken() ;
myaddVMWorkflowToken = MyAddVMWF.execute(addVMworkflowParameters);
myaddVMworkflowTokens = new Array();
myaddVMworkflowTokens.push(myaddVMWorkflowToken);
System.getModule("com.vmware.library.workflow").waitAllWorkflowComplete(myaddVMworkflowTokens)

while (MyNoNICS > 0)
{
ChangeVMNICworkflowParameters.put("INPVMNameString",vmname);
ChangeVMNICworkflowParameters.put("INPVMvApp",MyVappName);
ChangeVMNICworkflowParameters.put("INPVMvDC",myvdcname);
var myaddNICWorkflowToken = new WorkflowToken() ;
myaddNICWorkflowToken = AttNICWF.execute(ChangeVMNICworkflowParameters);
myaddNICWorkflowTokenS = new Array();
myaddNICWorkflowTokenS.push(myaddNICWorkflowToken);
System.getModule("com.vmware.library.workflow").waitAllWorkflowComplete(myaddNICWorkflowTokenS)

MyNoNICS = MyNoNICS – 1
}

Server.log("Adding Memory to" + " " + vmname);

ChangeVMmemworkflowParameters.put("INPVMNameString",vmname);
ChangeVMmemworkflowParameters.put("INPNewMem",MyNoMemory);
ChangeVMmemworkflowParameters.put("INPVMvApp",MyVappName);
ChangeVMmemworkflowParameters.put("INPVMvDC",myvdcname);
var ChangeVMmemWorkflowToken = new WorkflowToken() ;
ChangeVMmemWorkflowToken = AttMemWF.execute(ChangeVMmemworkflowParameters);
ChangeVMmemWorkflowTokens = new Array();
ChangeVMmemWorkflowTokens.push(ChangeVMmemWorkflowToken);
System.getModule("com.vmware.library.workflow").waitAllWorkflowComplete(ChangeVMmemWorkflowTokens)

Server.log("Adding CPU to" + " " + vmname);
ChangeVMCPUworkflowParameters.put("INPVMNameString",vmname);
ChangeVMCPUworkflowParameters.put("INPNewCPU",MyNoCPU);
ChangeVMCPUworkflowParameters.put("INPVMvApp",MyVappName);
ChangeVMCPUworkflowParameters.put("INPVMvDC",myvdcname);
var ChangeVMCPUWorkflowToken = new WorkflowToken() ;
ChangeVMCPUWorkflowToken = AttCPUWF.execute(ChangeVMCPUworkflowParameters);
ChangeVMCPUWorkflowTokens = new Array();
ChangeVMCPUWorkflowTokens.push(ChangeVMCPUWorkflowToken);
System.getModule("com.vmware.library.workflow").waitAllWorkflowComplete(ChangeVMCPUWorkflowToken)

Server.log("Adding NICs to" + " " + vmname);

var j = 5;

while (j < linelength)
{
Server.log ("i" + " " + i);
Server.log ("j" + " " + j);
Server.log ("linelength" + " " + linelength);
Server.log ("lineValues[j]" + " " + lineValues[j]);
var addHDworkflowParameters = new Properties();

addHDworkflowParameters.put("host",host);
addHDworkflowParameters.put("vdc",myvdcname);
addHDworkflowParameters.put("vapp",MyVappName);
addHDworkflowParameters.put("hdsize",lineValues[j]);
addHDworkflowParameters.put("vmname",vmname);
var myaddHDWorkflowToken = new WorkflowToken() ;
myaddHDWorkflowToken = MyAddVHDWF.execute(addHDworkflowParameters);
myaddHDworkflowTokens = new Array();
myaddHDworkflowTokens.push(myaddHDWorkflowToken);
System.getModule("com.vmware.library.workflow").waitAllWorkflowComplete(myaddHDworkflowTokens)
j = j + 1;

}
i = i + 1;
Server.log("looping");
line=fileReader.readLine();
}
fileReader.close;

 

no we are reading the file, storing the configuration in parameters, cloning the VM then looping through the configuration of NIC, vCPU and memory to configure the VM accordingly, let us take a look to what is happening exactly.

Adding NICs:

The following portion of the above code adds NICs to each VM, let us see how:

while (MyNoNICS > 0)
{
ChangeVMNICworkflowParameters.put("INPVMNameString",vmname);
ChangeVMNICworkflowParameters.put("INPVMvApp",MyVappName);
ChangeVMNICworkflowParameters.put("INPVMvDC",myvdcname);
var myaddNICWorkflowToken = new WorkflowToken() ;
myaddNICWorkflowToken = AttNICWF.execute(ChangeVMNICworkflowParameters);
myaddNICWorkflowTokenS = new Array();
myaddNICWorkflowTokenS.push(myaddNICWorkflowToken);
System.getModule("com.vmware.library.workflow").waitAllWorkflowComplete(myaddNICWorkflowTokenS)

MyNoNICS = MyNoNICS – 1
}

The above code creates a workflow parameters with VM name, VApp and vDC, then we create a token and execute The Caller script, the workflow then go and execute and return some parameters that is necessary for the Add NIC workflow and execute the Add NIC workflow.

the same goes for the rest of callers and java codes with different parameters and attributes.

once all set and done, you are ready to go, and this will read the CSV, create org, vdc, vapp, vms with the specifications you set in the CSV.

below is a copy of the workflows to import them and have a look.

https://skydrive.live.com/redir?resid=6B566FD2C47B21C4%21130

In Part 4 we will look on how to add approval and send email notifications.

 

Merry Christmas.

vCloud Orchestrator, Automating Org,vDC,vApp Provisioning via CSV Part1

Part 2  here: https://autodiscover.wordpress.com/2013/11/11/vcloud-orchestrator-automating-orgvdcvapp-provisioning-via-csv-part2reading-csv-files/

Hi There, during the past months I was working on this insane idea of automating the cloud provisioning. I eat,drink and breath (probably piss) cloud but I had a dream about automating the provisioning cycle and making it smart.

In this blogging series we will explore how you can use your vSphere, vCenter, vCloud and vCloud Orchestrator to provisioning clouds on the fly.

I know that there are a lot of blogs out there that speaks about how you can put some workflows to provide simple provisioning, this blog will take it a notch higher by doing it in a practical way. you don’t expect in real life people to login and supply cloud’s information, everything is in Excel.

By the end of this blog series we will:

– Configure cloud provisioning to provision Org, vDC and vApp using a single user input and link 3 workflows.

– Provision a VMs based on settings supplied in a CSV file, we will customize the HD sizes and no. of those VMs based on the settings supplied in the CSV.

– We will add a twist to add approval cycle to the process and send email to the approver with the CSV file to review and a link to approve the request, cool ha .

If you are old or new to orchestrator I guess you will like this series, we will touch on core concepts as well as new/advanced concepts with some Javascripting as well, so let us get started:

Part 1: Linking “Add Org” , “Add a vDC” and “Compose a vApp”:

before we get started, make a folder in your vCO and call it for example (Full cloud provisioning) and duplicate the “Add Org” , “Add a vDC” and “Compose a vApp”.

Create a workflow, call it Full Cloud Provisioning or any name you want, and add the workflows to it to create an org, then a vDC and a vApp, and link them together.

once the workflows are linked, we need to work our way through the in-parameters so we can have them set properly.

The workflows will required various inputs including the default vCloud host, Org/vDC name and various types of settings like lease duration, storage profile..etc, the most important thing for this blog post is the name, so link the inputs in the Add Org workflow as following:

you can set the other settings like (LDAP mode, SMTP host..etc) as attributes and pass them to the input parameters and that will make those inputs not needed thus not visible by the user, you can do the same of number of VMs, storage/memory/CPU limits, or you can have them in the presentation, for the purpose of my lab, I linked them to attributes in the my workflow, so they are static for every customer, but you might want to customize that based on your environment:

here is how I set my parameters:

Passing out-parameters to Input parameters in vCloud Orchestrator:

Before delving in this, let us discuss first the vCO attributes/input types. If you noticed in the above screenshot, you will find that we have (for example) an attribute called host and its type is vCloud: host, so what is the type thing?!

Attributes/inputs are defined in types based on their usage, so if you are doing same mathematical operations, you create a variable of type (Number), to hold text data you create a variable of type (String).

In vCO, we have additional set of types, like the vCloud:host or vCloud:vDC (those are for example, we have zillions of other types), which allows you to set these variables, parameters or inputs to carry specific type of data in them.

For Example, the host type, will always expect to carry in or out a vcloud:host data, you can’t pass text (string) to it but you can take it out as string, and same for a vDC, it will always expect to carry in/out a vDC.

Why I Explored this first, because if you set everything correctly and run the workflow, you will find that the add vDC workflow expects to be hosted within organization, and we need to specify the organization where this vDC will be hosted (and the same applies for vApp since it needs a vDC), but we can’t specify the vDC because it has been not create yet and the same is true for the organization, so how can we fix that ?!

I spent a month trying to figure this one out, I came from .net/Opalis background where we can pass the outs of one workflow/program to the inputs of another workflow/program, but this is not the case in vCO, you can’t do that.

To do that, you will need to create a parameter of the same type that you want to pass the out parameters to it , where it could be carried to the next workflow as in-parameters, you must pay attention to the type or the data won’t pass.

so for example, I created a parameter called MMvDC (I should have named it MyOrg) with the type vCloud: Organization, to carry the organization name from my “Add Org” workflow to the create vDC workflow, the same for mVDCName attribute of type vCloud:vDC to carry the vDC name from the Add a vDC workflow to the Compose a vApp workflow.

The next step is to bind the out parameters to those attribute, so to carry the organization name, I need to link this attribute with the out parameter, and since attributes are accessible across all workflows, I can use this attribute in any upcoming workflow and this is how we can skip entering organization/vDC names in subsequent workflows because they will receive those inputs with the correct types in there in parameters:

Now in the Add vDC workflow, you can see that it needs input (org) as type vCloud: organization, but I can supply this input on the fly from the previous workflow by linking the MMvDC attribute to the org input:

You can also see that I linked the vDCout parameter to the myVDCname attribute to carry the VDC to the compose vApp workflow.

by doing the above trick and working out the required inputs/parameters, the resulting workflow and presentation will be something like this:

So all what you need to do is supply organization name, and vCO will go and create in vCloud director an organization, a vDC and vApp inside that vDC to host VMs.

In the next blogpost, we will explore how we can read a CSV file and create VMs inside that vApp .