POWER Cloud intergrating IWD

Now that we have built our POWER Cloud frame-work we can look at integrating this in with IWD, I have been testing with both 3.1 and 3.  As things move on, I will be looking at testing this with Starter Kit for Cloud and IBM Service Delivery Manager.

- IWD Setup for POWER - 

- System Requirements - 
1/2/4 Procs Ent1 - Uncapped-168 - 4/4/8 Virt - 8/12/16Gb Mem - 30-60Gb storage, choose large if it is to be the NIM Master too.  AIX 6.1.5.3 
As we will be using IBM Workload Deployer this has been updated to match the cpu/memory/storage requirements, NIM Server requires Director Common Agent installed if it is not on the same node + VMControl sub agent POWER System 6/7 1 Dedicated HMC model CR5 and above.
 
- NOTES - 
In my blog post above 'POWER Cloud Install Notes' the pool sizes are rather small examples only, ideally you need about 40GB per deployed image, so even a small environment is going to be about 200-500GB in size.  Generally we will be pushing our installs to the cloud_boot pool and using the cloud_data for the System Director/NIM server so this should stay fairly static.  Though this of course is just the way I have set it up, so feel free to change it to suit your own needs 

- Final Configuration Updates - 
Once you have the POWER Cloud underlying infrastructure set-up you need to make the following updates to the system, this will ensure that there will be no issues between the POWER VMControl and IWD.
Check ulimit –a to ensure that “file(blocks)” is unlimited. The “nofiles(descriptor)” setting should be at least 6000 or unlimited. These parameters are set by editing /etc/security/limits; unlimited setting is “-1” in that file.
 
NIM tuning suggestions:

 # no -p -o tcp_recvspace=524288
# no -p -o tcp_sendspace=524288
# no -p -o udp_recvspace=655360
# no -p -o udp_sendspace=65536
# no -p -o arpqsize=128
# no -p -o rfc1323=1
# no -p -o sb_max=8388608
# chdev -l sys0 -a maxuproc='1024' 

The /tmp file system should be enlarged to have at least 15GB of free space. If you plan to store installation or fix pack files in /tmp, consider making the file system 20GB. The ISD will use the /tmp file system during IBM Workload Deployer deployments to temporarily cache catalog images as they are being passed to NIM. These catalog images can be larger than 10GB.
You will need a group of IPs (with resolvable hostnames) for your virtual system deployments. You will place these IP entries within an “IP group” within the IBM Workload Deployer appliance. In addition to the cloud system customers, these IPs/hostnames must be accessible from the HMC and from the IBM Workload Deployer appliance. Test the DNS entries for these IP address and hostnames using nslookup (for example). 

File systems (minimums) -
File system                 Minimum size     Comments
/  (root file system)     1GB
 /usr                             4GB
 /var                             2GB
 /opt                             6GB                   ISD installs in /opt & needs at least 4GB more.
/tmp                             15GB                 At least 1GB is required for ISD installation.

Workload Deployer V3.0, images will be staged here temporarily during deployment, so must have at least 15GB of free space at that time.
 
On the IBM Systems Director lpar, make these customizations: 
    Edit <sysdir_installation_root>/lwi/conf/overrides/USMIKernel.properties
    Add these two lines -

    Kernel.REST.Max.Concurrent.Requests=40

    max.cli.threads=17 

    Edit <sysdir_installation_root>/lwi/conf/overrides/director.javaopt 
     Change the Java heap to -         –Xms2048m –Xmx4096m

 This setting is based on an 8GB IBM Systems Director server, in my testing I have been running about 12GB initially.  Along with this I increased the processing to 2 full processors, 4 virtual. 
Set paging space for IBM Systems Director to be 3GB minimum. 

- IWD Deployment to POWER Issues - 
Now we did hit one issue once we connected to IWD with our system, in that the upload of the appliance to our image repository was successful once we had sufficient space, at the creation of the LPAR failed with the following message on IWD - 
Virtual machine could not be registered
VMControl Message: Start asynch work run for deploy of virtual appliance :7926
VMControl Message: Start asynch work run for deploy of virtual appliance :7926
VMControl Message: Workload, DB2 Enterprise 9.7 (PowerVM), was created. 

Looking at the agent logs - /opt/ibm/director/agent/logs/error-log-#.html - found the following -
       CLASS NetworkAdapter:
 Primary Adapter = true
 IP Address = 172.19.105.227
 Host Name = hiciwd227
 Gateway = 172.19.105.1
 Netmask = 255.255.248.0
 DNS1 IP Address = 172.19.105.30
 DNS2 IP Address = 172.19.105.30
 Domain Name =
 MAC Address = null
 Cable Type = N/A
 Speed = auto
 Duplex = auto
 Network Type = ent
 Display name= networkport.6]
 ::class.method=NIMMaster.validateDeploy() ::thread=HttpServer : 6 ::loggername=com.ibm.director.im.rf.nim.master
 ::class.method=unknown ::thread=HttpServer : 6 ::loggername=SystemErr
2011/11/24 17:08:41.730 INFO 2011/11/24 17:08:41.729 INFO Image /export/nim/appliances/4d3571a0-3ffe-4623-8026-d59e69a03ee6/image1.mksysb referenced by NIM Reso
urce appliance-1_image-1 was found. ::class.method=NIMMaster.validateAppliance() ::thread=HttpServer : 6 ::loggername=com.ibm.director.im.rf.nim.master
 ::class.method=unknown ::thread=HttpServer : 6 ::loggername=SystemErr
2011/11/24 17:08:42.239 INFO 2011/11/24 17:08:42.238 SEVERE DNZIMN620E Error occurred while trying to resolve hostname from IP 172.19.105.227. ::class.method=NI
MMaster.getResults() ::thread=HttpServer : 6 ::loggername=com.ibm.director.im.rf.nim.master
 ::class.method=unknown ::thread=HttpServer : 6 ::loggername=SystemErr 

As you can see it can't resolve the address that it has assigned to the new deployed LPAR, and this was because these addresses where not in DNS.  You can fix the issue by adding them in to DNS or /etc/hosts on the NIM server. 

- IWD Example Image -