Wednesday, 16 November 2011

POWER Cloud Build Notes

- Requirements -
- IBM Systems Director with VMControl -

    
1/2 Procs Ent1 - Uncapped - 2/4 Virt - 4Gb Mem - 4-8Gb storage, extra if it is to be the NIM Master too.  OS 5.3 - 6.1 - 7.1
NIM Server requires Director Common Agent installed if it is not on the same node + VMControl sub agent
POWER System 6/7
HMC
Integrated Virtualization Manager (IVM)

NOTES -

After a initial install I hit a number of problem getting the director server to install NIM agents on the NIM server and other issues such as complete System Director failure after it not being used for a week.  In addition there where further problems getting the VMControl to work and Director rediscovering and inventorying known systems correctly.  This revision should resolve those issues and get a system built ready to be integrated in any Cloud environment.
It should be stated that this is only the second revision, so some details have been take from memory is searches of history files from when I progressing round a problem.  I have plans for further systems and more test when more equipment arrives.
I will also be looking into a possible issue that I can't get VMControl working correctly due to the HMC and System Director/NIM Server being on different VLANs, the plan being to see if I get the same failure when the HMC is moved to the same VLAN as everything else.

- VIO/System Director server install -

First you will need to get a VIO server installed - normal details can be followed to do this.  Once its up and running and you can created a number of Virtual adapters to use you will need to create a partion for the systems director to be installed on using the specifications above, so we need some system pools created -
- Good VIO quick sheet - http://www.tablespace.net/quicksheet/powervm-quickstart.html

First lets create a boot volume group on the VIO Server, this is where we will keep all the boot/rootvg data, then followed by a data VG.

Create boot storage pool -
# mksp cloud_boot hdisk1

Then the data pool -

# mksp cloud_data hdisk2 hdisk3

Note- the disks for the storage pool can be anything you like, either from the POWER systems internal disk or SAN attached, its your choice depending on what is available.  In this case its all internal disk.


Then change the default storage pool from rootvg to cloud_boot -

# chsp -default cloud_boot

list those pools -

# lssp
Pool              Size(mb)   Free(mb)  Alloc Size(mb)    BDs Type     
rootvg                  69888      46848             128      0 LVPOOL   
cloud_boot           69888      69888              64      0 LVPOOL   
cloud_data          139776     139776              64      0 LVPOOL


Create a client disk for the System Director on adapter vhost0 from cloud_boot storage pool -

# mkbdsp -sp cloud_boot 20G -bd lv_pclm1_boot -vadapter vhost0

This server will also be a NIM server so we will need to create some data storage for it -

# mkbdsp -sp cloud_data 40G -bd lv_pclm1_data -vadapter vhost0

Note- keep the lv names no longer the 15 characters, else it will not map the disks correct to the vhost as is passes the command over, so 'lsmap -all' fails.


To list the details in the storage pool -

# lssp -bd                    - shows the default pool, in this case cloud_boot
# lssp -bd -sp cloud_data    - to list other pools.


Once these pools are created and assigned over to the server you can get it booted for the network install, at this stage you need to boot into SMS and select the network device and configure the ip details.


- System Director Install -

Obtain the latest System Director code, this example is 6.2.1.2 -
Extract the gzip/tar file into a relevant dir with enough space, about 3Gb in all.  Source file is - "
SysDir6_2_Server_AIX.tar"
Kick of the install with
./dirinstall.server
It should automatically expand the /opt filesystem to ensure that there is sufficient space for the install.

For proper environment setup for the database on this platform, please go to the IBM Systems Director Info Center:

http://publib.boulder.ibm.com/infocenter/director/v6r2x/topic/com.ibm.director.install.helps.doc/fqm0_t_install_config_database_application.html

You must configure the agent manager prior to starting the server.

To configure the agent manager, run
# /opt/ibm/director/bin/configAgtMgr.sh
This will allow you to set the System Director user (root) and the default passwords, in our working environment they all match that of 'root'.

To start the server manually, run

# /opt/ibm/director/bin/smstart    (this path can be added to the .profile PATH variable)

Monitor the status of the start with the following command as it can take some time to start -

# smstatus -r
Ctrl-c breaks out once the status changes to active.

- Install the IBM Systems Director VMControl 2.3.1 Plugin -


Confirm that IBM System Director 6.2.1.2 is installed on your IBM Systems Director server.

Open a browser to
https://<director>:8422/ibm/console and log on.
On the Welcome to IBM Systems Director panel, click on the Manage tab.  Confirm IBM Systems Director 6.2.1.2 is installed

Logout of IBM Systems Director.
Start a telnet (or ssh) session to your IBM Systems Director server and sign on.
Create a directory for installing VMControl Enterprise Edition 2.3.1 and then change directories to it.
# mkdir /tmp/sd
# cd /tmp/sd

The IBM System Director VMControl Enterprise Edition 2.3.1 for AIX 60 day trial can be downloaded at ->
http://www.ibm.com/systems/software/director/vmcontrol/enterprise.html
Unzip and decompress the IBM System Director VMControl Enterprise Edition 2.3.1 installation file
# gunzip –c /tmp/sd/SysDir_VMControl_2_3_1_Linux_AIX.tar.gz | tar xvf –

Using VI from PuTTY or other editor, edit the /tmp/sd/installer.properties file as follows and save it.

Installer_UI = silent
License _accepted = true


Install IBM System Director VMControl Enterprise Edition 2.3.1 as follows:

# /tmp/sd/IBMSystemsDirector-VMControl-Setup.sh –i silent

When completed successfully, stop and restart IBM System Director

 
# /opt/ibm/director/bin/smstop
# /opt/ibm/director/bin/smstart
# /opt/ibm/director/bin/smstatus –r


Once IBM Systems Director is active, point your browser to
https://<director>:8422/ibm/console and log on.
On the Welcome to IBM Systems Director panel, click on the Manage tab.  
You should find that VMControl 2.3.1 is now listed.

- Part 2: Installing the VMControl 2.3.1.2 fix pack -
Start a telnet (or ssh) session to your IBM Systems Director server and sign on.
Log into the System Director and create the following dir for the download to be installed to.
# mkdir /tmp/sd/vmc
# cd /tmp/sd/vmc


This fixpack can be downloaded from IBM Fix Central at ->
http://www-933.ibm.com/support/fixcentral/ - Package should be com.ibm.director.vmc.collection_2.3.1.2
Confirm that the VMControl 2.3.1.2 fix pack files are located in the /tmp/sd/vmc directory.
I have tried at this point installing this on the local server using the graphical interface, but so far it fails.  I have found that I need to run a up-to-date dicovery of the system -
# smcli collectinv -p "All Inventory" -n <node_name>

Once this has completed you can then update the system with the VMControl package.

# smcli importupd -vr /tmp/sd/vmc
# smcli installupd -v -n pclm1_sdvmc -u com.ibm.director.vmc.collection_2.3.1.2


When completed successfully, stop and restart IBM System Director

 
# /opt/ibm/director/bin/smstop
# /opt/ibm/director/bin/smstart
# /opt/ibm/director/bin/smstatus –r


When IBM Systems Director is active, point your browser to
https://<director>:8422/ibm/console and log on.

Confirm that VMControl is now at 2.3.1.2 via the Welcome to Systems Director/Manage tab
 - NIM Master install -
Build the LPAR you are going to install the NIM master on either a standalone LPAR or in this case the Service Director LPAR -

Install the AIX OS release that you want to the NIM master to serve, remembering that the NIM server has to be at the current or higher OS/patch level then the clients it serves.


Make sure you are happy with the network setup of the NIM master, speed and such, so you can get things set-up.  Remember you can always make changes later, but it is easier to ensure this is all correct first.


Install the following filesets if they are not installed already:

dsm.core
openssh.base.client
openssl.base
bos.sysmgt.nim.master
bos.sysmgt.nim.spot
bos.sysmgt.nim.client
bos.net.nfs.server


Now we can create the NIM master with the following script -

# /usr/sbin/nim_master_setup -a mk_resource=yes -a volume_group=nimvg -a device=/mnt/610TL06_lpp_res

In the example above I had mounted a known good lpp source from another nim server and used that, but you can use mksysb's or a DVD image (default).

# lsnim        - list the now created NIM resources.

From here you can then add other LPP sources and SPOTs but at the moment we want to make sure that the System Director picks up the NIM respository and installs the NIM agent.  So now you have created the NIM filesystem it would be best to increase the size to about 30-40Gig.

# chfs -a size=30G /export/nim

Then perform a inventory on the Director server to pick up the changes to the server.

# smcli collectinv -p "All Inventory" -n <node>


- Installing the VMControl NIM subagent -


In the IBM Systems Director navigation pane, expand Systems Configuration and then click VMControl -

Notice message
DNZIMC7081 – No image repositories were detected in IBM Systems Director.  Click Install Agents.

On the Welcome to Agent Installation Wizard panel, click Next.

Click on Common Agent Subagent Packages.

Select
CommonAgentSubagent_VMControl_NIM-2.3.1.2 and click Add then click Next.

Select your IBM Systems Director server and click Add. And then click Next.

Review the summary of the Install Agent Task and then click Finish.


Run the job now and display its properties.  Monitor the job for a successful completion.

Collect inventory on your IBM Systems Director (and NIM master) server

When complete, in the IBM Systems Director navigation pane, expand Systems Configuration and then click VMControl.


Ensure that message
DNZIMC709I is now displayed indicating that No virtual appliances were found.

Then ensure that you restart the Director service -

# smstop;smstart;smstatus -r

- NIM Server further configuration -


We can now update the bosinst_data resource so that we have hands free installation, we copy a file from the master system to the NIM volume group and define it. This time though, we also modify the resource. Read carefully!

# cp /bosinst.data /export/nim/bosinst_data/6100-05bid_ow
        This was the NIM resource - lsnim -l 6100-05bid_ow that the NIM creation script created.

For an automatic installation there are a number of lines we need to change:


The lines to change are from:


    PROMPT = yes
    RECOVER_DEVICES = Default
    ACCEPT_LICENSES =
    ACCEPT_SWMA =
    IMPORT_USER_VGS =
    CONSOLE = Default
    CREATE_JFS2_FS = Default

to:
    PROMPT = no
    RECOVER_DEVICES = no
    ACCEPT_LICENSES = yes
    ACCEPT_SWMA = yes
    IMPORT_USER_VGS = no
    CONSOLE = /dev/vty0
    CREATE_JFS2_FS = yes

add
  
  ENABLE_64BIT_KERNEL = yes

And simplify the target_disk_data: stanza from:

target_disk_data:
        PVID = 00f67207e26dbb8d
        PHYSICAL_LOCATION = U8233.E8B.107207P-V41-C21-T1-L8100000000000000
        CONNECTION = vscsi0//810000000000
        LOCATION =
        SIZE_MB = 70006
        HDISKNAME = hdisk0

to:
target_disk_data:
        HDISKNAME = hdisk0


Remember it is the file
/export/nim/bosinst_data/6100-05bid_ow that needs to be edited, though you can of course change any more lines that you might want.

If you want to create and add any other additional resources then the following commands should help -


First this a package check to pick out any duplicate files in any lpp sources -

# /usr/lib/instl/lppmgr -d /export/nim/lpp_source/<lpp_source>  -u -b -x -r

From a lpp source you can then create a SPOT, in this case from lpp 610TL06-lpp into a spot called 610TL06_spot.

# nim -o define -t spot -a server=master -a source=610TL06-lpp -a location=/export/nim/spot/610TL06_spot 610TL06_spot

To perform a command line discovery -

# smcli discover -i <ip-address>,<ip-address>

Then you should be able to perfrom a install via the NIM server

# nim -o bos_inst -a lpp_source=<lpp-source> -a spot=<spot> -a bosinst_data=<bos-inst> -a no_client_boot=no <server-name>
 

No comments:

Post a Comment