Handy SVC mini-scripts

In a former life, I was a storage administrator for Thomas Jefferson University Hospital in Center City, Philadelphia. Aside from the wide array of lunch options we had to choose from, another thing I really enjoyed was working with IBM's SVC, or SAN Volume Controller. As a sysadmin, there are few software packages that you truly enjoy working with and you think are really, really good, but the SVC was one of those. One of the reasons was because the command-line interface was just a restricted Bash shell - something I knew very, very well. This allowed me to do pretty amazing things, as the shell is a wonderfully capable programming language. Back then, I had a Wiki that I used as a notepad, and I had several pages on the cool things you could do with SVC on the command line. Here is one of those pages, cut-and-pasted from my internal webserver. Be warned, though - none of this has been updated since about 2007!

Thanks to the awesome Aussie Storage Blog for the inspiration to resurrect these old pages.

---

SVC's command line interface (CLI) is a restricted bash shell. You might not be able to cd to other directories or run commands like "grep" or "awk", (see below) but you can still do some really useful stuff using the shell builtins available to you.

Basic info

---

• The "for var in blah blah blah blah; do; done" and "while read; do; done" loops are very powerful tools available to the shell script programmer, and are the backbone of virtually every miniscript I write.
• SVC's bash is restricted, but otherwise the same as the bash you would find on any linux box. Prior to SVC 4.2, the bash version is 2.05. At 4.2, the bash is updated to 3.1. If you've got a linux system available to play with, you can try out these miniscripts there before you try them on your production SVC. If you want to test these scripts on a different linux system, you might want to see the SVC miniscript testing page.
• Before you actually run commands, ESPECIALLY on an SVC where you can cause some serious damage to your SAN, prefix the command with an "echo" so you can see the command it would run without actually running the command!
  
  So, before you run "svctask rmvdisk $x" from inside a loop; first run "echo svctask rmvdisk $x" and verify that it's going to do what you think it's going to do. Much better to see that you messed something up on the screen than let your SVC try to remove every vdisk you've got assigned. Clients will not be happy nor will they be too understanding when you tell them you were using some ultra-fancy "bash miniscripts" to make working on the SVC more efficient.

Possible gotchas

---

Beyond the obvious gotchas of really screwing up the configuration of your SVC if you get your miniscripts wrong, there are a couple little gotchas to be aware of.

* Variables set inside loops are not available outside of loops.

found=0
svcinfo lsvdisk -nohdr | while read line
do
    [[ $line == *DS4800* ]] && (( ++found ))
done
echo "Found $found vdisks on the DS4800"

This will always report 0 vdisks, even if it does find vdisks, because the loop executes in a subshell, and thus any modification to the variable won't be available to the parent shell. A workaround is to use command substitution and catch the STDOUT of the loop:

found=$(svcinfo lsvdisk -nohdr | while read x; do [[ $x == *DS4800* ]] && echo -n X; done)
echo "Found ${#found} vdisks on the DS4800"

For each disk I find, I echo an "X". At the end of the loop, I've got stored in the variable "found" something which looks like "XXXXX" for five vdisks. Echoing "${#found}" gives me the length of the variable, and thus, the number of vdisks. Painful workaround? Yep.

This is a bash thing -- don't blame SVC. Korn shell (ksh) works just fine either way, but we're not running on the ksh, are we?

Multi-line entry

---

To make things more readable, you can enter your mini scripts on multiple lines:

svcinfo lsmdisk -delim : | while read line
do
    echo "$line"
done

(this miniscript doesn't really accomplish anything... just echoes what it reads, as if we ran "svcinfo lsmdisk | cat")

Bash will convert your entry to a one-liner (separating lines with ";" as appropriate) when it enters it into the history file, so the above will look like this in the history file:

svcinfo lsmdisk -delim: | while read line; do echo "$line"; done

Because of this, when I'm writing mini-scripts, I just edit them in the one-line format. Your option.

Shortening command lines

---

if/then with only one command can be simplified:

if [[ $x = "yes" ]]
then
    echo "yes"
fi

can be simplified to:

[[ $x = "yes" ]] && echo "yes"

if/then/else with only one command can be simplified:

if [[ $x == "yes" ]]
then
    echo "yes"
else
    echo "no"
fi

can be shortened to:

[[ $x == "yes" ]] && echo "yes" || echo "no"

Multiple commands can be run as well:

if [[ $x == "yes" ]]
then
    echo "yes"
    runcommand
fi

can be shortened to:

[[ $x == "yes" ]] && echo "yes" && runcommand

• caveat: The shortened version will not work exactly the same as the long version. In the short version, "runcommand" is only run if the first command, echo "yes" is successful. Odds are good that an echo isn't going to fail, but if "runcommand" was first and it failed, the next command doesn't run. In the "long" if/then format, both commands will be run, even if the first fails.
• Basically, unless you really understand how the "&&" and "||" dividers work, stick to single commands in your shortened if/then commands. Anyhow, the if/then syntax for multiple commands (on one line) is still pretty easy:
  
  if test; then command1; command2; fi

No "ls"? No problem!

---

SVC's restricted bash shell gives you very, VERY few commands. You can still get by without some of them using the bash builtins. Here's how we can simulate the "ls" command:

echo /dumps/*

This lists the contents of the /dumps/ directory, but in a pretty ugly format... all the files are listed separated by a space, and wrapped around lines. A little hard to read. Here's how we can have something more like "ls -F" format: ("/" appended to the name of directories)

for file in /dumps/*
do
    [[ -d $file ]] && echo "$file/" || echo "$file"
done

I don't know of any bash builtins that could give us the equivalent output of "ls -l", though. No way to get file size, permissions, or ownership. You can check if your own permissions as related to the file (can I read, write, execute) but can't see who owns it. That said, if the file is there, as admin you will have permissions to access it

No "grep"? No worries!

---

Wouldn't it be nice to "svcinfo lsvdisk | grep MyDisk"? Here's how: (basic form)

svcinfo lsvdisk -nohdr -delim : | while read line
do
    [[ $line = *match* ]] && echo $line
done

• The bash used in SVC is kinda old (version 2.05 on SVC v4.1) which is too bad. If it were a more modern version (v3.x) there are many REALLY powerful pattern matching capabilities available as builtins... including regular expressions!
• WUHU! SVC 4.2 has upgraded bash to version 3.x! This opens up some REAL possibilities for powerful pattern matching. See the bash doc for more info.

No "awk"? No concerns!

---

Since most SVC output is field specific, you can use modify the "grep" code above bash to split up the stuff that you're reading:

svcinfo lsmdiskgrp -nohdr -delim : | while IFS=: read id name status nummdisk numvdisk size extsize free
do
    [[ $name == *4500* ]] &&  echo "MDiskGroup $name (id $id) has $free available of $size"
done

Note: Using the ":" delimiter and setting IFS allows us to handle blank fields. F'rinstance, "lsvdisk" often has blanks for FC_id, FC_name, RC_id, and RC_name. (unless you've got Flash Copy on that vdisk) but the vdisk UID might be important to you. If you don't set IFS and leave "delim" unset, there will just be extra blank space, and the vdisk UID will be set to the "FC_id". Setting the IFS to the delimiter makes it set the blank fields to blank. IF you don't have to deal with blank fields (like say lsmdiskgrp) then you can leave the delim unset and just use a read: (you should still turn off the header so you don't process that!)

svcinfo lsmdiskgrp -nohdr | while read id name status nummdisk numvdisk size extsize free
do
   ...

No "sleep"? Now that's a problem.

---

If you wanted to run a command repeatedly, separated by a certain time delay, I haven't figured out how to do that yet. The "sleep" command, oddly enough, isn't a bash builtin. (even though it seems like it could be, very easily... and not to harp on ksh93, but it's a builtin there as well, and takes fractional delays, to go with ksh93's capability for floating-point math... oh, and it leaves variables set in loops available to the original shell...)

One option I considered was checking the first field of /proc/uptime in a loop, and exiting the loop when the value equals (( initialvalue + sleeptime )), but the problem there is that bash will check the contents of the file as often as it can, many times a second, creating a possibly significant load on the system, which isn't a good idea.

• Interactive workaround for no "sleep" command:

Instead of having the system delay for a certain period of time, you can have the system wait for your input, using the "read" command to read a line of input. You can then CTRL-C or put in a specific exit string:

while read keyb
do
    svc command goes here
    [[ $keyb == x ]] && break
done

This will execute the command every time you hit ENTER, and if you type a single "x", it will return you to a command prompt. Don't use "exit" instead of "break" or you'll be logged out when you type "x".

Building associative arrays

---

One of the biggest pains in the butt with bash is the variable scoping. If you want to build an associative array of, say, mDisk Name mapping to its state, you can't just set up a while loop with the output of lsmdisk piped into it. The reason is that the pipe spawns a subshell and the array that you're merrily building belongs to the subshell and so ceases to exist once you exit the loop

I worked out the following method.

1. Build an array of your keys, using command substitution
2. Build an array of your values, using command substitution
3. Iterate over your values and generate your associative array

eg

vdiskIds=(`svcinfo lsvdisk -nohdr | while read id rest; do echo -n "$id "; done`)
vdiskNames=(`svcinfo lsvdisk -nohdr | while read id name rest; do echo -n "$name "; done`)
vdiskNameMap=()
for (( i = 0 ; i < ${#vdiskNames[

} ; i++ ))do

 vdiskNameMap[${vdiskIds[$i]}]=${vdiskNames[$i]}

done @]

Because no subshell was spawned for the last loop, vdiskNameMap is avaiable for later. I use this in my extent summary script

Functions in bash

---

If you wanted to use the "grep" code above and actually call it "grep", you could do that by defining a function:

function grep { while read line; do [[ $line == *$1* ]] && echo "$line"; done }

• Since you can't edit your .bash_profile or .bashrc, every time you log into the SVC, you'd have to re-enter the function. This could be done via cut & paste, or if you wanted to get fancy, an expect script which logged in, defined your functions, and then gave you interactive control.

Since we don't have the capability to set aliases in restricted shell, you can use this if you want to get rid of the annoying need to prefix everything with "svcinfo" and "svctask":

function lsmdiskgrp { svcinfo lsmdiskgrp $*; }

"lsmdiskgrp" now works just like "svcinfo lsmdiskgrp", taking options (like "-nohdr") and arguments (like the name or id or an mdiskgrp) without having to type "svcinfo" first.

• bash can be finicky when you define a function and put commands in { } brackets. Usually bash isn't like perl in needing each line terminated with ";", but if you have a simple command like we have here, end the command with a ";" before the closing "}". Watch your whitespace, too.

A more powerful miniscript

---

An example. If you're running extents migrations, the progress information given by "svcinfo lsmigrate" is a little hard to read:

IBM_2145:TJUH_SVC:admin>svcinfo lsmigrate
migrate_type MDisk_Extents_Migration
progress 33
migrate_vdisk_index 13
migrate_source_mdisk_index 10
migrate_target_mdisk_index 1
number_extents 160
max_thread_count 1
migrate_type MDisk_Extents_Migration
progress 90
migrate_vdisk_index 12
migrate_source_mdisk_index 10
migrate_target_mdisk_index 1
number_extents 60
max_thread_count 1

...and so forth. That's pretty ugly. Wouldn't it be nice if we could see output in a format like:

Vdisk  13  52% of  160 extents
Vdisk   1  10% of  819 extents
Vdisk  10  37% of   64 extents
Vdisk   7  22% of   96 extents
Vdisk   4   0% of  194 extents

Much nicer! It's actually pretty easy. Here's the command line:

svcinfo lsmigrate | while read x y; do
    [[ $x = progress ]] && p=$y
    [[ $x = migrate_vdisk_index ]] && vdisk=$y
    [[ $x = number_extents ]] && printf "Vdisk %3d %3d%% of %4d extents\n" "$vdisk" "$p" "$y"
done

Normally I'll just create the whole command line with one line and semicolons separating commands, since it's easier to return to that line and edit.

In this instance, 0% means it hasn't started the migration... it can only run four threads at once, so a vdisk at 0% is waiting. Let's fancy it up some:

svcinfo lsmigrate | while read x y; do \
    [[ $x = progress ]] && p=$y \
    [[ $x = migrate_vdisk_index ]] && vdisk=$y \
    if [[ $x = number_extents ]]; then \
        if [[ $p = 0 ]]
        then
             echo "Vdisk $vdisk is waiting..."
        else 
             printf "Vdisk %3d %3d%% of %4d extents\n" "$vdisk" "${p}" "$y"
        fi; fi
done

That produces output like this:

Vdisk   1  35% of  819 extents
Vdisk   4  85% of  194 extents
Vdisk   5  82% of  160 extents
Vdisk   0   6% of   16 extents
Vdisk 14 is waiting...
Vdisk 17 is waiting...
Vdisk 32 is waiting...

...and so forth!

As you can see, the command lines can get a little complicated, but you can do some REALLY powerful stuff.

More advanced stuff

---

• A progress bar could be simulated like so:
  ->

  bar="########################################"       # That's 40 pound signs
  barlen=${#bar}                                       # Sets "barlen" to 40 - the length of "bar"
  space="                                        "     # 40 spaces
  blocks=$((p*barlen/100))                             # How many blocks for that percentage.
  echo ">${bar:0:$blocks}${space:0:$((barlen-blocks))<"

  
  For a value of p=70; output would look like this:
  ->>############################ <
  pmwiki isn't showing all the spaces properly here... there would be 28 pound signs followed by 12 spaces, keeping the whole width between the > < characters at 40 characters.

Bash links

---

• http://www.gnu.org/software/bash/
  GNU's page on bash.
• http://www.gnu.org/software/bash/manual/bashref.html
  GNU's bash reference manual