Riaan's SysAdmin Blog

My tips, howtos, gotchas, snippets and stuff. Use at your own risk!

Solaris

Solaris Multipath Incorrect Totals

admin

From time to time we notice that some LUN's are not optimal. It could be because of off-lining a LUN, changes on the switches I am not sure why exactly it happens. If multipath is not showing the correct Path Counts you may need to run cfgadm.

See how some LUN's here are showing 4 paths only. We expect 8.

# mpathadm list lu
        /dev/rdsk/c0t5000CCA04385ED60d0s2
                Total Path Count: 1
                Operational Path Count: 1
[..]
        /dev/rdsk/c0t600144F09D7311B500005605A40C0006d0s2
                Total Path Count: 8
                Operational Path Count: 8
        /dev/rdsk/c0t600144F09D7311B50000561ED8AB0007d0s2
                Total Path Count: 8
                Operational Path Count: 8
[..]
        /dev/rdsk/c0t600144F09D7311B50000538507080021d0s2
                Total Path Count: 8
                Operational Path Count: 8
        /dev/rdsk/c0t600144F09D7311B50000534309C40011d0s2
                Total Path Count: 4
                Operational Path Count: 4
        /dev/rdsk/c0t600144F09D7311B500005342FE86000Fd0s2
                Total Path Count: 4
                Operational Path Count: 4
        /dev/rdsk/c0t600144F09D7311B5000053D13E130029d0s2
                Total Path Count: 4
                Operational Path Count: 4
        /dev/rdsk/c0t600144F09D7311B50000566AE1CC0008d0s2
                Total Path Count: 4
                Operational Path Count: 4

I tried a few things and it looks like the cfgadm worked. It could also be a couple other thing that triggered it like destroying an unused LUN or changing a recently added LUN's target group to be more restrictive but I doubt that is it. Most likely cfgadm.

# cfgadm -o show_SCSI_LUN -al
Ap_Id                          Type         Receptacle   Occupant     Condition
c1                             fc           connected    unconfigured unknown
c8                             fc-fabric    connected    configured   unknown
c8::20520002ac000f02,254       ESI          connected    configured   unknown
c8::21000024ff3db11d,0         disk         connected    configured   unknown
c8::21000024ff3db11d,1         disk         connected    configured   unknown
[..]
c8::21000024ff57d646,54        disk         connected    configured   unknown
c8::21000024ff57d646,56        disk         connected    configured   unknown
c8::21520002ac000f02,254       ESI          connected    configured   unknown
c8::22520002ac000f02,254       ESI          connected    configured   unknown
c8::23520002ac000f02,254       ESI          connected    configured   unknown
c9                             fc           connected    unconfigured unknown
c13                            fc-fabric    connected    configured   unknown
c13::20510002ac000f02,254      ESI          connected    configured   unknown
c13::21000024ff3db11c,0        disk         connected    configured   unknown
[..]
c13::21000024ff3db11c,44       disk         connected    configured   unknown
c13::21000024ff3db11c,46       disk         connected    configured   unknown
c13::21000024ff3db11c,48       disk         connected    configured   unknown
c13::21000024ff3db11c,50       disk         connected    unconfigured unknown
c13::21000024ff3db11c,52       disk         connected    unconfigured unknown
c13::21000024ff3db11c,54       disk         connected    unconfigured unknown
c13::21000024ff3db11c,56       disk         connected    configured   unknown
c13::21000024ff3db1b4,0        disk         connected    configured   unknown
c13::21000024ff3db1b4,1        disk         connected    configured   unknown
[..]
c13::21510002ac000f02,254      ESI          connected    configured   unknown
c13::22510002ac000f02,254      ESI          connected    configured   unknown
c13::23510002ac000f02,254      ESI          connected    configured   unknown

After I ran cfgadm..

# mpathadm list lu
        /dev/rdsk/c0t5000CCA04385ED60d0s2
                Total Path Count: 1
                Operational Path Count: 1
[..]
        /dev/rdsk/c0t600144F09D7311B500005605A40C0006d0s2
                Total Path Count: 8
                Operational Path Count: 8
        /dev/rdsk/c0t600144F09D7311B50000561ED8AB0007d0s2
                Total Path Count: 8
                Operational Path Count: 8
[..]
        /dev/rdsk/c0t600144F09D7311B5000053BE90620024d0s2
                Total Path Count: 8
                Operational Path Count: 8
        /dev/rdsk/c0t600144F09D7311B50000533C012A0009d0s2
                Total Path Count: 8
                Operational Path Count: 8
        /dev/rdsk/c0t600144F09D7311B50000533AAFF00007d0s2
                Total Path Count: 8
                Operational Path Count: 8
        /dev/rdsk/c0t600144F09D7311B50000538507080021d0s2
                Total Path Count: 8
                Operational Path Count: 8
        /dev/rdsk/c0t600144F09D7311B50000534309C40011d0s2
                Total Path Count: 8
                Operational Path Count: 8
        /dev/rdsk/c0t600144F09D7311B500005342FE86000Fd0s2
                Total Path Count: 8
                Operational Path Count: 8
        /dev/rdsk/c0t600144F09D7311B5000053D13E130029d0s2
                Total Path Count: 8
                Operational Path Count: 8
        /dev/rdsk/c0t600144F09D7311B50000566AE1CC0008d0s2
                Total Path Count: 8
                Operational Path Count: 8

I can also see the changes in messages...

# dmesg
Nov  9 03:59:21 solaris11 mac: [ID 469746 kern.info] NOTICE: ldoms-vsw0.vport15 registered
Nov  9 05:41:55 solaris11 scsi: [ID 583861 kern.info] ssd98 at scsi_vhci0: unit-address g600144f09d7311b50000534309c40011: f_tpgs
[..]
Dec 14 09:38:01 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b50000566ae1cc0008 (ssd101) multipath status: optimal: path 343 fp16/ssd@w21000024ff3db1b5,38 is standby
Dec 14 09:38:01 solaris11 last message repeated 1 time
Dec 14 09:38:01 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b5000053d13e130029 (ssd100) multipath status: optimal: path 344 fp16/ssd@w21000024ff3db1b5,36 is standby
Dec 14 09:38:01 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b500005342fe86000f (ssd99) multipath status: optimal: path 345 fp16/ssd@w21000024ff3db1b5,34 is standby
Dec 14 09:38:01 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b50000534309c40011 (ssd98) multipath status: optimal: path 346 fp16/ssd@w21000024ff3db1b5,32 is standby
Dec 14 09:38:07 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b50000566ae1cc0008 (ssd101) multipath status: optimal: path 347 fp16/ssd@w21000024ff3db11d,38 is standby
Dec 14 09:38:07 solaris11 last message repeated 1 time
Dec 14 09:38:07 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b5000053d13e130029 (ssd100) multipath status: optimal: path 348 fp16/ssd@w21000024ff3db11d,36 is standby
Dec 14 09:38:08 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b500005342fe86000f (ssd99) multipath status: optimal: path 349 fp16/ssd@w21000024ff3db11d,34 is standby
Dec 14 09:38:08 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b50000534309c40011 (ssd98) multipath status: optimal: path 350 fp16/ssd@w21000024ff3db11d,32 is standby
Dec 14 09:38:16 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b50000566ae1cc0008 (ssd101) multipath status: optimal: path 351 fp21/ssd@w21000024ff3db1b4,38 is standby
Dec 14 09:38:16 solaris11 last message repeated 1 time
Dec 14 09:38:17 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b5000053d13e130029 (ssd100) multipath status: optimal: path 352 fp21/ssd@w21000024ff3db1b4,36 is standby
Dec 14 09:38:17 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b500005342fe86000f (ssd99) multipath status: optimal: path 353 fp21/ssd@w21000024ff3db1b4,34 is standby
Dec 14 09:38:17 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b50000534309c40011 (ssd98) multipath status: optimal: path 354 fp21/ssd@w21000024ff3db1b4,32 is standby
Dec 14 09:38:22 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b50000566ae1cc0008 (ssd101) multipath status: optimal: path 355 fp21/ssd@w21000024ff3db11c,38 is standby
Dec 14 09:38:22 solaris11 last message repeated 1 time
Dec 14 09:38:22 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b5000053d13e130029 (ssd100) multipath status: optimal: path 356 fp21/ssd@w21000024ff3db11c,36 is standby
Dec 14 09:38:23 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b500005342fe86000f (ssd99) multipath status: optimal: path 357 fp21/ssd@w21000024ff3db11c,34 is standby
Dec 14 09:38:23 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b50000534309c40011 (ssd98) multipath status: optimal: path 358 fp21/ssd@w21000024ff3db11c,32 is standby

admin

Bio Info for Riaan