Solaris Multipath Incorrect Totals
From time to time we notice that some LUN's are not optimal. It could be because of off-lining a LUN, changes on the switches I am not sure why exactly it happens. If multipath is not showing the correct Path Counts you may need to run cfgadm.
See how some LUN's here are showing 4 paths only. We expect 8.
# mpathadm list lu /dev/rdsk/c0t5000CCA04385ED60d0s2 Total Path Count: 1 Operational Path Count: 1 [..] /dev/rdsk/c0t600144F09D7311B500005605A40C0006d0s2 Total Path Count: 8 Operational Path Count: 8 /dev/rdsk/c0t600144F09D7311B50000561ED8AB0007d0s2 Total Path Count: 8 Operational Path Count: 8 [..] /dev/rdsk/c0t600144F09D7311B50000538507080021d0s2 Total Path Count: 8 Operational Path Count: 8 /dev/rdsk/c0t600144F09D7311B50000534309C40011d0s2 Total Path Count: 4 Operational Path Count: 4 /dev/rdsk/c0t600144F09D7311B500005342FE86000Fd0s2 Total Path Count: 4 Operational Path Count: 4 /dev/rdsk/c0t600144F09D7311B5000053D13E130029d0s2 Total Path Count: 4 Operational Path Count: 4 /dev/rdsk/c0t600144F09D7311B50000566AE1CC0008d0s2 Total Path Count: 4 Operational Path Count: 4
I tried a few things and it looks like the cfgadm worked. It could also be a couple other thing that triggered it like destroying an unused LUN or changing a recently added LUN's target group to be more restrictive but I doubt that is it. Most likely cfgadm.
# cfgadm -o show_SCSI_LUN -al Ap_Id Type Receptacle Occupant Condition c1 fc connected unconfigured unknown c8 fc-fabric connected configured unknown c8::20520002ac000f02,254 ESI connected configured unknown c8::21000024ff3db11d,0 disk connected configured unknown c8::21000024ff3db11d,1 disk connected configured unknown [..] c8::21000024ff57d646,54 disk connected configured unknown c8::21000024ff57d646,56 disk connected configured unknown c8::21520002ac000f02,254 ESI connected configured unknown c8::22520002ac000f02,254 ESI connected configured unknown c8::23520002ac000f02,254 ESI connected configured unknown c9 fc connected unconfigured unknown c13 fc-fabric connected configured unknown c13::20510002ac000f02,254 ESI connected configured unknown c13::21000024ff3db11c,0 disk connected configured unknown [..] c13::21000024ff3db11c,44 disk connected configured unknown c13::21000024ff3db11c,46 disk connected configured unknown c13::21000024ff3db11c,48 disk connected configured unknown c13::21000024ff3db11c,50 disk connected unconfigured unknown c13::21000024ff3db11c,52 disk connected unconfigured unknown c13::21000024ff3db11c,54 disk connected unconfigured unknown c13::21000024ff3db11c,56 disk connected configured unknown c13::21000024ff3db1b4,0 disk connected configured unknown c13::21000024ff3db1b4,1 disk connected configured unknown [..] c13::21510002ac000f02,254 ESI connected configured unknown c13::22510002ac000f02,254 ESI connected configured unknown c13::23510002ac000f02,254 ESI connected configured unknown
After I ran cfgadm..
# mpathadm list lu /dev/rdsk/c0t5000CCA04385ED60d0s2 Total Path Count: 1 Operational Path Count: 1 [..] /dev/rdsk/c0t600144F09D7311B500005605A40C0006d0s2 Total Path Count: 8 Operational Path Count: 8 /dev/rdsk/c0t600144F09D7311B50000561ED8AB0007d0s2 Total Path Count: 8 Operational Path Count: 8 [..] /dev/rdsk/c0t600144F09D7311B5000053BE90620024d0s2 Total Path Count: 8 Operational Path Count: 8 /dev/rdsk/c0t600144F09D7311B50000533C012A0009d0s2 Total Path Count: 8 Operational Path Count: 8 /dev/rdsk/c0t600144F09D7311B50000533AAFF00007d0s2 Total Path Count: 8 Operational Path Count: 8 /dev/rdsk/c0t600144F09D7311B50000538507080021d0s2 Total Path Count: 8 Operational Path Count: 8 /dev/rdsk/c0t600144F09D7311B50000534309C40011d0s2 Total Path Count: 8 Operational Path Count: 8 /dev/rdsk/c0t600144F09D7311B500005342FE86000Fd0s2 Total Path Count: 8 Operational Path Count: 8 /dev/rdsk/c0t600144F09D7311B5000053D13E130029d0s2 Total Path Count: 8 Operational Path Count: 8 /dev/rdsk/c0t600144F09D7311B50000566AE1CC0008d0s2 Total Path Count: 8 Operational Path Count: 8
I can also see the changes in messages...
# dmesg Nov 9 03:59:21 solaris11 mac: [ID 469746 kern.info] NOTICE: ldoms-vsw0.vport15 registered Nov 9 05:41:55 solaris11 scsi: [ID 583861 kern.info] ssd98 at scsi_vhci0: unit-address g600144f09d7311b50000534309c40011: f_tpgs [..] Dec 14 09:38:01 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b50000566ae1cc0008 (ssd101) multipath status: optimal: path 343 fp16/ssd@w21000024ff3db1b5,38 is standby Dec 14 09:38:01 solaris11 last message repeated 1 time Dec 14 09:38:01 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b5000053d13e130029 (ssd100) multipath status: optimal: path 344 fp16/ssd@w21000024ff3db1b5,36 is standby Dec 14 09:38:01 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b500005342fe86000f (ssd99) multipath status: optimal: path 345 fp16/ssd@w21000024ff3db1b5,34 is standby Dec 14 09:38:01 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b50000534309c40011 (ssd98) multipath status: optimal: path 346 fp16/ssd@w21000024ff3db1b5,32 is standby Dec 14 09:38:07 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b50000566ae1cc0008 (ssd101) multipath status: optimal: path 347 fp16/ssd@w21000024ff3db11d,38 is standby Dec 14 09:38:07 solaris11 last message repeated 1 time Dec 14 09:38:07 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b5000053d13e130029 (ssd100) multipath status: optimal: path 348 fp16/ssd@w21000024ff3db11d,36 is standby Dec 14 09:38:08 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b500005342fe86000f (ssd99) multipath status: optimal: path 349 fp16/ssd@w21000024ff3db11d,34 is standby Dec 14 09:38:08 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b50000534309c40011 (ssd98) multipath status: optimal: path 350 fp16/ssd@w21000024ff3db11d,32 is standby Dec 14 09:38:16 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b50000566ae1cc0008 (ssd101) multipath status: optimal: path 351 fp21/ssd@w21000024ff3db1b4,38 is standby Dec 14 09:38:16 solaris11 last message repeated 1 time Dec 14 09:38:17 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b5000053d13e130029 (ssd100) multipath status: optimal: path 352 fp21/ssd@w21000024ff3db1b4,36 is standby Dec 14 09:38:17 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b500005342fe86000f (ssd99) multipath status: optimal: path 353 fp21/ssd@w21000024ff3db1b4,34 is standby Dec 14 09:38:17 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b50000534309c40011 (ssd98) multipath status: optimal: path 354 fp21/ssd@w21000024ff3db1b4,32 is standby Dec 14 09:38:22 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b50000566ae1cc0008 (ssd101) multipath status: optimal: path 355 fp21/ssd@w21000024ff3db11c,38 is standby Dec 14 09:38:22 solaris11 last message repeated 1 time Dec 14 09:38:22 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b5000053d13e130029 (ssd100) multipath status: optimal: path 356 fp21/ssd@w21000024ff3db11c,36 is standby Dec 14 09:38:23 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b500005342fe86000f (ssd99) multipath status: optimal: path 357 fp21/ssd@w21000024ff3db11c,34 is standby Dec 14 09:38:23 solaris11 genunix: [ID 483743 kern.info] /scsi_vhci/ssd@g600144f09d7311b50000534309c40011 (ssd98) multipath status: optimal: path 358 fp21/ssd@w21000024ff3db11c,32 is standby