Riaan's SysAdmin Blog

My tips, howtos, gotchas, snippets and stuff. Use at your own risk!

Solaris

Solaris Information On WWN

Recently I had some messages on a T5-2 hypervisor and I needed to find out exactly what it is complaining about.

Messages looked like this:

fctl: [ID 517869 kern.warning] WARNING: fp(5)::GPN_ID for D_ID=160003 failed
fctl: [ID 517869 kern.warning] WARNING: fp(5)::N_x Port with D_ID=160003, PWWN=23220002ac0012b4 disappeared from fabric
fctl: [ID 517869 kern.warning] WARNING: fp(6)::GPN_ID for D_ID=c0001 failed
fctl: [ID 517869 kern.warning] WARNING: fp(6)::N_x Port with D_ID=c0001, PWWN=23210002ac0012b4 disappeared from fabric
fctl: [ID 517869 kern.warning] WARNING: fp(6)::N_x Port with D_ID=c0001, PWWN=23210002ac0012b4 reappeared in fabric
fctl: [ID 517869 kern.warning] WARNING: fp(5)::N_x Port with D_ID=160003, PWWN=23220002ac0012b4 reappeared in fabric

First let's check the adapters and ports.

# fcinfo hba-port
HBA Port WWN: 21000024ff4e2a9c
        Port Mode: Initiator
        Port ID: 210600
        OS Device Name: /dev/cfg/c4
        Manufacturer: QLogic Corp.
        Model: 371-4325-02
        Firmware Version: 5.08.00
        FCode/BIOS Version:  BIOS: 2.02; fcode: 2.03; EFI: 2.01;
        Serial Number: 0402T00-1315130088
        Driver Name: qlc
        Driver Version: 20131114-4.03
        Type: N-port
        State: online
        Supported Speeds: 2Gb 4Gb 8Gb
        Current Speed: 8Gb
        Node WWN: 20000024ff4e2a9c
        Max NPIV Ports: 255
        NPIV port list:
HBA Port WWN: 21000024ff4e2a9d
        Port Mode: Initiator
        Port ID: 0
        OS Device Name: /dev/cfg/c5
        Manufacturer: QLogic Corp.
        Model: 371-4325-02
        Firmware Version: 5.08.00
        FCode/BIOS Version:  BIOS: 2.02; fcode: 2.03; EFI: 2.01;
        Serial Number: 0402T00-1315130088
        Driver Name: qlc
        Driver Version: 20131114-4.03
        Type: unknown
        State: offline
        Supported Speeds: 2Gb 4Gb 8Gb
        Current Speed: not established
        Node WWN: 20000024ff4e2a9d
        Max NPIV Ports: 255
        NPIV port list:
HBA Port WWN: 21000024ff4e29d6
        Port Mode: Initiator
        Port ID: 660700
        OS Device Name: /dev/cfg/c6
        Manufacturer: QLogic Corp.
        Model: 371-4325-02
        Firmware Version: 5.08.00
        FCode/BIOS Version:  BIOS: 2.02; fcode: 2.03; EFI: 2.01;
        Serial Number: 0402T00-1315129988
        Driver Name: qlc
        Driver Version: 20131114-4.03
        Type: N-port
        State: online
        Supported Speeds: 2Gb 4Gb 8Gb
        Current Speed: 8Gb
        Node WWN: 20000024ff4e29d6
        Max NPIV Ports: 255
        NPIV port list:
HBA Port WWN: 21000024ff4e29d7
        Port Mode: Initiator
        Port ID: 0
        OS Device Name: /dev/cfg/c7
        Manufacturer: QLogic Corp.
        Model: 371-4325-02
        Firmware Version: 5.08.00
        FCode/BIOS Version:  BIOS: 2.02; fcode: 2.03; EFI: 2.01;
        Serial Number: 0402T00-1315129988
        Driver Name: qlc
        Driver Version: 20131114-4.03
        Type: unknown
        State: offline
        Supported Speeds: 2Gb 4Gb 8Gb
        Current Speed: not established
        Node WWN: 20000024ff4e29d7
        Max NPIV Ports: 255
        NPIV port list:

Let's check which ports are connected.

# luxadm -e port
/devices/pci@340/pci@1/pci@0/pci@4/SUNW,qlc@0/fp@0,0:devctl        CONNECTED
/devices/pci@340/pci@1/pci@0/pci@5/SUNW,qlc@0/fp@0,0:devctl        CONNECTED
/devices/pci@340/pci@1/pci@0/pci@4/SUNW,qlc@0,1/fp@0,0:devctl      NOT CONNECTED
/devices/pci@340/pci@1/pci@0/pci@5/SUNW,qlc@0,1/fp@0,0:devctl      NOT CONNECTED

Now let's find devices on the above two connected ports. You should be able to see the original warning WWN's listed here.

# luxadm -e dump_map /devices/pci@340/pci@1/pci@0/pci@4/SUNW,qlc@0/fp@0,0:devctl
Pos  Port_ID Hard_Addr Port WWN         Node WWN         Type
0    160000  0         20220002ac0012b4 2ff70002ac0012b4 0x0  (Disk device)
1    160001  0         21220002ac0012b4 2ff70002ac0012b4 0x0  (Disk device)
2    160002  0         22220002ac0012b4 2ff70002ac0012b4 0x0  (Disk device)
3    160003  0         23220002ac0012b4 2ff70002ac0012b4 0x0  (Disk device)	<-- Messages show this one
4    210900  0         21000024ff57d60d 20000024ff57d60d 0x0  (Disk device)
5    210a00  0         21000024ff57d64d 20000024ff57d64d 0x0  (Disk device)
6    210b00  0         21000024ff57d649 20000024ff57d649 0x0  (Disk device)
7    210c00  0         21000024ff57d60b 20000024ff57d60b 0x0  (Disk device)
8    210600  0         21000024ff4e2a9c 20000024ff4e2a9c 0x1f (Unknown Type,Host Bus Adapter)

# luxadm -e dump_map /devices/pci@340/pci@1/pci@0/pci@5/SUNW,qlc@0/fp@0,0:devctl
Pos  Port_ID Hard_Addr Port WWN         Node WWN         Type
0    c0001   0         23210002ac0012b4 2ff70002ac0012b4 0x0  (Disk device)	<-- Messages show this one
1    c0002   0         22210002ac0012b4 2ff70002ac0012b4 0x0  (Disk device)
2    c0003   0         21210002ac0012b4 2ff70002ac0012b4 0x0  (Disk device)
3    c0004   0         20210002ac0012b4 2ff70002ac0012b4 0x0  (Disk device)
4    660000  0         21000024ff57d64c 20000024ff57d64c 0x0  (Disk device)
5    660100  0         21000024ff57d60a 20000024ff57d60a 0x0  (Disk device)
6    660200  0         21000024ff57d648 20000024ff57d648 0x0  (Disk device)
7    660a00  0         21000024ff57d60c 20000024ff57d60c 0x0  (Disk device)
8    660700  0         21000024ff4e29d6 20000024ff4e29d6 0x1f (Unknown Type,Host Bus Adapter)

You can try and see if the device is listed here. In my case it was not so it appears that we are not using this device but maybe the switch still have us zoned for the device.

# luxadm probe
No Network Array enclosures found in /dev/es

Found Fibre Channel device(s):
  Node WWN:20000024ff57d64d  Device Type:Disk device
    Logical Path:/dev/rdsk/c0t600144F086479F15000053DA5A03000Ad0s2
[..]

# luxadm probe | grep 2ac0012b4  <-- Not found so remote?

We probably did not need all steps above to get to the next command but I am listing everything since it depends on the situation and what you are tracing.

Here I can see the relevant WWN is a 3PAR SAN and in my case I knew I am not using 3PAR any longer so I can check if this is a zoning issue.

# fcinfo remote-port -ls -p 21000024ff4e29d6
Remote Port WWN: 20210002ac0012b4
        Active FC4 Types: SCSI
        SCSI Target: yes
        Port Symbolic Name: 1404788 - 0:2:1 - LPe12004
        Node WWN: 2ff70002ac0012b4
        Link Error Statistics:
                Link Failure Count: 0
                Loss of Sync Count: 12
                Loss of Signal Count: 0
                Primitive Seq Protocol Error Count: 0
                Invalid Tx Word Count: 48
                Invalid CRC Count: 0
        LUN: 254
          Vendor: 3PARdata
          Product: SES
          OS Device Name: /dev/es/ses0
Remote Port WWN: 21210002ac0012b4
        Active FC4 Types: SCSI
        SCSI Target: yes
        Port Symbolic Name: 1404788 - 1:2:1 - LPe12004
        Node WWN: 2ff70002ac0012b4
        Link Error Statistics:
                Link Failure Count: 0
                Loss of Sync Count: 6
                Loss of Signal Count: 0
                Primitive Seq Protocol Error Count: 0
                Invalid Tx Word Count: 32
                Invalid CRC Count: 0
        LUN: 254
          Vendor: 3PARdata
          Product: SES
          OS Device Name: /dev/es/ses1
Remote Port WWN: 22210002ac0012b4
        Active FC4 Types: SCSI
        SCSI Target: yes
        Port Symbolic Name: 1404788 - 2:2:1 - LPe12004
        Node WWN: 2ff70002ac0012b4
        Link Error Statistics:
                Link Failure Count: 0
                Loss of Sync Count: 0
                Loss of Signal Count: 0
                Primitive Seq Protocol Error Count: 0
                Invalid Tx Word Count: 16
                Invalid CRC Count: 0
        LUN: 254
          Vendor: 3PARdata
          Product: SES
          OS Device Name: /dev/es/ses2
Remote Port WWN: 23210002ac0012b4
        Active FC4 Types: SCSI
        SCSI Target: yes
        Port Symbolic Name: 1404788 - 3:2:1 - LPe12004
        Node WWN: 2ff70002ac0012b4
        Link Error Statistics:
                Link Failure Count: 0
                Loss of Sync Count: 0
                Loss of Signal Count: 0
                Primitive Seq Protocol Error Count: 0
                Invalid Tx Word Count: 16
                Invalid CRC Count: 0
        LUN: 254
          Vendor: 3PARdata
          Product: SES
          OS Device Name: /dev/es/ses3
[..]

# fcinfo remote-port -ls -p 21000024ff4e2a9c
Remote Port WWN: 20220002ac0012b4
        Active FC4 Types: SCSI
        SCSI Target: yes
        Port Symbolic Name: 1404788 - 0:2:2 - LPe12004
        Node WWN: 2ff70002ac0012b4
        Link Error Statistics:
                Link Failure Count: 0
                Loss of Sync Count: 6
                Loss of Signal Count: 0
                Primitive Seq Protocol Error Count: 0
                Invalid Tx Word Count: 32
                Invalid CRC Count: 0
        LUN: 254
          Vendor: 3PARdata
          Product: SES
          OS Device Name: /dev/es/ses4
Remote Port WWN: 21220002ac0012b4
        Active FC4 Types: SCSI
        SCSI Target: yes
        Port Symbolic Name: 1404788 - 1:2:2 - LPe12004
        Node WWN: 2ff70002ac0012b4
        Link Error Statistics:
                Link Failure Count: 0
                Loss of Sync Count: 0
                Loss of Signal Count: 0
                Primitive Seq Protocol Error Count: 0
                Invalid Tx Word Count: 16
                Invalid CRC Count: 0
        LUN: 254
          Vendor: 3PARdata
          Product: SES
          OS Device Name: /dev/es/ses5
Remote Port WWN: 22220002ac0012b4
        Active FC4 Types: SCSI
        SCSI Target: yes
        Port Symbolic Name: 1404788 - 2:2:2 - LPe12004
        Node WWN: 2ff70002ac0012b4
        Link Error Statistics:
                Link Failure Count: 0
                Loss of Sync Count: 0
                Loss of Signal Count: 0
                Primitive Seq Protocol Error Count: 0
                Invalid Tx Word Count: 16
                Invalid CRC Count: 0
        LUN: 254
          Vendor: 3PARdata
          Product: SES
          OS Device Name: /dev/es/ses6
Remote Port WWN: 23220002ac0012b4
        Active FC4 Types: SCSI
        SCSI Target: yes
        Port Symbolic Name: 1404788 - 3:2:2 - LPe12004
        Node WWN: 2ff70002ac0012b4
        Link Error Statistics:
                Link Failure Count: 0
                Loss of Sync Count: 0
                Loss of Signal Count: 0
                Primitive Seq Protocol Error Count: 0
                Invalid Tx Word Count: 16
                Invalid CRC Count: 0
        LUN: 254
          Vendor: 3PARdata
          Product: SES
          OS Device Name: /dev/es/ses7

admin

Bio Info for Riaan