sas_devices: incorrect association of block devices to enclosure bays

[fernandolcx]

sas_devices is making incorrect associations of enclosure bays:

root@r620:~# sas_devices -v
Found 2 SAS hosts
Found 1 SAS expanders
Found 23 SAS end devices
Resolved 1 enclosure groups
*** Enclosure group: [sg25:DELL MD1240, addr: 0x500c04f2620ec03d]
      BAY       TYPE                     WWID      DEVICE_MAPPER   BLOCK_DEVS      SG_DEVS PATHS   VENDOR            MODEL        SERIAL_NUMBER      REV    SIZE     TARGET    STATE
        0       disk     naa.5000c50097f3596f       [Not mapped]          sdc          sg3     1  SEAGATE      ST600MP0005             S7M1B68Y     VE06 600.1GB   5:0:23:0  running
        1       disk     naa.5000cca07d4e1980       [Not mapped]          sdd          sg4     1  IBM-SSG          H0YH600             0BHBYHXH     CF08 600.1GB   5:0:24:0  running
        2       disk     naa.5000cca01325b9b4       [Not mapped]          sde          sg5     1  IBM-SSG          HSNF300             XYVNS5HA     A3A0 300.0GB   5:0:25:0  running
        3       disk     naa.5000c500349f40f7       [Not mapped]          sdf          sg6     1  IBM-SSG          S7AQ3P0 Z290Q54400009145WCJ4     A058   3.0TB   5:0:26:0  running
        4       disk     naa.5000c50034e5e373       [Not mapped]          sdg          sg7     1  IBM-SSG          S7AQ3P0 Z290HT0T00009203797U     A058   3.0TB   5:0:27:0  running
        6       disk     naa.5000c500981f84cf       [Not mapped]          sdh          sg8     1  SEAGATE      ST600MP0005             S7M1CB94     VE06 600.1GB   5:0:28:0  running
        7       disk     naa.5000c50053c2686f       [Not mapped]          sdi          sg9     1  SEAGATE      ST3600057SS             6SL458CN     EN03 600.1GB   5:0:29:0  running
        8       disk     naa.5000c5009838bb97       [Not mapped]          sdj         sg10     1  SEAGATE      ST600MP0005             S7M1CRRH     VE06 600.1GB   5:0:30:0  running
        9       disk     naa.5000c50035000cdb       [Not mapped]          sdn         sg13     1  IBM-SSG          S7AQ3P0 Z2912XN900009147Z45Z     A058   3.0TB   5:0:33:0  running
       10       disk     naa.5000cca03e7d5208       [Not mapped]          sdm         sg12     1  IBM-SSG          H7AQ3P0             YVJ6XKWD     C3E0   3.0TB   5:0:32:0  running
       11       disk     naa.5000cca02f66bcd8       [Not mapped]          sdl         sg11     1  IBM-SSG          H0YH600             03HUHKJZ     CF09 600.1GB   5:0:31:0  running
       12       disk     naa.5000c5004cfc0077       [Not mapped]          sdw         sg22     1  SEAGATE      ST3600057SS             6SL4475C     EN03 600.1GB   5:0:42:0  running
       13       disk     naa.5000cca03eb96cf0       [Not mapped]          sdv         sg21     1  IBM-SSG          H7AQ3P0             YVK8Z9DK     C3E0   3.0TB   5:0:41:0  running
       14       disk     naa.5000c50035016bc3       [Not mapped]          sdq         sg16     1  IBM-SSG          S7AQ3P0 Z2913DXB00009205A9YW     A058   3.0TB   5:0:36:0  running
       16       disk     naa.5000c5004cfc073b       [Not mapped]          sdp         sg15     1  SEAGATE      ST3600057SS             6SL447B4     EN03 600.1GB   5:0:35:0  running
       17       disk     naa.5000c50034596c1b       [Not mapped]          sdo         sg14     1  SEAGATE     ST32000444SS             9WM68BDV     KS68   2.0TB   5:0:34:0  running
       18       disk     naa.5000c500336d2697       [Not mapped]          sdt         sg19     1  SEAGATE      ST3450857SS 6SK08QZ70000M130BZLN     000B 450.1GB   5:0:39:0  running
       19       disk     naa.5000039478cadfec       [Not mapped]          sdu         sg20     1  TOSHIBA       MK2001TRKB         Z2K0A061FLD5     DCA8   2.0TB   5:0:40:0  running
       20       disk     naa.5000c5003500ebff       [Not mapped]          sdx         sg23     1  IBM-SSG          S7AQ3P0 Z2913AQR00009205NA9A     A058   3.0TB   5:0:43:0  running
       21       disk     naa.5000c50034e8ef53       [Not mapped]          sdz         sg24     1  IBM-SSG          S7AQ3P0 Z290WSXB00009149B9S5     A058   3.0TB   5:0:44:0  running
       22       disk     naa.5002538e30137db0       [Not mapped]          sdr         sg17     1      ATA  Samsung SSD 860      S599NJ0N111961B     4B6Q   1.0TB   5:0:37:0  running
       23       disk     naa.50014ee6b3773904       [Not mapped]          sds         sg18     1      ATA WDC WD10SPZX-75Z         WXP1A68K1F70     1A02   1.0TB   5:0:38:0  running
Total: 22 devices
*** Standalone devices
      BAY       TYPE                     WWID      SG_DEVS PATHS   VENDOR            MODEL      REV     TARGET    STATE
       36  enclosure     naa.500c04f2620ec000         sg25     1     DELL           MD1240     1.06   5:0:45:0  running
Total: 1 devices

This is the correct assignment:

root@r620:~# ledctl --list-slots --controller-type=scsi
slot: sg2-2           led state: NORMAL          device: (empty)
slot: sg25-0          led state: NORMAL          device: /dev/sdd
slot: sg25-1          led state: NORMAL          device: /dev/sdl
slot: sg25-2          led state: NORMAL          device: /dev/sdo
slot: sg25-3          led state: LOCATE          device: /dev/sds
slot: sg25-4          led state: NORMAL          device: /dev/sdc
slot: sg25-5          led state: NORMAL          device: /dev/sdj
slot: sg25-6          led state: NORMAL          device: /dev/sdp
slot: sg25-7          led state: NORMAL          device: /dev/sdt
slot: sg25-8          led state: NORMAL          device: (empty)
slot: sg25-9          led state: NORMAL          device: /dev/sdi
slot: sg25-10         led state: NORMAL          device: (empty)
slot: sg25-11         led state: NORMAL          device: /dev/sdu
slot: sg25-12         led state: NORMAL          device: /dev/sde
slot: sg25-13         led state: NORMAL          device: /dev/sdh
slot: sg25-14         led state: NORMAL          device: /dev/sdw
slot: sg25-15         led state: NORMAL          device: /dev/sdr
slot: sg25-16         led state: NORMAL          device: /dev/sdf
slot: sg25-17         led state: NORMAL          device: /dev/sdm
slot: sg25-18         led state: NORMAL          device: /dev/sdv
slot: sg25-19         led state: NORMAL          device: /dev/sdz
slot: sg25-20         led state: NORMAL          device: /dev/sdg
slot: sg25-21         led state: NORMAL          device: /dev/sdn
slot: sg25-22         led state: NORMAL          device: /dev/sdq
slot: sg25-23         led state: NORMAL          device: /dev/sdx
root@r620:~#

[thiell]

Likely in that case the kernel is not returning the correct bay_identifier:

You can pick a sg device from this list and get its end_device (for example end_device-9:7:6):

ls -l /sys/class/sas_device/end_device-*/device/target*/*/generic

and then get its bay identifier from the output of (replace X:Y:Z):

grep "" /sys/class/sas_device/end_device-*/bay_identifier | grep end_device-X:Y:Z

this should match what sas_devices returns.

[fernandolcx]

root@r620:/sys/class/sas_device# ls -l /sys/class/sas_device/end_device-*/device/target*/*/generic
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:0/device/target5:0:23/5:0:23:0/generic -> scsi_generic/sg3
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:10/device/target5:0:33/5:0:33:0/generic -> scsi_generic/sg13
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:11/device/target5:0:34/5:0:34:0/generic -> scsi_generic/sg14
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:12/device/target5:0:35/5:0:35:0/generic -> scsi_generic/sg15
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:13/device/target5:0:36/5:0:36:0/generic -> scsi_generic/sg16
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:14/device/target5:0:37/5:0:37:0/generic -> scsi_generic/sg17
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:15/device/target5:0:38/5:0:38:0/generic -> scsi_generic/sg18
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:16/device/target5:0:39/5:0:39:0/generic -> scsi_generic/sg19
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:17/device/target5:0:40/5:0:40:0/generic -> scsi_generic/sg20
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:18/device/target5:0:41/5:0:41:0/generic -> scsi_generic/sg21
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:19/device/target5:0:42/5:0:42:0/generic -> scsi_generic/sg22
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:1/device/target5:0:24/5:0:24:0/generic -> scsi_generic/sg4
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:20/device/target5:0:43/5:0:43:0/generic -> scsi_generic/sg23
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:21/device/target5:0:44/5:0:44:0/generic -> scsi_generic/sg24
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:22/device/target5:0:45/5:0:45:0/generic -> scsi_generic/sg25
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:2/device/target5:0:25/5:0:25:0/generic -> scsi_generic/sg5
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:3/device/target5:0:26/5:0:26:0/generic -> scsi_generic/sg6
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:4/device/target5:0:27/5:0:27:0/generic -> scsi_generic/sg7
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:5/device/target5:0:28/5:0:28:0/generic -> scsi_generic/sg8
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:6/device/target5:0:29/5:0:29:0/generic -> scsi_generic/sg9
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:7/device/target5:0:30/5:0:30:0/generic -> scsi_generic/sg10
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:8/device/target5:0:31/5:0:31:0/generic -> scsi_generic/sg11
lrwxrwxrwx 1 root root 0 Nov 22 21:08 /sys/class/sas_device/end_device-5:1:9/device/target5:0:32/5:0:32:0/generic -> scsi_generic/sg12
root@r620:/sys/class/sas_device#

root@r620:/sys/class/sas_device# grep "" /sys/class/sas_device/end_device-*/bay_identifier
/sys/class/sas_device/end_device-5:1:0/bay_identifier:0
/sys/class/sas_device/end_device-5:1:10/bay_identifier:9
/sys/class/sas_device/end_device-5:1:11/bay_identifier:17
/sys/class/sas_device/end_device-5:1:12/bay_identifier:16
/sys/class/sas_device/end_device-5:1:13/bay_identifier:14
/sys/class/sas_device/end_device-5:1:14/bay_identifier:22
/sys/class/sas_device/end_device-5:1:15/bay_identifier:23
/sys/class/sas_device/end_device-5:1:16/bay_identifier:18
/sys/class/sas_device/end_device-5:1:17/bay_identifier:19
/sys/class/sas_device/end_device-5:1:18/bay_identifier:13
/sys/class/sas_device/end_device-5:1:19/bay_identifier:12
/sys/class/sas_device/end_device-5:1:1/bay_identifier:1
/sys/class/sas_device/end_device-5:1:20/bay_identifier:20
/sys/class/sas_device/end_device-5:1:21/bay_identifier:21
/sys/class/sas_device/end_device-5:1:22/bay_identifier:36
/sys/class/sas_device/end_device-5:1:2/bay_identifier:2
/sys/class/sas_device/end_device-5:1:3/bay_identifier:3
/sys/class/sas_device/end_device-5:1:4/bay_identifier:4
/sys/class/sas_device/end_device-5:1:5/bay_identifier:6
/sys/class/sas_device/end_device-5:1:6/bay_identifier:7
/sys/class/sas_device/end_device-5:1:7/bay_identifier:8
/sys/class/sas_device/end_device-5:1:8/bay_identifier:11
/sys/class/sas_device/end_device-5:1:9/bay_identifier:10
root@r620:/sys/class/sas_device#

[fernandolcx]

Is there any workaround for this ?