[Airship-discuss] RAID Config Implementation
Zainub Wahid
zainub.wahid at xflowresearch.com
Thu Jun 3 10:49:43 UTC 2021
Darren,
Thank you so much for your time and reply. These are very helpful and
important points you mentioned in your email.
For [1], we are already taking care of this. Exceptions will be thrown if
there is any problem with Disks.
For [2], it seems a very interesting point but according to my knowledge,
these are OS level configurations. We will look into it in future but right
now, it is beyond our scope.
Thank you for showing interest, once again.
Warm Regards,
Zainub Wahid
Network Design Engineer II,
xFlow Research Inc.
+923044506900
zainub.wahid at xflowresearch.com
www.xFlowResearch.com
On Tue, Jun 1, 2021 at 7:20 PM DEJAEGER, DARREN R <dd118r at att.com> wrote:
> Matt,
>
>
>
> I agree with your thoughts on making available both approaches. A couple
> things to consider:
>
>
>
> 1. Regarding disk failures: would these disks even be available to be
> selected for any sort of RAID if they’re bad? If not, we’d quickly hit an
> issue with a mismatched number of disks vs. what’s declared (at least
> today).
> 2. Can we allow other disk/raid-related config to be tuned? I’m
> thinking Stripe size, Read policy, Write policy?
>
>
>
> Just some food for thought.
>
>
>
> Regards,
>
> Darren
>
>
>
> *From: *"MCEUEN, MATT" <MM9745 at att.com>
> *Date: *Tuesday, June 1, 2021 at 10:04 AM
> *To: *Zainub Wahid <zainub.wahid at xflowresearch.com>, "
> airship-discuss at lists.airshipit.org" <airship-discuss at lists.airshipit.org>,
> "SPHICAS, PHIL G" <ps3910 at att.com>, "OBERFIELD, GREG" <go4243 at att.com>,
> "DEJAEGER, DARREN R" <dd118r at att.com>, "STEIN, JARED W" <js5175 at att.com>
> *Cc: *"Williams, J T" <J.T.Williams at dell.com>, Airship <
> airship at xflowresearch.com>, "richard_pioso at dell.com" <
> richard_pioso at dell.com>
> *Subject: *RE: [Airship-discuss] RAID Config Implementation
>
>
>
> Refreshing this in post-holiday inboxes: Phil, Craig, Greg, Darren, (&
> anyone else with experience with RAID config today) – please take a look at
> my thoughts below and let me know if that lines up with what you would find
> most useful as well. Thanks!
>
>
>
>
>
> Also, Merging Mahnoor’s comments with the email thread:
>
> (quote)
>
> Thank you for your email, Matt.
>
>
>
> About the Airship use cases:
>
> 1. The strongly declarative one would be easy to support via Metal3, to
> the best of my knowledge. This would be perfect as long as the Airship user
> is okay with finding out the disk and RAID controller names through other
> means (using iDRAC, for example). It would also be an easier and valuable
> implementation.
>
> 2. Metal3 will perform the disk selection itself (or delegate to Ironic).
>
>
>
> Providing some level of guarantee for the disk failures would contribute
> to a reliable user experience. For the declarative case, we can perhaps
> ensure that the specified disks are functional. For the specification with
> hints, I suggest making the disk capacity a metric that must always be
> satisfied (e.g. x GB always provided).
>
>
>
> Input on these requirements is much appreciated, so we can carry the
> discussion forward with the Metal3 community and get to implementation.
>
> Thank you so much!
>
> (end quote)
>
>
>
> *From:* MCEUEN, MATT
> *Sent:* Wednesday, May 26, 2021 4:17 PM
> *To:* Zainub Wahid <zainub.wahid at xflowresearch.com>;
> airship-discuss at lists.airshipit.org; SPHICAS, PHIL G <ps3910 at att.com>;
> OBERFIELD, GREG <go4243 at att.com>; DEJAEGER, DARREN R <dd118r at att.com>;
> STEIN, JARED W <js5175 at att.com>
> *Cc:* Williams, J T <J.T.Williams at dell.com>; Airship <
> airship at xflowresearch.com>; richard_pioso at dell.com
> *Subject:* RE: [Airship-discuss] RAID Config Implementation
>
>
>
> Thanks for sharing & summarizing this, Zainub.
>
>
>
> Phil, Craig, Greg, Darren, (& anyone else with experience with RAID config
> today) – here are my thoughts; please refine/agree/disagree with them so we
> can give the Metal3 community good feedback on the interface we need for
> declarative RAID config.
>
>
>
> First: long term, we want to be able to declaratively configure all the
> BIOS settings that need to be configured. Steps like this RAID config are
> a valuable step in that direction.
>
>
>
> Second: I think there are use cases for both of these setups, and I’d
> advocate for M3 facilitating both of them in the long run*:
>
> 1. Strongly declarative: I want a RAID-N array on ControllerX with
> disks [DiskY, DiskZ]
> 2. Declarative with hints: I want a RAID-N array with two SSDs (and I
> don’t care what disk or what controller – you figure it out, M3!)
>
> * Rook supports something similar – “use all disks:yes/no”, or alternately
> “use these specific disks:…”
>
>
>
> Third: I think that #1 is a good fit for our most common use cases, where
> we put a lot of work into a reference definition, and then stamp it out
> many times on *identical* hardware. This is also the simplest
> implementation and would probably get something usable into Airship’s hands
> fastest.
>
>
>
> Does that sound right to you all?
>
>
>
> The other thing to consider is thresholding when disks are bad. I.e. a
> config item that says something like, “tolerate disk failures as long as I
> get Xgb” or “as long as I get 3 disks” or something. Do you think this is
> critical enough to be needed on Day 1?
>
>
>
> Thanks!
>
> Matt
>
>
>
> *From:* Zainub Wahid <zainub.wahid at xflowresearch.com>
> *Sent:* Tuesday, May 25, 2021 11:50 AM
> *To:* airship-discuss at lists.airshipit.org
> *Cc:* Williams, J T <J.T.Williams at dell.com>; Airship <
> airship at xflowresearch.com>; richard_pioso at dell.com
> *Subject:* [Airship-discuss] RAID Config Implementation
>
>
>
> Hello everyone,
>
>
>
> As discussed in today's meeting, this is related to the demo represented
> by Noor, last year, related to performing RAID configurations in Airship
> via Metal3 (and Ironic).
> Add RAID to example hardwareprofile (I50c82ce5) · Gerrit Code Review (
> opendev.org
> <https://urldefense.com/v3/__http:/opendev.org__;!!BhdT!0FCxfaRXqTKY8FTua2QbHU5D5562OD_9g_BLbFq-DtlLGrsxsjRowGYyvEP2e5U$>
> )
>
> Through Airship, the user should be able to perform RAID configurations on
> their hardware. The user can specify the RAID controller names and their
> associated disks, along with the required RAID level, the desired number of
> disks and the amount of storage required.
> Here is the proposal for specifying the RAID configuration for a bare
> metal host via Metal3:
> https://github.com/metal3-io/metal3-docs/pull/148/files
> <https://urldefense.com/v3/__https:/github.com/metal3-io/metal3-docs/pull/148/files__;!!BhdT!0FCxfaRXqTKY8FTua2QbHU5D5562OD_9g_BLbFq-DtlLGrsxsjRowGYyGgo40Mo$>
>
> In terms of the CRD, the current implementation of the RAID has the
> following sections in it, for hardware RAID:
> spec:
> raid:
> hardwareRAIDVolumes:
> - sizeGibiBytes: 1000
> level: 1
> name: volume-name
> rotational: true
> numberOfPhysicalDisks: 2
> We propose to add fields into the CRD, in the following manner:
> spec:
> raid:
> hardwareRAIDVolumes:
> - sizeGibiBytes: 1000
> level: 1
> name: volume-1
> rotational: true
> numberOfPhysicalDisks: 2
> physicalDisks:
> - diskName: Disk-1
> - diskName: Disk-2
> controller: primary-controller
> - sizeGibiBytes: 2000
> level: 0
> name: volume-2
> rotational: false
> numberOfPhysicalDisks: 2
> physicalDisks:
> - diskName: Disk-3
> - diskName: Disk-4
> controller: secondary-controller-2
>
> However, it remains unclear how the user would have the in-depth knowledge
> of the RAID controllers available to them, along with the vendor-specific
> disk names.
> Example disk names (Dell):
> ['Disk.Bay.0:Enclosure.Internal.0-1:RAID.Intergrated.1-1',
> 'Disk.Bay.2:Enclosure.Internal.0-1:RAID.Integrated.1-1',
> 'Disk.Bay.3:Enclosure.Internal.0-1:RAID.Integrated.1-1',...]
> Example disk names (HP):
> ['2I:1:1', '2I:1:2', '2I:1:3',...]
>
> Example RAID Controllers (Dell):
> [‘Boss-F1 RAID Controller’, 'Dell PowerEdge RAID Controller H740P', ' Dell
> PowerEdge RAID Controller H830',...]
> Example RAID controller (HP):
> ['HP Smart Array P400 Controller', 'HPE Smart Array E208i-a Controller',
> 'HPE Smart Array P816i-a Controller',...]
>
> One possible solution is to let the user specify the required RAID level,
> number of disks and storage capacity needed. Metal3 could then validate
> this request using the Ironic drivers (e.g. Redfish driver, iDRAC driver
> etc.) in order to ensure the required hardware is available in the cluster.
> If it is available, the RAID configuration can be applied. Ifnot, the user
> can be informed about the error.
>
> Another solution is that Airship can perform inventory of the storage
> subsystems available in the cluster, and so the user can construct a RAID
> configuration accordingly. Conversely, the logic to select the physical
> disks and associated controllers may be implemented in Airship, whilst the
> user just specifies the high level requirements.
>
> One issue is that Redfish requires the explicit RAID controller and disk
> names in order to do the RAID configuration, but it would be more
> convenient if the user did not have to know the low level details of their
> system.
> Here is the Redfish Action URI: {Base URI of target
> resource}/Actions/Volume.ChangeRAIDLayout
> (Provided in 6.109.4.2 of Redfish Resource and Schema Guide (dmtf.org
> <https://urldefense.com/v3/__http:/dmtf.org__;!!BhdT!0FCxfaRXqTKY8FTua2QbHU5D5562OD_9g_BLbFq-DtlLGrsxsjRowGYys3w9hc0$>
> ))
>
> The disk names can be generated using the Redfish logic constructing a
> human friendly identifier or label for hardware devices.
> https://github.com/DMTF/Redfish-Tacklebox/blob/e429f70a79cfe288756618498ce485ab4be37131/redfish_utilities/inventory.py#L192
> <https://urldefense.com/v3/__https:/github.com/DMTF/Redfish-Tacklebox/blob/e429f70a79cfe288756618498ce485ab4be37131/redfish_utilities/inventory.py*L192__;Iw!!BhdT!0FCxfaRXqTKY8FTua2QbHU5D5562OD_9g_BLbFq-DtlLGrsxsjRowGYyuJLYIII$>
>
> Disk selection logic needs to be implemented in the most appropriate
> place. Airship may require that the user be unaware of the storage
> subsystem details when requesting the RAID config. Ironic requires a vendor
> independent implementation for this purpose.
>
> Here is relevant work done in metal3 to support RAID configuration in
> baremetal servers, using the Fujitsu iRMC driver:
> https://github.com/metal3-io/baremetal-operator/issues/206
> <https://urldefense.com/v3/__https:/github.com/metal3-io/baremetal-operator/issues/206__;!!BhdT!0FCxfaRXqTKY8FTua2QbHU5D5562OD_9g_BLbFq-DtlLGrsxsjRowGYyAS9JYzU$>
> https://github.com/metal3-io/baremetal-operator/pull/292
> <https://urldefense.com/v3/__https:/github.com/metal3-io/baremetal-operator/pull/292__;!!BhdT!0FCxfaRXqTKY8FTua2QbHU5D5562OD_9g_BLbFq-DtlLGrsxsjRowGYym4_G3ew$>
>
>
>
>
>
> Warm Regards,
>
>
>
> Zainub Wahid
>
> Network Design Engineer II,
>
> xFlow Research Inc.
>
> +923044506900
>
> zainub.wahid at xflowresearch.com
>
> www.xFlowResearch.com
> <https://urldefense.com/v3/__http:/www.xFlowResearch.com__;!!BhdT!0FCxfaRXqTKY8FTua2QbHU5D5562OD_9g_BLbFq-DtlLGrsxsjRowGYyG48Ww0I$>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.airshipit.org/pipermail/airship-discuss/attachments/20210603/c83a477f/attachment-0001.html>
More information about the Airship-discuss
mailing list