<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
        {font-family:SimSun;
        panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
        {font-family:"Cambria Math";
        panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
        {font-family:"Calibri Light";
        panose-1:2 15 3 2 2 2 4 3 2 4;}
@font-face
        {font-family:Calibri;
        panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
        {font-family:"Segoe UI";
        panose-1:2 11 5 2 4 2 4 2 2 3;}
@font-face
        {font-family:Tahoma;
        panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
        {font-family:Consolas;
        panose-1:2 11 6 9 2 2 4 3 2 4;}
@font-face
        {font-family:"Segoe Script";
        panose-1:3 11 5 4 2 0 0 0 0 3;}
@font-face
        {font-family:"\@SimSun";
        panose-1:2 1 6 0 3 1 1 1 1 1;}
@font-face
        {font-family:"Monotype Corsiva";
        panose-1:3 1 1 1 1 2 1 1 1 1;}
@font-face
        {font-family:DengXian;
        panose-1:0 0 0 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
        {margin:0in;
        margin-bottom:.0001pt;
        font-size:12.0pt;
        font-family:"Times New Roman",serif;}
h1
        {mso-style-priority:9;
        mso-style-link:"Heading 1 Char";
        mso-margin-top-alt:auto;
        margin-right:0in;
        mso-margin-bottom-alt:auto;
        margin-left:0in;
        font-size:24.0pt;
        font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
        {mso-style-priority:99;
        color:blue;
        text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
        {mso-style-priority:99;
        color:purple;
        text-decoration:underline;}
span.Heading1Char
        {mso-style-name:"Heading 1 Char";
        mso-style-priority:9;
        mso-style-link:"Heading 1";
        font-family:"Calibri Light",sans-serif;
        color:#2E74B5;}
p.msonormal0, li.msonormal0, div.msonormal0
        {mso-style-name:msonormal;
        mso-margin-top-alt:auto;
        margin-right:0in;
        mso-margin-bottom-alt:auto;
        margin-left:0in;
        font-size:12.0pt;
        font-family:SimSun;}
span.EmailStyle19
        {mso-style-type:personal;
        font-family:"Calibri",sans-serif;
        color:#1F497D;}
span.EmailStyle20
        {mso-style-type:personal;
        font-family:"DengXian",serif;
        color:windowtext;}
span.EmailStyle21
        {mso-style-type:personal-reply;
        font-family:"Calibri",sans-serif;
        color:#1F497D;}
.MsoChpDefault
        {mso-style-type:export-only;
        font-size:10.0pt;}
@page WordSection1
        {size:8.5in 11.0in;
        margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
        {page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Hey!
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Have a look if ceph is healthy in general with following command (cluster status)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">kubectl exec –it –n ceph ceph-mon-h2gsm -- ceph -s<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I still see that you have 2 instances of ceph-mgr from your terminal outputs<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-mgr-6dc44fc75b-jtv7w                                   0/1       Pending     0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-mgr-6dc44fc75b-jvstm                                   1/1       Running     0          22h</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">There are 2 ceph-client charts, 1 for initial single node/genesis, and ceph-client-update for final state (when additional nodes join). Feel free to explore
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><a href="https://github.com/openstack/airship-treasuremap/tree/master/global/software/charts/ucp/ceph">https://github.com/openstack/airship-treasuremap/tree/master/global/software/charts/ucp/ceph</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I think simplest way for you to get ceph running (if you are using single control plane) would be to just stay with initial ceph as it will also keep crush rules
 and other settings as needed (final state requires 3 domains/hosts)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I would say change
<a href="https://github.com/openstack/airship-treasuremap/blob/master/global/software/manifests/full-site.yaml#L22">
https://github.com/openstack/airship-treasuremap/blob/master/global/software/manifests/full-site.yaml#L22</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">From  `</span><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">ucp-ceph-update</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">`
 to just `</span><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D">ucp-ceph</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">` instead; this will keep using initial single node/genesis ceph and keep it healthy.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">For RGW – is similar – you will need to set it to 1 replica (right now there is no site override so you would need to craft one or set on global level)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><a href="https://github.com/openstack/airship-treasuremap/blob/master/global/software/charts/ucp/ceph/ceph-rgw.yaml#L133">https://github.com/openstack/airship-treasuremap/blob/master/global/software/charts/ucp/ceph/ceph-rgw.yaml#L133</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Kindly, Kaspars<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Tu, Qiaolin (NSB - CN/Hangzhou) [mailto:qiaolin.tu@nokia-sbell.com]
<br>
<b>Sent:</b> Thursday, November 8, 2018 1:25 AM<br>
<b>To:</b> SKELS, KASPARS <ks3019@att.com>; Li, Maxwell (NSB - CN/Hangzhou) <maxwell.li@nokia-sbell.com><br>
<b>Cc:</b> GORSHUNOV, ROMAN <roman.gorshunov@att.com>; MEADOWS, ALAN <am240k@att.com>; MCEUEN, MATT <MM9745@att.com>; PACHECO, RODOLFO J <rp2723@att.com>; 'airship-discuss@lists.airshipit.org' <airship-discuss@lists.airshipit.org><br>
<b>Subject:</b> RE: Airship installation Questions<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Hi,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Thanks very much for your reply.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">I adjust the replica count on site level (set 1, instead of 2) and<b><span style="color:red"> ceph-rgw pod</span></b> still have
<b><span style="color:red">getsockopt: connection refused error</span></b>. Below is the log detail:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN">root@cab23-r720-11:~# kubectl describe pod ceph-rgw-5b6645c456-rsv4k -n ceph<o:p></o:p></span></b></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Name:           ceph-rgw-5b6645c456-rsv4k<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Namespace:      ceph<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Node:           cab23-r720-11/10.23.22.11<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Start Time:     Wed, 07 Nov 2018 23:07:06 +0000<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Labels:         application=ceph<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">                component=rgw<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">                pod-template-hash=1622017012<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">                release_group=airship-ucp-ceph-rgw<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Annotations:    configmap-bin-hash=aee34fa624622fb03c48b85c014e805a1012f74f772bc0f3a1e91597839ce49e<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">                configmap-etc-client-hash=38eb0b2eb23d31bcd3c4238c2bf9f6b8acd335659d94b3fcfff6c333ed1339b2<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Status:         Running<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">IP:             10.97.38.77<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Controlled By:  ReplicaSet/ceph-rgw-5b6645c456<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Events:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">  Type     Reason     Age                  From                    Message<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">  ----     ------     ----                 ----                    -------<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">  Warning  BackOff    5m (x737 over 7h)    kubelet, cab23-r720-11  Back-off restarting failed container<o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN">  Warning  Unhealthy  39s (x1154 over 7h)  kubelet, cab23-r720-11  Readiness probe failed: Get
<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__10.97.38.77-3A8088_&d=DwQGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=D4ltt0SIk5Pwy68QAIKWn9EVFOIaNEjb0kjv1bpo724&s=IA_GBVFM-BWoCOlZ7Pt-MR2fraxaMFqve_EXGWlkvo4&e=">
http://10.97.38.77:8088/</a>: dial tcp 10.97.38.77:8088: getsockopt: connection refused<o:p></o:p></span></b></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN">root@cab23-r720-11:~# kubectl logs -f ceph-rgw-5b6645c456-rsv4k -n ceph<o:p></o:p></span></b></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">+ export LC_ALL=C<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">+ LC_ALL=C<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">+ : 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">++ uname -n<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">+ : ceph-rgw-5b6645c456-rsv4k<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">+ : ''<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">+ : ''<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">+ : 0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">+ : 9000<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">+ : 0.0.0.0<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">+ : /etc/ceph/ceph.client.admin.keyring<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">+ : /var/lib/ceph/radosgw/ceph-rgw-5b6645c456-rsv4k/keyring<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">+ : /var/lib/ceph/bootstrap-rgw/ceph.keyring<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">+ [[ ! -e /etc/ceph/ceph.conf ]]<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">+ '[' 0 -eq 1 ']'<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">+ '[' '!' -e /var/lib/ceph/radosgw/ceph-rgw-5b6645c456-rsv4k/keyring ']'<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">+ RGW_FRONTENDS='civetweb port=8088'<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">+ '[' 0 -eq 1 ']'<o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN">+ /usr/bin/radosgw --cluster ceph --setuser ceph --setgroup ceph -d -n client.rgw.ceph-rgw-5b6645c456-rsv4k -k /var/lib/ceph/radosgw/ceph-rgw-5b6645c456-rsv4k/keyring
 --rgw-socket-path= --rgw-zonegroup= --rgw-zone= '--rgw-frontends=civetweb port=8088'<o:p></o:p></span></b></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">2018-11-08 06:53:53.769321 7f3c7826de80  0 deferred set uid:gid to 64045:64045 (ceph:ceph)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">2018-11-08 06:53:53.769371 7f3c7826de80  0 ceph version 12.2.3 (2dab17a455c09584f2a85e6b10888337d1ec8949) luminous (stable), process (unknown), pid 8<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN">root@cab23-r720-11:~# kubectl get pods -n ceph</span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"><o:p></o:p></span></b></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">NAME                                                        READY     STATUS      RESTARTS   AGE<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">airship-ucp-ceph-provisioners-ceph-ns-key-generator-8vm8b   0/1       Completed   0          20h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-bootstrap-mp5tn                                        0/1       Completed   0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-cephfs-client-key-generator-4sqrd                      0/1       Completed   0          20h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-cephfs-provisioner-676684f6bd-48fq8                    1/1       Running     0          20h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-cephfs-provisioner-676684f6bd-mqwnj                    1/1       Running     0          20h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-mds-6bfb74d9c7-gqgtl                                   1/1       Running     0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-mds-6bfb74d9c7-sk4pw                                   1/1       Running     0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-mds-keyring-generator-m2rjj                            0/1       Completed   0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-mgr-6dc44fc75b-jtv7w                                   0/1       Pending     0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-mgr-6dc44fc75b-jvstm                                   1/1       Running     0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-mgr-keyring-generator-m56p5                            0/1       Completed   0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-mon-check-6db6b569b6-flg76                             1/1       Running     0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-mon-h2gsm                                              1/1       Running     0          21h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-mon-keyring-generator-5phjm                            0/1       Completed   0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-osd-default-64779b8c-c74ms                             1/1       Running     0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-osd-default-6ea9de2c-hn4tm                             1/1       Running     0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-osd-default-70a54190-t54l7                             1/1       Running     0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-osd-default-7544b6da-mxfj6                             1/1       Running     1          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-osd-default-7cfc44c1-vbqs4                             1/1       Running     0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-osd-default-83945928-66m9s                             1/1       Running     0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-osd-default-be8e8cc4-2vnxb                             1/1       Running     0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-osd-default-f9249fa9-8gw25                             1/1       Running     0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-osd-keyring-generator-fnrgk                            0/1       Completed   0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-rbd-pool-4dzjz                                         0/1       Completed   0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-rbd-provisioner-84bc5c88c7-smfcn                       1/1       Running     0          20h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-rbd-provisioner-84bc5c88c7-vtc5g                       1/1       Running     0          20h<o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN">ceph-rgw-5b6645c456-rsv4k                                   0/1       Running     74         8h<o:p></o:p></span></b></p>
<p class="MsoNormal"><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN">ceph-rgw-5b6645c456-sgzpt                                   0/1       Running     74         8h<o:p></o:p></span></b></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-rgw-storage-init-pqf6q                                 0/1       Completed   0          8h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-storage-keys-generator-9fx4k                           0/1       Completed   0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ingress-65dc849968-9zlqn                                    1/1       Running     0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ingress-65dc849968-cc8xp                                    1/1       Running     0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ingress-error-pages-796b76c856-dnpcp                        1/1       Running     0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ingress-error-pages-796b76c856-tt9x4                        1/1       Running     0          22h<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-align:justify;background:white"><b><span style="font-size:10.5pt;font-family:"Monotype Corsiva";color:#1F497D;mso-fareast-language:ZH-CN">Best Regards!</span></b><span style="font-size:10.5pt;font-family:"Segoe UI",sans-serif;mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-align:justify;background:white"><b><span style="font-size:10.5pt;font-family:"Monotype Corsiva";color:#1F497D;mso-fareast-language:ZH-CN">Qiaolin Tu</span></b><span style="font-size:10.5pt;font-family:"Segoe UI",sans-serif;mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-align:justify;background:white"><span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:#7F7F7F;mso-fareast-language:ZH-CN">NSB MN 5G ECE HZ CN2 SG04</span><span style="font-size:10.5pt;font-family:"Segoe UI",sans-serif;mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-align:justify;background:white"><span style="font-size:8.0pt;font-family:"Arial",sans-serif;color:gray;mso-fareast-language:ZH-CN">Mobile:
</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:gray;mso-fareast-language:ZH-CN"> +86 138 057 59684</span><span style="font-size:10.5pt;font-family:"Segoe UI",sans-serif;mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="text-align:justify;background:white"><span style="font-size:8.0pt;font-family:"Arial",sans-serif;color:gray;mso-fareast-language:ZH-CN">E-Mail:  
<a href="mailto:qiaolin.tu@nokia-sbell.com">qiaolin.tu@nokia-sbell.com</a></span><span style="font-size:10.5pt;font-family:"Segoe UI",sans-serif;mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:ZH-CN">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:ZH-CN"> SKELS, KASPARS <<a href="mailto:ks3019@att.com">ks3019@att.com</a>>
<br>
<b>Sent:</b> Wednesday, November 07, 2018 4:28 AM<br>
<b>To:</b> Tu, Qiaolin (NSB - CN/Hangzhou) <<a href="mailto:qiaolin.tu@nokia-sbell.com">qiaolin.tu@nokia-sbell.com</a>>; Li, Maxwell (NSB - CN/Hangzhou) <<a href="mailto:maxwell.li@nokia-sbell.com">maxwell.li@nokia-sbell.com</a>><br>
<b>Cc:</b> GORSHUNOV, ROMAN <<a href="mailto:roman.gorshunov@att.com">roman.gorshunov@att.com</a>>; MEADOWS, ALAN <<a href="mailto:am240k@att.com">am240k@att.com</a>>; MCEUEN, MATT <<a href="mailto:MM9745@att.com">MM9745@att.com</a>>; PACHECO, RODOLFO J <<a href="mailto:rp2723@att.com">rp2723@att.com</a>>;
 'airship-discuss@lists.airshipit.org' <<a href="mailto:airship-discuss@lists.airshipit.org">airship-discuss@lists.airshipit.org</a>><br>
<b>Subject:</b> RE: Airship installation Questions<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span style="mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN">Hi Qiaolin/Maxwell,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN">it seems like you may be using 1 control node for your deployment.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN">The
</span><span style="font-size:11.0pt;font-family:Consolas;color:#1F497D;mso-fareast-language:ZH-CN">ceph-mgr</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN"> chart is set to have 2 replicas and
 is trying to launch them (here you get a port clash).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN">Generally speaking, the manifests (by default) are targeted for 3 control node (HA) configuration, including ceph to reflect production-like
 HA deployment<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openstack_airship-2Dtreasuremap_blob_master_global_software_charts_ucp_ceph_ceph-2Dclient-2Dupdate.yaml-23L145&d=DwMGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=D4ltt0SIk5Pwy68QAIKWn9EVFOIaNEjb0kjv1bpo724&s=ryDNfEpcHGDMGGbSjUmlkjoTcINDh6JT426VPeJDY78&e=">https://github.com/openstack/airship-treasuremap/blob/master/global/software/charts/ucp/ceph/ceph-client-update.yaml#L145</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN">That said, you may adjust the replica count on site level (set 1, instead of 2) as well, as well as OSD count here by doing overrides
 (matching how many OSDs you have)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openstack_airship-2Dtreasuremap_blob_master_site_airship-2Dseaworthy_software_charts_ucp_ceph_ceph-2Dclient-2Dupdate.yaml&d=DwMGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=D4ltt0SIk5Pwy68QAIKWn9EVFOIaNEjb0kjv1bpo724&s=1_cNRZV5b8U3J20kYkk-BWxvlkUVVQdYQIdjjjVvMW4&e=">https://github.com/openstack/airship-treasuremap/blob/master/site/airship-seaworthy/software/charts/ucp/ceph/ceph-client-update.yaml</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN">And also, adjustments in other charts may be required to set this up for a single control plane node, here is more info on reference
 site<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__airship-2Dtreasuremap.readthedocs.io_en_latest_seaworthy.html&d=DwMGaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=D4ltt0SIk5Pwy68QAIKWn9EVFOIaNEjb0kjv1bpo724&s=ezzAGQJQOJd6evJtTSDs8QVivNbRIw5b2DtLrHtd0k8&e=">https://airship-treasuremap.readthedocs.io/en/latest/seaworthy.html</a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN">Kindly, Kaspars<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:ZH-CN">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:ZH-CN"> Kaspars Skels [<a href="mailto:kaspars.skels@gmail.com">mailto:kaspars.skels@gmail.com</a>]
<br>
<b>Sent:</b> Tuesday, November 6, 2018 2:10 PM<br>
<b>To:</b> <a href="mailto:qiaolin.tu@nokia-sbell.com">qiaolin.tu@nokia-sbell.com</a><br>
<b>Cc:</b> MEADOWS, ALAN <<a href="mailto:am240k@att.com">am240k@att.com</a>>; MONTEIRO, FELIPE C <<a href="mailto:fm577c@att.com">fm577c@att.com</a>>; BARTRA, RICK <<a href="mailto:rb560u@att.com">rb560u@att.com</a>>; REDDY, CHINASUBBA <<a href="mailto:cr3938@att.com">cr3938@att.com</a>>;
 GORSHUNOV, ROMAN <<a href="mailto:roman.gorshunov@att.com">roman.gorshunov@att.com</a>>; KHUNTIA, SOUMITRA <<a href="mailto:sk698p@att.com">sk698p@att.com</a>>; KABANOV, DMITRII <<a href="mailto:dk370c@att.com">dk370c@att.com</a>>; Mark Burnett <<a href="mailto:mark.m.burnett@gmail.com">mark.m.burnett@gmail.com</a>>;
 VOLKOV, ANDREY <<a href="mailto:av903u@att.com">av903u@att.com</a>>; <a href="mailto:avolkov@mirantis.com">
avolkov@mirantis.com</a>; Chris Wedgwood <<a href="mailto:cw@f00f.org">cw@f00f.org</a>>; PALLAV GUPTA <<a href="mailto:pallavgupta84@gmail.com">pallavgupta84@gmail.com</a>>; GUPTA, SANGEET <<a href="mailto:sg774j@att.com">sg774j@att.com</a>>;
<a href="mailto:zuul@review01.openstack.org">zuul@review01.openstack.org</a>; HUSSEY, SCOTT T <<a href="mailto:sh8121@att.com">sh8121@att.com</a>>; PACHECO, RODOLFO J <<a href="mailto:rp2723@att.com">rp2723@att.com</a>>;
<a href="mailto:maxwell.li@nokia-sbell.com">maxwell.li@nokia-sbell.com</a>; SKELS, KASPARS <<a href="mailto:ks3019@att.com">ks3019@att.com</a>><br>
<b>Subject:</b> Re: Airship installation Questions<o:p></o:p></span></p>
<p class="MsoNormal"><span style="mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span style="mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span style="mso-fareast-language:ZH-CN"><o:p> </o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span style="mso-fareast-language:ZH-CN">On Tue, Nov 6, 2018 at 3:42 AM Tu, Qiaolin (NSB - CN/Hangzhou) <<a href="mailto:qiaolin.tu@nokia-sbell.com">qiaolin.tu@nokia-sbell.com</a>> wrote:<o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Hi,</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">I also found that pod(<b><span style="color:red">ceph-mgr-6dc44fc75b-25n24</span></b>) is
 always on the pending state. It report error as below: </span><b><span style="font-size:10.5pt;color:red;mso-fareast-language:ZH-CN">  </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN">Warning</span></b><b><span style="font-size:10.5pt;color:red;mso-fareast-language:ZH-CN"> </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN">
 FailedScheduling</span></b><b><span style="font-size:10.5pt;color:red;mso-fareast-language:ZH-CN"> </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN"> 1m (x125 over 36m)</span></b><b><span style="font-size:10.5pt;color:red;mso-fareast-language:ZH-CN"> </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN">
 default-scheduler</span></b><b><span style="font-size:10.5pt;color:red;mso-fareast-language:ZH-CN"> </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN"> 0/1 nodes are available: 1 node(s) didn't have
 free ports for the requested pod ports.</span></b><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">root@cab23-r720-11:~# kubectl get pods --all-namespaces</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">NAMESPACE</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">    </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 NAME</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">                                                       </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> READY</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">    </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 STATUS</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> RESTARTS</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">  </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 AGE</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">airship-ucp-ceph-provisioners-ceph-ns-key-generator-n5vzj</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">  </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 0/1</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Completed</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">  </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 0</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 20m</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-cephfs-client-key-generator-656rd</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">                     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 0/1</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 Completed</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">  </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 0</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 20m</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-cephfs-provisioner-676684f6bd-trf5h</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">                   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 1/1</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 Running</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">    </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 0</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 20m</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-cephfs-provisioner-676684f6bd-zfk2s</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">                   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 1/1</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 Running</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">    </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 0</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 20m</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-mds-6bfb74d9c7-btbt5</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">                                  </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 1/1</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Running</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">    </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 0</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 37m</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-mds-6bfb74d9c7-qw4xl</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">                                  </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 1/1</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 Running</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">    </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 0</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 37m</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-mds-keyring-generator-dlqjt</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">                           </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 0/1</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 Completed</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">  </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 0</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 53m</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN">ceph</span></b><b><span style="font-size:10.5pt;color:red;mso-fareast-language:ZH-CN">         </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN">
 ceph-mgr-6dc44fc75b-25n24</span></b><b><span style="font-size:10.5pt;color:red;mso-fareast-language:ZH-CN">                                  </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN"> 0/1</span></b><b><span style="font-size:10.5pt;color:red;mso-fareast-language:ZH-CN">      </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN">
 Pending</span></b><b><span style="font-size:10.5pt;color:red;mso-fareast-language:ZH-CN">    </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN"> 0</span></b><b><span style="font-size:10.5pt;color:red;mso-fareast-language:ZH-CN">         </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN">
 37m</span></b><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph</span></b><b><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-mgr-6dc44fc75b-42mqz</span></b><b><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">                                  </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 1/1</span></b><b><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 Running</span></b><b><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">    </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 0</span></b><b><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 37m</span></b><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-mgr-keyring-generator-7dftx</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">                           </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 0/1</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 Completed</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">  </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 0</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 53m</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-mon-cfrf6</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">                                             </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 1/1</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 Running</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">    </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 0</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 41m</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-mon-check-6db6b569b6-67vvm</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">                        </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">    </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">1/1</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 Running</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">    </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 0</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 53m</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-mon-keyring-generator-55ps8</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">                           </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 0/1</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 Completed</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">  </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> 0</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 53m</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">root@cab23-r720-11:~#
<b><span style="color:red">kubectl describe pod ceph-mgr-6dc44fc75b-25n24 -n ceph</span></b></span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Name:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">          </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-mgr-6dc44fc75b-25n24</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Namespace:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Node:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">          </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 <none></span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Labels:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">        </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 application=ceph</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">               </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> component=mgr</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">               </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> pod-template-hash=2870097316</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">               </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> release_group=airship-ucp-ceph-client</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Annotations:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 configmap-bin-hash=d23610f5cc67014b596eb45b9d47ddb97997d6ec5fd3289fb264429a260f39a9</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">               </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> configmap-etc-client-hash=0174be85f29398aa5cea5fccb9537e0e0f25ea2d2f280182e9e2506a1be1b115</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Status:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">        </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 Pending</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">IP:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">            </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Controlled By:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ReplicaSet/ceph-mgr-6dc44fc75b</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Init Containers:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> init:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Image:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__quay.io_stackanetes_kubernetes-2Dentrypoint-3Av0.3.1&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=Yfn1URqa4p-w7xoxiueJJP2SNYUZZCsOA07XxDCOxEM&s=QVq3KXWVyPVji_O3s5wgjBk7wnbkw3swE9ddmYytPOw&e=" target="_blank">
quay.io/stackanetes/kubernetes-entrypoint:v0.3.1</a></span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Port:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 <none></span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Host Port:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 <none></span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Command:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> kubernetes-entrypoint</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Environment:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> POD_NAME:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">       </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-mgr-6dc44fc75b-25n24 (v1:<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__metadata.name&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=Yfn1URqa4p-w7xoxiueJJP2SNYUZZCsOA07XxDCOxEM&s=DIlvVxm8OFCSVZEOPMgsZQb4rkRx8zql64UE_2OnMjU&e=" target="_blank">metadata.name</a>)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> NAMESPACE:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">            </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph (v1:metadata.namespace)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> INTERFACE_NAME:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">       </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 eth0</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> PATH:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">                 </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> DEPENDENCY_SERVICE:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph:ceph-mon</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> DEPENDENCY_JOBS:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-storage-keys-generator,ceph-mgr-keyring-generator</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> DEPENDENCY_DAEMONSET:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">DEPENDENCY_CONTAINER:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">DEPENDENCY_POD_JSON:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">  </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">COMMAND:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">              </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 echo done</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Mounts:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> /var/run/secrets/<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__kubernetes.io_serviceaccount&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=Yfn1URqa4p-w7xoxiueJJP2SNYUZZCsOA07XxDCOxEM&s=S6tMkfnBpnWWGsjH58zRIhEcnxQ0dJINNWuoTXXeNyI&e=" target="_blank">kubernetes.io/serviceaccount</a>
 from ceph-mgr-token-pgbwp (ro)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> ceph-init-dirs:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Image:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__docker.io_ceph_daemon-3Atag-2Dbuild-2Dmaster-2Dluminous-2Dubuntu-2D16.04&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=Yfn1URqa4p-w7xoxiueJJP2SNYUZZCsOA07XxDCOxEM&s=wD6kGoJq_VjM3q9MaytwbkA4gjV0HuHeKt7ZJ_419m4&e=" target="_blank">
docker.io/ceph/daemon:tag-build-master-luminous-ubuntu-16.04</a></span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Port:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 <none></span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Host Port:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 <none></span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Command:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> /tmp/init-dirs.sh</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Environment:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> CLUSTER:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Mounts:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> /etc/ceph from pod-etc-ceph
 (rw)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> /run from pod-run (rw)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> /tmp/init-dirs.sh from
 ceph-client-bin (ro)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> /var/lib/ceph from
 pod-var-lib-ceph (rw)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> /var/run/secrets/<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__kubernetes.io_serviceaccount&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=Yfn1URqa4p-w7xoxiueJJP2SNYUZZCsOA07XxDCOxEM&s=S6tMkfnBpnWWGsjH58zRIhEcnxQ0dJINNWuoTXXeNyI&e=" target="_blank">kubernetes.io/serviceaccount</a>
 from ceph-mgr-token-pgbwp (ro)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Containers:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> ceph-mgr:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Image:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__docker.io_ceph_daemon-3Atag-2Dbuild-2Dmaster-2Dluminous-2Dubuntu-2D16.04&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=Yfn1URqa4p-w7xoxiueJJP2SNYUZZCsOA07XxDCOxEM&s=wD6kGoJq_VjM3q9MaytwbkA4gjV0HuHeKt7ZJ_419m4&e=" target="_blank">
docker.io/ceph/daemon:tag-build-master-luminous-ubuntu-16.04</a></span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Ports:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 7000/TCP, 9283/TCP</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Host Ports:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 7000/TCP, 9283/TCP</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Command:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> /mgr-start.sh</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">  </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Liveness:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">  </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 exec [/tmp/mgr-check.sh liveness] delay=30s timeout=5s period=10s #success=1 #failure=3</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Readiness:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 exec [/tmp/mgr-check.sh readiness] delay=30s timeout=5s period=10s #success=1 #failure=3</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Environment:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> CLUSTER:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">         </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">  </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ENABLED_MODULES:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 restful</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">status</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">prometheus</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Mounts:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> /etc/ceph from pod-etc-ceph
 (rw)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> /etc/ceph/ceph.client.admin.keyring
 from ceph-client-admin-keyring (ro)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> /etc/ceph/ceph.conf
 from ceph-client-etc (ro)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> /mgr-start.sh from
 ceph-client-bin (ro)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> /run from pod-run (rw)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> /tmp/mgr-check.sh from
 ceph-client-bin (ro)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> /var/lib/ceph from
 pod-var-lib-ceph (rw)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> /var/lib/ceph/bootstrap-mgr/ceph.keyring
 from ceph-bootstrap-mgr-keyring (rw)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> /var/run/secrets/<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__kubernetes.io_serviceaccount&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=Yfn1URqa4p-w7xoxiueJJP2SNYUZZCsOA07XxDCOxEM&s=S6tMkfnBpnWWGsjH58zRIhEcnxQ0dJINNWuoTXXeNyI&e=" target="_blank">kubernetes.io/serviceaccount</a>
 from ceph-mgr-token-pgbwp (ro)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Conditions:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Type</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">          </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 Status</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> PodScheduled</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">  </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 False </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Volumes:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> pod-etc-ceph:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Type:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 EmptyDir (a temporary directory that shares a pod's lifetime)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Medium:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">  </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">ceph-client-bin:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Type:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ConfigMap (a volume populated by a ConfigMap)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Name:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-client-bin</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Optional:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 false</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> ceph-client-etc:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Type:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ConfigMap (a volume populated by a ConfigMap)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Name:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">     </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-client-etc</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Optional:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 false</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> pod-var-lib-ceph:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Type:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 EmptyDir (a temporary directory that shares a pod's lifetime)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Medium:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">  </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">pod-run:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Type:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 EmptyDir (a temporary directory that shares a pod's lifetime)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Medium:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 Memory</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> ceph-client-admin-keyring:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Type:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">       </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 Secret (a volume populated by a Secret)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> SecretName:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-client-admin-keyring</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Optional:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 false</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> ceph-bootstrap-mgr-keyring:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Type:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">       </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 Secret (a volume populated by a Secret)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> SecretName:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-bootstrap-mgr-keyring</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">  </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Optional:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 false</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> ceph-mgr-token-pgbwp:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Type:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">       </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 Secret (a volume populated by a Secret)</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> SecretName:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-mgr-token-pgbwp</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Optional:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">   </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 false</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">QoS Class:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">      </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 BestEffort</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Node-Selectors:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ceph-mgr=enabled</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Tolerations:</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">    </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__node.kubernetes.io_not-2Dready-3ANoExecute&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=Yfn1URqa4p-w7xoxiueJJP2SNYUZZCsOA07XxDCOxEM&s=7UGHV8v22mtkm09xG6MoMxOg7RO9aa7lQkwQd3yYYqw&e=" target="_blank">
node.kubernetes.io/not-ready:NoExecute</a> for 300s</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">                </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__node.kubernetes.io_unreachable-3ANoExecute&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=Yfn1URqa4p-w7xoxiueJJP2SNYUZZCsOA07XxDCOxEM&s=U8nhQAM3YHPjGEUgzGDzxxM8T5YYO0V9_PBd7yfaXpU&e=" target="_blank">
node.kubernetes.io/unreachable:NoExecute</a> for 300s</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Events:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Type</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">    </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 Reason</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">           </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Age</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">                </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 From</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">              </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> Message</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> ----</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">    </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ------</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">           </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> ----</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">               </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">
 ----</span><span style="font-size:10.5pt;mso-fareast-language:ZH-CN">              </span><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN"> -------</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:10.5pt;color:red;mso-fareast-language:ZH-CN"> </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN">
 Warning</span></b><b><span style="font-size:10.5pt;color:red;mso-fareast-language:ZH-CN"> </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN"> FailedScheduling</span></b><b><span style="font-size:10.5pt;color:red;mso-fareast-language:ZH-CN"> </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN">
 1m (x125 over 36m)</span></b><b><span style="font-size:10.5pt;color:red;mso-fareast-language:ZH-CN"> </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN"> default-scheduler</span></b><b><span style="font-size:10.5pt;color:red;mso-fareast-language:ZH-CN"> </span></b><b><span style="font-size:10.5pt;font-family:"DengXian",serif;color:red;mso-fareast-language:ZH-CN">
 0/1 nodes are available: 1 node(s) didn't have free ports for the requested pod ports.</span></b><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-align:justify;background:white">
<b><span style="font-size:10.5pt;font-family:"Monotype Corsiva";color:#1F497D;mso-fareast-language:ZH-CN">Best Regards!</span></b><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-align:justify;background:white">
<b><span style="font-size:10.5pt;font-family:"Monotype Corsiva";color:#1F497D;mso-fareast-language:ZH-CN">Qiaolin Tu</span></b><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-align:justify;background:white">
<span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:#7F7F7F;mso-fareast-language:ZH-CN">NSB MN 5G ECE HZ CN2 SG04</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-align:justify;background:white">
<span style="font-size:8.0pt;font-family:"Arial",sans-serif;color:gray;mso-fareast-language:ZH-CN">Mobile:
</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:gray;mso-fareast-language:ZH-CN"> +86 138 057 59684</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-align:justify;background:white">
<span style="font-size:8.0pt;font-family:"Arial",sans-serif;color:gray;mso-fareast-language:ZH-CN">E-Mail:  
<a href="mailto:qiaolin.tu@nokia-sbell.com" target="_blank">qiaolin.tu@nokia-sbell.com</a></span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:ZH-CN">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:ZH-CN">
 Tu, Qiaolin (NSB - CN/Hangzhou) <br>
<b>Sent:</b> Tuesday, November 06, 2018 1:46 PM<br>
<b>To:</b> 'MEADOWS, ALAN' <<a href="mailto:am240k@att.com" target="_blank">am240k@att.com</a>>; Kaspars Skels <<a href="mailto:kaspars.skels@gmail.com" target="_blank">kaspars.skels@gmail.com</a>><br>
<b>Cc:</b> MONTEIRO, FELIPE C <<a href="mailto:fm577c@att.com" target="_blank">fm577c@att.com</a>>; BARTRA, RICK <<a href="mailto:rb560u@att.com" target="_blank">rb560u@att.com</a>>; REDDY, CHINASUBBA <<a href="mailto:cr3938@att.com" target="_blank">cr3938@att.com</a>>;
 GORSHUNOV, ROMAN <<a href="mailto:roman.gorshunov@att.com" target="_blank">roman.gorshunov@att.com</a>>; KHUNTIA, SOUMITRA <<a href="mailto:sk698p@att.com" target="_blank">sk698p@att.com</a>>; KABANOV, DMITRII <<a href="mailto:dk370c@att.com" target="_blank">dk370c@att.com</a>>;
<a href="mailto:mark.m.burnett@gmail.com" target="_blank">mark.m.burnett@gmail.com</a>; VOLKOV, ANDREY <<a href="mailto:av903u@att.com" target="_blank">av903u@att.com</a>>;
<a href="mailto:avolkov@mirantis.com" target="_blank">avolkov@mirantis.com</a>; <a href="mailto:cw@f00f.org" target="_blank">
cw@f00f.org</a>; <a href="mailto:pallavgupta84@gmail.com" target="_blank">pallavgupta84@gmail.com</a>; GUPTA, SANGEET <<a href="mailto:sg774j@att.com" target="_blank">sg774j@att.com</a>>;
<a href="mailto:pete@port.direct" target="_blank">pete@port.direct</a>; <a href="mailto:zuul@review01.openstack.org" target="_blank">
zuul@review01.openstack.org</a>; HUSSEY, SCOTT T <<a href="mailto:sh8121@att.com" target="_blank">sh8121@att.com</a>>; PACHECO, RODOLFO J <<a href="mailto:rp2723@att.com" target="_blank">rp2723@att.com</a>>; Li, Maxwell (NSB - CN/Hangzhou) <<a href="mailto:maxwell.li@nokia-sbell.com" target="_blank">maxwell.li@nokia-sbell.com</a>><br>
<b>Subject:</b> Airship installation Questions</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="mso-fareast-language:ZH-CN"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Hi,</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">Thanks very much for your kindly support.</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:"DengXian",serif;mso-fareast-language:ZH-CN">I also encountered a problem when install multi-node airship. It blocked by 2 ceph-rgw pods.
 They crash off and run again and again. Pods error log as bellow:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN">root@cab23-r720-11:~#
<b><span style="color:#DF402A">kubectl describe pod ceph-rgw-6ff5ff866d-4x66n -n ceph</span></b></span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN">Events:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN">Type Reason Age From Message</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN">---- ------ ---- ---- -------</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN">Normal Scheduled 1m default-scheduler Successfully assigned ceph-rgw-6ff5ff866d-4x66n to cab23-r720-11</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN">Normal SuccessfulMountVolume 1m kubelet, cab23-r720-11 MountVolume.SetUp succeeded for volume "pod-etc-ceph"</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN">Normal SuccessfulMountVolume 1m kubelet, cab23-r720-11 MountVolume.SetUp succeeded for volume "pod-var-lib-ceph"</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN">Normal SuccessfulMountVolume 1m kubelet, cab23-r720-11 MountVolume.SetUp succeeded for volume "pod-run"</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN">Normal SuccessfulMountVolume 1m kubelet, cab23-r720-11 MountVolume.SetUp succeeded for volume "ceph-rgw-etc"</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN">Normal SuccessfulMountVolume 1m kubelet, cab23-r720-11 MountVolume.SetUp succeeded for volume "ceph-rgw-bin"</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN">Normal SuccessfulMountVolume 1m kubelet, cab23-r720-11 MountVolume.SetUp succeeded for volume "ceph-bootstrap-rgw-keyring"</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN">Normal SuccessfulMountVolume 1m kubelet, cab23-r720-11 MountVolume.SetUp succeeded for volume "ceph-rgw-token-sjg9x"</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN">Normal Pulled 1m kubelet, cab23-r720-11 Container image "<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__quay.io_stackanetes_kubernetes-2Dentrypoint-3Av0.3.1&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=Yfn1URqa4p-w7xoxiueJJP2SNYUZZCsOA07XxDCOxEM&s=QVq3KXWVyPVji_O3s5wgjBk7wnbkw3swE9ddmYytPOw&e=" target="_blank">quay.io/stackanetes/kubernetes-entrypoint:v0.3.1</a>"
 already present on machine</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN">Normal Started 1m kubelet, cab23-r720-11 Started container</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN">Normal Pulled 1m kubelet, cab23-r720-11 Container image "<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__docker.io_ceph_daemon-3Atag-2Dbuild-2Dmaster-2Dluminous-2Dubuntu-2D16.04&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=Yfn1URqa4p-w7xoxiueJJP2SNYUZZCsOA07XxDCOxEM&s=wD6kGoJq_VjM3q9MaytwbkA4gjV0HuHeKt7ZJ_419m4&e=" target="_blank">docker.io/ceph/daemon:tag-build-master-luminous-ubuntu-16.04</a>"
 already present on machine</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN">Normal Created 1m kubelet, cab23-r720-11 Created container</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN">Normal Started 1m kubelet, cab23-r720-11 Started container</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;font-family:SimSun;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:10.5pt;font-family:SimSun;color:#DF402A;mso-fareast-language:ZH-CN">Warning Unhealthy 18s (x6 over 1m) kubelet, cab23-r720-11 Readiness probe failed: Get
<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__10.97.38.105-3A8088_&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=Yfn1URqa4p-w7xoxiueJJP2SNYUZZCsOA07XxDCOxEM&s=N6CGj_cMP5_eYfe9DOVCi3m-z6jWPpRQNrRRkYg6kjA&e=" target="_blank">
http://10.97.38.105:8088/</a>: dial tcp <a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__10.97.38.105-3A8088&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=Yfn1URqa4p-w7xoxiueJJP2SNYUZZCsOA07XxDCOxEM&s=_MBsJiSpie7BI64_sUyyqznHkaLMa1UEOyINiKSIfEE&e=" target="_blank">
10.97.38.105:8088</a>: getsockopt: connection refused</span></b><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-align:justify;background:white">
<b><span style="font-size:10.5pt;font-family:"Monotype Corsiva";color:#1F497D;mso-fareast-language:ZH-CN">Best Regards!</span></b><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-align:justify;background:white">
<b><span style="font-size:10.5pt;font-family:"Monotype Corsiva";color:#1F497D;mso-fareast-language:ZH-CN">Qiaolin Tu</span></b><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-align:justify;background:white">
<span style="font-size:10.0pt;font-family:"Tahoma",sans-serif;color:#7F7F7F;mso-fareast-language:ZH-CN">NSB MN 5G ECE HZ CN2 SG04</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-align:justify;background:white">
<span style="font-size:8.0pt;font-family:"Arial",sans-serif;color:gray;mso-fareast-language:ZH-CN">Mobile:
</span><span style="font-size:10.0pt;font-family:"Arial",sans-serif;color:gray;mso-fareast-language:ZH-CN"> +86 138 057 59684</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;text-align:justify;background:white">
<span style="font-size:8.0pt;font-family:"Arial",sans-serif;color:gray;mso-fareast-language:ZH-CN">E-Mail:  
<a href="mailto:qiaolin.tu@nokia-sbell.com" target="_blank">qiaolin.tu@nokia-sbell.com</a></span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:10.5pt;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:ZH-CN">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:ZH-CN">
 MEADOWS, ALAN <<a href="mailto:am240k@att.com" target="_blank">am240k@att.com</a>>
<br>
<b>Sent:</b> Saturday, November 03, 2018 5:39 AM<br>
<b>To:</b> Kaspars Skels <<a href="mailto:kaspars.skels@gmail.com" target="_blank">kaspars.skels@gmail.com</a>>; Li, Maxwell (NSB - CN/Hangzhou) <<a href="mailto:maxwell.li@nokia-sbell.com" target="_blank">maxwell.li@nokia-sbell.com</a>><br>
<b>Cc:</b> MONTEIRO, FELIPE C <<a href="mailto:fm577c@att.com" target="_blank">fm577c@att.com</a>>; BARTRA, RICK <<a href="mailto:rb560u@att.com" target="_blank">rb560u@att.com</a>>; REDDY, CHINASUBBA <<a href="mailto:cr3938@att.com" target="_blank">cr3938@att.com</a>>;
 GORSHUNOV, ROMAN <<a href="mailto:roman.gorshunov@att.com" target="_blank">roman.gorshunov@att.com</a>>; KHUNTIA, SOUMITRA <<a href="mailto:sk698p@att.com" target="_blank">sk698p@att.com</a>>; KABANOV, DMITRII <<a href="mailto:dk370c@att.com" target="_blank">dk370c@att.com</a>>;
<a href="mailto:mark.m.burnett@gmail.com" target="_blank">mark.m.burnett@gmail.com</a>; VOLKOV, ANDREY <<a href="mailto:av903u@att.com" target="_blank">av903u@att.com</a>>;
<a href="mailto:avolkov@mirantis.com" target="_blank">avolkov@mirantis.com</a>; <a href="mailto:cw@f00f.org" target="_blank">
cw@f00f.org</a>; <a href="mailto:pallavgupta84@gmail.com" target="_blank">pallavgupta84@gmail.com</a>; GUPTA, SANGEET <<a href="mailto:sg774j@att.com" target="_blank">sg774j@att.com</a>>;
<a href="mailto:pete@port.direct" target="_blank">pete@port.direct</a>; <a href="mailto:zuul@review01.openstack.org" target="_blank">
zuul@review01.openstack.org</a>; HUSSEY, SCOTT T <<a href="mailto:sh8121@att.com" target="_blank">sh8121@att.com</a>>; Tu, Qiaolin (NSB - CN/Hangzhou) <<a href="mailto:qiaolin.tu@nokia-sbell.com" target="_blank">qiaolin.tu@nokia-sbell.com</a>>; PACHECO, RODOLFO
 J <<a href="mailto:rp2723@att.com" target="_blank">rp2723@att.com</a>><br>
<b>Subject:</b> RE: Airship Version Questions</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="mso-fareast-language:ZH-CN"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN">Hi Maxwell!</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN">Great to see additional people trying Airship out.</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN">Treasuremap does change quite frequently.  In fact, this is from a lot of the great
 work Kaspars has been doing – building a CI/CD pipeline to not only keep treasuremap fresh and referencing the latest working versions of Airship components but also validating that the YAML documents are able to provision a multi-node baremetal physical environment
 by way of third-party gates. </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN">This gives us a high degree of confidence that no matter what version of treasuremap
 you have decided to start working with to deploy your own environments it should work.</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN">To be sure, we do understand when getting started it is nice to work with a stable
 target that isn’t constantly changing.    Kaspars has introduced tagging to provide a human readable reference (using dates).  These are not on any particular cadence, but usually follow more in-depth tested release beyond just automated CI/CD using the manifests:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openstack_airship-2Dtreasuremap_releases&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=Yfn1URqa4p-w7xoxiueJJP2SNYUZZCsOA07XxDCOxEM&s=ZyIbYVxKvMan1A5Fw3GshH0lifNZHRUnU_40MqtumxA&e=" target="_blank"><span style="color:windowtext">https://github.com/openstack/airship-treasuremap/releases</span></a></span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN">For right now with regard to pegleg, the best way to leverage the version of pegleg
 that has been tested with the treasuremap manifests is to pull it the one used in the baremetal CI/CD gate test directly from the target tag Jenkinsfile.  For example, for v18.11.01:</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN"><a href="https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_openstack_airship-2Dtreasuremap_blob_v18.11.01_tools_gate_Jenkinsfile-23L15&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=Yfn1URqa4p-w7xoxiueJJP2SNYUZZCsOA07XxDCOxEM&s=S0oZ2OKnKI07o5yGWGvwz6_gUToysqjc9W_EKPR8eTY&e=" target="_blank"><span style="color:windowtext">https://github.com/openstack/airship-treasuremap/blob/v18.11.01/tools/gate/Jenkinsfile#L15</span></a></span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN">Please feel free to get on the airship mailing list (<a href="https://urldefense.proofpoint.com/v2/url?u=http-3A__airshipit.org&d=DwMFaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=L2WcX1Ge33cpIHXX6krAgw&m=Yfn1URqa4p-w7xoxiueJJP2SNYUZZCsOA07XxDCOxEM&s=9mXi3m93VupawhG1pIMsk99_Ng5NFUCbs-6ip-2NxTY&e=" target="_blank">airshipit.org</a>)
 to connect with a wider audience or our IRC channel!</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Calibri Light",sans-serif;mso-fareast-language:ZH-CN">Alan Meadows </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D;mso-fareast-language:ZH-CN"> </span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:ZH-CN">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;mso-fareast-language:ZH-CN">
 Kaspars Skels [<a href="mailto:kaspars.skels@gmail.com" target="_blank">mailto:kaspars.skels@gmail.com</a>]
<br>
<b>Sent:</b> Friday, November 02, 2018 2:27 PM<br>
<b>To:</b> <a href="mailto:maxwell.li@nokia-sbell.com" target="_blank">maxwell.li@nokia-sbell.com</a><br>
<b>Cc:</b> MONTEIRO, FELIPE C <<a href="mailto:fm577c@att.com" target="_blank">fm577c@att.com</a>>; BARTRA, RICK <<a href="mailto:rb560u@att.com" target="_blank">rb560u@att.com</a>>; REDDY, CHINASUBBA <<a href="mailto:cr3938@att.com" target="_blank">cr3938@att.com</a>>;
 GORSHUNOV, ROMAN <<a href="mailto:roman.gorshunov@att.com" target="_blank">roman.gorshunov@att.com</a>>; KHUNTIA, SOUMITRA <<a href="mailto:sk698p@att.com" target="_blank">sk698p@att.com</a>>; KABANOV, DMITRII <<a href="mailto:dk370c@att.com" target="_blank">dk370c@att.com</a>>;
<a href="mailto:mark.m.burnett@gmail.com" target="_blank">mark.m.burnett@gmail.com</a>; VOLKOV, ANDREY <<a href="mailto:av903u@att.com" target="_blank">av903u@att.com</a>>;
<a href="mailto:avolkov@mirantis.com" target="_blank">avolkov@mirantis.com</a>; <a href="mailto:cw@f00f.org" target="_blank">
cw@f00f.org</a>; <a href="mailto:pallavgupta84@gmail.com" target="_blank">pallavgupta84@gmail.com</a>; GUPTA, SANGEET <<a href="mailto:sg774j@att.com" target="_blank">sg774j@att.com</a>>;
<a href="mailto:pete@port.direct" target="_blank">pete@port.direct</a>; <a href="mailto:zuul@review01.openstack.org" target="_blank">
zuul@review01.openstack.org</a>; HUSSEY, SCOTT T <<a href="mailto:sh8121@att.com" target="_blank">sh8121@att.com</a>>;
<a href="mailto:qiaolin.tu@nokia-sbell.com" target="_blank">qiaolin.tu@nokia-sbell.com</a>; MEADOWS, ALAN <<a href="mailto:am240k@att.com" target="_blank">am240k@att.com</a>>; PACHECO, RODOLFO J <<a href="mailto:rp2723@att.com" target="_blank">rp2723@att.com</a>><br>
<b>Subject:</b> Re: Airship Version Questions</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="mso-fareast-language:ZH-CN"> <o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="mso-fareast-language:ZH-CN">+ Alan/Rodolfo<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="mso-fareast-language:ZH-CN"> <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="mso-fareast-language:ZH-CN"> <o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="mso-fareast-language:ZH-CN"> <o:p></o:p></span></p>
<div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="mso-fareast-language:ZH-CN">On Thu, Nov 1, 2018 at 3:37 AM Li, Maxwell (NSB - CN/Hangzhou) <<a href="mailto:maxwell.li@nokia-sbell.com" target="_blank">maxwell.li@nokia-sbell.com</a>>
 wrote:<o:p></o:p></span></p>
</div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<div>
<div>
<h1 style="margin-top:0in;background:white"><span style="mso-fareast-language:ZH-CN">Hi Airship Team:<o:p></o:p></span></h1>
<h1 style="margin-top:0in;background:white"><span style="mso-fareast-language:ZH-CN"> <o:p></o:p></span></h1>
<h1 style="margin-top:0in;background:white"><span style="mso-fareast-language:ZH-CN">I have a problem when I deploy airship on my server. There are new commits in airship-pegleg and airship-treasuremap, almost erveryday. And there are no tags or branches in
 these code repositories. Could I get a commit ID about these repositories so that I could deploy airship.<o:p></o:p></span></h1>
<h1 style="margin-top:0in;background:white"><span style="mso-fareast-language:ZH-CN"> <o:p></o:p></span></h1>
<h1 style="margin-top:0in;background:white"><span style="mso-fareast-language:ZH-CN">Thanks a lot!<o:p></o:p></span></h1>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="mso-fareast-language:ZH-CN"> <o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Segoe Script";mso-fareast-language:ZH-CN">Best Regards!</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="font-family:"Segoe Script";mso-fareast-language:ZH-CN">Maxwell Li</span><span style="mso-fareast-language:ZH-CN"><o:p></o:p></span></p>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto"><span style="mso-fareast-language:ZH-CN"> <o:p></o:p></span></p>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</body>
</html>