[Airship-discuss] Armada Compatibility with Keystone

Thu Jan 17 14:56:13 UTC 2019

Hey Shoaib!

I'll give your patch a look today and see if I have any feedback to offer.

Regards,
Drew Walters

On Tue, Jan 15, 2019 at 11:59 AM Nasir, Shoaib <Shoaib.Nasir at windriver.com>
wrote:

> Hi Matt
>
>
>
> Sorry I was a bit late in getting to this message
>
> So far, I only added a change in the CLI to pass a user’s bearertoken and
> didn’t make any changes to the API. That can be added later if people need
> it.
>
> I will fix the Zuul errors and put a new patch set as soon as I do that.
>
> But in the meantime, do you have any suggestions/ feedback regarding what
> I have proposed ?
>
>
>
> Thanks
>
> Shoaib
>
>
>
> *From:* MCEUEN, MATT [mailto:MM9745 at att.com]
> *Sent:* Monday, January 14, 2019 11:44 AM
> *To:* Nasir, Shoaib; EAGAN, SEAN; airship-discuss at lists.airshipit.org
> *Subject:* RE: Armada Compatibility with Keystone
>
>
>
> Looking forward to seeing what you have in mind, Shoaib.  If you’d like to
> discuss in tomorrow’s team meeting, feel free to add the topic to the
> etherpad:
>
>
>
> https://etherpad.openstack.org/p/airship-meeting-2019-01-15
>
>
>
> Thanks,
> Matt
>
>
>
> *From:* Nasir, Shoaib <Shoaib.Nasir at windriver.com>
> *Sent:* Thursday, January 10, 2019 9:26 AM
> *To:* EAGAN, SEAN <se136c at att.com>; airship-discuss at lists.airshipit.org
> *Subject:* Re: [Airship-discuss] Armada Compatibility with Keystone
>
>
>
> Hi Sean
>
>
>
> After digging further into the issue, I realized that armada does not
> support passing end users bearer-tokens.
>
> Armada can be authenticated by keystone for incoming API requests.
>
>
>
> But in the scenario, where I want to use armada to bring up openstack-helm
> pods on a kubernetes cluster that’s already setup to be authenticated by
> keystone using the k8s-keystone-auth Webhook application, armada is not
> compatible.
>
> Therefore, I have made some changes to the armada CLI such that a user can
> pass its bearer-token to tiller via armada using the CLI.
>
> I will be proposing this upstream to armada soon
>
>
>
> Thanks
>
> Shoaib Nasir
>
>
>
> *From:* EAGAN, SEAN [mailto:se136c at att.com <se136c at att.com>]
> *Sent:* Wednesday, January 09, 2019 1:02 PM
> *To:* Nasir, Shoaib; airship-discuss at lists.airshipit.org
> *Subject:* RE: Armada Compatibility with Keystone
>
>
>
>
>
> Shoaib,
>
>
>
> It looks like that feature of k8s-keystone-auth [0] is implemented by
> kubernetes/client-go [1] [2].
>
>
>
> kubernetes/client-go is used by helm so that’s why it works there.
>
>
>
> However, armada is using kubernetes-client/python, and as far as I can
> tell this feature is not implemented there. Unless I am missing it.
>
> I am not sure what the easiest path to getting that implemented will be.
>
>
>
> [0]:
> https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/using-keystone-webhook-authenticator-and-authorizer.md#clientkubectl-configuration
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_kubernetes_cloud-2Dprovider-2Dopenstack_blob_master_docs_using-2Dkeystone-2Dwebhook-2Dauthenticator-2Dand-2Dauthorizer.md-23clientkubectl-2Dconfiguration&d=DwQFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=qVl9v7EluGfsovbPYxi9J0-eSktkyFZ-inq7sKmgtWY&s=v7fsUjtnEjd10n9wUMyCRFRvFMCvhs4kpYCKfJn3NfE&e=>
>
> [1]:
> https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/using-client-keystone-auth.md
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_kubernetes_cloud-2Dprovider-2Dopenstack_blob_master_docs_using-2Dclient-2Dkeystone-2Dauth.md&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=qVl9v7EluGfsovbPYxi9J0-eSktkyFZ-inq7sKmgtWY&s=OcvR-HBkecGK7QMmr-UCS5zU5dlxeSHg1LbvHgfR6Es&e=>
>
> [2]: https://github.com/kubernetes/kubernetes/pull/64346
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_kubernetes_kubernetes_pull_64346&d=DwQFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=qVl9v7EluGfsovbPYxi9J0-eSktkyFZ-inq7sKmgtWY&s=DIUfb2shPrZdP8mQ3IxLfbjPIEV-f9WI-oWGJ9VoMHY&e=>
>
>
>
> Sean Eagan
>
>
>
>
>
> *From:* Nasir, Shoaib <Shoaib.Nasir at windriver.com>
> *Sent:* Thursday, December 13, 2018 12:35 PM
> *To:* airship-discuss at lists.airshipit.org
> *Subject:* [Airship-discuss] Armada Compatibility with Keystone
>
>
>
> Hi
>
>
>
> I have a kubernetes cluster that is setup with Keystone as its
> Authentication/Authorization Backend (via the k8s-keystone-auth Webhook pod)
>
> With that, any application such as helm now needs to pass user credentials
> or a keystone token when talking to the kubernetes-apiserver
>
>
>
> If I set up env OS_ variables and use any helm commands (such as helm
> init, or helm lists etc), everything works fine as these OS_ user
> credentials are authorized via the kubernetes keystone backend
>
>
>
> However, I run into problems with Armada. After bringing up the armada
> docker container, and logging into it to execute any armada apply commands,
> and using the –token argument to pass along a valid keystone token, armada
> still fails to access the kubernetes cluster.”User “system:anonymous”
> suggests that the token that armada cli gets, does not pass along to the
> kubernetes-apiserver.
>
> I’m very new to armada and need some assistance in figuring out how to
> setup the armada container or any other configurations required to make it
> pass the keystone token to the kubernetes-apiserver.
>
>
>
> Each and every armada command fails similar to the one below. Any
> assistance in this regard would be greatly appreciated ?
>
>
>
> *armada at c7f42c9263ad:~$ armada --token $TOKEN --debug apply*
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli [-] Caught unexpected
> exception: kubernetes.client.rest.ApiException: (403)
>
> Reason: Forbidden
>
> HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 13 Dec 2018 18:10:13
> GMT', 'Content-Length': '265', 'Content-Type': 'application/json',
> 'X-Content-Type-Options': 'nosniff'})
>
> HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods
> is forbidden: User \"system:anonymous\" cannot list resource \"pods\" in
> API group \"\" in the namespace
> \"kube-system\"","reason":"Forbidden","details":{"kind":"pods"},"code":403}
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli Traceback (most recent call
> last):
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli   File
> "/usr/local/lib/python3.5/site-packages/armada/cli/__init__.py", line 39,
> in safe_invoke
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli     self.invoke()
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli   File
> "/usr/local/lib/python3.5/site-packages/armada/cli/apply.py", line 215, in
> invoke
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli
> target_manifest=self.target_manifest)
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli   File
> "/usr/local/lib/python3.5/site-packages/armada/handlers/armada.py", line
> 103, in __init__
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli     dry_run=dry_run)
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli   File
> "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line
> 95, in __init__
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli     self.channel =
> self.get_channel()
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli   File
> "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line
> 117, in get_channel
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli     tiller_ip =
> self._get_tiller_ip()
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli   File
> "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line
> 163, in _get_tiller_ip
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli     pod =
> self._get_tiller_pod()
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli   File
> "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line
> 141, in _get_tiller_pod
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli
> CONF.tiller_pod_labels).items
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli   File
> "/usr/local/lib/python3.5/site-packages/armada/handlers/k8s.py", line 215,
> in get_namespace_pod
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli     namespace,
> label_selector=label_selector)
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli   File
> "/usr/local/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py",
> line 12310, in list_namespaced_pod
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli     (data) =
> self.list_namespaced_pod_with_http_info(namespace, **kwargs)
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli   File
> "/usr/local/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py",
> line 12413, in list_namespaced_pod_with_http_info
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli
> collection_formats=collection_formats)
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli   File
> "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py",
> line 321, in call_api
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli     _return_http_data_only,
> collection_formats, _preload_content, _request_timeout)
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli   File
> "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py",
> line 155, in __call_api
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli
> _request_timeout=_request_timeout)
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli   File
> "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py",
> line 342, in request
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli     headers=headers)
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli   File
> "/usr/local/lib/python3.5/site-packages/kubernetes/client/rest.py", line
> 231, in GET
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli     query_params=query_params)
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli   File
> "/usr/local/lib/python3.5/site-packages/kubernetes/client/rest.py", line
> 222, in request
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli     raise
> ApiException(http_resp=r)
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli
> kubernetes.client.rest.ApiException: (403)
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli Reason: Forbidden
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli HTTP response headers:
> HTTPHeaderDict({'Date': 'Thu, 13 Dec 2018 18:10:13 GMT', 'Content-Length':
> '265', 'Content-Type': 'application/json', 'X-Content-Type-Options':
> 'nosniff'})
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli HTTP response body:
> {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods
> is forbidden: User \"system:anonymous\" cannot list resource \"pods\" in
> API group \"\" in the namespace
> \"kube-system\"","reason":"Forbidden","details":{"kind":"pods"},"code":403}
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli
>
> 2018-12-13 18:10:13.656 72 ERROR armada.cli
>
>
>
>
>
>
> _______________________________________________
> Airship-discuss mailing list
> Airship-discuss at lists.airshipit.org
> http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.airshipit.org/pipermail/airship-discuss/attachments/20190117/de3e9e5d/attachment-0001.html>