Armada Compatibility with Keystone
Hi
I have a kubernetes cluster that is setup with Keystone as its Authentication/Authorization Backend (via the k8s-keystone-auth Webhook pod) With that, any application such as helm now needs to pass user credentials or a keystone token when talking to the kubernetes-apiserver
If I set up env OS_ variables and use any helm commands (such as helm init, or helm lists etc), everything works fine as these OS_ user credentials are authorized via the kubernetes keystone backend
However, I run into problems with Armada. After bringing up the armada docker container, and logging into it to execute any armada apply commands, and using the -token argument to pass along a valid keystone token, armada still fails to access the kubernetes cluster."User "system:anonymous" suggests that the token that armada cli gets, does not pass along to the kubernetes-apiserver. I'm very new to armada and need some assistance in figuring out how to setup the armada container or any other configurations required to make it pass the keystone token to the kubernetes-apiserver.
Each and every armada command fails similar to the one below. Any assistance in this regard would be greatly appreciated ?
armada@c7f42c9263ad:~$ armada --token $TOKEN --debug apply 2018-12-13 18:10:13.656 72 ERROR armada.cli [-] Caught unexpected exception: kubernetes.client.rest.ApiException: (403) Reason: Forbidden HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 13 Dec 2018 18:10:13 GMT', 'Content-Length': '265', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff'}) HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" in the namespace "kube-system"","reason":"Forbidden","details":{"kind":"pods"},"code":403} 2018-12-13 18:10:13.656 72 ERROR armada.cli Traceback (most recent call last): 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/__init__.py", line 39, in safe_invoke 2018-12-13 18:10:13.656 72 ERROR armada.cli self.invoke() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/apply.py", line 215, in invoke 2018-12-13 18:10:13.656 72 ERROR armada.cli target_manifest=self.target_manifest) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/armada.py", line 103, in __init__ 2018-12-13 18:10:13.656 72 ERROR armada.cli dry_run=dry_run) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 95, in __init__ 2018-12-13 18:10:13.656 72 ERROR armada.cli self.channel = self.get_channel() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 117, in get_channel 2018-12-13 18:10:13.656 72 ERROR armada.cli tiller_ip = self._get_tiller_ip() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 163, in _get_tiller_ip 2018-12-13 18:10:13.656 72 ERROR armada.cli pod = self._get_tiller_pod() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 141, in _get_tiller_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli CONF.tiller_pod_labels).items 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/k8s.py", line 215, in get_namespace_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli namespace, label_selector=label_selector) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py", line 12310, in list_namespaced_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli (data) = self.list_namespaced_pod_with_http_info(namespace, **kwargs) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py", line 12413, in list_namespaced_pod_with_http_info 2018-12-13 18:10:13.656 72 ERROR armada.cli collection_formats=collection_formats) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 321, in call_api 2018-12-13 18:10:13.656 72 ERROR armada.cli _return_http_data_only, collection_formats, _preload_content, _request_timeout) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 155, in __call_api 2018-12-13 18:10:13.656 72 ERROR armada.cli _request_timeout=_request_timeout) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 342, in request 2018-12-13 18:10:13.656 72 ERROR armada.cli headers=headers) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/rest.py", line 231, in GET 2018-12-13 18:10:13.656 72 ERROR armada.cli query_params=query_params) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/rest.py", line 222, in request 2018-12-13 18:10:13.656 72 ERROR armada.cli raise ApiException(http_resp=r) 2018-12-13 18:10:13.656 72 ERROR armada.cli kubernetes.client.rest.ApiException: (403) 2018-12-13 18:10:13.656 72 ERROR armada.cli Reason: Forbidden 2018-12-13 18:10:13.656 72 ERROR armada.cli HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 13 Dec 2018 18:10:13 GMT', 'Content-Length': '265', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff'}) 2018-12-13 18:10:13.656 72 ERROR armada.cli HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" in the namespace "kube-system"","reason":"Forbidden","details":{"kind":"pods"},"code":403} 2018-12-13 18:10:13.656 72 ERROR armada.cli 2018-12-13 18:10:13.656 72 ERROR armada.cli 2018-12-13 18:10:13.656 72 ERROR armada.cli
Shoaib,
It looks like that feature of k8s-keystone-auth [0] is implemented by kubernetes/client-go [1] [2].
kubernetes/client-go is used by helm so that's why it works there.
However, armada is using kubernetes-client/python, and as far as I can tell this feature is not implemented there. Unless I am missing it. I am not sure what the easiest path to getting that implemented will be.
[0]: https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/usin... [1]: https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/usin... [2]: https://github.com/kubernetes/kubernetes/pull/64346
Sean Eagan
From: Nasir, Shoaib Shoaib.Nasir@windriver.com Sent: Thursday, December 13, 2018 12:35 PM To: airship-discuss@lists.airshipit.org Subject: [Airship-discuss] Armada Compatibility with Keystone
Hi
I have a kubernetes cluster that is setup with Keystone as its Authentication/Authorization Backend (via the k8s-keystone-auth Webhook pod) With that, any application such as helm now needs to pass user credentials or a keystone token when talking to the kubernetes-apiserver
If I set up env OS_ variables and use any helm commands (such as helm init, or helm lists etc), everything works fine as these OS_ user credentials are authorized via the kubernetes keystone backend
However, I run into problems with Armada. After bringing up the armada docker container, and logging into it to execute any armada apply commands, and using the -token argument to pass along a valid keystone token, armada still fails to access the kubernetes cluster."User "system:anonymous" suggests that the token that armada cli gets, does not pass along to the kubernetes-apiserver. I'm very new to armada and need some assistance in figuring out how to setup the armada container or any other configurations required to make it pass the keystone token to the kubernetes-apiserver.
Each and every armada command fails similar to the one below. Any assistance in this regard would be greatly appreciated ?
armada@c7f42c9263ad:~$ armada --token $TOKEN --debug apply 2018-12-13 18:10:13.656 72 ERROR armada.cli [-] Caught unexpected exception: kubernetes.client.rest.ApiException: (403) Reason: Forbidden HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 13 Dec 2018 18:10:13 GMT', 'Content-Length': '265', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff'}) HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" in the namespace "kube-system"","reason":"Forbidden","details":{"kind":"pods"},"code":403} 2018-12-13 18:10:13.656 72 ERROR armada.cli Traceback (most recent call last): 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/__init__.py", line 39, in safe_invoke 2018-12-13 18:10:13.656 72 ERROR armada.cli self.invoke() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/apply.py", line 215, in invoke 2018-12-13 18:10:13.656 72 ERROR armada.cli target_manifest=self.target_manifest) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/armada.py", line 103, in __init__ 2018-12-13 18:10:13.656 72 ERROR armada.cli dry_run=dry_run) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 95, in __init__ 2018-12-13 18:10:13.656 72 ERROR armada.cli self.channel = self.get_channel() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 117, in get_channel 2018-12-13 18:10:13.656 72 ERROR armada.cli tiller_ip = self._get_tiller_ip() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 163, in _get_tiller_ip 2018-12-13 18:10:13.656 72 ERROR armada.cli pod = self._get_tiller_pod() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 141, in _get_tiller_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli CONF.tiller_pod_labels).items 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/k8s.py", line 215, in get_namespace_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli namespace, label_selector=label_selector) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py", line 12310, in list_namespaced_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli (data) = self.list_namespaced_pod_with_http_info(namespace, **kwargs) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py", line 12413, in list_namespaced_pod_with_http_info 2018-12-13 18:10:13.656 72 ERROR armada.cli collection_formats=collection_formats) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 321, in call_api 2018-12-13 18:10:13.656 72 ERROR armada.cli _return_http_data_only, collection_formats, _preload_content, _request_timeout) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 155, in __call_api 2018-12-13 18:10:13.656 72 ERROR armada.cli _request_timeout=_request_timeout) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 342, in request 2018-12-13 18:10:13.656 72 ERROR armada.cli headers=headers) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/rest.py", line 231, in GET 2018-12-13 18:10:13.656 72 ERROR armada.cli query_params=query_params) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/rest.py", line 222, in request 2018-12-13 18:10:13.656 72 ERROR armada.cli raise ApiException(http_resp=r) 2018-12-13 18:10:13.656 72 ERROR armada.cli kubernetes.client.rest.ApiException: (403) 2018-12-13 18:10:13.656 72 ERROR armada.cli Reason: Forbidden 2018-12-13 18:10:13.656 72 ERROR armada.cli HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 13 Dec 2018 18:10:13 GMT', 'Content-Length': '265', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff'}) 2018-12-13 18:10:13.656 72 ERROR armada.cli HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" in the namespace "kube-system"","reason":"Forbidden","details":{"kind":"pods"},"code":403} 2018-12-13 18:10:13.656 72 ERROR armada.cli 2018-12-13 18:10:13.656 72 ERROR armada.cli 2018-12-13 18:10:13.656 72 ERROR armada.cli
Hi Sean
After digging further into the issue, I realized that armada does not support passing end users bearer-tokens. Armada can be authenticated by keystone for incoming API requests.
But in the scenario, where I want to use armada to bring up openstack-helm pods on a kubernetes cluster that's already setup to be authenticated by keystone using the k8s-keystone-auth Webhook application, armada is not compatible. Therefore, I have made some changes to the armada CLI such that a user can pass its bearer-token to tiller via armada using the CLI. I will be proposing this upstream to armada soon
Thanks Shoaib Nasir
From: EAGAN, SEAN [mailto:se136c@att.com] Sent: Wednesday, January 09, 2019 1:02 PM To: Nasir, Shoaib; airship-discuss@lists.airshipit.org Subject: RE: Armada Compatibility with Keystone
Shoaib,
It looks like that feature of k8s-keystone-auth [0] is implemented by kubernetes/client-go [1] [2].
kubernetes/client-go is used by helm so that's why it works there.
However, armada is using kubernetes-client/python, and as far as I can tell this feature is not implemented there. Unless I am missing it. I am not sure what the easiest path to getting that implemented will be.
[0]: https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/usin... [1]: https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/usin... [2]: https://github.com/kubernetes/kubernetes/pull/64346
Sean Eagan
From: Nasir, Shoaib Shoaib.Nasir@windriver.com Sent: Thursday, December 13, 2018 12:35 PM To: airship-discuss@lists.airshipit.org Subject: [Airship-discuss] Armada Compatibility with Keystone
Hi
I have a kubernetes cluster that is setup with Keystone as its Authentication/Authorization Backend (via the k8s-keystone-auth Webhook pod) With that, any application such as helm now needs to pass user credentials or a keystone token when talking to the kubernetes-apiserver
If I set up env OS_ variables and use any helm commands (such as helm init, or helm lists etc), everything works fine as these OS_ user credentials are authorized via the kubernetes keystone backend
However, I run into problems with Armada. After bringing up the armada docker container, and logging into it to execute any armada apply commands, and using the -token argument to pass along a valid keystone token, armada still fails to access the kubernetes cluster."User "system:anonymous" suggests that the token that armada cli gets, does not pass along to the kubernetes-apiserver. I'm very new to armada and need some assistance in figuring out how to setup the armada container or any other configurations required to make it pass the keystone token to the kubernetes-apiserver.
Each and every armada command fails similar to the one below. Any assistance in this regard would be greatly appreciated ?
armada@c7f42c9263ad:~$ armada --token $TOKEN --debug apply 2018-12-13 18:10:13.656 72 ERROR armada.cli [-] Caught unexpected exception: kubernetes.client.rest.ApiException: (403) Reason: Forbidden HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 13 Dec 2018 18:10:13 GMT', 'Content-Length': '265', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff'}) HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" in the namespace "kube-system"","reason":"Forbidden","details":{"kind":"pods"},"code":403} 2018-12-13 18:10:13.656 72 ERROR armada.cli Traceback (most recent call last): 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/__init__.py", line 39, in safe_invoke 2018-12-13 18:10:13.656 72 ERROR armada.cli self.invoke() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/apply.py", line 215, in invoke 2018-12-13 18:10:13.656 72 ERROR armada.cli target_manifest=self.target_manifest) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/armada.py", line 103, in __init__ 2018-12-13 18:10:13.656 72 ERROR armada.cli dry_run=dry_run) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 95, in __init__ 2018-12-13 18:10:13.656 72 ERROR armada.cli self.channel = self.get_channel() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 117, in get_channel 2018-12-13 18:10:13.656 72 ERROR armada.cli tiller_ip = self._get_tiller_ip() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 163, in _get_tiller_ip 2018-12-13 18:10:13.656 72 ERROR armada.cli pod = self._get_tiller_pod() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 141, in _get_tiller_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli CONF.tiller_pod_labels).items 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/k8s.py", line 215, in get_namespace_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli namespace, label_selector=label_selector) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py", line 12310, in list_namespaced_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli (data) = self.list_namespaced_pod_with_http_info(namespace, **kwargs) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py", line 12413, in list_namespaced_pod_with_http_info 2018-12-13 18:10:13.656 72 ERROR armada.cli collection_formats=collection_formats) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 321, in call_api 2018-12-13 18:10:13.656 72 ERROR armada.cli _return_http_data_only, collection_formats, _preload_content, _request_timeout) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 155, in __call_api 2018-12-13 18:10:13.656 72 ERROR armada.cli _request_timeout=_request_timeout) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 342, in request 2018-12-13 18:10:13.656 72 ERROR armada.cli headers=headers) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/rest.py", line 231, in GET 2018-12-13 18:10:13.656 72 ERROR armada.cli query_params=query_params) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/rest.py", line 222, in request 2018-12-13 18:10:13.656 72 ERROR armada.cli raise ApiException(http_resp=r) 2018-12-13 18:10:13.656 72 ERROR armada.cli kubernetes.client.rest.ApiException: (403) 2018-12-13 18:10:13.656 72 ERROR armada.cli Reason: Forbidden 2018-12-13 18:10:13.656 72 ERROR armada.cli HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 13 Dec 2018 18:10:13 GMT', 'Content-Length': '265', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff'}) 2018-12-13 18:10:13.656 72 ERROR armada.cli HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" in the namespace "kube-system"","reason":"Forbidden","details":{"kind":"pods"},"code":403} 2018-12-13 18:10:13.656 72 ERROR armada.cli 2018-12-13 18:10:13.656 72 ERROR armada.cli 2018-12-13 18:10:13.656 72 ERROR armada.cli
Looking forward to seeing what you have in mind, Shoaib. If you'd like to discuss in tomorrow's team meeting, feel free to add the topic to the etherpad:
https://etherpad.openstack.org/p/airship-meeting-2019-01-15
Thanks, Matt
From: Nasir, Shoaib Shoaib.Nasir@windriver.com Sent: Thursday, January 10, 2019 9:26 AM To: EAGAN, SEAN se136c@att.com; airship-discuss@lists.airshipit.org Subject: Re: [Airship-discuss] Armada Compatibility with Keystone
Hi Sean
After digging further into the issue, I realized that armada does not support passing end users bearer-tokens. Armada can be authenticated by keystone for incoming API requests.
But in the scenario, where I want to use armada to bring up openstack-helm pods on a kubernetes cluster that's already setup to be authenticated by keystone using the k8s-keystone-auth Webhook application, armada is not compatible. Therefore, I have made some changes to the armada CLI such that a user can pass its bearer-token to tiller via armada using the CLI. I will be proposing this upstream to armada soon
Thanks Shoaib Nasir
From: EAGAN, SEAN [mailto:se136c@att.com] Sent: Wednesday, January 09, 2019 1:02 PM To: Nasir, Shoaib; airship-discuss@lists.airshipit.orgmailto:airship-discuss@lists.airshipit.org Subject: RE: Armada Compatibility with Keystone
Shoaib,
It looks like that feature of k8s-keystone-auth [0] is implemented by kubernetes/client-go [1] [2].
kubernetes/client-go is used by helm so that's why it works there.
However, armada is using kubernetes-client/python, and as far as I can tell this feature is not implemented there. Unless I am missing it. I am not sure what the easiest path to getting that implemented will be.
[0]: https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/usin...https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_kubernetes_cloud-2Dprovider-2Dopenstack_blob_master_docs_using-2Dkeystone-2Dwebhook-2Dauthenticator-2Dand-2Dauthorizer.md-23clientkubectl-2Dconfiguration&d=DwQFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=qVl9v7EluGfsovbPYxi9J0-eSktkyFZ-inq7sKmgtWY&s=v7fsUjtnEjd10n9wUMyCRFRvFMCvhs4kpYCKfJn3NfE&e= [1]: https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/usin...https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_kubernetes_cloud-2Dprovider-2Dopenstack_blob_master_docs_using-2Dclient-2Dkeystone-2Dauth.md&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=qVl9v7EluGfsovbPYxi9J0-eSktkyFZ-inq7sKmgtWY&s=OcvR-HBkecGK7QMmr-UCS5zU5dlxeSHg1LbvHgfR6Es&e= [2]: https://github.com/kubernetes/kubernetes/pull/64346https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_kubernetes_kubernetes_pull_64346&d=DwQFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=qVl9v7EluGfsovbPYxi9J0-eSktkyFZ-inq7sKmgtWY&s=DIUfb2shPrZdP8mQ3IxLfbjPIEV-f9WI-oWGJ9VoMHY&e=
Sean Eagan
From: Nasir, Shoaib <Shoaib.Nasir@windriver.commailto:Shoaib.Nasir@windriver.com> Sent: Thursday, December 13, 2018 12:35 PM To: airship-discuss@lists.airshipit.orgmailto:airship-discuss@lists.airshipit.org Subject: [Airship-discuss] Armada Compatibility with Keystone
Hi
I have a kubernetes cluster that is setup with Keystone as its Authentication/Authorization Backend (via the k8s-keystone-auth Webhook pod) With that, any application such as helm now needs to pass user credentials or a keystone token when talking to the kubernetes-apiserver
If I set up env OS_ variables and use any helm commands (such as helm init, or helm lists etc), everything works fine as these OS_ user credentials are authorized via the kubernetes keystone backend
However, I run into problems with Armada. After bringing up the armada docker container, and logging into it to execute any armada apply commands, and using the -token argument to pass along a valid keystone token, armada still fails to access the kubernetes cluster."User "system:anonymous" suggests that the token that armada cli gets, does not pass along to the kubernetes-apiserver. I'm very new to armada and need some assistance in figuring out how to setup the armada container or any other configurations required to make it pass the keystone token to the kubernetes-apiserver.
Each and every armada command fails similar to the one below. Any assistance in this regard would be greatly appreciated ?
armada@c7f42c9263ad:~$ armada --token $TOKEN --debug apply 2018-12-13 18:10:13.656 72 ERROR armada.cli [-] Caught unexpected exception: kubernetes.client.rest.ApiException: (403) Reason: Forbidden HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 13 Dec 2018 18:10:13 GMT', 'Content-Length': '265', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff'}) HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" in the namespace "kube-system"","reason":"Forbidden","details":{"kind":"pods"},"code":403} 2018-12-13 18:10:13.656 72 ERROR armada.cli Traceback (most recent call last): 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/__init__.py", line 39, in safe_invoke 2018-12-13 18:10:13.656 72 ERROR armada.cli self.invoke() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/apply.py", line 215, in invoke 2018-12-13 18:10:13.656 72 ERROR armada.cli target_manifest=self.target_manifest) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/armada.py", line 103, in __init__ 2018-12-13 18:10:13.656 72 ERROR armada.cli dry_run=dry_run) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 95, in __init__ 2018-12-13 18:10:13.656 72 ERROR armada.cli self.channel = self.get_channel() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 117, in get_channel 2018-12-13 18:10:13.656 72 ERROR armada.cli tiller_ip = self._get_tiller_ip() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 163, in _get_tiller_ip 2018-12-13 18:10:13.656 72 ERROR armada.cli pod = self._get_tiller_pod() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 141, in _get_tiller_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli CONF.tiller_pod_labels).items 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/k8s.py", line 215, in get_namespace_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli namespace, label_selector=label_selector) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py", line 12310, in list_namespaced_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli (data) = self.list_namespaced_pod_with_http_info(namespace, **kwargs) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py", line 12413, in list_namespaced_pod_with_http_info 2018-12-13 18:10:13.656 72 ERROR armada.cli collection_formats=collection_formats) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 321, in call_api 2018-12-13 18:10:13.656 72 ERROR armada.cli _return_http_data_only, collection_formats, _preload_content, _request_timeout) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 155, in __call_api 2018-12-13 18:10:13.656 72 ERROR armada.cli _request_timeout=_request_timeout) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 342, in request 2018-12-13 18:10:13.656 72 ERROR armada.cli headers=headers) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/rest.py", line 231, in GET 2018-12-13 18:10:13.656 72 ERROR armada.cli query_params=query_params) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/rest.py", line 222, in request 2018-12-13 18:10:13.656 72 ERROR armada.cli raise ApiException(http_resp=r) 2018-12-13 18:10:13.656 72 ERROR armada.cli kubernetes.client.rest.ApiException: (403) 2018-12-13 18:10:13.656 72 ERROR armada.cli Reason: Forbidden 2018-12-13 18:10:13.656 72 ERROR armada.cli HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 13 Dec 2018 18:10:13 GMT', 'Content-Length': '265', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff'}) 2018-12-13 18:10:13.656 72 ERROR armada.cli HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" in the namespace "kube-system"","reason":"Forbidden","details":{"kind":"pods"},"code":403} 2018-12-13 18:10:13.656 72 ERROR armada.cli 2018-12-13 18:10:13.656 72 ERROR armada.cli 2018-12-13 18:10:13.656 72 ERROR armada.cli
Hi Matt
Sorry I was a bit late in getting to this message So far, I only added a change in the CLI to pass a user's bearertoken and didn't make any changes to the API. That can be added later if people need it. I will fix the Zuul errors and put a new patch set as soon as I do that. But in the meantime, do you have any suggestions/ feedback regarding what I have proposed ?
Thanks Shoaib
From: MCEUEN, MATT [mailto:MM9745@att.com] Sent: Monday, January 14, 2019 11:44 AM To: Nasir, Shoaib; EAGAN, SEAN; airship-discuss@lists.airshipit.org Subject: RE: Armada Compatibility with Keystone
Looking forward to seeing what you have in mind, Shoaib. If you'd like to discuss in tomorrow's team meeting, feel free to add the topic to the etherpad:
https://etherpad.openstack.org/p/airship-meeting-2019-01-15
Thanks, Matt
From: Nasir, Shoaib Shoaib.Nasir@windriver.com Sent: Thursday, January 10, 2019 9:26 AM To: EAGAN, SEAN se136c@att.com; airship-discuss@lists.airshipit.org Subject: Re: [Airship-discuss] Armada Compatibility with Keystone
Hi Sean
After digging further into the issue, I realized that armada does not support passing end users bearer-tokens. Armada can be authenticated by keystone for incoming API requests.
But in the scenario, where I want to use armada to bring up openstack-helm pods on a kubernetes cluster that's already setup to be authenticated by keystone using the k8s-keystone-auth Webhook application, armada is not compatible. Therefore, I have made some changes to the armada CLI such that a user can pass its bearer-token to tiller via armada using the CLI. I will be proposing this upstream to armada soon
Thanks Shoaib Nasir
From: EAGAN, SEAN [mailto:se136c@att.com] Sent: Wednesday, January 09, 2019 1:02 PM To: Nasir, Shoaib; airship-discuss@lists.airshipit.orgmailto:airship-discuss@lists.airshipit.org Subject: RE: Armada Compatibility with Keystone
Shoaib,
It looks like that feature of k8s-keystone-auth [0] is implemented by kubernetes/client-go [1] [2].
kubernetes/client-go is used by helm so that's why it works there.
However, armada is using kubernetes-client/python, and as far as I can tell this feature is not implemented there. Unless I am missing it. I am not sure what the easiest path to getting that implemented will be.
[0]: https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/usin...https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_kubernetes_cloud-2Dprovider-2Dopenstack_blob_master_docs_using-2Dkeystone-2Dwebhook-2Dauthenticator-2Dand-2Dauthorizer.md-23clientkubectl-2Dconfiguration&d=DwQFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=qVl9v7EluGfsovbPYxi9J0-eSktkyFZ-inq7sKmgtWY&s=v7fsUjtnEjd10n9wUMyCRFRvFMCvhs4kpYCKfJn3NfE&e= [1]: https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/usin...https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_kubernetes_cloud-2Dprovider-2Dopenstack_blob_master_docs_using-2Dclient-2Dkeystone-2Dauth.md&d=DwMFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=qVl9v7EluGfsovbPYxi9J0-eSktkyFZ-inq7sKmgtWY&s=OcvR-HBkecGK7QMmr-UCS5zU5dlxeSHg1LbvHgfR6Es&e= [2]: https://github.com/kubernetes/kubernetes/pull/64346https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_kubernetes_kubernetes_pull_64346&d=DwQFAg&c=LFYZ-o9_HUMeMTSQicvjIg&r=_C5hC_103uW491yNPPpNmA&m=qVl9v7EluGfsovbPYxi9J0-eSktkyFZ-inq7sKmgtWY&s=DIUfb2shPrZdP8mQ3IxLfbjPIEV-f9WI-oWGJ9VoMHY&e=
Sean Eagan
From: Nasir, Shoaib <Shoaib.Nasir@windriver.commailto:Shoaib.Nasir@windriver.com> Sent: Thursday, December 13, 2018 12:35 PM To: airship-discuss@lists.airshipit.orgmailto:airship-discuss@lists.airshipit.org Subject: [Airship-discuss] Armada Compatibility with Keystone
Hi
I have a kubernetes cluster that is setup with Keystone as its Authentication/Authorization Backend (via the k8s-keystone-auth Webhook pod) With that, any application such as helm now needs to pass user credentials or a keystone token when talking to the kubernetes-apiserver
If I set up env OS_ variables and use any helm commands (such as helm init, or helm lists etc), everything works fine as these OS_ user credentials are authorized via the kubernetes keystone backend
However, I run into problems with Armada. After bringing up the armada docker container, and logging into it to execute any armada apply commands, and using the -token argument to pass along a valid keystone token, armada still fails to access the kubernetes cluster."User "system:anonymous" suggests that the token that armada cli gets, does not pass along to the kubernetes-apiserver. I'm very new to armada and need some assistance in figuring out how to setup the armada container or any other configurations required to make it pass the keystone token to the kubernetes-apiserver.
Each and every armada command fails similar to the one below. Any assistance in this regard would be greatly appreciated ?
armada@c7f42c9263ad:~$ armada --token $TOKEN --debug apply 2018-12-13 18:10:13.656 72 ERROR armada.cli [-] Caught unexpected exception: kubernetes.client.rest.ApiException: (403) Reason: Forbidden HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 13 Dec 2018 18:10:13 GMT', 'Content-Length': '265', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff'}) HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" in the namespace "kube-system"","reason":"Forbidden","details":{"kind":"pods"},"code":403} 2018-12-13 18:10:13.656 72 ERROR armada.cli Traceback (most recent call last): 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/__init__.py", line 39, in safe_invoke 2018-12-13 18:10:13.656 72 ERROR armada.cli self.invoke() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/apply.py", line 215, in invoke 2018-12-13 18:10:13.656 72 ERROR armada.cli target_manifest=self.target_manifest) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/armada.py", line 103, in __init__ 2018-12-13 18:10:13.656 72 ERROR armada.cli dry_run=dry_run) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 95, in __init__ 2018-12-13 18:10:13.656 72 ERROR armada.cli self.channel = self.get_channel() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 117, in get_channel 2018-12-13 18:10:13.656 72 ERROR armada.cli tiller_ip = self._get_tiller_ip() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 163, in _get_tiller_ip 2018-12-13 18:10:13.656 72 ERROR armada.cli pod = self._get_tiller_pod() 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 141, in _get_tiller_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli CONF.tiller_pod_labels).items 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/k8s.py", line 215, in get_namespace_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli namespace, label_selector=label_selector) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py", line 12310, in list_namespaced_pod 2018-12-13 18:10:13.656 72 ERROR armada.cli (data) = self.list_namespaced_pod_with_http_info(namespace, **kwargs) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py", line 12413, in list_namespaced_pod_with_http_info 2018-12-13 18:10:13.656 72 ERROR armada.cli collection_formats=collection_formats) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 321, in call_api 2018-12-13 18:10:13.656 72 ERROR armada.cli _return_http_data_only, collection_formats, _preload_content, _request_timeout) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 155, in __call_api 2018-12-13 18:10:13.656 72 ERROR armada.cli _request_timeout=_request_timeout) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 342, in request 2018-12-13 18:10:13.656 72 ERROR armada.cli headers=headers) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/rest.py", line 231, in GET 2018-12-13 18:10:13.656 72 ERROR armada.cli query_params=query_params) 2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/rest.py", line 222, in request 2018-12-13 18:10:13.656 72 ERROR armada.cli raise ApiException(http_resp=r) 2018-12-13 18:10:13.656 72 ERROR armada.cli kubernetes.client.rest.ApiException: (403) 2018-12-13 18:10:13.656 72 ERROR armada.cli Reason: Forbidden 2018-12-13 18:10:13.656 72 ERROR armada.cli HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 13 Dec 2018 18:10:13 GMT', 'Content-Length': '265', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff'}) 2018-12-13 18:10:13.656 72 ERROR armada.cli HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" in the namespace "kube-system"","reason":"Forbidden","details":{"kind":"pods"},"code":403} 2018-12-13 18:10:13.656 72 ERROR armada.cli 2018-12-13 18:10:13.656 72 ERROR armada.cli 2018-12-13 18:10:13.656 72 ERROR armada.cli
Hey Shoaib!
I'll give your patch a look today and see if I have any feedback to offer.
Regards, Drew Walters
On Tue, Jan 15, 2019 at 11:59 AM Nasir, Shoaib Shoaib.Nasir@windriver.com wrote:
Hi Matt
Sorry I was a bit late in getting to this message
So far, I only added a change in the CLI to pass a user’s bearertoken and didn’t make any changes to the API. That can be added later if people need it.
I will fix the Zuul errors and put a new patch set as soon as I do that.
But in the meantime, do you have any suggestions/ feedback regarding what I have proposed ?
Thanks
Shoaib
*From:* MCEUEN, MATT [mailto:MM9745@att.com] *Sent:* Monday, January 14, 2019 11:44 AM *To:* Nasir, Shoaib; EAGAN, SEAN; airship-discuss@lists.airshipit.org *Subject:* RE: Armada Compatibility with Keystone
Looking forward to seeing what you have in mind, Shoaib. If you’d like to discuss in tomorrow’s team meeting, feel free to add the topic to the etherpad:
https://etherpad.openstack.org/p/airship-meeting-2019-01-15
Thanks, Matt
*From:* Nasir, Shoaib Shoaib.Nasir@windriver.com *Sent:* Thursday, January 10, 2019 9:26 AM *To:* EAGAN, SEAN se136c@att.com; airship-discuss@lists.airshipit.org *Subject:* Re: [Airship-discuss] Armada Compatibility with Keystone
Hi Sean
After digging further into the issue, I realized that armada does not support passing end users bearer-tokens.
Armada can be authenticated by keystone for incoming API requests.
But in the scenario, where I want to use armada to bring up openstack-helm pods on a kubernetes cluster that’s already setup to be authenticated by keystone using the k8s-keystone-auth Webhook application, armada is not compatible.
Therefore, I have made some changes to the armada CLI such that a user can pass its bearer-token to tiller via armada using the CLI.
I will be proposing this upstream to armada soon
Thanks
Shoaib Nasir
*From:* EAGAN, SEAN [mailto:se136c@att.com se136c@att.com] *Sent:* Wednesday, January 09, 2019 1:02 PM *To:* Nasir, Shoaib; airship-discuss@lists.airshipit.org *Subject:* RE: Armada Compatibility with Keystone
Shoaib,
It looks like that feature of k8s-keystone-auth [0] is implemented by kubernetes/client-go [1] [2].
kubernetes/client-go is used by helm so that’s why it works there.
However, armada is using kubernetes-client/python, and as far as I can tell this feature is not implemented there. Unless I am missing it.
I am not sure what the easiest path to getting that implemented will be.
Sean Eagan
*From:* Nasir, Shoaib Shoaib.Nasir@windriver.com *Sent:* Thursday, December 13, 2018 12:35 PM *To:* airship-discuss@lists.airshipit.org *Subject:* [Airship-discuss] Armada Compatibility with Keystone
Hi
I have a kubernetes cluster that is setup with Keystone as its Authentication/Authorization Backend (via the k8s-keystone-auth Webhook pod)
With that, any application such as helm now needs to pass user credentials or a keystone token when talking to the kubernetes-apiserver
If I set up env OS_ variables and use any helm commands (such as helm init, or helm lists etc), everything works fine as these OS_ user credentials are authorized via the kubernetes keystone backend
However, I run into problems with Armada. After bringing up the armada docker container, and logging into it to execute any armada apply commands, and using the –token argument to pass along a valid keystone token, armada still fails to access the kubernetes cluster.”User “system:anonymous” suggests that the token that armada cli gets, does not pass along to the kubernetes-apiserver.
I’m very new to armada and need some assistance in figuring out how to setup the armada container or any other configurations required to make it pass the keystone token to the kubernetes-apiserver.
Each and every armada command fails similar to the one below. Any assistance in this regard would be greatly appreciated ?
*armada@c7f42c9263ad:~$ armada --token $TOKEN --debug apply*
2018-12-13 18:10:13.656 72 ERROR armada.cli [-] Caught unexpected exception: kubernetes.client.rest.ApiException: (403)
Reason: Forbidden
HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 13 Dec 2018 18:10:13 GMT', 'Content-Length': '265', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" in the namespace "kube-system"","reason":"Forbidden","details":{"kind":"pods"},"code":403}
2018-12-13 18:10:13.656 72 ERROR armada.cli Traceback (most recent call last):
2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/__init__.py", line 39, in safe_invoke
2018-12-13 18:10:13.656 72 ERROR armada.cli self.invoke()
2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/cli/apply.py", line 215, in invoke
2018-12-13 18:10:13.656 72 ERROR armada.cli target_manifest=self.target_manifest)
2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/armada.py", line 103, in __init__
2018-12-13 18:10:13.656 72 ERROR armada.cli dry_run=dry_run)
2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 95, in __init__
2018-12-13 18:10:13.656 72 ERROR armada.cli self.channel = self.get_channel()
2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 117, in get_channel
2018-12-13 18:10:13.656 72 ERROR armada.cli tiller_ip = self._get_tiller_ip()
2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 163, in _get_tiller_ip
2018-12-13 18:10:13.656 72 ERROR armada.cli pod = self._get_tiller_pod()
2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/tiller.py", line 141, in _get_tiller_pod
2018-12-13 18:10:13.656 72 ERROR armada.cli CONF.tiller_pod_labels).items
2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/armada/handlers/k8s.py", line 215, in get_namespace_pod
2018-12-13 18:10:13.656 72 ERROR armada.cli namespace, label_selector=label_selector)
2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py", line 12310, in list_namespaced_pod
2018-12-13 18:10:13.656 72 ERROR armada.cli (data) = self.list_namespaced_pod_with_http_info(namespace, **kwargs)
2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/apis/core_v1_api.py", line 12413, in list_namespaced_pod_with_http_info
2018-12-13 18:10:13.656 72 ERROR armada.cli collection_formats=collection_formats)
2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 321, in call_api
2018-12-13 18:10:13.656 72 ERROR armada.cli _return_http_data_only, collection_formats, _preload_content, _request_timeout)
2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 155, in __call_api
2018-12-13 18:10:13.656 72 ERROR armada.cli _request_timeout=_request_timeout)
2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/api_client.py", line 342, in request
2018-12-13 18:10:13.656 72 ERROR armada.cli headers=headers)
2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/rest.py", line 231, in GET
2018-12-13 18:10:13.656 72 ERROR armada.cli query_params=query_params)
2018-12-13 18:10:13.656 72 ERROR armada.cli File "/usr/local/lib/python3.5/site-packages/kubernetes/client/rest.py", line 222, in request
2018-12-13 18:10:13.656 72 ERROR armada.cli raise ApiException(http_resp=r)
2018-12-13 18:10:13.656 72 ERROR armada.cli kubernetes.client.rest.ApiException: (403)
2018-12-13 18:10:13.656 72 ERROR armada.cli Reason: Forbidden
2018-12-13 18:10:13.656 72 ERROR armada.cli HTTP response headers: HTTPHeaderDict({'Date': 'Thu, 13 Dec 2018 18:10:13 GMT', 'Content-Length': '265', 'Content-Type': 'application/json', 'X-Content-Type-Options': 'nosniff'})
2018-12-13 18:10:13.656 72 ERROR armada.cli HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"pods is forbidden: User "system:anonymous" cannot list resource "pods" in API group "" in the namespace "kube-system"","reason":"Forbidden","details":{"kind":"pods"},"code":403}
2018-12-13 18:10:13.656 72 ERROR armada.cli
2018-12-13 18:10:13.656 72 ERROR armada.cli
2018-12-13 18:10:13.656 72 ERROR armada.cli
Airship-discuss mailing list Airship-discuss@lists.airshipit.org http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss
participants (4)
-
Drew Walters
-
EAGAN, SEAN
-
MCEUEN, MATT
-
Nasir, Shoaib