[Airship-discuss] [SIG-YAML] Robust variable substitution

[Dmitry Ukov]

Hi Matt

> Define variables+values in regular CRDs/Kustomize resources (rather than
standalone transformer files)
We can do it only with default implementation of 'vars'

> Use normal Kustomize layering to override variable definitions an
arbitrary number of times (e.g. function, composite, type levels)
Default vars can not be refined multiple times due to this piece of code
[0] which is used  here [1]

> Wait till layering is fully rendered before doing variable substitution
This is actually how it's implemented for default 'vars' [1]. Unfortunately
kustomize transformer plugins are executed at resource accumulation phase.
If resource is a directory new instance of KustTarget object is created [2]
so there is no connection between parent and child plugins. Therefore we
have 2 options:
1. put all variables at highest possible layer to avoid overrides
2. move common parts of functions to template without defined vars and
include this template to child function which has defined variables. This
actually leads to some portion of code duplication

> Avoid deploying these variable catalog CRDs to the cluster, via e.g. via
an “airshipit.org/no-deploy” label
Not an issue and can be implemented as a part of document filtering.


[0]
https://github.com/kubernetes-sigs/kustomize/blob/v3.2.0/pkg/types/var.go#L142
[1]
https://github.com/kubernetes-sigs/kustomize/blob/v3.2.0/pkg/target/kusttarget.go#L243
[2]
https://github.com/kubernetes-sigs/kustomize/blob/v3.2.0/pkg/target/kusttarget.go#L327

On Wed, Feb 12, 2020 at 10:04 PM MCEUEN, MATT <MM9745 at att.com> wrote:

> Team, I wanted to continue the discussion around Kustomize-based variable
> substitution from Monday’s SIG-YAML design call.
>
>
>
> There are two main Deckhand-like features we need out of Kustomize:
> Layering and Substitution.
>
>    - Layering is something Kustomize does really well, via strategic
>    merge.
>    - Substitution is what's presenting some challenges, between custom
>    plugins being unable to do late binding on variables (i.e., you only get
>    one shot at a tree substitution using our VarReference plugin), and the
>    fact that the Kustomize team is not really interested in enhancing variable
>    capabilities overall.
>
>
>
> For one-shot-tree-substitution:  can we reframe the substitution problem
> as a layering problem?  If we:
>
>    1. Define variables+values in regular CRDs/Kustomize resources (rather
>    than standalone transformer files)
>    2. Use normal Kustomize layering to override variable definitions an
>    arbitrary number of times (e.g. function, composite, type levels)
>    3. Wait till layering is fully rendered before doing variable
>    substitution
>    4. Avoid deploying these variable catalog CRDs to the cluster, via
>    e.g. via an “airshipit.org/no-deploy” label
>
>
>
> Then I think we would be able to accomplish multiple overrides for YAML
> tree variables, and recursive variable references, at the same time.
>
>
>
> #3 above then would require either a new plugin (Dmitry: does that sound
> possible?) or some kind of post-Kustomize step to fill in variables.
>
>
>
> Am I missing anything?
>
>
>
> Thanks,
>
> Matt
> _______________________________________________
> Airship-discuss mailing list
> Airship-discuss at lists.airshipit.org
> http://lists.airshipit.org/cgi-bin/mailman/listinfo/airship-discuss



-- 
Kind regards
Dmitry Ukov
Principal Deployment Engineer
Mirantis, Inc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.airshipit.org/pipermail/airship-discuss/attachments/20200213/9a3b5758/attachment.html>