Mapstack Details

YAML data source

Structure

When a single YAML file is loaded, it should return a configuration dictionary structured as follows:

• values provides the actual formula configuration, representing a layer

• strategy (optional) allows to influence how this layer is merged with the previous ones. Please see slsutil.merge for available values.

• merge_lists (optional) specifies whether lists in this layer should be appended to lists from previous layers or override them.

# A YAML data source showcasing defaults.
strategy: smart
merge_lists: false
values: {}

Templating

YAML data sources are always rendered using the jinja renderer, meaning you have full access to all of Salt’s functionality.

Hint

This is true for both .yaml.jinja and plain .yaml files.

Additionally, the following variables are provided:

• tplroot: The formula root directory relative to the Salt fileserver root.

• mapdata: Merged configuration from the previous layers.

• custom_data: A custom mapping passed by the caller and used by the U@ matcher.

post-map.jinja

This template, relative to the formula root, is executed before returning the layered configuration. It should modify mapdata in place.

Suppose a formula installs borg from GitHub releases. It allows to set the version to latest, but must know which specific version corresponds latest to correctly identify needed changes. A salt://borg/post-map.jinja file could check whether mapdata.version is set to latest, and if it is, query the GitHub API to discover the specific version and replace latest with it:

{%- if mapdata.version == "latest" %}
{%-   set latest_version = salt["http.query"](mapdata.lookup.latest_release_uri, decode=true, decode_type="json").dict.latest.version %}
{%-   do mapdata.update({"version": latest_version}) %}
{%- endif %}

Meta configuration

Formula authors and users can influence many aspects of the layering process:

• Default layering configuration can be passed to map.data directly.

• Formulae can provide custom map_jinja.yaml files.

• Users can add custom formula-specific or global map_jinja.yaml files.

map_jinja.yaml

Before starting the configuration layering process, meta configuration is loaded using the same process. By default, a formula named vault would aggregate the following paths into the meta configuration:

• salt://parameters/map_jinja.yaml

• salt://parameters/map_jinja.yaml.jinja

• salt://vault/parameters/map_jinja.yaml

• salt://vault/parameters/map_jinja.yaml.jinja

Since it uses the regular layering process, the root structure should follow the YAML data source format. Inside values, all optional parameters to saltext.formula.modules.map.data() can be overridden.

# A map_jinja.yaml file showcasing defaults.
strategy: smart
merge_lists: false
values:
  config_get_strategy: null
  default_merge_strategy: null
  default_merge_lists: false
  parameter_dirs:
    - {{ tplroot }}/parameters
  post_map: post-map.jinja
  post_map_template: jinja
  sources:
    - Y!G@osarch
    - Y!G@os_family
    - Y!G@os
    - Y!G@osfinger
    - C@{{ tplroot }}
    - Y!G@id

Layer definition

A configuration layer is specified as a list of one or more matcher definitions that describe where to derive the configuration from and how to derive it.

Added in version 0.3.0: Multiple matcher definitions can be chained via the | separator. See Data source chains.

Spec

A data source describes a single configuration layer by chaining one or more matchers: [Y!]<single_def_0>[|<single_def_1>[|...]]

Y! prefix

Indicates that the returns of the following matchers should be used to render paths to parameter files. They are rendered using Jinja and loaded as YAML.

single_def

A single matcher definition. Multiple ones are separated by a |.

Single matcher spec

A matcher definition references a lookup context and defines a lookup key: [<type>[:<option>[:<delimiter>]]@]<query>

type

The matcher lookup context. Defaults to C.

C

Query config.get and use its return as this layer’s configuration.

G

Query grains.get and use its return as this layer’s configuration.

I

Query pillar.get and use its return as this layer’s configuration.

P

Specify that this query is a static one (represents a static path, technically: identity lookup).

M

Traverse the currently rendered configuration. Mostly useful in chains.

U

Traverse the dictionary passed by the caller as custom_dict. Useful in chains or when enforcing specific overrides.

option

An option, valid values can depend on the type.

This can be set to SUB, which nests the result inside the key defined by query instead of merging it into the stack root.

delimiter

The delimiter used in the query to separate nested keys. Defaults to :.

Hint

Specifying a custom delimiter can be important when loading YAML files via the Y! prefix on Windows minions: The rendered YAML path contains the query value and the default delimiter (:) is disallowed in paths on Windows. Suggested replacement: !

query

The value to look up with type.

Examples

I@mysql

Run salt["pillar.get"]("mysql") and merge the resulting dictionary into the stack.

Y!C@roles

Run salt["config.get"]("roles"), build a list of paths, render and load the files and merge the resulting dictionary into the stack.

Shorthands: Y!roles, roles

For example, if a minion has a pillar value roles containing [db, db_master], the following files relative to configurable directories are tried:

• roles/db.yaml

• roles/db.yaml.jinja

• roles/db_master.yaml

• roles/db_master.yaml.jinja

Note

Notice how lists are expanded into separate paths. The same works for dictionary keys.

Y!I@roles

Does the same as Y!C@roles/Y!roles, but restricts the value source to the pillar.

Y!P@defaults.yaml

Don’t query metadata, always load this static path as a YAML data source.

Y!G::!@selinux!enabled

Run salt["config.get"]("selinux!enabled", delimiter="!"), build a list of paths, render and load the files and merge the resulting dictionary into the stack. If SELinux is enabled on the minion, the tried paths are:

• selinux!enabled/True.yaml

• selinux!enabled/True.yaml.jinja

Y!G@os_family|I@roles

See YAML chaining.

C@tplroot:variant_defaults|M@variant

See raw chaining.

Differences to template-formula with libmapstack.jinja

Breaking

• The default slsutil.merge and config.get merge strategy was looked up via salt["config.get"]("{tplroot}:strategy"). This behavior was dropped in favor of configuring this in map_jinja.yaml only. This keeps formula configuration in one place and allows to specify both strategies separately from each other.

Added features

• Cached configuration, reducing rendering times significantly

• All meta configuration parameters are read from map_jinja.yaml

• The name of post-map.jinja and its template language are configurable

• Added U (custom data)/P (static path)/M (mapdata) matchers

• Allowed to chain multiple matchers to load parameter paths that vary in two or more variables

• Allowed to chain multiple matchers to filter raw (non-YAML) matcher returns

Changed

• An obvious file data source like defaults.yaml was first tried via salt["config.get"]. This has been removed.

• If a data source query did not yield any results, it was tried as a YAML file path instead. This has been removed.

• The Y: type matcher has been deprecated. The Y! prefix should be used instead.