Skip to content

Configuration

This page describes how Huginn picks up its settings: command-line flags, HUGINN_* environment variables, and project defaults in the [tool.huginn] table of pyproject.toml. For the full option list of each command, see the CLI Reference.

Overview

Huginn has three configuration sources:

  • Command-line flags, such as --plan or --results-dir, for a single invocation.
  • HUGINN_* environment variables, such as HUGINN_PLAN, for a shell session or CI job.
  • [tool.huginn] in pyproject.toml, for defaults that belong to the repository: where the test plan, testbed and output directories live, logging, and which plugins are active.

A CLI flag overrides its environment variable, which overrides the [tool.huginn] key, which overrides the built-in default. See Precedence.

Execution behavior is not configured here. Concurrency comes from the strategy of each group or phase in the test plan, and connection timeouts and transport options come from each connection in the testbed.

Project defaults in [tool.huginn]

A repository can store its paths, logging and warning settings in [tool.huginn], so you don't have to pass -p and -t on every command:

[tool.huginn]
test_plan = "test_plan"
testbed = "inventory/lab.yaml"
parameters_dir = "state/parameters"
results_dir = "state/results"
output_dir = "state/artifacts"
log_file = "logs/huginn.log"
log_level = "INFO"

With this table in place, huginn run -m testing uses the plan, testbed and directories above.

Supported keys

Each key sets the default for one CLI option. A command uses a key only if it has that option.

Key CLI option Commands
test_plan -p/--plan run, validate, relearn, reconcile, prune, inject new/into
testbed -t/--testbed run, validate, relearn, execute
inventory_plugin -i/--inventory-plugin run, validate, relearn
parameters_dir --parameters-dir run, relearn, reconcile
results_dir --results-dir run, relearn, reconcile, prune
output_dir --output-dir run, relearn
log_file --log-file run, validate, relearn, reconcile, prune, execute
log_level --log-level run, validate, relearn, reconcile, prune, execute
unknown_key_warnings --no-unknown-key-warnings run, validate, relearn, reconcile, prune, execute, inject new/into

Every value is a string, except unknown_key_warnings, which is a TOML boolean (true or false, not quoted). log_level must be one of DEBUG, INFO, WARNING or ERROR, in any case. mode is not supported, because a learning default would silently overwrite baselines.

unknown_key_warnings = false turns off the warnings for keys that Huginn does not read in the test plan and testbed (see Test Plan Specification - Unknown keys). It is the default for --no-unknown-key-warnings, so HUGINN_NO_UNKNOWN_KEY_WARNINGS=false turns the warnings back on for one shell or CI job. true, or leaving the key out, keeps them on.

There is no key for the data model. Set data_model.path in the test plan, or pass --data-model on the command line.

Precedence

For each option, the first of these that is set wins:

  1. The CLI flag
  2. The HUGINN_* environment variable, for example HUGINN_PLAN or HUGINN_RESULTS_DIR
  3. The key in [tool.huginn]
  4. The built-in default, for example ./test_plan or ./results/

huginn <command> --help marks each default that comes from pyproject.toml, for example [default: (state/results from pyproject.toml)].

Discovery and path resolution

Huginn reads pyproject.toml from the current working directory only. It doesn't search parent directories, and there is no option to point at a different file. If the file or its [tool.huginn] table is missing, only the built-in defaults apply.

Relative paths resolve against the directory that contains pyproject.toml, which is the current working directory. Absolute paths are used as written.

Testbed and inventory plugin

testbed and inventory_plugin are mutually exclusive, the same as --testbed and --inventory-plugin:

  • Setting both keys in [tool.huginn] is an error.
  • If you pass one on the CLI or through its environment variable, it overrides the other key from [tool.huginn] for that run. For example, huginn run -m testing -i file:inventory/staging.yaml ignores a testbed key.
  • Passing both on the CLI, or one on the CLI and the other through its environment variable, is a usage error.

inventory_plugin takes the same <plugin>:<config> value as --inventory-plugin. The built-in file plugin loads a testbed file, for example file:inventory/lab.yaml. Other plugin names are looked up in the huginn.inventory entry point group, and the text after the colon is passed to the plugin as its configuration.

Errors

Huginn validates [tool.huginn] before running any command except version. The command prints the error and exits with code 1 if:

  • pyproject.toml is not valid TOML
  • [tool.huginn] or [tool.huginn.plugins] is not a table
  • the table has a key that is not in the table above or plugins, for example mode, parallel_tests or hooks
  • a value is not a non-empty string, or unknown_key_warnings is not a boolean
  • log_level is outside the accepted levels
  • both testbed and inventory_plugin are set

For example, a leftover mode key fails with:

ERROR: Unknown key(s) in [tool.huginn]: mode

[tool.huginn.plugins] is validated in the same way. The command also fails if:

  • the plugins table has a key other than brokers, reporters, hooks or config
  • brokers, reporters or hooks is not a list of non-empty strings
  • config is not a table, or a value inside it is not a table

An enabled key, from older documentation, fails with a hint to use the current keys:

ERROR: Unknown key(s) in [tool.huginn.plugins]: enabled. Use 'brokers', 'reporters' or 'hooks' instead of 'enabled'.

Plugins in [tool.huginn.plugins]

Huginn discovers plugins through Python entry points. [tool.huginn.plugins] limits which discovered plugins are active and passes options to them:

[tool.huginn.plugins]
brokers = ["ssh", "netconf"]
reporters = ["html"]
hooks = []
Key Type When absent Description
brokers list of strings Every discovered broker Connection brokers a run may use. Requiring a broker that is installed but not listed fails the run.
reporters list of strings Every discovered reporter Reporters that run after run and relearn. An empty list disables reporting.
hooks list of strings Every discovered hook Hook plugins that run and relearn call. An empty list disables all hooks; see Hooks.
config table of tables No options Per-plugin options, one sub-table per plugin name, for example [tool.huginn.plugins.config.html].

Each name matches an entry point name, not a package name. run and relearn resolve brokers, reporters and hooks through this table. validate resolves only the built-in file inventory plugin, and execute uses the built-in brokers directly.

Built-in plugins

Huginn registers these entry points itself:

Group Name Plugin
huginn.brokers ssh SSH broker for CLI commands
huginn.brokers http HTTP broker for REST APIs
huginn.brokers netconf NETCONF broker
huginn.inventory file Loads a testbed YAML file
huginn.reporters html HTML dashboard written under reports/, with latest/

An installed package adds plugins by registering entry points in the same groups. See Connection Broker - Plugin Discovery for how brokers are resolved.

Plugin options

config.<name> holds options for the plugin whose entry point is <name>:

[tool.huginn.plugins.config.html]
title = "DC1 lab"

Huginn passes the table to the plugin unchanged. A reporter receives it as the config argument of generate_report(), and a hook plugin as the config argument of its constructor. The plugin decides which options it reads. The built-in html reporter reads no options, so a config.html table is accepted but has no effect today.

Inventory plugins do not read config.<name>. They take their configuration from the text after the colon in --inventory-plugin or inventory_plugin.

Hooks

A hook plugin runs code at lifecycle points of huginn run and huginn relearn, such as before each test case or after the run. It can skip a phase, group or test case before it runs, or stop the whole run, which then exits with code 1. Hook plugins are classes registered in the huginn.hooks entry point group of an installed package:

# pyproject.toml of the package that provides the hook
[project.entry-points."huginn.hooks"]
change-window = "acme_hooks.change_window:ChangeWindowHook"

Every installed hook plugin is active by default. A project limits them with hooks under [tool.huginn.plugins] and passes them options with config.<name>:

[tool.huginn.plugins]
hooks = ["change-window"]

[tool.huginn.plugins.config.change-window]
calendar = "network-changes"

The hooks are loaded once per run. hooks = [] disables them all. huginn execute does not dispatch hooks. A hook that raises prints a warning and the run continues with the same exit code. Plugin options are passed as written: ${VAR} references are not expanded, so a plugin that needs a secret should take the name of an environment variable and read it itself.

See Hook Plugins for the protocol, the events and their payloads, skipping and aborting, and complete plugins for run notifications, a testbed lock and job telemetry.

Environment variables

Every HUGINN_* variable sets the CLI option of the same name on each command that has it. A flag given on the command line takes precedence over its variable, and the variable takes precedence over [tool.huginn].

Variable CLI option Commands
HUGINN_MODE -m/--mode run
HUGINN_PLAN -p/--plan run, validate, relearn, reconcile, prune, inject new, inject into
HUGINN_TESTBED -t/--testbed run, validate, relearn, execute
HUGINN_INVENTORY_PLUGIN -i/--inventory-plugin run, validate, relearn
HUGINN_DATA_MODEL -d/--data-model run, validate, relearn
HUGINN_TAGS --tags run, validate
HUGINN_EXCLUDE_TAGS --exclude-tags run, validate
HUGINN_SCENARIO --scenario run, validate, relearn, reconcile
HUGINN_PHASE --phase run, validate, relearn, reconcile
HUGINN_TEST_CASE_GROUP --test-case-group run, validate
HUGINN_TEST_ID --test-id run, validate
HUGINN_TEST_ID_PATTERN --test-id-pattern run, validate
HUGINN_RESULTS_DIR --results-dir run, relearn, reconcile, prune
HUGINN_PARAMETERS_DIR --parameters-dir run, relearn, reconcile
HUGINN_OUTPUT_DIR --output-dir run, relearn
HUGINN_DEBUG --debug run, validate, relearn, reconcile, prune, execute
HUGINN_LOG_LEVEL --log-level run, validate, relearn, reconcile, prune, execute
HUGINN_SHOW_LOGS --show-logs run, validate, relearn, reconcile, prune, execute
HUGINN_LOG_FILE --log-file run, validate, relearn, reconcile, prune, execute
HUGINN_NO_UNKNOWN_KEY_WARNINGS --no-unknown-key-warnings run, validate, relearn, reconcile, prune, execute, inject new, inject into

See CLI Reference - Environment variables for how boolean and repeatable values are written.

CLI overrides

Any flag overrides the environment and [tool.huginn] for one invocation. With the Complete Example below in place:

# Run a different plan against the staging testbed
huginn run -m testing -p plans/smoke/ -t inventory/staging.yaml

# Load the testbed through the file inventory plugin instead of the testbed key
huginn run -m testing -i file:inventory/staging.yaml

# Log at DEBUG for one learning run
huginn run -m learning --log-level DEBUG

# Use a different data model directory than the plan's data_model.path
huginn run -m testing -d nac/staging/

# Keep this run's results apart from the configured results_dir
HUGINN_RESULTS_DIR=/var/tmp/huginn-results huginn run -m testing

# Validate the plan against the staging testbed before a run
huginn validate -t inventory/staging.yaml

Complete Example

# pyproject.toml

[project]
name = "dc1-network-tests"
version = "0.1.0"
requires-python = ">=3.10"
dependencies = [
    "huginn-framework",
]

[tool.huginn]
test_plan = "plans/full-validation/"
testbed = "inventory/production.yaml"
parameters_dir = "state/parameters"
results_dir = "state/results"
output_dir = "state/artifacts"
log_file = "logs/huginn.log"
log_level = "INFO"

[tool.huginn.plugins]
brokers = ["ssh", "netconf"]
reporters = ["html"]

From the project root, huginn run -m testing then runs plans/full-validation/ against inventory/production.yaml, reads parameters from state/parameters/, writes results under state/results/, and logs to logs/huginn.log. Only the SSH and NETCONF brokers are available, and the HTML report is written under reports/.