config_syntax.md

MONAI Bundle Configuration

The monai.bundle module supports building Python-based workflows via structured configurations.

The main benefits are threefold:

• it provides good readability and usability by separating system parameter settings from the Python code.
• it describes workflow at a relatively high level and allows for different low-level implementations.
• learning paradigms at a higher level such as federated learning and AutoML can be decoupled from the component details.

Content:

• A basic example
• Syntax examples explained
  • @ to reference Python objects in configurations
  • $ to evaluate as Python expressions
  • % to textually replace configuration elements
  • _target_ (_disabled_, _desc_, _requires_, _mode_) to instantiate a Python object
• The command line interface
• Recommendations

A basic example

Components as part of a workflow can be specified using JSON or YAML syntax, for example, a network architecture definition could be stored in a demo_config.json file with the following content:

{
  "demo_net": {
    "_target_": "monai.networks.nets.BasicUNet",
    "spatial_dims": 3,
    "in_channels": 1,
    "out_channels": 2,
    "features": [16, 16, 32, 32, 64, 64]
  }
}

or alternatively, in YAML format (demo_config.yaml):

demo_net:
  _target_: monai.networks.nets.BasicUNet
  spatial_dims: 3
  in_channels: 1
  out_channels: 2
  features: [16, 16, 32, 32, 64, 64]

The configuration parser can instantiate the component as a Python object:

>>> from monai.bundle import ConfigParser
>>> config = ConfigParser()
>>> config.read_config("demo_config.json")
>>> net = config.get_parsed_content("demo_net", instantiate=True)
BasicUNet features: (16, 16, 32, 32, 64, 64).
>>> print(type(net))
<class 'monai.networks.nets.basic_unet.BasicUNet'>

or additionally, tune the input parameters then instantiate the component:

>>> config["demo_net"]["features"] = [32, 32, 32, 64, 64, 64]
>>> net = config.get_parsed_content("demo_net", instantiate=True)
BasicUNet features: (32, 32, 32, 64, 64, 64).

For more details on the ConfigParser API, please see monai.bundle.ConfigParser.

Syntax examples explained

A few characters and keywords are interpreted beyond the plain texts, here are examples of the syntax:

To reference Python objects in configurations

"@preprocessing::transforms::keys"

Description: @ character indicates a reference to another configuration value defined at preprocessing::transforms::keys. where :: indicates a sub-structure of this configuration file. (# is a synonym for ::, preprocessing#transforms#keys refers to the same object.)

"@preprocessing::1"

Description: 1 is referencing as an integer, which is used to index (zero-based indexing) the preprocessing sub-structure.

Relative reference is supported by starting the reference with #. For example, @#A is to use A at the same config structure level, and @##A refers to A at one level above.

To evaluate as Python expressions

"$print(42)"

Description: $ is a special character to indicate evaluating print(42) at runtime.

"$[i for i in @datalist]"

Description: Create a list at runtime using the values in datalist as input.

"$from torchvision.models import resnet18"

Description: $ followed by an import statement is handled slightly differently from the Python expressions. The imported module resnet18 will be available as a global variable to the other configuration sections. This is to simplify the use of external modules in the configuration.

The config expressions may use @ to reference other config items. For example, in $lambda x: x + @a + @b, @a and @b are references to other Python objects and are made available to the anonymous function as 'globals'. It's therefore possible to modify the Python objects within an expression, for example, $lambda x: @my_list.pop() + x will pop the last element from @my_list and add it to x.

To textually replace configuration elements

"%demo_config.json::demo_net::in_channels"

Description: % character indicates a macro to replace the current configuration element with the texts at demo_net::in_channels in the demo_config.json file. The replacement is done before instantiating or evaluating the components.

Instantiate a Python object

{
  "demo_name":{
    "_target_": "my.python.module.Class",
    "args1": "string",
    "args2": 42}
}

Description: This dictionary defines an object with a reference name demo_name, with an instantiable type specified at _target_ and with input arguments args1 and args2. This dictionary will be instantiated as a Pytorch object at runtime.

_target_ is a required key by monai bundle syntax for the Python object name. args1 and args2 should be compatible with the Python object to instantiate.

{
  "component_name": {
    "_target_": "my.module.Class",
    "_desc_": "this is a customized class which also triggers 'cudnn_opt' reference",
    "_requires_": "@cudnn_opt",
    "_disabled_": "true",
    "_mode_":  "default"}
}

Description: _requires_, _disabled_, _desc_, and _mode_ are optional keys.

• _requires_ specifies references (string starts with @) or Python expression that will be evaluated/instantiated before _target_ object is instantiated. It is useful when the component does not explicitly depend on the other ConfigItems via its arguments, but requires the dependencies to be instantiated/evaluated beforehand.
• _disabled_ specifies a flag to indicate whether to skip the instantiation.
• _desc_ can be used for providing free text descriptions.
• _mode_ specifies the operating mode when the component is instantiated or the callable is called. it currently supports the following values:
  • "default" (default) -- return the return value of _target_(**kwargs)
  • "partial" -- return a partial function of functools.partial(_target_, **kwargs) (this is often useful when some portion of the full set of arguments are supplied to the _target_, and the user wants to call it with additional arguments later).
  • "debug" -- execute with debug prompt and return the return value of pdb.runcall(_target_, **kwargs), see also pdb.runcall.

The command line interface

In addition to the Pythonic APIs, a few command line interfaces (CLI) are provided to interact with the bundle. The primary usage is:

python -m monai.bundle COMMANDS

where COMMANDS is one of the following: run, verify_metadata, ckpt_export, ... (please see python -m monai.bundle --help for a list of available options).

The CLI supports flexible use cases, such as overriding configs at runtime and predefining arguments in a file. To display a usage page for a command, for example run:

python -m monai.bundle run -- --help

The support is provided by Python Fire, please make sure the optional dependency is installed, for example, using pip install monai[fire] or pip install fire. Details on the CLI argument parsing is provided in the Python Fire Guide.

Recommendations

• Both YAML and JSON are supported, but the advanced features of these formats are not supported.
• Using meaningful names for the configuration elements can improve the readability.
• While it is possible to build complex configurations with the bundle syntax, simple structures with sparse uses of expressions or references are preferred.
• For $import <module> in the configuration, please make sure there are instructions for the users to install the <module> if it is not a (optional) dependency of MONAI.
• As #, ::, and $ might be interpreted differently by the shell or CLI tools, may need to add escape characters or quotes for them in the command line, like: "\$torch.device('cuda:1')", "'train_part#trainer'".
• For more details and examples, please see the tutorials.