Skip to content

Add ability to use variables from grid file in input file expressions#3222

Open
ZedThree wants to merge 21 commits intonextfrom
field-variable-expression
Open

Add ability to use variables from grid file in input file expressions#3222
ZedThree wants to merge 21 commits intonextfrom
field-variable-expression

Conversation

@ZedThree
Copy link
Copy Markdown
Member

@ZedThree ZedThree commented Dec 2, 2025

This adds the ability to read fields (2D and 3D) and doubles* from the grid file (mesh:file) and use them in expressions in the input file, for example to use coordinates directly from the grid generator:

[input:grid_variables]
rho = field2d
theta = field2d
scale = boutreal

[mesh]
B = (scale / rho) * cos(theta)

* ints can also be read, but the expression parser only handles doubles


This could also be used for something like region labels to the input file by having hypnotoad make a field which is 1.0 in that region and 0.0 elsewhere:

region_pfr = 1.0 if is_in_pfr else 0.0

and using like

[input:grid_variables]
region_pfr = field2d

[some_source]
function = region_pfr * cos(x) * whatever

We could probably also do this in BOUT++ directly, making fields for each boundary region.

Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

return std::make_shared<FieldValuePtr>(ptr);
}

BOUT_ENUM_CLASS(GridVariableFunction, field3d, field2d, boutreal);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: enum 'GridVariableFunction' uses a larger base type ('int', size: 4 bytes) than necessary for its value set, consider using 'std::uint8_t' (1 byte) as the base type to reduce its size [performance-enum-size]

BOUT_ENUM_CLASS(GridVariableFunction, field3d, field2d, boutreal);
                ^

return std::make_shared<FieldValuePtr>(ptr);
}

BOUT_ENUM_CLASS(GridVariableFunction, field3d, field2d, boutreal);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'GridVariableFunctionFromString' can be made static or moved into an anonymous namespace to enforce internal linkage [misc-use-internal-linkage]

BOUT_ENUM_CLASS(GridVariableFunction, field3d, field2d, boutreal);
^

expanded from here

return std::make_shared<FieldValuePtr>(ptr);
}

BOUT_ENUM_CLASS(GridVariableFunction, field3d, field2d, boutreal);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: parameter 'UNUSED_similar_to' is unused [misc-unused-parameters]

BOUT_ENUM_CLASS(GridVariableFunction, field3d, field2d, boutreal);
^
Additional context

include/bout/bout_enum_class.hxx:98: expanded from macro 'BOUT_ENUM_CLASS'

  inline enumname Options::as<enumname>(const enumname&) const {               \
                                                       ^

class GridVariable : public FieldGenerator {
public:
GridVariable(T var, std::string name)
: variable(std::move(var)), name(std::move(name)) {}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: no header providing "std::move" is directly included [misc-include-cleaner]

src/field/fieldgenerators.hxx:18:

+ #include <utility>

}

FieldGeneratorPtr clone(const std::list<FieldGeneratorPtr> args) override {
if (args.size() != 0) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: the 'empty' method should be used to check for emptiness instead of 'size' [readability-container-size-empty]

Suggested change
if (args.size() != 0) {
if (!args.empty()) {
Additional context

/usr/include/c++/13/bits/stl_list.h:1142: method 'list'::empty() defined here

      empty() const _GLIBCXX_NOEXCEPT
      ^

dschwoerer
dschwoerer previously approved these changes Feb 4, 2026
Copy link
Copy Markdown
Contributor

@dschwoerer dschwoerer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks for cleaning up 👍

@ZedThree ZedThree dismissed dschwoerer’s stale review February 4, 2026 12:24

The merge-base changed after approval.

dschwoerer
dschwoerer previously approved these changes Feb 4, 2026
@ZedThree ZedThree dismissed dschwoerer’s stale review February 4, 2026 13:48

The merge-base changed after approval.

dschwoerer
dschwoerer previously approved these changes Feb 27, 2026
@ZedThree ZedThree dismissed dschwoerer’s stale review February 27, 2026 11:27

The merge-base changed after approval.

bendudson
bendudson previously approved these changes Mar 10, 2026
Copy link
Copy Markdown
Contributor

@bendudson bendudson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ZedThree ! This looks really useful. I think this is good as-is, but for future if this becomes widely used:

  1. These expressions are in the global scope rather than being in the mesh namespace.
  2. A nice-to-have might be renaming imports analogous to "import x as y" to avoid name conflicts.
    These may over-complicate things, so I think this simplest implementation is good.

@ZedThree ZedThree dismissed bendudson’s stale review March 10, 2026 18:45

The merge-base changed after approval.

The constructor argument `localmesh` can be null but in that
case `fieldmesh` is set to `bout::globals::mesh`.
`BoutMesh::load` creates a FieldFactory to read variables such as
mesh sizes. This then tries to load grid variables, which in turn
starts trying to create coordinates, use regions etc., none of which
has been constructed at that point.

This defers reading until it's needed, at which point it reads the
variable once and re-uses in subsequent calls.
@bendudson
Copy link
Copy Markdown
Contributor

Hey @ZedThree ! This functionality is really useful :) I had to make the loading lazy to avoid issues where a FieldFactory is created during BoutMesh::load and before mesh->get works.

Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

return std::make_shared<FieldValuePtr>(ptr);
}

BOUT_ENUM_CLASS(GridVariableFunction, field3d, field2d, boutreal);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'operator<<' can be made static or moved into an anonymous namespace to enforce internal linkage [misc-use-internal-linkage]

BOUT_ENUM_CLASS(GridVariableFunction, field3d, field2d, boutreal);
^
Additional context

include/bout/bout_enum_class.hxx:102: expanded from macro 'BOUT_ENUM_CLASS'

  inline std::ostream& operator<<(std::ostream& out, const enumname& e) {      \
                       ^

return std::make_shared<FieldValuePtr>(ptr);
}

BOUT_ENUM_CLASS(GridVariableFunction, field3d, field2d, boutreal);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: function 'toString' can be made static or moved into an anonymous namespace to enforce internal linkage [misc-use-internal-linkage]

BOUT_ENUM_CLASS(GridVariableFunction, field3d, field2d, boutreal);
^
Additional context

include/bout/bout_enum_class.hxx:70: expanded from macro 'BOUT_ENUM_CLASS'

  inline std::string toString(enumname e) {                                    \
                     ^

FieldTanhHat(FieldGeneratorPtr xin, FieldGeneratorPtr widthin,
FieldGeneratorPtr centerin, FieldGeneratorPtr steepnessin)
: X(xin), width(widthin), center(centerin), steepness(steepnessin){};
: X(xin), width(widthin), center(centerin), steepness(steepnessin) {};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: parameter 'centerin' is passed by value and only copied once; consider moving it to avoid unnecessary copies [performance-unnecessary-value-param]

Suggested change
: X(xin), width(widthin), center(centerin), steepness(steepnessin) {};
: X(xin), width(widthin), center(std::move(centerin)), steepness(steepnessin) {};

FieldTanhHat(FieldGeneratorPtr xin, FieldGeneratorPtr widthin,
FieldGeneratorPtr centerin, FieldGeneratorPtr steepnessin)
: X(xin), width(widthin), center(centerin), steepness(steepnessin){};
: X(xin), width(widthin), center(centerin), steepness(steepnessin) {};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: parameter 'steepnessin' is passed by value and only copied once; consider moving it to avoid unnecessary copies [performance-unnecessary-value-param]

Suggested change
: X(xin), width(widthin), center(centerin), steepness(steepnessin) {};
: X(xin), width(widthin), center(centerin), steepness(std::move(steepnessin)) {};

FieldTanhHat(FieldGeneratorPtr xin, FieldGeneratorPtr widthin,
FieldGeneratorPtr centerin, FieldGeneratorPtr steepnessin)
: X(xin), width(widthin), center(centerin), steepness(steepnessin){};
: X(xin), width(widthin), center(centerin), steepness(steepnessin) {};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: parameter 'widthin' is passed by value and only copied once; consider moving it to avoid unnecessary copies [performance-unnecessary-value-param]

Suggested change
: X(xin), width(widthin), center(centerin), steepness(steepnessin) {};
: X(xin), width(std::move(widthin)), center(centerin), steepness(steepnessin) {};

FieldTanhHat(FieldGeneratorPtr xin, FieldGeneratorPtr widthin,
FieldGeneratorPtr centerin, FieldGeneratorPtr steepnessin)
: X(xin), width(widthin), center(centerin), steepness(steepnessin){};
: X(xin), width(widthin), center(centerin), steepness(steepnessin) {};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: parameter 'xin' is passed by value and only copied once; consider moving it to avoid unnecessary copies [performance-unnecessary-value-param]

Suggested change
: X(xin), width(widthin), center(centerin), steepness(steepnessin) {};
: X(std::move(xin)), width(widthin), center(centerin), steepness(steepnessin) {};

public:
FieldWhere(FieldGeneratorPtr test, FieldGeneratorPtr gt0, FieldGeneratorPtr lt0)
: test(test), gt0(gt0), lt0(lt0){};
: test(test), gt0(gt0), lt0(lt0) {};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: parameter 'gt0' is passed by value and only copied once; consider moving it to avoid unnecessary copies [performance-unnecessary-value-param]

Suggested change
: test(test), gt0(gt0), lt0(lt0) {};
: test(test), gt0(std::move(gt0)), lt0(lt0) {};

public:
FieldWhere(FieldGeneratorPtr test, FieldGeneratorPtr gt0, FieldGeneratorPtr lt0)
: test(test), gt0(gt0), lt0(lt0){};
: test(test), gt0(gt0), lt0(lt0) {};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: parameter 'lt0' is passed by value and only copied once; consider moving it to avoid unnecessary copies [performance-unnecessary-value-param]

Suggested change
: test(test), gt0(gt0), lt0(lt0) {};
: test(test), gt0(gt0), lt0(std::move(lt0)) {};

public:
FieldWhere(FieldGeneratorPtr test, FieldGeneratorPtr gt0, FieldGeneratorPtr lt0)
: test(test), gt0(gt0), lt0(lt0){};
: test(test), gt0(gt0), lt0(lt0) {};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: parameter 'test' is passed by value and only copied once; consider moving it to avoid unnecessary copies [performance-unnecessary-value-param]

Suggested change
: test(test), gt0(gt0), lt0(lt0) {};
: test(std::move(test)), gt0(gt0), lt0(lt0) {};

const int myg = (m->LocalNy - (m->yend - m->ystart + 1)) / 2;
///Check that ghost region widths are in fact integers
// Check grid has cells
ASSERT1(m->LocalNx > 0);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: no header providing "ASSERT1" is directly included [misc-include-cleaner]

src/mesh/data/gridfromfile.cxx:1:

- #include "bout/traits.hxx"
+ #include "bout/assert.hxx"
+ #include "bout/traits.hxx"

Something like this is already in `next`
No need to get coordinates before they are needed. Calling
this can result in problems if mesh is not fully constructed.
Not strictly necessary because variables are checked
before creating a GridVariable, but doesn't hurt.
@bendudson bendudson force-pushed the field-variable-expression branch from b146562 to cd31e42 Compare March 27, 2026 04:57
@bendudson
Copy link
Copy Markdown
Contributor

Sorry, I force-pushed to remove a bad merge with next. It introduced all kinds of exciting memory errors.

Missing argument to exception, misc. headers.
Unit tests now include Y guard cells.
FieldFactory can create Fields when bout::mesh::global is non-null
but the size of the mesh (LocalNx etc.) is not set, because
FieldFactory is created inside BoutMesh::load() via Options.
Addressing clang-tidy comments and Clang compiler warnings (sprintf use).
Set field sizes in the same way, but get and check sizes on allocation.
Copy link
Copy Markdown
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clang-tidy made some suggestions

There were too many comments to post at once. Showing the first 25 out of 56. Check the log or trigger a new build to see more.


// New value of phi at boundary, relaxing towards phivalue
BoutReal const newvalue = weight * oldvalue + (1. - weight) * phivalue;
const BoutReal newvalue = weight * oldvalue + (1. - weight) * phivalue;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: '*' has higher precedence than '+'; add parentheses to explicitly specify the order of operations [readability-math-missing-parentheses]

Suggested change
const BoutReal newvalue = weight * oldvalue + (1. - weight) * phivalue;
const BoutReal newvalue = (weight * oldvalue) + (1. - weight) * phivalue;


// New value of phi at boundary, relaxing towards phivalue
BoutReal const newvalue = weight * oldvalue + (1. - weight) * phivalue;
const BoutReal newvalue = weight * oldvalue + (1. - weight) * phivalue;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: '*' has higher precedence than '+'; add parentheses to explicitly specify the order of operations [readability-math-missing-parentheses]

Suggested change
const BoutReal newvalue = weight * oldvalue + (1. - weight) * phivalue;
const BoutReal newvalue = weight * oldvalue + ((1. - weight) * phivalue);


// New value of phi at boundary, relaxing towards phivalue
BoutReal const newvalue = weight * oldvalue + (1. - weight) * phivalue;
const BoutReal newvalue = weight * oldvalue + (1. - weight) * phivalue;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: '*' has higher precedence than '+'; add parentheses to explicitly specify the order of operations [readability-math-missing-parentheses]

Suggested change
const BoutReal newvalue = weight * oldvalue + (1. - weight) * phivalue;
const BoutReal newvalue = (weight * oldvalue) + (1. - weight) * phivalue;


// New value of phi at boundary, relaxing towards phivalue
BoutReal const newvalue = weight * oldvalue + (1. - weight) * phivalue;
const BoutReal newvalue = weight * oldvalue + (1. - weight) * phivalue;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: '*' has higher precedence than '+'; add parentheses to explicitly specify the order of operations [readability-math-missing-parentheses]

Suggested change
const BoutReal newvalue = weight * oldvalue + (1. - weight) * phivalue;
const BoutReal newvalue = weight * oldvalue + ((1. - weight) * phivalue);


/// Size of the mesh on this processor including guard/boundary cells
int LocalNx, LocalNy, LocalNz;
int LocalNx{0}, LocalNy{0}, LocalNz{0};
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: member variable 'LocalNx' has public visibility [cppcoreguidelines-non-private-member-variables-in-classes]

  int LocalNx{0}, LocalNy{0}, LocalNz{0};
      ^

ny = fieldmesh->LocalNy;
nz = fieldmesh->LocalNz;
Field3D::Field3D(const Field2D& f)
: Field(f), nx(fieldmesh->LocalNx), ny(fieldmesh->LocalNy), nz(fieldmesh->LocalNz) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: slicing object from type 'Field2D' to 'Field' discards override 'size' [cppcoreguidelines-slicing]

    : Field(f), nx(fieldmesh->LocalNx), ny(fieldmesh->LocalNy), nz(fieldmesh->LocalNz) {
      ^

BOUT_OMP_SAFE(master)
{ output << "Num threads = " << omp_get_num_threads() << endl; }
{
output << "Num threads = " << omp_get_num_threads() << endl;
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: do not use 'endl' with streams; use '\n' instead [performance-avoid-endl]

Suggested change
output << "Num threads = " << omp_get_num_threads() << endl;
output << "Num threads = " << omp_get_num_threads() << '\n';

FILE* outf;
char outfile[256];
sprintf(outfile, "test_matF_%d.mat", kMG->rProcI);
std::string outfile = fmt::format("test_matF_{:d}.mat", kMG->rProcI);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: no header providing "fmt::format" is directly included [misc-include-cleaner]

src/invert/laplace/impls/multigrid/multigrid_laplace.cxx:30:

- #include "bout/build_defines.hxx"
+ #include "fmt/format.h"
+ #include "bout/build_defines.hxx"

FILE* outf;
char outfile[256];
sprintf(outfile, "test_matF_%d.mat", kMG->rProcI);
std::string outfile = fmt::format("test_matF_{:d}.mat", kMG->rProcI);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: variable 'outfile' of type 'std::string' (aka 'basic_string') can be declared 'const' [misc-const-correctness]

Suggested change
std::string outfile = fmt::format("test_matF_{:d}.mat", kMG->rProcI);
std::string const outfile = fmt::format("test_matF_{:d}.mat", kMG->rProcI);

std::string outfile = fmt::format("test_matF_{:d}.mat", kMG->rProcI);
output << "Out file= " << outfile << endl;
outf = fopen(outfile, "w");
FILE* outf = fopen(outfile.c_str(), "w");
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warning: initializing non-owner 'FILE *' (aka '_IO_FILE *') with a newly created 'gsl::owner<>' [cppcoreguidelines-owning-memory]

    FILE* outf = fopen(outfile.c_str(), "w");
    ^

@bendudson
Copy link
Copy Markdown
Contributor

Probably the easiest way to namespace variables is to append something like "mesh:" (or "input:"?) to the start of the first argument to addGenerator here: https://github.com/boutproject/BOUT-dev/blob/field-variable-expression/src/field/field_factory.cxx#L97
i.e.

factory.addGenerator(std::string("mesh:" + name, std::make_shared<GridVariable<T>>(&mesh, name));

The generator lookup is a simple dictionary key search that happens before any splitting into section names: They only have one namespace.

An alternative is more complex but might simplify code elsewhere: To store fields, or lazily loaded fields, in the Options tree and allow them to be used in expressions. Currently only strings can be resolved by findOption (https://github.com/boutproject/BOUT-dev/blob/field-variable-expression/src/field/field_factory.cxx#L452) but perhaps that could be extended. Seems like a bigger refactor with more thinking needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants