[conda] conda compiler's FFLAGS et al / jinja2 syntax in meta.yaml

[Paul Madden]

Two hopefully quick questions:

When I use the Anaconda compilers in a recipe (e.g. via "{{
compiler('fortran') }}" in my meta.yaml), I get environment variables like
FFLAGS in my build environment. Are these meant to be used? I ask because,
in particular, they include a -I component that yields a "Warning:
Nonexistent include directory" from gfortran during the build if I do
"$(FC) $(FFLAGS)" in my Makefile.

Also about the jinja2 syntax for the compilers, is there a benefit to using
this vs specifying the actual compiler packages (e.g. "gfortran_linux-64
7.2.0") in meta.yaml. I ask because (1) I am always concerned about changes
due to unspecified versions, and (2) meta.yaml is no longer valid yaml, so
I cannot read it with other tools, which is useful to me (though I can
quote the jinja2 syntax, so it's only a minor annoyance).

thanks,
paul

[Paul Madden]

Hi Michael,

Thanks for the helpful feedback.

Yes, FFLAGS is generally meant to be picked up by tools such as make.

Non-existent include dirs are fine to ignore, so long as you aren't getting
errors about things not actually being found. We don't deal with fortran
much, so things might be a little off just due to lack of exercise of this
functionality. We may need to tweak FFLAGS in our recipe.

It would be nice if conda-build created $PREFIX/include. That would address
the warning, and since bin, lib and share are all there, I was surprised to
find include missing.

I hate to leave warnings unaddressed because they're a broken-windows
phenomenon: Once people start ignoring warnings that are ok to ignore, they
start ignoring ones they shouldn't, too, because it becomes difficult to
separate the signal from the noise.

I'll open a GitHub issue on this for your consideration.

There is huge benefit to the jinja2:
Those are compelling reasons, especially in combination with the
"fortran_compiler_version", which I'll test and adopt. Thank you.

In general, you should not aim to directly interpret meta.yaml. That is

crippling. Instead, you should run it through the "conda render" cli
command, or conda build's api.render function.

Also thanks for the "conda render" pointer. I'll check it out and likely
use it. Since jinja2 is apparently fine with its syntax appearing inside
quotes, this also wouldn't seem to preclude programmatically generating
meta.yaml, either. Not that I have a use case for that.

If I can bother you with one more question: Is there anything more
idiomatic/correct to use than $CONDA_PREFIX/include to access Fortran .mod
files from a build-requirement package that installed them into its own
$PREFIX/include during build? That is, I have a package with the structure

info/hash_input.json
info/files
info/index.json
info/paths.json
info/about.json
info/git
include/sigio_module.mod
include/sigio_r_module.mod
lib/libsigio.a

and other code I'm packaging (and that has this package as a build
requirement) requires those .mod files at build time. My thought would be
to add -I$(CONDA_PREFIX)/include to my gfortran invocation. This seems to
work, but I wondered if there might be better ways to access include files
from other packages. I'll poke around some published recipes to see if I
can find something better, but would appreciate any thoughts you might have.

thanks,
paul

[Paul Madden]

To close the loop here for potential future readers, TIL:

1. Which top-level directories are created under $PREFIX during a build
(e.g. bin/, include/, lib/, share/) are determined by the required packages
defined in meta.yaml. For example, if one of those packages includes a
top-level include/ directory, a top-level include/ directory will be
created under $PREFIX. conda-build has no opinion about these folders and
only creates what packages require.

2. When using the {{ compiler() }} jinja2 function, the "build" section
under "requirements" in meta.yaml should contain only build tools; library
packages required for the build should be placed in the "host" section,
which will result in the correct top-level directories being created under
$PREFIX.

3. Environment variables like CFLAGS and FFLAGS that are added to the build
environment, while reported by conda-build, are not defined by conda-build
-- they're defined by the compiler packages. So, for example, the presence
of a -I$PREFIX/include switch in FFLAGS is due to the compiler package, not
due to conda-build.

I hope this helps someone.