0

I am using xarray with cfgrib to load grib files in Python. I have custom grib definitions, which I am providing to eccodes (backend for cfgrib) via the environment variable GRIB_DEFINITION_PATH. This setup works well, as long as I run the Python script in an environment where the variable was already set.

Now I want to be more flexible with my setup and provide the environment variable from within Python using os.environ (see the example below). But somehow when setting up the environment like this, the variable gets ignored and I don't understand why. Can anyone provide me some insight into this mystery? Thanks in advance!

Here an "MRE" of the setting.

import xarray as xr
import os

grib_definitions_path = "/paths/to/definitions:/split/like/this"
os.environ["GRIB_DEFINITION_PATH"] = grib_definitions_path

grib_file = '/path/to/grib/file'
backend_args = {
    "filter_by_keys": {"shortName": "P"}
}
array = xr.open_dataset(grib_file, engine="cfgrib", encode_cf=("geography", "vertical"), backend_kwargs=backend_args)["P"]
print(array.dims)

Executing the above code in a terminal fails for me with KeyError: 'P'. If I however first run

export GRIB_DEFINITION_PATH="/paths/to/definitions:/split/like/this"

the dimensions of array are being printed as expected.

4
  • 1
    Try setting the environment before you import xarray Commented Oct 3, 2022 at 20:44
  • How in the world?... I already tried that but it didn't work earlier. But now it somehow seems to work. Thanks a lot! Commented Oct 3, 2022 at 20:47
  • You still need to run that line after os.environ["GRIB_DEFINITION_PATH"] = grib_definitions_path Add the following subprocess.check_call(['sqsub', '-np', sys.argv[1], grib_definitions_path ],env=dict(os.environ, SQSUB_VAR="visible in this subprocess")) Commented Oct 4, 2022 at 7:57
  • @blackbird Can you explain why? I don't have sqsub and I also don't pass any arguments to my run script, hence sys.argv[1] would be undefined. Commented Oct 10, 2022 at 15:37

0

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.