How to get ERA5 constants?

[arthurfeeney]

TL;DR: It is not clear to me how to download the ERA5 constants.

I'm trying to download ERA5 for use in my own project. I want to do "direct-forecasting" and I am using the cl.data.IterDataModule.

In the docs, it says that land_sea_mask, orography, and lattitude should be downloaded as constants. If I include "constants" in my list of variables, it seems to work fine and downloads a file called "constants.nc". The docs for preprocessing say that "constants should not be included in variables and it handles it automatically", and the preprocessing script seems to run fine.

However, when I try to run training, I get an error saying it cannot find the three constants. If I remove the three constants from the data loader, my training script runs fine. (I just have 138 input channels instead of 141.)

So I think the issue is me doing something wrong when I am downloading the constants...

This is what I am using to download and preprocess the data. Is there something obvious I am doing wrong?

Download script

import climate_learn as cl

root_directory = "./era5_5625"

variables = [
    'temperature',
    '2m_temperature',
    'temperature_850'
    'geopotential',
    'geopotential_500',
    'u_component_of_wind',
    'v_component_of_wind',
    '10m_u_component_of_wind',
    '10m_v_component_of_wind',
    'relative_humidity',
    'specific_humidity',
    'toa_incident_solar_radiation',
    'constants'
]

for variable in variables:
    cl.data.download_weatherbench(
        dst=f"{root_directory}/{variable}",
        dataset="era5",
        variable=variable,
        # options: 1.40625, 2.8125, and (default) 5.625
        resolution=5.625
    )

Preprocessing Script

from climate_learn.data.processing.nc2npz import convert_nc2npz

variables = [
    'temperature',
    '2m_temperature',
    'geopotential',
    'u_component_of_wind',
    'v_component_of_wind',
    '10m_u_component_of_wind',
    '10m_v_component_of_wind',
    'relative_humidity',
    'specific_humidity',
    'toa_incident_solar_radiation'
]

convert_nc2npz(
    root_dir="/path/to/climatelearn/era5_5625/",
    save_dir="/path/to/climatelearn/era5_5625/processed/",
    variables=variables,
    start_train_year=1979,
    start_val_year=2015,
    start_test_year=2017,
    end_year=2018,
    num_shards=16
)

[arthurfeeney]

oops correction, the constants ARE downloaded into constants.nc, but for some reason they are not getting included in the preprocessed .npz files. :-(