Skip to content

Further ease writing custom slice sources #82

@forman

Description

@forman

Is your feature request related to a problem? Please describe.

Within zappend, a slice item, either given as file path or URL or as custom slice source, is converted into a context manager and then used as such (https://github.com/bcdev/zappend/blob/main/zappend/processor.py#L98). Writing custom slice sources could therefore be further simplified if a slice item is a context manager.

Describe the solution you'd like

Allow slice items to be context managers, that is, they implement contextlib.AbstractContextManager or they provide the dunder methods __enter__() and __extit__(). Slice context managers must yield a xarray.Dataset - the slice dataset. Note, this excludes slice items that are already a xarray.Dataset, because xarray datasets are themselves context managers.

With this extension, a custom slice source can be written as follows:

from contextlib import contextmanager
import xarray as xr

@contextmanager
def get_dataset(ds_path, unused_var_names):
    # Code to acquire a dataset from somewhere
    # and optionally bind other resources.
    ds = xr.open_dataset(ds_path)
    try:
        # Further process dataset
        ds_subset = ds.drop_vars(unused_var_names)
        # Return processed dataset
        yield ds_subset 
    finally:
        # Close original dataset:
        ds.close()
        # Optionally release other resources

EDIT

Two additional settings could simplify writing of slice sources and make passing of parameters easier and more comprehensible:

  1. slice_source_kwargs to pass extra keyword arguments to every slice source together with its slice item
  2. extra to pass arbitrary configuration to a slice source. A syntactical alternative to keyword arguments passed in every call.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions