Datasets API

Auto-generated from source — a multi-backend Keras port of torchgeo.datasets, covering the base geospatial/non-geospatial dataset classes plus the full catalog of concrete leaf datasets (EuroSAT, So2Sat, Sentinel, SpaceNet, Copernicus-Bench, ...). See Data for a narrative guide to loading, sampling, and transforming data.

Image/mask samples use the channels-last layout (H x W x C), a deliberate deviation from torchgeo's channels-first convention to match the rest of keras_climate.

Core Framework

keras_climate.datasets.errors.DatasetNotFoundError

DatasetNotFoundError(dataset)

Bases: FileNotFoundError

Raised when a dataset is requested but doesn't exist.

Source code in keras_climate/datasets/errors.py

def __init__(self, dataset):
    msg = "Dataset not found"

    if hasattr(dataset, "root"):
        var = "root"
        val = dataset.root
    elif hasattr(dataset, "paths"):
        var = "paths"
        val = dataset.paths
    else:
        super().__init__(f"{msg}.")
        return

    msg += f" in `{var}={val!r}` and "

    if hasattr(dataset, "download") and not dataset.download:
        msg += "`download=False`"
    else:
        msg += "cannot be automatically downloaded"

    msg += f", either specify a different `{var}` or "

    if hasattr(dataset, "download") and not dataset.download:
        msg += "use `download=True` to automatically"
    else:
        msg += "manually"

    msg += " download the dataset."

    super().__init__(msg)

keras_climate.datasets.errors.DependencyNotFoundError

Bases: Exception

Raised when an optional dataset dependency is not installed.

keras_climate.datasets.errors.RGBBandsMissingError

RGBBandsMissingError()

Bases: ValueError

Raised when a dataset is missing RGB bands for plotting.

Source code in keras_climate/datasets/errors.py

def __init__(self):
    super().__init__("Dataset does not contain some of the RGB bands")

keras_climate.datasets.geo.GeoDataset

Bases: ABC, PlottingMixin

Abstract base class for datasets containing geospatial information.

Geospatial information includes things like coordinates, CRS, and resolution. Unlike :class:NonGeoDataset, the presence of geospatial information allows two or more datasets to be combined based on latitude/longitude, via dataset1 & dataset2 (:class:IntersectionDataset) or dataset1 | dataset2 (:class:UnionDataset).

__getitem__ abstractmethod

__getitem__(index)

Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

Source code in keras_climate/datasets/geo.py

@abc.abstractmethod
def __getitem__(self, index):
    """Retrieve input, target, and/or metadata indexed by spatiotemporal slice."""

keras_climate.datasets.geo.IntersectionDataset

IntersectionDataset(dataset1, dataset2, spatial_only=False, collate_fn=concat_samples, transforms=None)

Bases: GeoDataset

Dataset representing the intersection of two GeoDatasets.

Initialize a new IntersectionDataset instance.

When computing the intersection between two datasets that both contain model inputs (images) or model outputs (masks), the default behavior is to stack the data along the channel dimension. The collate_fn parameter can be used to change this behavior.

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    dataset1,
    dataset2,
    spatial_only=False,
    collate_fn=concat_samples,
    transforms=None,
):
    """Initialize a new IntersectionDataset instance.

    When computing the intersection between two datasets that both
    contain model inputs (images) or model outputs (masks), the default
    behavior is to stack the data along the channel dimension. The
    *collate_fn* parameter can be used to change this behavior.
    """
    self.datasets = [dataset1, dataset2]
    self.collate_fn = collate_fn
    self.transforms = transforms

    for ds in self.datasets:
        if not isinstance(ds, GeoDataset):
            raise TypeError("IntersectionDataset only supports GeoDatasets")

    dataset2.crs = dataset1.crs
    dataset2.res = dataset1.res

    index1 = dataset1.index.reset_index()
    index2 = dataset2.index.reset_index()
    self.index = gpd.overlay(index1, index2, how="intersection", keep_geom_type=True)

    if self.index.empty:
        raise RuntimeError("Datasets have no spatial intersection")

    columns = ["filepath_1", "filepath_2"]
    self.index.drop(columns=columns, inplace=True, errors="ignore")

    name = "datetime"
    datetime_1 = pd.IntervalIndex(list(self.index.pop("datetime_1")), name=name)
    datetime_2 = pd.IntervalIndex(list(self.index.pop("datetime_2")), name=name)
    self.index.index = datetime_1

    if not spatial_only:
        mint = np.maximum(datetime_1.left, datetime_2.left)
        maxt = np.minimum(datetime_1.right, datetime_2.right)
        valid = maxt >= mint
        mint = mint[valid]
        maxt = maxt[valid]
        self.index = self.index[valid]
        self.index.index = pd.IntervalIndex.from_arrays(
            mint, maxt, closed="both", name="datetime"
        )

        if self.index.empty:
            msg = "Datasets have no temporal intersection. Use `spatial_only=True`"
            msg += " if you want to ignore temporal intersection"
            raise RuntimeError(msg)

keras_climate.datasets.geo.NonGeoClassificationDataset

NonGeoClassificationDataset(root='data', transforms=None, loader=None, is_valid_file=None)

Bases: NonGeoDataset

Abstract base class for classification datasets lacking geospatial info.

This base class is designed for datasets with pre-defined image chips separated into separate folders per class (an ImageFolder-style layout).

Initialize a new NonGeoClassificationDataset instance.

Args: root: root directory where dataset can be found transforms: a function/transform that takes input sample and its target as entry and returns a transformed version loader: a callable that takes a path to an image and returns a PIL Image or numpy array (defaults to a PIL-based loader) is_valid_file: a function that takes the path of an image file and checks if the file is a valid file

Source code in keras_climate/datasets/geo.py

def __init__(self, root="data", transforms=None, loader=None, is_valid_file=None):
    """Initialize a new NonGeoClassificationDataset instance.

    Args:
        root: root directory where dataset can be found
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        loader: a callable that takes a path to an image and returns a
            PIL Image or numpy array (defaults to a PIL-based loader)
        is_valid_file: a function that takes the path of an image file
            and checks if the file is a valid file
    """
    self.root = str(root)
    self.loader = loader or _pil_loader
    self.classes, self.class_to_idx = _find_classes(self.root)
    self.samples = _make_dataset(self.root, self.class_to_idx, is_valid_file)
    self.imgs = self.samples
    self.tg_transforms = transforms

keras_climate.datasets.geo.NonGeoDataset

Bases: ABC, PlottingMixin

Abstract base class for datasets lacking geospatial information.

This base class is designed for datasets with pre-defined image chips.

__getitem__ abstractmethod

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/geo.py

@abc.abstractmethod
def __getitem__(self, index):
    """Return an index within the dataset."""

__len__ abstractmethod

__len__()

Return the length of the dataset.

Source code in keras_climate/datasets/geo.py

@abc.abstractmethod
def __len__(self):
    """Return the length of the dataset."""

keras_climate.datasets.geo.RasterDataset

RasterDataset(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: GeoDataset

Abstract base class for :class:GeoDataset stored as raster files.

Initialize a new RasterDataset instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS (defaults to the resolution of the first file found) bands: bands to return (defaults to all bands) transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling time_series: if True, stack data along the time series dimension ([T, H, W, C]). If False, merge data into a mosaic ([H, W, C]). For mask-style datasets (is_image=False), single-band data has the channel dimension squeezed.

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new RasterDataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a mosaic
            (``[H, W, C]``). For mask-style datasets (``is_image=False``),
            single-band data has the channel dimension squeezed.
    """
    self.paths = paths
    self.bands = bands or self.all_bands
    self.transforms = transforms
    self.cache = cache
    self.time_series = time_series

    if self.all_bands:
        assert set(self.bands) <= set(self.all_bands)

    filename_regex = re.compile(self.filename_regex, re.VERBOSE)
    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        match = re.match(filename_regex, os.path.basename(filepath))
        if match is not None:
            vrt = None
            try:
                vrt = self._load_warp_file(filepath=filepath, crs=crs)
                if self.cmap is None:
                    try:
                        colors = np.array(list(vrt.colormap(1).values())) / 255
                        self.cmap = ListedColormap(colors)
                    except ValueError:
                        pass
                if crs is None:
                    with rasterio.Env(OSR_WKT_FORMAT="WKT2_2018"):
                        crs = PROJ_CRS.from_user_input(vrt.crs)
                footprint = self.footprint_from_datasource(vrt)
                if footprint is None:
                    footprint = shapely.box(*vrt.bounds)
                geometries.append(footprint)
                if res is None:
                    res = vrt.res
            except rasterio.errors.RasterioIOError:
                continue
            else:
                filepaths.append(filepath)
                mint, maxt = self._filepath_to_timestamp(filepath)
                datetimes.append((mint, maxt))
            finally:
                if vrt is not None:
                    vrt.close()

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if not self.separate_files:
        self.band_indexes = None
        if self.bands:
            if self.all_bands:
                self.band_indexes = [
                    self.all_bands.index(i) + 1 for i in self.bands
                ]
            else:
                msg = (
                    f"{self.__class__.__name__} is missing an `all_bands` "
                    "attribute, so `bands` cannot be specified."
                )
                raise AssertionError(msg)

    if res is not None:
        if isinstance(res, (int, float)):
            res = (res, res)

        self._res = res

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

footprint_from_datasource

footprint_from_datasource(datasource)

Compute the spatial footprint of the dataset from a file handle.

Override this in subclasses to compute a more precise footprint than just the raster bounds.

Source code in keras_climate/datasets/geo.py

def footprint_from_datasource(self, datasource):
    """Compute the spatial footprint of the dataset from a file handle.

    Override this in subclasses to compute a more precise footprint than
    just the raster bounds.
    """
    return

keras_climate.datasets.geo.UnionDataset

UnionDataset(dataset1, dataset2, collate_fn=merge_samples, transforms=None)

Bases: GeoDataset

Dataset representing the union of two GeoDatasets.

Initialize a new UnionDataset instance.

When computing the union between two datasets that both contain model inputs (images) or model outputs (masks), the default behavior is to merge the data to create a single image/mask. The collate_fn parameter can be used to change this behavior.

Source code in keras_climate/datasets/geo.py

def __init__(self, dataset1, dataset2, collate_fn=merge_samples, transforms=None):
    """Initialize a new UnionDataset instance.

    When computing the union between two datasets that both contain
    model inputs (images) or model outputs (masks), the default
    behavior is to merge the data to create a single image/mask. The
    *collate_fn* parameter can be used to change this behavior.
    """
    self.datasets = [dataset1, dataset2]
    self.collate_fn = collate_fn
    self.transforms = transforms

    for ds in self.datasets:
        if not isinstance(ds, GeoDataset):
            raise TypeError("UnionDataset only supports GeoDatasets")

    dataset2.crs = dataset1.crs
    dataset2.res = dataset1.res

    self.index = pd.concat([dataset1.index, dataset2.index])

keras_climate.datasets.geo.VectorDataset

VectorDataset(paths='data', crs=None, res=(0.0001, 0.0001), transforms=None, label_name=None, task='semantic_segmentation', layer=None)

Bases: GeoDataset

Abstract base class for :class:GeoDataset stored as vector files.

Initialize a new VectorDataset instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS transforms: a function/transform that takes input sample and its target as entry and returns a transformed version label_name: name of the dataset property that has the label to be rasterized into the mask task: one of 'object_detection', 'semantic_segmentation', 'instance_segmentation' layer: if the input is a multilayer vector dataset, such as a geopackage, specify which layer to use

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=(0.0001, 0.0001),
    transforms=None,
    label_name=None,
    task="semantic_segmentation",
    layer=None,
):
    """Initialize a new VectorDataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        label_name: name of the dataset property that has the label to
            be rasterized into the mask
        task: one of 'object_detection', 'semantic_segmentation',
            'instance_segmentation'
        layer: if the input is a multilayer vector dataset, such as a
            geopackage, specify which layer to use
    """
    self.paths = paths
    self.transforms = transforms
    self.label_name = label_name
    allowed_tasks = [
        "semantic_segmentation",
        "object_detection",
        "instance_segmentation",
    ]
    if task not in allowed_tasks:
        raise ValueError(f"Invalid task: {task!r}. Must be one of {allowed_tasks}")
    self.task = task
    self.layer = layer
    filename_regex = re.compile(self.filename_regex, re.VERBOSE)
    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        match = re.match(filename_regex, os.path.basename(filepath))
        if match is not None:
            try:
                if pathlib.Path(filepath).suffix.lower() == ".parquet":
                    src = gpd.read_parquet(filepath)
                else:
                    src = gpd.read_file(filepath, layer=layer)
                crs = crs or src.crs or PROJ_CRS.from_epsg(4326)
                if src.crs is None:
                    src.set_crs(crs, inplace=True)
                elif src.crs != crs:
                    src.to_crs(crs, inplace=True)

                minx, miny, maxx, maxy = src.total_bounds
                geom = shapely.box(minx, miny, maxx, maxy)
                geometries.append(geom)
            except (RuntimeError, ValueError):
                continue
            else:
                filepaths.append(filepath)

                mint = pd.Timestamp.min
                maxt = pd.Timestamp.max
                if "date" in match.groupdict():
                    date = match.group("date")
                    mint, maxt = disambiguate_timestamp(date, self.date_format)

                datetimes.append((mint, maxt))

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if isinstance(res, (int, float)):
        res = (res, res)

    self._res = res

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

get_label

get_label(feature)

Get label value to use for rendering a feature.

Source code in keras_climate/datasets/geo.py

def get_label(self, feature):
    """Get label value to use for rendering a feature."""
    if self.label_name:
        return int(feature[self.label_name])
    return 1

keras_climate.datasets.geo.XarrayDataset

XarrayDataset(paths='data', crs=None, res=None, data_vars=None, transforms=None)

Bases: GeoDataset

Abstract base class for :class:GeoDataset stored as raster files (xarray).

.. warning:: This dataset is considered experimental and subject to change.

Source code in keras_climate/datasets/geo.py

def __init__(self, paths="data", crs=None, res=None, data_vars=None, transforms=None):
    lazy_import("rioxarray")
    xr = lazy_import("xarray")
    self.paths = paths
    self.transforms = transforms

    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        try:
            with xr.open_dataset(filepath, decode_coords="all") as src:
                crs = crs or src.rio.crs or PROJ_CRS.from_epsg(4326)
                res = res or src.rio.resolution()
                data_vars = data_vars or list(src.data_vars.keys())
                tmin = pd.Timestamp(src.time.values.min())
                tmax = pd.Timestamp(src.time.values.max())

                if src.rio.crs is None:
                    warnings.warn(
                        f"Unable to decode coordinates of '{filepath}', "
                        f"defaulting to {crs}. Set `crs` if this is incorrect.",
                        UserWarning,
                    )
                    src = src.rio.write_crs(crs)

                if src.rio.crs != crs:
                    src = src.rio.reproject(crs)

                filepaths.append(filepath)
                datetimes.append((tmin, tmax))
                geometries.append(shapely.box(*src.rio.bounds()))
        except (OSError, ValueError):
            continue

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if res is not None:
        if isinstance(res, (int, float)):
            res = (res, res)

        self._res = res

    if data_vars is not None:
        self.data_vars = data_vars

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

keras_climate.datasets.mixins.PlottingMixin

Mixin for dataset plotting.

keras_climate.datasets.splits.random_bbox_assignment

random_bbox_assignment(dataset, lengths, generator=None)

Split a GeoDataset randomly assigning its index's objects.

Source code in keras_climate/datasets/splits.py

def random_bbox_assignment(dataset, lengths, generator=None):
    """Split a GeoDataset randomly assigning its index's objects."""
    rng = np.random.default_rng() if generator is None else generator

    if not (isclose(sum(lengths), 1) or isclose(sum(lengths), len(dataset))):
        raise ValueError(
            "Sum of input lengths must equal 1 or the length of dataset's index."
        )

    if any(n <= 0 for n in lengths):
        raise ValueError("All items in input lengths must be greater than 0.")

    if isclose(sum(lengths), 1):
        lengths = _fractions_to_lengths(lengths, len(dataset))

    indices = rng.permutation(sum(lengths))

    new_datasets = []
    for offset, length in zip(itertools.accumulate(lengths), lengths):
        ds = deepcopy(dataset)
        ds.index = dataset.index.iloc[indices[offset - length : offset]]
        new_datasets.append(ds)

    return new_datasets

keras_climate.datasets.splits.random_bbox_splitting

random_bbox_splitting(dataset, fractions, generator=None)

Split a GeoDataset randomly splitting its index's objects.

Source code in keras_climate/datasets/splits.py

def random_bbox_splitting(dataset, fractions, generator=None):
    """Split a GeoDataset randomly splitting its index's objects."""
    rng = np.random.default_rng() if generator is None else generator

    if not isclose(sum(fractions), 1):
        raise ValueError("Sum of input fractions must equal 1.")

    if any(n <= 0 for n in fractions):
        raise ValueError("All items in input fractions must be greater than 0.")

    i_geom = dataset.index.columns.get_loc("geometry")
    new_datasets = [deepcopy(dataset) for _ in fractions]

    for i in range(len(dataset)):
        geometry_remaining = dataset.index.geometry.iloc[i]
        fraction_remaining = 1.0

        horizontal, flip = rng.integers(0, 2, (2,))
        for j, fraction in enumerate(fractions):
            if isclose(fraction_remaining, fraction):
                new_geometry = geometry_remaining
            else:
                minx, miny, maxx, maxy = geometry_remaining.bounds

                if flip:
                    frac = fraction_remaining - fraction
                else:
                    frac = fraction

                if horizontal:
                    splity = miny + (maxy - miny) * frac / fraction_remaining
                    line = LineString([(minx, splity), (maxx, splity)])
                else:
                    splitx = minx + (maxx - minx) * frac / fraction_remaining
                    line = LineString([(splitx, miny), (splitx, maxy)])

                geom1, geom2 = shapely.ops.split(geometry_remaining, line).geoms
                if horizontal:
                    if flip:
                        if geom1.centroid.y < splity:
                            geometry_remaining, new_geometry = geom1, geom2
                        else:
                            new_geometry, geometry_remaining = geom1, geom2
                    else:
                        if geom1.centroid.y < splity:
                            new_geometry, geometry_remaining = geom1, geom2
                        else:
                            geometry_remaining, new_geometry = geom1, geom2
                else:
                    if flip:
                        if geom1.centroid.x < splitx:
                            geometry_remaining, new_geometry = geom1, geom2
                        else:
                            new_geometry, geometry_remaining = geom1, geom2
                    else:
                        if geom1.centroid.x < splitx:
                            new_geometry, geometry_remaining = geom1, geom2
                        else:
                            geometry_remaining, new_geometry = geom1, geom2
            new_datasets[j].index.iloc[i, i_geom] = new_geometry

            fraction_remaining -= fraction
            horizontal = not horizontal

    return new_datasets

keras_climate.datasets.splits.random_grid_cell_assignment

random_grid_cell_assignment(dataset, fractions, grid_size=6, generator=None)

Overlays a grid over a GeoDataset and randomly assigns cells to new GeoDatasets.

Source code in keras_climate/datasets/splits.py

def random_grid_cell_assignment(dataset, fractions, grid_size=6, generator=None):
    """Overlays a grid over a GeoDataset and randomly assigns cells to new GeoDatasets."""
    rng = np.random.default_rng() if generator is None else generator

    if not isclose(sum(fractions), 1):
        raise ValueError("Sum of input fractions must equal 1.")

    if any(n <= 0 for n in fractions):
        raise ValueError("All items in input fractions must be greater than 0.")

    if grid_size < 2:
        raise ValueError("Input grid_size must be greater than 1.")

    left = []
    right = []
    rows = []
    geometry = []
    for index, row in dataset.index.iterrows():
        minx, miny, maxx, maxy = row.geometry.bounds

        stridex = (maxx - minx) / grid_size
        stridey = (maxy - miny) / grid_size

        for x in range(grid_size):
            for y in range(grid_size):
                geom = shapely.box(
                    minx + x * stridex,
                    miny + y * stridey,
                    minx + (x + 1) * stridex,
                    miny + (y + 1) * stridey,
                )
                if geom := shapely.intersection(row.geometry, geom):
                    left.append(index.left)
                    right.append(index.right)
                    rows.append(row)
                    geometry.append(geom)

    lengths = _fractions_to_lengths(fractions, len(rows))

    indexes_sr = pd.IntervalIndex.from_arrays(left, right, closed="both", name="datetime")
    rows_df = pd.DataFrame(rows)
    geometry_sr = pd.Series(geometry)

    indices = rng.permutation(len(rows))

    new_datasets = []
    for offset, length in zip(itertools.accumulate(lengths), lengths):
        ds = deepcopy(dataset)
        idx = indices[offset - length : offset].tolist()
        ds.index = GeoDataFrame(
            data=rows_df.iloc[idx].values,
            index=indexes_sr[idx],
            geometry=geometry_sr[idx].values,
            crs=dataset.crs,
        )
        new_datasets.append(ds)

    return new_datasets

keras_climate.datasets.splits.roi_split

roi_split(dataset, rois)

Split a GeoDataset intersecting it with a ROI for each desired new GeoDataset.

Source code in keras_climate/datasets/splits.py

def roi_split(dataset, rois):
    """Split a GeoDataset intersecting it with a ROI for each desired new GeoDataset."""
    new_datasets = []
    for i, roi in enumerate(rois):
        if any(
            shapely.intersects(roi, x) and not shapely.touches(roi, x)
            for x in rois[i + 1 :]
        ):
            raise ValueError("ROIs in input rois can't overlap.")

        ds = deepcopy(dataset)
        ds.index = geopandas.clip(dataset.index, roi)
        new_datasets.append(ds)

    return new_datasets

keras_climate.datasets.splits.time_series_split

time_series_split(dataset, lengths)

Split a GeoDataset on its time dimension to create non-overlapping GeoDatasets.

Source code in keras_climate/datasets/splits.py

def time_series_split(dataset, lengths):
    """Split a GeoDataset on its time dimension to create non-overlapping GeoDatasets."""
    _, _, t = dataset.bounds

    totalt = t.stop - t.start

    if all(isinstance(x, (int, float)) for x in lengths):
        if any(n <= 0 for n in lengths):
            raise ValueError("All items in input lengths must be greater than 0.")

        if not isclose(sum(lengths), 1):
            raise ValueError(
                "Sum of input lengths must equal 1 or the dataset's time length."
            )

        lengths = [totalt * f for f in lengths]

    if all(isinstance(x, pd.Timedelta) for x in lengths):
        lengths = [
            pd.Interval(t.start + offset - length, t.start + offset, closed="neither")
            for offset, length in zip(accumulate(lengths), lengths)
        ]

    _totalt = pd.Timedelta(0)
    new_datasets = []
    for i, interval in enumerate(lengths):
        start = interval.left
        end = interval.right

        offset = (
            pd.Timedelta(0) if i == len(lengths) - 1 else pd.Timedelta(1, unit="us")
        )

        if start < t.start or end > t.stop:
            raise ValueError(
                "Pairs of timestamps in lengths can't be out of dataset's time bounds."
            )

        for other in lengths:
            left = other.left
            right = other.right
            if start < left < end or start < right < end:
                raise ValueError("Pairs of timestamps in lengths can't overlap.")

        ds = deepcopy(dataset)
        ds.index = dataset.index.iloc[dataset.index.index.overlaps(interval)]
        new_index = []
        for xy in ds.index.index:
            left = xy.left
            right = xy.right
            left = max(start, left)
            right = min(end - offset, right - offset)
            new_index.append(pd.Interval(left, right, closed="neither"))
        ds.index.index = pd.IntervalIndex(new_index, closed="neither", name="datetime")
        new_datasets.append(ds)
        _totalt += end - start

    if not _totalt == totalt:
        raise ValueError(
            "Pairs of timestamps in lengths must cover dataset's time bounds."
        )

    return new_datasets

keras_climate.datasets.utils.BoundingBox dataclass

BoundingBox(minx: float, maxx: float, miny: float, maxy: float, mint: datetime, maxt: datetime)

Data class for indexing spatiotemporal data.

.. deprecated:: Use GeoSlice or shapely.Polygon instead.

keras_climate.datasets.utils.concat_samples

concat_samples(samples)

Concatenate a list of samples along an existing axis.

Concatenates along the last axis (the channel axis for channels-last image/mask samples). For 1-D metadata values (e.g. bounds, transform) the last axis is the only axis, so this matches the previous (and torchgeo's channels-first) axis-0 behavior for those keys.

Source code in keras_climate/datasets/utils.py

def concat_samples(samples):
    """Concatenate a list of samples along an existing axis.

    Concatenates along the last axis (the channel axis for channels-last
    ``image``/``mask`` samples). For 1-D metadata values (e.g. ``bounds``,
    ``transform``) the last axis is the only axis, so this matches the
    previous (and torchgeo's channels-first) axis-0 behavior for those keys.
    """
    uncollated = _list_dict_to_dict_list(samples)
    collated = {}
    for key, value in uncollated.items():
        collated[key] = ops.concatenate(value, axis=-1)
    return collated

keras_climate.datasets.utils.merge_samples

merge_samples(samples)

Merge a list of samples, taking the elementwise maximum for shared keys.

Source code in keras_climate/datasets/utils.py

def merge_samples(samples):
    """Merge a list of samples, taking the elementwise maximum for shared keys."""
    collated = {}
    for sample in samples:
        for key, value in sample.items():
            if key in collated:
                # Take the maximum so that nodata values (zeros) get replaced
                # by data values whenever possible
                collated[key] = ops.maximum(collated[key], value)
            else:
                collated[key] = value
    return collated

keras_climate.datasets.utils.stack_samples

stack_samples(samples)

Stack a list of samples along a new axis.

Source code in keras_climate/datasets/utils.py

def stack_samples(samples):
    """Stack a list of samples along a new axis."""
    uncollated = _list_dict_to_dict_list(samples)
    collated = {}
    for key, value in uncollated.items():
        collated[key] = ops.stack(value)
    return collated

keras_climate.datasets.utils.unbind_samples

unbind_samples(sample)

Reverse of :func:stack_samples.

Source code in keras_climate/datasets/utils.py

def unbind_samples(sample):
    """Reverse of :func:`stack_samples`."""
    uncollated = {}
    for key, values in sample.items():
        uncollated[key] = [values[i] for i in range(values.shape[0])]
    return _dict_list_to_list_dict(uncollated)

Copernicus-Bench

keras_climate.datasets.copernicus.CopernicusBenchAQNO2S5P

CopernicusBenchAQNO2S5P(root='data', split='train', mode='annual', bands=None, transforms=None, download=False, checksum=True)

Bases: CopernicusBenchBase

Copernicus-Bench AQ-NO2-S5P dataset.

AQ-NO2-S5P is a regression dataset based on Sentinel-5P NO2 images and EEA air quality data products. Specifically, this dataset combines 2021 measurements of NO2 (annual average concentration) from EEA with S5P NO2 ("tropospheric NO2 column number density") from GEE.

This benchmark supports both annual (1 image/location) and seasonal (4 images/location) modes, the former is used in the original benchmark.

If you use this dataset in your research, please cite the following papers:

• https://arxiv.org/abs/2503.11849
• https://www.researchgate.net/profile/Jan-Horalek/publication/389165501_Air_quality_maps_of_EEA_member_and_cooperating_countries_for_2022/links/67b72628207c0c20fa8ec116/Air-quality-maps-of-EEA-member-and-cooperating-countries-for-2022.pdf

Initialize a new CopernicusBenchAQNO2S5P instance.

Args: root: Root directory where dataset can be found. split: One of 'train', 'val', or 'test'. mode: One of 'annual' or 'seasonal'. bands: Sequence of band names to load (defaults to all bands). transforms: A function/transform that takes input sample and its target as entry and returns a transformed version. download: If True, download dataset and store it in the root directory. checksum: If True, verify the checksum of the downloaded files (may be slow).

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/copernicus/aq_no2_s5p.py

def __init__(
    self,
    root="data",
    split="train",
    mode="annual",
    bands=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new CopernicusBenchAQNO2S5P instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', or 'test'.
        mode: One of 'annual' or 'seasonal'.
        bands: Sequence of band names to load (defaults to all bands).
        transforms: A function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: If True, download dataset and store it in the root
            directory.
        checksum: If True, verify the checksum of the downloaded files
            (may be slow).

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.mode = mode
    super().__init__(root, split, bands, transforms, download, checksum)

keras_climate.datasets.copernicus.CopernicusBenchAQO3S5P

CopernicusBenchAQO3S5P(root='data', split='train', mode='annual', bands=None, transforms=None, download=False, checksum=True)

Bases: CopernicusBenchBase

Copernicus-Bench AQ-O3-S5P dataset.

AQ-O3-S5P is a regression dataset based on Sentinel-5P O3 images and EEA air quality data products. Specifically, this dataset combines 2021 measurements of O3 (93.2 percentile of maximum daily 8-hour means, SOMO35) from EEA with S5P O3 ("O3 column number density") from GEE.

This benchmark supports both annual (1 image/location) and seasonal (4 images/location) modes, the former is used in the original benchmark.

If you use this dataset in your research, please cite the following papers:

• https://arxiv.org/abs/2503.11849
• https://www.researchgate.net/profile/Jan-Horalek/publication/389165501_Air_quality_maps_of_EEA_member_and_cooperating_countries_for_2022/links/67b72628207c0c20fa8ec116/Air-quality-maps-of-EEA-member-and-cooperating-countries-for-2022.pdf

Initialize a new CopernicusBenchAQO3S5P instance.

Args: root: Root directory where dataset can be found. split: One of 'train', 'val', or 'test'. mode: One of 'annual' or 'seasonal'. bands: Sequence of band names to load (defaults to all bands). transforms: A function/transform that takes input sample and its target as entry and returns a transformed version. download: If True, download dataset and store it in the root directory. checksum: If True, verify the checksum of the downloaded files (may be slow).

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/copernicus/aq_o3_s5p.py

def __init__(
    self,
    root="data",
    split="train",
    mode="annual",
    bands=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new CopernicusBenchAQO3S5P instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', or 'test'.
        mode: One of 'annual' or 'seasonal'.
        bands: Sequence of band names to load (defaults to all bands).
        transforms: A function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: If True, download dataset and store it in the root
            directory.
        checksum: If True, verify the checksum of the downloaded files
            (may be slow).

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.mode = mode
    super().__init__(root, split, bands, transforms, download, checksum)

keras_climate.datasets.copernicus.CopernicusBenchBase

CopernicusBenchBase(root='data', split='train', bands=None, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset, ABC

Abstract base class for all Copernicus-Bench datasets.

If you use this dataset in your research, please cite: https://arxiv.org/abs/2503.11849

Initialize a new CopernicusBenchBase instance.

Args: root: Root directory where dataset can be found. split: One of 'train', 'val', or 'test'. bands: Sequence of band names to load (defaults to all bands). transforms: A function/transform that takes input sample and its target as entry and returns a transformed version. download: If True, download dataset and store it in the root directory. checksum: If True, verify the checksum of the downloaded files (may be slow).

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/copernicus/base.py

def __init__(
    self,
    root="data",
    split="train",
    bands=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new CopernicusBenchBase instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', or 'test'.
        bands: Sequence of band names to load (defaults to all bands).
        transforms: A function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: If True, download dataset and store it in the root
            directory.
        checksum: If True, verify the checksum of the downloaded files
            (may be slow).

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    import pandas as pd

    self.root = root
    self.split = split
    self.bands = bands or self.all_bands
    self.band_indices = [self.all_bands.index(i) + 1 for i in self.bands]
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    filepath = os.path.join(root, self.directory, self.filename.format(split))
    self.files = pd.read_csv(filepath, header=None)[0]

url abstractmethod property

url

Download URL.

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/copernicus/base.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise RGBBandsMissingError()

    # Static -> time series. Channels-last: (H, W, C) or (T, H, W, C).
    images = sample["image"]
    if len(images.shape) == 3:
        images = ops.expand_dims(images, axis=0)

    ncols = images.shape[0]
    if "mask" in sample:
        ncols += 1
        if "prediction" in sample:
            ncols += 1

    fig, ax = plt.subplots(ncols=ncols, squeeze=False)

    title = "Image"
    if "label" in sample:
        label_arr = ops.convert_to_numpy(sample["label"])
        if label_arr.ndim == 0:
            # Multiclass classification
            label = self.classes[int(label_arr)]
            if "prediction" in sample:
                prediction = self.classes[
                    int(ops.convert_to_numpy(sample["prediction"]))
                ]
        else:
            # Multilabel classification
            label = label_arr.nonzero()[0]
            if "prediction" in sample:
                prediction = ops.convert_to_numpy(
                    sample["prediction"]
                ).nonzero()[0]

        title = f"Label: {label}"
        if "prediction" in sample:
            title += f"\nPrediction: {prediction}"

    images = ops.take(images, ops.convert_to_tensor(rgb_indices), axis=-1)
    if set(self.rgb_bands) <= {"VV", "VH", "HH", "HV"}:
        # SAR
        vv = images[..., 0]
        vh = images[..., 1]
        images = ops.stack([vv, vh, (vv + vh) / 2], axis=-1)
        images = quantile_normalization(images)

    images = quantile_normalization(images)
    images = ops.convert_to_numpy(images)
    for i in range(len(images)):
        ax[0, i].imshow(images[i])
        ax[0, i].axis("off")
        if show_titles:
            ax[0, i].set_title(title)

    if "mask" in sample:
        kwargs = {"cmap": self.cmap}
        if self.classes:
            kwargs |= {
                "vmin": 0,
                "vmax": len(self.classes) - 1,
                "interpolation": "none",
            }
        mask = ops.convert_to_numpy(sample["mask"])
        ax[0, i + 1].imshow(mask, **kwargs)
        ax[0, i + 1].axis("off")
        if show_titles:
            ax[0, i + 1].set_title("Mask")

        if "prediction" in sample:
            prediction = ops.convert_to_numpy(sample["prediction"])
            ax[0, i + 2].imshow(prediction, **kwargs)
            ax[0, i + 2].axis("off")
            if show_titles:
                ax[0, i + 2].set_title("Prediction")

    if suptitle is not None:
        fig.suptitle(suptitle)

    fig.tight_layout()

    return fig

keras_climate.datasets.copernicus.CopernicusBenchBigEarthNetS1

CopernicusBenchBigEarthNetS1(root='data', split='train', bands=None, transforms=None, download=False, checksum=True)

Bases: CopernicusBenchBase

Copernicus-Bench BigEarthNet-S1 dataset.

BigEarthNet-S1 is a multilabel land use/land cover classification dataset composed of 5% of the Sentinel-1 data of BigEarthNet-v2.

If you use this dataset in your research, please cite the following papers:

• https://arxiv.org/abs/2503.11849
• https://arxiv.org/abs/2407.03653

Initialize a new CopernicusBenchBigEarthNetS1 instance.

Args: root: Root directory where dataset can be found. split: One of 'train', 'val', or 'test'. bands: Sequence of band names to load (defaults to all bands). transforms: A function/transform that takes input sample and its target as entry and returns a transformed version. download: If True, download dataset and store it in the root directory. checksum: If True, verify the checksum of the downloaded files (may be slow).

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/copernicus/bigearthnet_s1.py

def __init__(
    self,
    root="data",
    split="train",
    bands=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new CopernicusBenchBigEarthNetS1 instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', or 'test'.
        bands: Sequence of band names to load (defaults to all bands).
        transforms: A function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: If True, download dataset and store it in the root
            directory.
        checksum: If True, verify the checksum of the downloaded files
            (may be slow).

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    super().__init__(root, split, bands, transforms, download, checksum)
    filepath = os.path.join(root, self.directory, self.filename.format(split))
    self.files = pd.read_csv(filepath)

keras_climate.datasets.copernicus.CopernicusBenchBigEarthNetS2

CopernicusBenchBigEarthNetS2(root='data', split='train', bands=None, transforms=None, download=False, checksum=True)

Bases: CopernicusBenchBase

Copernicus-Bench BigEarthNet-S2 dataset.

BigEarthNet-S2 is a multilabel land use/land cover classification dataset composed of 5% of the Sentinel-2 data of BigEarthNet-v2.

If you use this dataset in your research, please cite the following papers:

• https://arxiv.org/abs/2503.11849
• https://arxiv.org/abs/2407.03653

Initialize a new CopernicusBenchBigEarthNetS2 instance.

Args: root: Root directory where dataset can be found. split: One of 'train', 'val', or 'test'. bands: Sequence of band names to load (defaults to all bands). transforms: A function/transform that takes input sample and its target as entry and returns a transformed version. download: If True, download dataset and store it in the root directory. checksum: If True, verify the checksum of the downloaded files (may be slow).

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/copernicus/bigearthnet_s2.py

def __init__(
    self,
    root="data",
    split="train",
    bands=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new CopernicusBenchBigEarthNetS2 instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', or 'test'.
        bands: Sequence of band names to load (defaults to all bands).
        transforms: A function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: If True, download dataset and store it in the root
            directory.
        checksum: If True, verify the checksum of the downloaded files
            (may be slow).

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    super().__init__(root, split, bands, transforms, download, checksum)
    filepath = os.path.join(root, self.directory, self.filename.format(split))
    self.files = pd.read_csv(filepath)

keras_climate.datasets.copernicus.CopernicusBenchBiomassS3

CopernicusBenchBiomassS3(root='data', split='train', mode='static', bands=None, transforms=None, download=False, checksum=True)

Bases: CopernicusBenchBase

Copernicus-Bench Biomass-S3 dataset.

Biomass-S3 is a regression dataset based on Sentinel-3 OLCI images and CCI biomass. The biomass product is part of the European Space Agency's Climate Change Initiative (CCI) program and delivers global forest above-ground biomass at 100 m spatial resolution.

This benchmark supports both static (1 image/location) and time series (1-4 images/location) modes, the former is used in the original benchmark.

If you use this dataset in your research, please cite the following papers:

• https://arxiv.org/abs/2503.11849
• https://catalogue.ceda.ac.uk/uuid/02e1b18071ad45a19b4d3e8adafa2817/

Initialize a new CopernicusBenchBiomassS3 instance.

Args: root: Root directory where dataset can be found. split: One of 'train', 'val', or 'test'. mode: One of 'static' or 'time-series'. bands: Sequence of band names to load (defaults to all bands). transforms: A function/transform that takes input sample and its target as entry and returns a transformed version. download: If True, download dataset and store it in the root directory. checksum: If True, verify the checksum of the downloaded files (may be slow).

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/copernicus/biomass_s3.py

def __init__(
    self,
    root="data",
    split="train",
    mode="static",
    bands=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new CopernicusBenchBiomassS3 instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', or 'test'.
        mode: One of 'static' or 'time-series'.
        bands: Sequence of band names to load (defaults to all bands).
        transforms: A function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: If True, download dataset and store it in the root
            directory.
        checksum: If True, verify the checksum of the downloaded files
            (may be slow).

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.mode = mode
    super().__init__(root, split, bands, transforms, download, checksum)
    filepath = os.path.join(root, self.directory, self.filename.format(split))
    self.files = pd.read_csv(filepath, header=None)

keras_climate.datasets.copernicus.CopernicusBenchCloudS2

CopernicusBenchCloudS2(root='data', split='train', bands=None, transforms=None, download=False, checksum=True)

Bases: CopernicusBenchBase

Copernicus-Bench Cloud-S2 dataset.

Cloud-S2 is a multi-class cloud segmentation dataset derived from CloudSEN12+ <https://www.sciencedirect.com/science/article/pii/S2352340924008163>_, one of the largest Sentinel-2 cloud and cloud shadow detection datasets with expert-labeled pixels. We take 25% samples with high-quality labels, and split them into 1699/567/551 train/val/test subsets.

Classes:

• 0 - Clear: Pixels without cloud and cloud shadow contamination.
• 1 - Thick Cloud: Opaque clouds that block all the reflected light from the Earth's surface.
• 2 - Thin Cloud: Semitransparent clouds that alter the surface spectral signal but still allow to recognize the background. This is the hardest class to identify.
• 3 - Cloud Shadow: Dark pixels where light is occluded by thick or thin clouds.

If you use this dataset in your research, please cite the following papers:

• https://arxiv.org/abs/2503.11849
• https://doi.org/10.1016/j.dib.2024.110852

Source code in keras_climate/datasets/copernicus/base.py

def __init__(
    self,
    root="data",
    split="train",
    bands=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new CopernicusBenchBase instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', or 'test'.
        bands: Sequence of band names to load (defaults to all bands).
        transforms: A function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: If True, download dataset and store it in the root
            directory.
        checksum: If True, verify the checksum of the downloaded files
            (may be slow).

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    import pandas as pd

    self.root = root
    self.split = split
    self.bands = bands or self.all_bands
    self.band_indices = [self.all_bands.index(i) + 1 for i in self.bands]
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    filepath = os.path.join(root, self.directory, self.filename.format(split))
    self.files = pd.read_csv(filepath, header=None)[0]

keras_climate.datasets.copernicus.CopernicusBenchCloudS3

CopernicusBenchCloudS3(root='data', split='train', mode='multi', bands=None, transforms=None, download=False, checksum=True)

Bases: CopernicusBenchBase

Copernicus-Bench Cloud-S3 dataset.

Cloud-S3 is a cloud segmentation dataset with raw images from Sentinel-3 OLCI and labels from the IdePix <https://step.esa.int/main/snap-supported-plugins/idepix-tool/>__ classification algorithm.

This dataset has two modes:

Multiclass classification:

• 0 - Invalid: Invalid pixels, should be ignored during training.
• 1 - Clear: Land, coastline, or water pixels.
• 2 - Cloud-Ambiguous: Semi-transparent clouds, or clouds where the detection level is uncertain.
• 3 - Cloud-Sure: Fully-opaque clouds with full confidence of their detection.
• 4 - Cloud Shadow: Pixels are affected by a cloud shadow.
• 5 - Snow/Ice: Clear snow/ice pixels.

Binary classification:

• 0 - Invalid: Invalid pixels, should be ignored during training.
• 1 - Clear: Land, coastline, water, snow, or ice pixels.
• 2 - Cloud: Pixels which are either cloud-sure or cloud-ambiguous.

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2503.11849

Initialize a new CopernicusBenchCloudS3 instance.

Args: root: Root directory where dataset can be found. split: One of 'train', 'val', or 'test'. mode: One of 'binary' or 'multi'. bands: Sequence of band names to load (defaults to all bands). transforms: A function/transform that takes input sample and its target as entry and returns a transformed version. download: If True, download dataset and store it in the root directory. checksum: If True, verify the checksum of the downloaded files (may be slow).

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/copernicus/cloud_s3.py

def __init__(
    self,
    root="data",
    split="train",
    mode="multi",
    bands=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new CopernicusBenchCloudS3 instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', or 'test'.
        mode: One of 'binary' or 'multi'.
        bands: Sequence of band names to load (defaults to all bands).
        transforms: A function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: If True, download dataset and store it in the root
            directory.
        checksum: If True, verify the checksum of the downloaded files
            (may be slow).

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.mode = mode
    if mode == "binary":
        self.classes = ("Invalid", "Clear", "Cloud")
        self.cmap = ListedColormap(["red", "gray", "white"])
    super().__init__(root, split, bands, transforms, download, checksum)

keras_climate.datasets.copernicus.CopernicusBenchDFC2020S1

CopernicusBenchDFC2020S1(root='data', split='train', bands=None, transforms=None, download=False, checksum=True)

Bases: CopernicusBenchBase

Copernicus-Bench DFC2020-S1 dataset.

DFC2020-S1 is a land use/land cover segmentation datasets derived from the IEEE GRSS Data Fusion Contest 2020 (DFC2020).

If you use this dataset in your research, please cite the following papers:

• https://arxiv.org/abs/2503.11849
• https://ieee-dataport.org/competitions/2020-ieee-grss-data-fusion-contest

Source code in keras_climate/datasets/copernicus/base.py

def __init__(
    self,
    root="data",
    split="train",
    bands=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new CopernicusBenchBase instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', or 'test'.
        bands: Sequence of band names to load (defaults to all bands).
        transforms: A function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: If True, download dataset and store it in the root
            directory.
        checksum: If True, verify the checksum of the downloaded files
            (may be slow).

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    import pandas as pd

    self.root = root
    self.split = split
    self.bands = bands or self.all_bands
    self.band_indices = [self.all_bands.index(i) + 1 for i in self.bands]
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    filepath = os.path.join(root, self.directory, self.filename.format(split))
    self.files = pd.read_csv(filepath, header=None)[0]

keras_climate.datasets.copernicus.CopernicusBenchDFC2020S2

CopernicusBenchDFC2020S2(root='data', split='train', bands=None, transforms=None, download=False, checksum=True)

Bases: CopernicusBenchBase

Copernicus-Bench DFC2020-S2 dataset.

DFC2020-S2 is a land use/land cover segmentation datasets derived from the IEEE GRSS Data Fusion Contest 2020 (DFC2020).

If you use this dataset in your research, please cite the following papers:

• https://arxiv.org/abs/2503.11849
• https://ieee-dataport.org/competitions/2020-ieee-grss-data-fusion-contest

Source code in keras_climate/datasets/copernicus/base.py

def __init__(
    self,
    root="data",
    split="train",
    bands=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new CopernicusBenchBase instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', or 'test'.
        bands: Sequence of band names to load (defaults to all bands).
        transforms: A function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: If True, download dataset and store it in the root
            directory.
        checksum: If True, verify the checksum of the downloaded files
            (may be slow).

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    import pandas as pd

    self.root = root
    self.split = split
    self.bands = bands or self.all_bands
    self.band_indices = [self.all_bands.index(i) + 1 for i in self.bands]
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    filepath = os.path.join(root, self.directory, self.filename.format(split))
    self.files = pd.read_csv(filepath, header=None)[0]

keras_climate.datasets.copernicus.CopernicusBenchEuroSATS1

CopernicusBenchEuroSATS1(root='data', split='train', bands=None, transforms=None, download=False, checksum=True)

Bases: CopernicusBenchBase

Copernicus-Bench EuroSAT-S1 dataset.

EuroSAT-S1 is a multi-class land use/land cover classification dataset, and is functionally identical to EuroSAT-SAR.

If you use this dataset in your research, please cite the following papers:

• https://arxiv.org/abs/2503.11849
• https://doi.org/10.1109/JSTARS.2024.3493237

Source code in keras_climate/datasets/copernicus/base.py

def __init__(
    self,
    root="data",
    split="train",
    bands=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new CopernicusBenchBase instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', or 'test'.
        bands: Sequence of band names to load (defaults to all bands).
        transforms: A function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: If True, download dataset and store it in the root
            directory.
        checksum: If True, verify the checksum of the downloaded files
            (may be slow).

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    import pandas as pd

    self.root = root
    self.split = split
    self.bands = bands or self.all_bands
    self.band_indices = [self.all_bands.index(i) + 1 for i in self.bands]
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    filepath = os.path.join(root, self.directory, self.filename.format(split))
    self.files = pd.read_csv(filepath, header=None)[0]

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/copernicus/eurosat_s1.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    file = str(self.files[index]).replace(".jpg", ".tif")
    classname = file.split("_")[0]
    path = os.path.join(self.root, self.directory, "all_imgs", classname, file)
    sample = self._load_image(path)
    sample["label"] = ops.convert_to_tensor(
        np.array(self.classes.index(classname), dtype="int64")
    )

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

keras_climate.datasets.copernicus.CopernicusBenchEuroSATS2

CopernicusBenchEuroSATS2(root='data', split='train', bands=None, transforms=None, download=False, checksum=True)

Bases: CopernicusBenchBase

Copernicus-Bench EuroSAT-S2 dataset.

EuroSAT-S2 is a multi-class land use/land cover classification dataset, and is functionally identical to EuroSAT-MS.

If you use this dataset in your research, please cite the following papers:

• https://arxiv.org/abs/2503.11849
• https://ieeexplore.ieee.org/document/8736785
• https://ieeexplore.ieee.org/document/8519248

Source code in keras_climate/datasets/copernicus/base.py

def __init__(
    self,
    root="data",
    split="train",
    bands=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new CopernicusBenchBase instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', or 'test'.
        bands: Sequence of band names to load (defaults to all bands).
        transforms: A function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: If True, download dataset and store it in the root
            directory.
        checksum: If True, verify the checksum of the downloaded files
            (may be slow).

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    import pandas as pd

    self.root = root
    self.split = split
    self.bands = bands or self.all_bands
    self.band_indices = [self.all_bands.index(i) + 1 for i in self.bands]
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    filepath = os.path.join(root, self.directory, self.filename.format(split))
    self.files = pd.read_csv(filepath, header=None)[0]

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/copernicus/eurosat_s2.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    file = str(self.files[index]).replace(".jpg", ".tif")
    classname = file.split("_")[0]
    path = os.path.join(self.root, self.directory, "all_imgs", classname, file)
    sample = self._load_image(path)
    sample["label"] = ops.convert_to_tensor(
        np.array(self.classes.index(classname), dtype="int64")
    )

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

keras_climate.datasets.copernicus.CopernicusBenchFloodS1

CopernicusBenchFloodS1(root='data', split='train', mode=1, bands=None, transforms=None, download=False, checksum=True)

Bases: CopernicusBenchBase

Copernicus-Bench Flood-S1 dataset.

Flood-S1 is a flood segmentation dataset extracted from a large flood mapping dataset Kuro Siwo.

If you use this dataset in your research, please cite the following papers:

• https://arxiv.org/abs/2503.11849
• https://arxiv.org/abs/2311.12056

Initialize a new CopernicusBenchFloodS1 instance.

Args: root: Root directory where dataset can be found. split: One of 'train', 'val', or 'test'. mode: Number of pre-flood images, 1 or 2. bands: Sequence of band names to load (defaults to all bands). transforms: A function/transform that takes input sample and its target as entry and returns a transformed version. download: If True, download dataset and store it in the root directory. checksum: If True, verify the checksum of the downloaded files (may be slow).

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/copernicus/flood_s1.py

def __init__(
    self,
    root="data",
    split="train",
    mode=1,
    bands=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new CopernicusBenchFloodS1 instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', or 'test'.
        mode: Number of pre-flood images, 1 or 2.
        bands: Sequence of band names to load (defaults to all bands).
        transforms: A function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: If True, download dataset and store it in the root
            directory.
        checksum: If True, verify the checksum of the downloaded files
            (may be slow).

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.root = root
    self.split = split
    self.mode = mode
    self.bands = bands or self.all_bands
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    filepath = os.path.join(root, self.directory, self.filename.format(split))
    with open(filepath) as f:
        self.metadata = json.load(f)
    self.files = pd.Series(sorted(self.metadata.keys()))

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/copernicus/flood_s1.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    key = self.files[index]
    path = self.metadata[key]["path"]
    directory = os.path.join(self.root, self.directory, "data", path)
    mask_path = glob.glob(os.path.join(directory, "MK0_MLU*.tif"))[0]
    sample = self._load_image(directory) | self._load_mask(mask_path)

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

keras_climate.datasets.copernicus.CopernicusBenchLC100ClsS3

CopernicusBenchLC100ClsS3(root='data', split='train', mode='static', bands=None, transforms=None, download=False, checksum=True)

Bases: CopernicusBenchBase

Copernicus-Bench LC100Cls-S3 dataset.

LC100Cls-S3 is a multilabel land use/land cover classification dataset based on Sentinel-3 OLCI images and CGLS-LC100 land cover maps. CGLS-LC100 is a product in the Copernicus Global Land Service (CGLS) portfolio and delivers a global 23-class land cover map at 100 m spatial resolution.

This benchmark supports both static (1 image/location) and time series (1-4 images/location) modes, the former is used in the original benchmark.

Classes: 0 Unknown, 20 Shrubs, 30 Herbaceous vegetation, 40 Cultivated and managed vegetation/agriculture, 50 Urban/built up, 60 Bare/sparse vegetation, 70 Snow and ice, 80 Permanent water bodies, 90 Herbaceous wetland, 100 Moss and lichen, 111-116 Closed forest variants, 121-126 Open forest variants, 200 Oceans/seas.

If you use this dataset in your research, please cite the following papers:

• https://arxiv.org/abs/2503.11849
• https://doi.org/10.5281/zenodo.3939049

Initialize a new CopernicusBenchLC100ClsS3 instance.

Args: root: Root directory where dataset can be found. split: One of 'train', 'val', or 'test'. mode: One of 'static' or 'time-series'. bands: Sequence of band names to load (defaults to all bands). transforms: A function/transform that takes input sample and its target as entry and returns a transformed version. download: If True, download dataset and store it in the root directory. checksum: If True, verify the checksum of the downloaded files (may be slow).

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/copernicus/lc100cls_s3.py

def __init__(
    self,
    root="data",
    split="train",
    mode="static",
    bands=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new CopernicusBenchLC100ClsS3 instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', or 'test'.
        mode: One of 'static' or 'time-series'.
        bands: Sequence of band names to load (defaults to all bands).
        transforms: A function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: If True, download dataset and store it in the root
            directory.
        checksum: If True, verify the checksum of the downloaded files
            (may be slow).

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.mode = mode
    super().__init__(root, split, bands, transforms, download, checksum)
    filepath = os.path.join(root, self.directory, self.filename.format(split))
    self.files = pd.read_csv(filepath)
    if mode == "static":
        filepath = os.path.join(root, self.directory, f"static_fnames-{split}.csv")
        self.static_files = pd.read_csv(filepath, header=None)

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/copernicus/lc100cls_s3.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    row = self.files.iloc[index].values
    if self.mode == "static":
        pid, file = self.static_files.iloc[index]
        path = os.path.join(self.root, self.directory, "s3_olci", pid, file)
        sample = self._load_image(path)
        sample["label"] = ops.convert_to_tensor(row[1:].astype(np.int64))
    elif self.mode == "time-series":
        pid = row[0]
        paths = os.path.join(self.root, self.directory, "s3_olci", pid, "*.tif")
        samples = [self._load_image(path) for path in sorted(glob.glob(paths))]
        sample = stack_samples(samples)
        sample["label"] = ops.convert_to_tensor(row[1:].astype(np.int64))
    else:
        raise ValueError(f"Invalid mode: {self.mode!r}")

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

keras_climate.datasets.copernicus.CopernicusBenchLC100SegS3

CopernicusBenchLC100SegS3(root='data', split='train', mode='static', bands=None, transforms=None, download=False, checksum=True)

Bases: CopernicusBenchBase

Copernicus-Bench LC100Seg-S3 dataset.

LC100Seg-S3 is a multilabel land use/land cover segmentation dataset based on Sentinel-3 OLCI images and CGLS-LC100 land cover maps. CGLS-LC100 is a product in the Copernicus Global Land Service (CGLS) portfolio and delivers a global 23-class land cover map at 100 m spatial resolution.

This benchmark supports both static (1 image/location) and time series (1-4 images/location) modes, the former is used in the original benchmark.

Classes: 0 Unknown, 20 Shrubs, 30 Herbaceous vegetation, 40 Cultivated and managed vegetation/agriculture, 50 Urban/built up, 60 Bare/sparse vegetation, 70 Snow and ice, 80 Permanent water bodies, 90 Herbaceous wetland, 100 Moss and lichen, 111-116 Closed forest variants, 121-126 Open forest variants, 200 Oceans/seas.

If you use this dataset in your research, please cite the following papers:

• https://arxiv.org/abs/2503.11849
• https://doi.org/10.5281/zenodo.3939049

Initialize a new CopernicusBenchLC100SegS3 instance.

Args: root: Root directory where dataset can be found. split: One of 'train', 'val', or 'test'. mode: One of 'static' or 'time-series'. bands: Sequence of band names to load (defaults to all bands). transforms: A function/transform that takes input sample and its target as entry and returns a transformed version. download: If True, download dataset and store it in the root directory. checksum: If True, verify the checksum of the downloaded files (may be slow).

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/copernicus/lc100seg_s3.py

def __init__(
    self,
    root="data",
    split="train",
    mode="static",
    bands=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new CopernicusBenchLC100SegS3 instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', or 'test'.
        mode: One of 'static' or 'time-series'.
        bands: Sequence of band names to load (defaults to all bands).
        transforms: A function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: If True, download dataset and store it in the root
            directory.
        checksum: If True, verify the checksum of the downloaded files
            (may be slow).

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.mode = mode
    super().__init__(root, split, bands, transforms, download, checksum)
    filepath = os.path.join(root, self.directory, self.filename.format(split))
    self.files = pd.read_csv(filepath)
    if mode == "static":
        filepath = os.path.join(root, self.directory, f"static_fnames-{split}.csv")
        self.static_files = pd.read_csv(filepath, header=None)

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/copernicus/lc100seg_s3.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    row = self.files.iloc[index].values
    if self.mode == "static":
        pid, file = self.static_files.iloc[index]
        path = os.path.join(self.root, self.directory, "s3_olci", pid, file)
        sample = self._load_image(path)
    elif self.mode == "time-series":
        pid = row[0]
        paths = os.path.join(self.root, self.directory, "s3_olci", pid, "*.tif")
        samples = [self._load_image(path) for path in sorted(glob.glob(paths))]
        sample = stack_samples(samples)
    else:
        raise ValueError(f"Invalid mode: {self.mode!r}")

    path = os.path.join(self.root, self.directory, "lc100", f"{pid}.tif")
    sample = sample | self._load_mask(path)
    mask = ops.convert_to_numpy(sample["mask"])
    remapped = mask.copy()
    for before, after in self.class_map.items():
        remapped[mask == before] = after
    sample["mask"] = ops.cast(ops.convert_to_tensor(remapped), self.dtype)

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

keras_climate.datasets.copernicus.CopernicusBenchLCZS2

CopernicusBenchLCZS2(root='data', split='train', bands=None, transforms=None, download=False, checksum=True)

Bases: CopernicusBenchBase

Copernicus-Bench LCZ-S2 dataset.

LCZ-S2 is a multi-class scene classification dataset derived from So2Sat-LCZ42, a large-scale local climate zone classification dataset.

If you use this dataset in your research, please cite the following papers:

• https://arxiv.org/abs/2503.11849
• https://doi.org/10.1109/MGRS.2020.2964708

.. note::

This dataset requires the following additional library to be installed:

• <https://pypi.org/project/h5py/>_ to load the dataset.

Initialize a new CopernicusBenchLCZS2 instance.

Args: root: Root directory where dataset can be found. split: One of 'train', 'val', or 'test'. bands: Sequence of band names to load (defaults to all bands). transforms: A function/transform that takes input sample and its target as entry and returns a transformed version. download: If True, download dataset and store it in the root directory. checksum: If True, verify the checksum of the downloaded files (may be slow).

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/copernicus/lcz_s2.py

def __init__(
    self,
    root="data",
    split="train",
    bands=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new CopernicusBenchLCZS2 instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', or 'test'.
        bands: Sequence of band names to load (defaults to all bands).
        transforms: A function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: If True, download dataset and store it in the root
            directory.
        checksum: If True, verify the checksum of the downloaded files
            (may be slow).

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    h5py = lazy_import("h5py")

    self.root = root
    self.split = split
    self.bands = bands or self.all_bands
    self.band_indices = [self.all_bands.index(i) for i in self.bands]
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    self.filepath = os.path.join(root, self.filename.format(split))
    with h5py.File(self.filepath, "r") as f:
        self.length = f["label"].shape[0]

__len__

__len__()

Return the length of the dataset.

Source code in keras_climate/datasets/copernicus/lcz_s2.py

def __len__(self):
    """Return the length of the dataset."""
    return self.length

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/copernicus/lcz_s2.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    h5py = lazy_import("h5py")

    with h5py.File(self.filepath, "r") as f:
        # Already stored channels-last (H x W x C)
        sen2 = f["sen2"][index][:, :, self.band_indices]
        label = f["label"][index].argmax()

    sample = {
        "image": ops.convert_to_tensor(sen2.astype("float32")),
        "label": ops.convert_to_tensor(np.array(label, dtype="int64")),
    }

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

keras_climate.datasets.copernicus.CopernicusEmbed

CopernicusEmbed(paths='data', crs=None, res=None, transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: RasterDataset

Copernicus-Embed dataset.

Copernicus-Embed <https://github.com/zhu-xlab/Copernicus-FM/tree/main/Copernicus-Embed-025deg>__ is an embedding dataset that gives each 0.25x0.25 grid one embedding vector, aggregated over all available modalities from the whole Copernicus-Pretrain dataset (721x1440x768, filling empty ocean grids with 0). This dataset can be seen as a semantic representation product that integrates various sources of satellite observations at an extremely high compression ratio. It also makes it very convenient to link Earth's surface to the atmosphere (e.g., as improved static variables adding to ERA5), unlocking new possibilities in the development of weather/climate foundation models.

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2503.11849

Initialize a new CopernicusEmbed instance.

Args: paths: one or more root directories to search or files to load crs: coordinate reference system to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS in (xres, yres) format. If a single float is provided, it is used for both the x and y resolution (defaults to the resolution of the first file found) transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow) time_series: if True, stack data along the time series dimension (T x H x W x C). If False, merge data into an H x W x C mosaic.

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/copernicus/embed.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new CopernicusEmbed instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: coordinate reference system to warp to (defaults to the CRS
            of the first file found)
        res: resolution of the dataset in units of CRS in (xres, yres)
            format. If a single float is provided, it is used for both
            the x and y resolution (defaults to the resolution of the
            first file found)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)
        time_series: if True, stack data along the time series dimension
            (``T x H x W x C``). If False, merge data into an ``H x W x C``
            mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.paths = paths
    self.download = download
    self.checksum = checksum

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

.. warning:: Visualizations are generated using PCA on each image individually, and are thus not comparable across images. The plot method is provided for visualization purposes only and should not be used to draw conclusions.

Source code in keras_climate/datasets/copernicus/embed.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    .. warning::
       Visualizations are generated using PCA on each image
       *individually*, and are thus not comparable across images. The
       plot method is provided for visualization purposes only and
       should not be used to draw conclusions.
    """
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"]).astype(np.float64)
    h, w, c = image.shape
    A = image.reshape(h * w, c)

    # Use PCA to project embeddings from a high-dimensional to a 3D space
    valid = A.sum(axis=1) != 0
    invalid = ~valid

    centered = A[valid] - A[valid].mean(axis=0, keepdims=True)
    _, _, vt = np.linalg.svd(centered, full_matrices=False)
    v = vt[:3].T
    b = A @ v

    b -= b[valid].min(axis=0, keepdims=True)
    b /= b[valid].max(axis=0, keepdims=True)
    b[invalid] = 1
    image = b.reshape(h, w, 3)

    fig, ax = plt.subplots()
    ax.imshow(image)
    ax.axis("off")

    if show_titles:
        ax.set_title("Embedding")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.copernicus.CopernicusPretrain

CopernicusPretrain(*args, **kwargs)

Copernicus-Pretrain dataset.

Copernicus-Pretrain is an extension of the SSL4EO-S12 dataset to all major Sentinel missions (S1-S5P). The images are organized into ~310K regional grids (0.25x0.25 degrees, consistent with ERA5), densely covering the whole land surface and near-land ocean with time series from eight distinct Sentinel modalities.

This dataset class uses WebDataset for efficient data loading in distributed environments, which returns a Python iterable that streams samples on demand.

The full dataset has a varying number of modalities, S1/2 local patches, and timestamps for different grids. It also contains metadata including the filenames all images are derived from. For simplicity, the current dataset class provides a minimum example:

• only use grids with all modalities (220k)
• sample one local patch for S1 and S2
• sample one timestamp for each modality

Therefore, each sample contains 8 tensors (S1, S2, S3, S5P NO2/CO/SO2/O3, DEM).

Example:

.. code-block:: python

dataset = CopernicusPretrain( urls='data/example-{000000..000009}.tar', shardshuffle=True, resampled=True )

# Check the first sample sample = next(iter(dataset)) s1 = sample['s1_grd.pth'] s2 = sample['s2_toa.pth'] s3 = sample['s3_olci.pth'] s5p_co = sample['s5p_co.pth'] s5p_no2 = sample['s5p_no2.pth'] s5p_o3 = sample['s5p_o3.pth'] s5p_so2 = sample['s5p_so2.pth'] dem = sample['dem.pth']

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2503.11849

.. note::

This dataset requires the following additional library to be installed:

• <https://pypi.org/project/webdataset/>_ to load the dataset.

Initialize a new CopernicusPretrain instance.

Args: *args: Arguments passed to the WebDataset base class. **kwargs: Keyword arguments passed to the WebDataset base class.

Source code in keras_climate/datasets/copernicus/pretrain.py

def __init__(self, *args, **kwargs):
    """Initialize a new CopernicusPretrain instance.

    Args:
        *args: Arguments passed to the WebDataset base class.
        **kwargs: Keyword arguments passed to the WebDataset base class.
    """
    # https://github.com/torchgeo/torchgeo/security/advisories/GHSA-6gm9-8jxc-p862
    os.environ["WDS_PYTORCH_WEIGHTS_ONLY"] = "1"

    wds = lazy_import("webdataset")

    self.dataset = (
        wds.WebDataset(*args, **kwargs)
        .shuffle(10)  # shuffle individual samples before batching
        .decode()  # decode binary data
        .map(self._drop_metadata)  # remove non-tensor metadata
        .select(self._has_all_modalities)  # select samples with all modalities
        .map(self._sample_one_local_patch)  # sample one local patch for S1 and S2
        .map(self._sample_one_time_stamp)  # sample one timestamp for all modalities
        .map(self._to_tensors)  # convert to Keras tensors
    )

__iter__

__iter__()

Iterate over images in the dataset.

Yields: sample of images

Source code in keras_climate/datasets/copernicus/pretrain.py

def __iter__(self):
    """Iterate over images in the dataset.

    Yields:
        sample of images
    """
    return iter(self.dataset)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Args: sample: A sample returned by :meth:__iter__. show_titles: Flag indicating whether to show titles above each panel. suptitle: Optional string to use as a suptitle.

Returns: A matplotlib Figure with the rendered sample.

Source code in keras_climate/datasets/copernicus/pretrain.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Args:
        sample: A sample returned by :meth:`__iter__`.
        show_titles: Flag indicating whether to show titles above each panel.
        suptitle: Optional string to use as a suptitle.

    Returns:
        A matplotlib Figure with the rendered sample.
    """
    import matplotlib.pyplot as plt

    fig, ax = plt.subplots(nrows=2, ncols=4)

    # Channels-first (C, H, W) samples -> channels-last for display.
    image = sample["s1_grd.pth"]
    vv = image[0]
    vh = image[1]
    image = ops.stack([vv, vh, (vv + vh) / 2], axis=-1)
    image = quantile_normalization(image)
    ax[0, 0].imshow(ops.convert_to_numpy(image))
    ax[0, 0].axis("off")

    rgb_bands = [3, 2, 1]
    image = ops.take(sample["s2_toa.pth"], ops.convert_to_tensor(rgb_bands), axis=0)
    image = ops.cast(image, "float32")
    image = ops.transpose(image, (1, 2, 0))
    image = quantile_normalization(image)
    ax[0, 1].imshow(ops.convert_to_numpy(image))
    ax[0, 1].axis("off")

    rgb_bands = [7, 5, 3]
    image = ops.take(sample["s3_olci.pth"], ops.convert_to_tensor(rgb_bands), axis=0)
    image = ops.transpose(image, (1, 2, 0))
    image = quantile_normalization(image)
    ax[0, 2].imshow(ops.convert_to_numpy(image))
    ax[0, 2].axis("off")

    image = ops.convert_to_numpy(sample["dem.pth"])
    ax[0, 3].imshow(image, cmap="terrain")
    ax[0, 3].axis("off")

    image = ops.convert_to_numpy(sample["s5p_co.pth"])[0]
    ax[1, 0].imshow(image, cmap="Wistia")
    ax[1, 0].axis("off")

    image = ops.convert_to_numpy(sample["s5p_no2.pth"])[0]
    ax[1, 1].imshow(image, cmap="Wistia")
    ax[1, 1].axis("off")

    image = ops.convert_to_numpy(sample["s5p_o3.pth"])[0]
    ax[1, 2].imshow(image, cmap="Wistia")
    ax[1, 2].axis("off")

    image = ops.convert_to_numpy(sample["s5p_so2.pth"])[0]
    ax[1, 3].imshow(image, cmap="Wistia")
    ax[1, 3].axis("off")

    if show_titles:
        ax[0, 0].set_title("S1 GRD")
        ax[0, 1].set_title("S2 TOA")
        ax[0, 2].set_title("S3 OLCI")
        ax[0, 3].set_title("DEM")
        ax[1, 0].set_title("S5P CO")
        ax[1, 1].set_title("S5P NO$_2$")
        ax[1, 2].set_title("S5P O$_3$")
        ax[1, 3].set_title("S5P SO$_2$")

    if suptitle is not None:
        fig.suptitle(suptitle)

    fig.tight_layout()

    return fig

SpaceNet

keras_climate.datasets.spacenet.SpaceNet

SpaceNet(root='data', split='train', aois=None, image=None, mask=None, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset, ABC

Abstract base class for the SpaceNet datasets.

The SpaceNet <https://spacenet.ai/datasets/>__ datasets are a set of datasets that all together contain >11M building footprints and ~20,000 km of road labels mapped over high-resolution satellite imagery obtained from a variety of sensors such as Worldview-2, Worldview-3 and Dove.

.. note::

The SpaceNet datasets require the following additional library to be installed: AWS CLI <https://aws.amazon.com/cli/>_, to download the dataset from AWS.

Initialize a new SpaceNet Dataset instance.

Args: root: root directory where dataset can be found split: 'train' or 'test' split aois: areas of interest image: image selection mask: mask selection transforms: a function/transform that takes input sample and its target as entry and returns a transformed version. download: if True, download dataset and store it in the root directory. checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: If any invalid arguments are passed. DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/spacenet/base.py

def __init__(
    self,
    root="data",
    split="train",
    aois=None,
    image=None,
    mask=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new SpaceNet Dataset instance.

    Args:
        root: root directory where dataset can be found
        split: 'train' or 'test' split
        aois: areas of interest
        image: image selection
        mask: mask selection
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: if True, download dataset and store it in the root
            directory.
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: If any invalid arguments are passed.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    if aois is None:
        aois = []
    self.root = root
    self.split = split
    self.aois = aois or self.valid_aois[split]
    self.image = image or self.valid_images[split][0]
    self.mask = mask or self.valid_masks[0]
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    assert self.split in {"train", "test"}
    assert set(self.aois) <= set(self.valid_aois[split])
    assert self.image in self.valid_images[split]
    assert self.mask in self.valid_masks

    self._verify()

    if self.split == "train":
        assert len(self.images) == len(self.masks)

dataset_id abstractmethod property

dataset_id

Dataset ID.

tarballs abstractmethod property

tarballs

Mapping of tarballs[split][aoi] to a list of tarball names.

md5s abstractmethod property

md5s

Mapping of md5s[split][aoi] to a list of md5 checksums.

valid_aois abstractmethod property

valid_aois

Mapping of valid_aois[split] = [aois].

valid_images abstractmethod property

valid_images

Mapping of valid_images[split] = [images].

valid_masks abstractmethod property

valid_masks

List of valid masks.

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/spacenet/base.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image = quantile_normalization(sample["image"][..., :3])
    image = ops.convert_to_numpy(image)

    ncols = 1
    show_mask = "mask" in sample
    show_predictions = "prediction" in sample

    if show_mask:
        mask = ops.convert_to_numpy(sample["mask"])
        ncols += 1

    if show_predictions:
        prediction = ops.convert_to_numpy(sample["prediction"])
        ncols += 1

    fig, axs = plt.subplots(ncols=ncols, squeeze=False, figsize=(ncols * 8, 8))
    axs[0, 0].imshow(image)
    axs[0, 0].axis("off")
    if show_titles:
        axs[0, 0].set_title("Image")

    if show_mask:
        axs[0, 1].imshow(mask, interpolation="none")
        axs[0, 1].axis("off")
        if show_titles:
            axs[0, 1].set_title("Label")

    if show_predictions:
        axs[0, 2].imshow(prediction, interpolation="none")
        axs[0, 2].axis("off")
        if show_titles:
            axs[0, 2].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.spacenet.SpaceNet1

SpaceNet1(root='data', split='train', aois=None, image=None, mask=None, transforms=None, download=False, checksum=True)

Bases: SpaceNet

SpaceNet 1: Building Detection v1 Dataset.

SpaceNet 1 <https://spacenet.ai/spacenet-buildings-dataset-v1/>_ is a dataset of building footprints over the city of Rio de Janeiro.

Dataset features:

• No. of images: 6940 (8 Band) + 6940 (RGB)
• No. of polygons: 382,534 building labels
• Area Coverage: 2544 sq km
• GSD: 1 m (8 band), 50 cm (rgb)
• Chip size: 102 x 110 (8 band), 406 x 439 (rgb)

Dataset format:

• Imagery - Worldview-2 GeoTIFFs

  • 8Band.tif (Multispectral)
  • RGB.tif (Pansharpened RGB)

• Labels - GeoJSON

  • labels.geojson

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/1807.01232

Source code in keras_climate/datasets/spacenet/base.py

def __init__(
    self,
    root="data",
    split="train",
    aois=None,
    image=None,
    mask=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new SpaceNet Dataset instance.

    Args:
        root: root directory where dataset can be found
        split: 'train' or 'test' split
        aois: areas of interest
        image: image selection
        mask: mask selection
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: if True, download dataset and store it in the root
            directory.
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: If any invalid arguments are passed.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    if aois is None:
        aois = []
    self.root = root
    self.split = split
    self.aois = aois or self.valid_aois[split]
    self.image = image or self.valid_images[split][0]
    self.mask = mask or self.valid_masks[0]
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    assert self.split in {"train", "test"}
    assert set(self.aois) <= set(self.valid_aois[split])
    assert self.image in self.valid_images[split]
    assert self.mask in self.valid_masks

    self._verify()

    if self.split == "train":
        assert len(self.images) == len(self.masks)

keras_climate.datasets.spacenet.SpaceNet2

SpaceNet2(root='data', split='train', aois=None, image=None, mask=None, transforms=None, download=False, checksum=True)

Bases: SpaceNet

SpaceNet 2: Building Detection v2 Dataset.

SpaceNet 2 <https://spacenet.ai/spacenet-buildings-dataset-v2/>_ is a dataset of building footprints over the cities of Las Vegas, Paris, Shanghai and Khartoum.

Collection features (AOI, area km2, # images, # buildings):

• Las Vegas: 216, 3850, 151,367
• Paris: 1030, 1148, 23,816
• Shanghai: 1000, 4582, 92,015
• Khartoum: 765, 1012, 35,503

Imagery features (GSD in meters, chip size in px):

• PAN: 0.31 GSD, 650 x 650
• MS: 1.24 GSD, 163 x 163
• PS-MS: 0.30 GSD, 650 x 650
• PS-RGB: 0.30 GSD, 650 x 650

Dataset format:

• Imagery - Worldview-3 GeoTIFFs

  • PAN.tif (Panchromatic)
  • MS.tif (Multispectral)
  • PS-MS (Pansharpened Multispectral)
  • PS-RGB (Pansharpened RGB)

• Labels - GeoJSON

  • label.geojson

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/1807.01232

Source code in keras_climate/datasets/spacenet/base.py

def __init__(
    self,
    root="data",
    split="train",
    aois=None,
    image=None,
    mask=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new SpaceNet Dataset instance.

    Args:
        root: root directory where dataset can be found
        split: 'train' or 'test' split
        aois: areas of interest
        image: image selection
        mask: mask selection
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: if True, download dataset and store it in the root
            directory.
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: If any invalid arguments are passed.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    if aois is None:
        aois = []
    self.root = root
    self.split = split
    self.aois = aois or self.valid_aois[split]
    self.image = image or self.valid_images[split][0]
    self.mask = mask or self.valid_masks[0]
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    assert self.split in {"train", "test"}
    assert set(self.aois) <= set(self.valid_aois[split])
    assert self.image in self.valid_images[split]
    assert self.mask in self.valid_masks

    self._verify()

    if self.split == "train":
        assert len(self.images) == len(self.masks)

keras_climate.datasets.spacenet.SpaceNet3

SpaceNet3(root='data', split='train', aois=None, image=None, mask=None, transforms=None, download=False, checksum=True)

Bases: SpaceNet

SpaceNet 3: Road Network Detection.

SpaceNet 3 <https://spacenet.ai/spacenet-roads-dataset/>_ is a dataset of road networks over the cities of Las Vegas, Paris, Shanghai, and Khartoum.

Collection features (AOI, area km2, # images, road network labels in km):

• Vegas: 216, 854, 3685
• Paris: 1030, 257, 425
• Shanghai: 1000, 1028, 3537
• Khartoum: 765, 283, 1030

Imagery features (GSD in meters, chip size in px):

• PAN: 0.31 GSD, 1300 x 1300
• MS: 1.24 GSD, 325 x 325
• PS-MS: 0.30 GSD, 1300 x 1300
• PS-RGB: 0.30 GSD, 1300 x 1300

Dataset format:

• Imagery - Worldview-3 GeoTIFFs

  • PAN.tif (Panchromatic)
  • MS.tif (Multispectral)
  • PS-MS (Pansharpened Multispectral)
  • PS-RGB (Pansharpened RGB)

• Labels - GeoJSON

  • labels.geojson

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/1807.01232

Source code in keras_climate/datasets/spacenet/base.py

def __init__(
    self,
    root="data",
    split="train",
    aois=None,
    image=None,
    mask=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new SpaceNet Dataset instance.

    Args:
        root: root directory where dataset can be found
        split: 'train' or 'test' split
        aois: areas of interest
        image: image selection
        mask: mask selection
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: if True, download dataset and store it in the root
            directory.
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: If any invalid arguments are passed.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    if aois is None:
        aois = []
    self.root = root
    self.split = split
    self.aois = aois or self.valid_aois[split]
    self.image = image or self.valid_images[split][0]
    self.mask = mask or self.valid_masks[0]
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    assert self.split in {"train", "test"}
    assert set(self.aois) <= set(self.valid_aois[split])
    assert self.image in self.valid_images[split]
    assert self.mask in self.valid_masks

    self._verify()

    if self.split == "train":
        assert len(self.images) == len(self.masks)

keras_climate.datasets.spacenet.SpaceNet4

SpaceNet4(root='data', split='train', aois=None, image=None, mask=None, transforms=None, download=False, checksum=True)

Bases: SpaceNet

SpaceNet 4: Off-Nadir Buildings Dataset.

SpaceNet 4 <https://spacenet.ai/off-nadir-building-detection/>_ is a dataset of 27 WV-2 imagery captured at varying off-nadir angles and associated building footprints over the city of Atlanta. The off-nadir angle ranges from 7 degrees to 54 degrees.

Dataset features:

• No. of chipped images: 28,728 (PAN/MS/PS-RGBNIR)
• No. of label files: 1064
• No. of building footprints: >120,000
• Area Coverage: 665 sq km
• Chip size: 225 x 225 (MS), 900 x 900 (PAN/PS-RGBNIR)

Dataset format:

• Imagery - Worldview-2 GeoTIFFs

  • PAN.tif (Panchromatic)
  • MS.tif (Multispectral)
  • PS-RGBNIR (Pansharpened RGBNIR)

• Labels - GeoJSON

  • labels.geojson

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/1903.12239

Source code in keras_climate/datasets/spacenet/base.py

def __init__(
    self,
    root="data",
    split="train",
    aois=None,
    image=None,
    mask=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new SpaceNet Dataset instance.

    Args:
        root: root directory where dataset can be found
        split: 'train' or 'test' split
        aois: areas of interest
        image: image selection
        mask: mask selection
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: if True, download dataset and store it in the root
            directory.
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: If any invalid arguments are passed.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    if aois is None:
        aois = []
    self.root = root
    self.split = split
    self.aois = aois or self.valid_aois[split]
    self.image = image or self.valid_images[split][0]
    self.mask = mask or self.valid_masks[0]
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    assert self.split in {"train", "test"}
    assert set(self.aois) <= set(self.valid_aois[split])
    assert self.image in self.valid_images[split]
    assert self.mask in self.valid_masks

    self._verify()

    if self.split == "train":
        assert len(self.images) == len(self.masks)

keras_climate.datasets.spacenet.SpaceNet5

SpaceNet5(root='data', split='train', aois=None, image=None, mask=None, transforms=None, download=False, checksum=True)

Bases: SpaceNet3

SpaceNet 5: Automated Road Network Extraction and Route Travel Time Estimation.

SpaceNet 5 <https://spacenet.ai/sn5-challenge/>_ is a dataset of road networks over the cities of Moscow, Mumbai and San Juan (unavailable).

Collection features (AOI, area km2, # images, road network labels in km):

• Moscow: 1353, 1353, 3066
• Mumbai: 1021, 1016, 1951

Imagery features (GSD in meters, chip size in px):

• PAN: 0.31 GSD, 1300 x 1300
• MS: 1.24 GSD, 325 x 325
• PS-MS: 0.30 GSD, 1300 x 1300
• PS-RGB: 0.30 GSD, 1300 x 1300

Dataset format:

• Imagery - Worldview-3 GeoTIFFs

  • PAN.tif (Panchromatic)
  • MS.tif (Multispectral)
  • PS-MS (Pansharpened Multispectral)
  • PS-RGB (Pansharpened RGB)

• Labels - GeoJSON

  • labels.geojson

If you use this dataset in your research, please use the following citation:

• The SpaceNet Partners, "SpaceNet5: Automated Road Network Extraction and Route Travel Time Estimation from Satellite Imagery", https://spacenet.ai/sn5-challenge/

Source code in keras_climate/datasets/spacenet/base.py

def __init__(
    self,
    root="data",
    split="train",
    aois=None,
    image=None,
    mask=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new SpaceNet Dataset instance.

    Args:
        root: root directory where dataset can be found
        split: 'train' or 'test' split
        aois: areas of interest
        image: image selection
        mask: mask selection
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: if True, download dataset and store it in the root
            directory.
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: If any invalid arguments are passed.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    if aois is None:
        aois = []
    self.root = root
    self.split = split
    self.aois = aois or self.valid_aois[split]
    self.image = image or self.valid_images[split][0]
    self.mask = mask or self.valid_masks[0]
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    assert self.split in {"train", "test"}
    assert set(self.aois) <= set(self.valid_aois[split])
    assert self.image in self.valid_images[split]
    assert self.mask in self.valid_masks

    self._verify()

    if self.split == "train":
        assert len(self.images) == len(self.masks)

keras_climate.datasets.spacenet.SpaceNet6

SpaceNet6(root='data', split='train', aois=None, image=None, mask=None, transforms=None, download=False, checksum=True)

Bases: SpaceNet

SpaceNet 6: Multi-Sensor All-Weather Mapping.

SpaceNet 6 <https://spacenet.ai/sn6-challenge/>_ is a dataset of optical and SAR imagery over the city of Rotterdam.

Collection features: Rotterdam covers 120 sq km, 3401 images, ~48000 building footprint labels.

Imagery features (GSD in meters, chip size in px):

• PAN: 0.5 GSD, 900 x 900
• RGBNIR: 2.0 GSD, 450 x 450
• PS-RGB: 0.5 GSD, 900 x 900
• PS-RGBNIR: 0.5 GSD, 900 x 900
• SAR-Intensity: 0.5 GSD, 900 x 900

Dataset format:

• Imagery - GeoTIFFs from Worldview-2 (optical) and Capella Space (SAR)

  • PAN.tif (Panchromatic)
  • RGBNIR.tif (Multispectral)
  • PS-RGB (Pansharpened RGB)
  • PS-RGBNIR (Pansharpened RGBNIR)
  • SAR-Intensity (SAR Intensity)

• Labels - GeoJSON

  • labels.geojson

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2004.06500

Source code in keras_climate/datasets/spacenet/base.py

def __init__(
    self,
    root="data",
    split="train",
    aois=None,
    image=None,
    mask=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new SpaceNet Dataset instance.

    Args:
        root: root directory where dataset can be found
        split: 'train' or 'test' split
        aois: areas of interest
        image: image selection
        mask: mask selection
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: if True, download dataset and store it in the root
            directory.
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: If any invalid arguments are passed.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    if aois is None:
        aois = []
    self.root = root
    self.split = split
    self.aois = aois or self.valid_aois[split]
    self.image = image or self.valid_images[split][0]
    self.mask = mask or self.valid_masks[0]
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    assert self.split in {"train", "test"}
    assert set(self.aois) <= set(self.valid_aois[split])
    assert self.image in self.valid_images[split]
    assert self.mask in self.valid_masks

    self._verify()

    if self.split == "train":
        assert len(self.images) == len(self.masks)

keras_climate.datasets.spacenet.SpaceNet7

SpaceNet7(root='data', split='train', aois=None, image=None, mask=None, transforms=None, download=False, checksum=True)

Bases: SpaceNet

SpaceNet 7: Multi-Temporal Urban Development Challenge.

SpaceNet 7 <https://spacenet.ai/sn7-challenge/>_ is a dataset which consist of medium resolution (4.0m) satellite imagery mosaics acquired from Planet Labs' Dove constellation between 2017 and 2020. It includes ~24 images (one per month) covering > 100 unique geographies, and comprises > 40,000 km2 of imagery and exhaustive polygon labels of building footprints therein, totaling over 11M individual annotations.

Dataset features:

• No. of train samples: 1423
• No. of test samples: 466
• No. of building footprints: 11,080,000
• Area Coverage: 41,000 sq km
• Chip size: 1024 x 1024
• GSD: ~4m

Dataset format:

• Imagery - Planet Dove GeoTIFF

  • mosaic.tif

• Labels - GeoJSON

  • labels.geojson

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2102.04420

Source code in keras_climate/datasets/spacenet/base.py

def __init__(
    self,
    root="data",
    split="train",
    aois=None,
    image=None,
    mask=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new SpaceNet Dataset instance.

    Args:
        root: root directory where dataset can be found
        split: 'train' or 'test' split
        aois: areas of interest
        image: image selection
        mask: mask selection
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: if True, download dataset and store it in the root
            directory.
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: If any invalid arguments are passed.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    if aois is None:
        aois = []
    self.root = root
    self.split = split
    self.aois = aois or self.valid_aois[split]
    self.image = image or self.valid_images[split][0]
    self.mask = mask or self.valid_masks[0]
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    assert self.split in {"train", "test"}
    assert set(self.aois) <= set(self.valid_aois[split])
    assert self.image in self.valid_images[split]
    assert self.mask in self.valid_masks

    self._verify()

    if self.split == "train":
        assert len(self.images) == len(self.masks)

keras_climate.datasets.spacenet.SpaceNet8

SpaceNet8(root='data', split='train', aois=None, image=None, mask=None, transforms=None, download=False, checksum=True)

Bases: SpaceNet

SpaceNet8: Flood Detection Challenge Using Multiclass Segmentation.

SpaceNet 8 <https://spacenet.ai/sn8-challenge/>_ is a dataset focusing on infrastructure and flood mapping related to hurricanes and heavy rains that cause route obstructions and significant damage.

If you use this dataset in your research, please cite the following paper:

• https://openaccess.thecvf.com/content/CVPR2022W/EarthVision/html/Hansch_SpaceNet_8_-_The_Detection_of_Flooded_Roads_and_Buildings_CVPRW_2022_paper.html

Source code in keras_climate/datasets/spacenet/base.py

def __init__(
    self,
    root="data",
    split="train",
    aois=None,
    image=None,
    mask=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new SpaceNet Dataset instance.

    Args:
        root: root directory where dataset can be found
        split: 'train' or 'test' split
        aois: areas of interest
        image: image selection
        mask: mask selection
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: if True, download dataset and store it in the root
            directory.
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: If any invalid arguments are passed.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    if aois is None:
        aois = []
    self.root = root
    self.split = split
    self.aois = aois or self.valid_aois[split]
    self.image = image or self.valid_images[split][0]
    self.mask = mask or self.valid_masks[0]
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    assert self.split in {"train", "test"}
    assert set(self.aois) <= set(self.valid_aois[split])
    assert self.image in self.valid_images[split]
    assert self.mask in self.valid_masks

    self._verify()

    if self.split == "train":
        assert len(self.images) == len(self.masks)

Dataset Catalog

keras_climate.datasets.advance.ADVANCE

ADVANCE(root='data', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

ADVANCE dataset.

The ADVANCE <https://akchen.github.io/ADVANCE-DATASET/>__ dataset is a dataset for audio visual scene recognition.

Dataset features:

• 5,075 pairs of geotagged audio recordings and images
• three spectral bands - RGB (512x512 px)
• 10-second audio recordings

Dataset classes:

1. airport, 1. beach, 2. bridge, 3. farmland, 4. forest, 5. grassland,
2. harbour, 7. lake, 8. orchard, 9. residential, 10. sparse shrub land,
3. sports land, 12. train station

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1007/978-3-030-58586-0_5

.. note:: This dataset requires scipy to be installed to load the audio files.

Initialize a new ADVANCE dataset instance.

Args: root: root directory where dataset can be found transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False. DependencyNotFoundError: If scipy is not installed.

Source code in keras_climate/datasets/advance.py

def __init__(self, root="data", transforms=None, download=False, checksum=True):
    """Initialize a new ADVANCE dataset instance.

    Args:
        root: root directory where dataset can be found
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
        DependencyNotFoundError: If scipy is not installed.
    """
    lazy_import("scipy.io.wavfile")

    self.root = root
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    if download:
        self._download()

    if not self._check_integrity():
        raise DatasetNotFoundError(self)

    self.files = self._load_files(self.root)
    self.classes = tuple(sorted({f["cls"] for f in self.files}))
    self.class_to_idx = {c: i for i, c in enumerate(self.classes)}

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/advance.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"])
    label = int(ops.convert_to_numpy(sample["label"]))
    label_class = self.classes[label]

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = int(ops.convert_to_numpy(sample["prediction"]))
        prediction_class = self.classes[prediction]

    fig, ax = plt.subplots(figsize=(4, 4))
    ax.imshow(image.astype("uint8"))
    ax.axis("off")
    if show_titles:
        title = f"Label: {label_class}"
        if showing_predictions:
            title += f"\nPrediction: {prediction_class}"
        ax.set_title(title)

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.agb_live_woody_density.AbovegroundLiveWoodyBiomassDensity

AbovegroundLiveWoodyBiomassDensity(paths='data', crs=None, res=None, transforms=None, download=False, cache=True, time_series=False)

Bases: RasterDataset

Aboveground Live Woody Biomass Density dataset.

The Aboveground Live Woody Biomass Density dataset <https://data.globalforestwatch.org/datasets/gfw::aboveground-live-woody -biomass-density/about>_ is a global-scale, wall-to-wall map of aboveground biomass at ~30m resolution for the year 2000.

Dataset format:

• geojson file that contains download links to tif files
• single-channel geotiffs with the pixel values representing biomass density

If you use this dataset in your research, please give credit to:

• Global Forest Watch <https://data.globalforestwatch.org/>_

Initialize a new Dataset instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS (defaults to the resolution of the first file found) transforms: a function/transform that takes an input sample and returns a transformed version download: if True, download dataset and store it in the root directory cache: if True, cache file handle to speed up repeated sampling time_series: if True, stack data along the time series dimension

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/agb_live_woody_density.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    transforms=None,
    download=False,
    cache=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.paths = paths
    self.download = download

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/agb_live_woody_density.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt
    import numpy as np
    from keras import ops

    mask = np.squeeze(ops.convert_to_numpy(sample["mask"]))
    ncols = 1

    showing_predictions = "prediction" in sample
    if showing_predictions:
        pred = np.squeeze(ops.convert_to_numpy(sample["prediction"]))
        ncols = 2

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 4, 4))

    if showing_predictions:
        axs[0].imshow(mask)
        axs[0].axis("off")
        axs[1].imshow(pred)
        axs[1].axis("off")
        if show_titles:
            axs[0].set_title("Mask")
            axs[1].set_title("Prediction")
    else:
        axs.imshow(mask)
        axs.axis("off")
        if show_titles:
            axs.set_title("Mask")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.agrifieldnet.AgriFieldNet

AgriFieldNet(paths='data', crs=None, res=None, classes=None, bands=AgriFieldNetImage.all_bands, transforms=None, cache=True, download=False, time_series=False)

Bases: IntersectionDataset

AgriFieldNet India Challenge dataset.

The AgriFieldNet India Challenge <https://zindi.world/competitions/agrifieldnet-india-challenge>__ dataset includes satellite imagery from Sentinel-2 cloud free composites (single snapshot) and labels for crop type that were collected by ground survey. The Sentinel-2 data are then matched with corresponding labels. The dataset contains 7081 fields, split into training and test sets (5551 fields train, 1530 fields test). Satellite imagery and labels are tiled into 256x256 chips adding up to 1217 tiles. If the field ID for a pixel is set to 0 it means that pixel is not included in either the train or test set (and correspondingly the crop label will be 0 as well). The original dataset can be downloaded from Source Cooperative <https://source.coop/radiantearth/agrifieldnet-competition>__.

Dataset format:

• images are 12-band Sentinel-2 data
• masks are tiff images with unique values representing the class and field id

Dataset classes:

• 1. No-Data
• 1. Wheat
• 1. Mustard
• 1. Lentil
• 1. No Crop/Fallow
• 1. Green pea
• 1. Sugarcane
• 1. Garlic
• 1. Maize
• 1. Gram
• 1. Coriander
• 1. Potato
• 1. Berseem
• 1. Rice

If you use this dataset in your research, please cite the following dataset:

• https://doi.org/10.34911/rdnt.wu92p1

.. note::

This dataset requires the following additional library to be installed:

• azcopy <https://github.com/Azure/azure-storage-azcopy>_: to download the dataset from Source Cooperative.

Initialize a new AgriFieldNet dataset instance.

Args: paths: one or more root directories to search for files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS (defaults to the resolution of the first file found) classes: list of classes to include, the rest will be mapped to 0 (defaults to all classes) bands: the subset of bands to load transforms: a function/transform that takes input sample and its target as entry and returns a transformed version cache: if True, cache the dataset in memory download: if True, download dataset and store it in the root directory time_series: if True, stack data along the time series dimension

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/agrifieldnet.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    classes=None,
    bands=AgriFieldNetImage.all_bands,
    transforms=None,
    cache=True,
    download=False,
    time_series=False,
):
    """Initialize a new AgriFieldNet dataset instance.

    Args:
        paths: one or more root directories to search for files to load
        crs: CRS to warp to (defaults to the CRS of the first file
            found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        classes: list of classes to include, the rest will be mapped to
            0 (defaults to all classes)
        bands: the subset of bands to load
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        cache: if True, cache the dataset in memory
        download: if True, download dataset and store it in the root
            directory
        time_series: if True, stack data along the time series dimension

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    if classes is None:
        classes = list(AgriFieldNetMask.valid_classes)
    assert set(classes) <= set(AgriFieldNetMask.valid_classes), (
        f"Only the following classes are valid: {AgriFieldNetMask.valid_classes}."
    )
    assert 0 in classes, "Classes must include the background class: 0"

    self.paths = paths
    self.download = download
    AgriFieldNetImage.filename_glob = AgriFieldNetImage.filename_glob.format(
        bands[0]
    )

    self._verify()

    self.image = AgriFieldNetImage(
        paths, crs, res, bands, transforms, cache, time_series
    )
    self.mask = AgriFieldNetMask(
        paths, crs, res, None, transforms, cache, time_series
    )

    super().__init__(self.image, self.mask)

    # Ignore unintentional partial overlap
    self.index = self.image.index

    # Map chosen classes to ordinal numbers, all others mapped to background class
    self.ordinal_map = np.zeros(
        self.mask.valid_classes[-1] + 1, dtype=self.mask.dtype
    )
    self.inverse_map = np.zeros(len(classes), dtype=self.mask.dtype)
    for v, k in enumerate(classes):
        self.ordinal_map[k] = v
        self.inverse_map[v] = k

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/agrifieldnet.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.image.rgb_bands:
        if band in self.image.bands:
            rgb_indices.append(self.image.bands.index(band))
        else:
            raise RGBBandsMissingError()

    image = ops.take(sample["image"], rgb_indices, axis=-1)
    image = quantile_normalization(image)
    image = ops.convert_to_numpy(image)

    mask_np = ops.convert_to_numpy(sample["mask"]).astype("int64")
    mask = self.inverse_map[mask_np]
    ncols = 2

    showing_prediction = "prediction" in sample
    if showing_prediction:
        pred_np = ops.convert_to_numpy(sample["prediction"]).astype("int64")
        pred = self.inverse_map[pred_np]
        ncols += 1

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 4, 4))
    kwargs = {
        "cmap": self.mask.cmap,
        "vmin": self.mask.valid_classes[0],
        "vmax": self.mask.valid_classes[-1],
        "interpolation": "none",
    }

    axs[0].imshow(image)
    axs[0].axis("off")
    axs[1].imshow(mask, **kwargs)
    axs[1].axis("off")
    if show_titles:
        axs[0].set_title("Image")
        axs[1].set_title("Mask")

    if showing_prediction:
        axs[2].imshow(pred, **kwargs)
        axs[2].axis("off")
        if show_titles:
            axs[2].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.agrifieldnet.AgriFieldNetImage

AgriFieldNetImage(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset

AgriFieldNet Sentinel-2 imagery.

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new RasterDataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a mosaic
            (``[H, W, C]``). For mask-style datasets (``is_image=False``),
            single-band data has the channel dimension squeezed.
    """
    self.paths = paths
    self.bands = bands or self.all_bands
    self.transforms = transforms
    self.cache = cache
    self.time_series = time_series

    if self.all_bands:
        assert set(self.bands) <= set(self.all_bands)

    filename_regex = re.compile(self.filename_regex, re.VERBOSE)
    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        match = re.match(filename_regex, os.path.basename(filepath))
        if match is not None:
            vrt = None
            try:
                vrt = self._load_warp_file(filepath=filepath, crs=crs)
                if self.cmap is None:
                    try:
                        colors = np.array(list(vrt.colormap(1).values())) / 255
                        self.cmap = ListedColormap(colors)
                    except ValueError:
                        pass
                if crs is None:
                    with rasterio.Env(OSR_WKT_FORMAT="WKT2_2018"):
                        crs = PROJ_CRS.from_user_input(vrt.crs)
                footprint = self.footprint_from_datasource(vrt)
                if footprint is None:
                    footprint = shapely.box(*vrt.bounds)
                geometries.append(footprint)
                if res is None:
                    res = vrt.res
            except rasterio.errors.RasterioIOError:
                continue
            else:
                filepaths.append(filepath)
                mint, maxt = self._filepath_to_timestamp(filepath)
                datetimes.append((mint, maxt))
            finally:
                if vrt is not None:
                    vrt.close()

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if not self.separate_files:
        self.band_indexes = None
        if self.bands:
            if self.all_bands:
                self.band_indexes = [
                    self.all_bands.index(i) + 1 for i in self.bands
                ]
            else:
                msg = (
                    f"{self.__class__.__name__} is missing an `all_bands` "
                    "attribute, so `bands` cannot be specified."
                )
                raise AssertionError(msg)

    if res is not None:
        if isinstance(res, (int, float)):
            res = (res, res)

        self._res = res

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

keras_climate.datasets.agrifieldnet.AgriFieldNetMask

AgriFieldNetMask(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset

AgriFieldNet masks.

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new RasterDataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a mosaic
            (``[H, W, C]``). For mask-style datasets (``is_image=False``),
            single-band data has the channel dimension squeezed.
    """
    self.paths = paths
    self.bands = bands or self.all_bands
    self.transforms = transforms
    self.cache = cache
    self.time_series = time_series

    if self.all_bands:
        assert set(self.bands) <= set(self.all_bands)

    filename_regex = re.compile(self.filename_regex, re.VERBOSE)
    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        match = re.match(filename_regex, os.path.basename(filepath))
        if match is not None:
            vrt = None
            try:
                vrt = self._load_warp_file(filepath=filepath, crs=crs)
                if self.cmap is None:
                    try:
                        colors = np.array(list(vrt.colormap(1).values())) / 255
                        self.cmap = ListedColormap(colors)
                    except ValueError:
                        pass
                if crs is None:
                    with rasterio.Env(OSR_WKT_FORMAT="WKT2_2018"):
                        crs = PROJ_CRS.from_user_input(vrt.crs)
                footprint = self.footprint_from_datasource(vrt)
                if footprint is None:
                    footprint = shapely.box(*vrt.bounds)
                geometries.append(footprint)
                if res is None:
                    res = vrt.res
            except rasterio.errors.RasterioIOError:
                continue
            else:
                filepaths.append(filepath)
                mint, maxt = self._filepath_to_timestamp(filepath)
                datetimes.append((mint, maxt))
            finally:
                if vrt is not None:
                    vrt.close()

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if not self.separate_files:
        self.band_indexes = None
        if self.bands:
            if self.all_bands:
                self.band_indexes = [
                    self.all_bands.index(i) + 1 for i in self.bands
                ]
            else:
                msg = (
                    f"{self.__class__.__name__} is missing an `all_bands` "
                    "attribute, so `bands` cannot be specified."
                )
                raise AssertionError(msg)

    if res is not None:
        if isinstance(res, (int, float)):
            res = (res, res)

        self._res = res

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

keras_climate.datasets.air_quality.AirQuality

AirQuality(root='data', *, input_steps=3, target_steps=1, input_features=None, target_features=None, download=False)

Bases: NonGeoDataset

Air Quality dataset.

The Air Quality dataset <https://archive.ics.uci.edu/dataset/360/air+quality>_ from the UCI Machine Learning Repository is a multivariate time series dataset containing air quality measurements from an Italian city.

Dataset Format:

• .csv file containing date, time and air quality measurements

Dataset Features:

• hourly averaged sensor responses and reference analyzer ground truth over one year (2004-2005)
• contains missing features, gap filled using linear interpolation

.. note:: There are actually two different versions of this dataset with major formatting differences, including comma-delimited vs. semicolon-delimited, empty rows and columns, and differences in datetime formatting. This dataset currently only supports the comma-delimited version.

If you use this dataset in your research, please cite:

• https://doi.org/10.1016/J.SNB.2007.09.060

Initialize a new Dataset instance.

Args: root: Root directory where dataset can be found. input_steps: Number of input time steps to use. target_steps: Number of target time steps to use. input_features: List of input features to load (uses all features by default). target_features: List of target features to load (uses all features by default). download: If True, download dataset and store it in the root directory.

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/air_quality.py

def __init__(
    self,
    root="data",
    *,
    input_steps=3,
    target_steps=1,
    input_features=None,
    target_features=None,
    download=False,
):
    """Initialize a new Dataset instance.

    Args:
        root: Root directory where dataset can be found.
        input_steps: Number of input time steps to use.
        target_steps: Number of target time steps to use.
        input_features: List of input features to load (uses all
            features by default).
        target_features: List of target features to load (uses all
            features by default).
        download: If True, download dataset and store it in the root
            directory.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.root = pathlib.Path(root)
    self.download = download
    self.input_steps = input_steps
    self.target_steps = target_steps
    self.input_features = input_features
    self.target_features = target_features
    self._load_data()

plot

plot(sample, show_titles=True, suptitle=None, features=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/air_quality.py

def plot(self, sample, show_titles=True, suptitle=None, features=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    ylabel = {
        "CO(GT)": "CO (mg/m$^3$)",
        "PT08.S1(CO)": "CO",
        "NMHC(GT)": "NMHC (μg/m$^3$)",
        "C6H6(GT)": "C$_6$H$_6$ (μg/m$^3$)",
        "PT08.S2(NMHC)": "NHMC",
        "NOx(GT)": "NO$_x$ (ppb)",
        "PT08.S3(NOx)": "NO$_x$",
        "NO2(GT)": "NO$_2$ (μg/m$^3$)",
        "PT08.S4(NO2)": "NO$_2$",
        "PT08.S5(O3)": "O$_3$",
        "T": "Temperature (°C)",
        "RH": "Relative Humidity (%)",
        "AH": "Absolute Humidity",
    }

    sample_input = ops.convert_to_numpy(sample["input"])
    sample_target = ops.convert_to_numpy(sample["target"])

    input_steps = range(len(sample_input))
    target_steps = range(len(sample_input), len(sample_input) + len(sample_target))

    features = features or self.target_data.columns
    n_features = len(features)

    ncols = math.ceil(math.sqrt(n_features))
    nrows = math.ceil(n_features / ncols)

    fig, axes = plt.subplots(
        nrows, ncols, figsize=(5 * ncols, 3 * nrows), squeeze=False
    )
    axes = axes.ravel()

    for ax, feature in zip(axes, features):
        if show_titles:
            ax.set_title(feature)

        if feature in self.input_data:
            idx = self.input_data.columns.get_loc(feature)
            data = sample_input[:, idx]
            ax.plot(input_steps, data, label="Input", marker="o")

        if feature in self.target_data:
            idx = self.target_data.columns.get_loc(feature)
            data = sample_target[:, idx]
            ax.plot(target_steps, data, label="Target", marker="x")

            if "prediction" in sample:
                data = ops.convert_to_numpy(sample["prediction"])[:, idx]
                ax.plot(target_steps, data, label="Prediction", marker="^")

        ax.legend()
        if feature in ylabel:
            ax.set_ylabel(ylabel[feature])

    for ax in axes[n_features:]:
        ax.set_visible(False)

    if suptitle is not None:
        plt.suptitle(suptitle)

    fig.tight_layout()
    return fig

keras_climate.datasets.airphen.Airphen

Airphen(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset

Airphen dataset.

Airphen <https://ideol.sakura.ne.jp/img/20170123_HiphenAirphenKeyfeatures.pdf>__ is a multispectral scientific camera developed by agronomists and photonics engineers at Hiphen <https://www.hiphen-plant.com/>_ to match plant measurement needs and constraints.

Main characteristics:

• 6 Synchronized global shutter sensors
• Sensor resolution 1280 x 960 pixels
• Data format (.tiff, 12 bit)
• SD card storage
• Metadata information: Exif and XMP
• Internal or external GPS
• Synchronization with different sensors (TIR, RGB, others)

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.34133/2021/9892647

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new RasterDataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a mosaic
            (``[H, W, C]``). For mask-style datasets (``is_image=False``),
            single-band data has the channel dimension squeezed.
    """
    self.paths = paths
    self.bands = bands or self.all_bands
    self.transforms = transforms
    self.cache = cache
    self.time_series = time_series

    if self.all_bands:
        assert set(self.bands) <= set(self.all_bands)

    filename_regex = re.compile(self.filename_regex, re.VERBOSE)
    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        match = re.match(filename_regex, os.path.basename(filepath))
        if match is not None:
            vrt = None
            try:
                vrt = self._load_warp_file(filepath=filepath, crs=crs)
                if self.cmap is None:
                    try:
                        colors = np.array(list(vrt.colormap(1).values())) / 255
                        self.cmap = ListedColormap(colors)
                    except ValueError:
                        pass
                if crs is None:
                    with rasterio.Env(OSR_WKT_FORMAT="WKT2_2018"):
                        crs = PROJ_CRS.from_user_input(vrt.crs)
                footprint = self.footprint_from_datasource(vrt)
                if footprint is None:
                    footprint = shapely.box(*vrt.bounds)
                geometries.append(footprint)
                if res is None:
                    res = vrt.res
            except rasterio.errors.RasterioIOError:
                continue
            else:
                filepaths.append(filepath)
                mint, maxt = self._filepath_to_timestamp(filepath)
                datetimes.append((mint, maxt))
            finally:
                if vrt is not None:
                    vrt.close()

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if not self.separate_files:
        self.band_indexes = None
        if self.bands:
            if self.all_bands:
                self.band_indexes = [
                    self.all_bands.index(i) + 1 for i in self.bands
                ]
            else:
                msg = (
                    f"{self.__class__.__name__} is missing an `all_bands` "
                    "attribute, so `bands` cannot be specified."
                )
                raise AssertionError(msg)

    if res is not None:
        if isinstance(res, (int, float)):
            res = (res, res)

        self._res = res

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/airphen.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise RGBBandsMissingError()

    image = ops.take(sample["image"], rgb_indices, axis=-1)
    image = ops.cast(image, "float32")
    image = quantile_normalization(image)

    fig, ax = plt.subplots(1, 1, figsize=(4, 4))
    ax.imshow(ops.convert_to_numpy(image))
    ax.axis("off")

    if show_titles:
        ax.set_title("Image")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.astergdem.AsterGDEM

AsterGDEM(paths='data', crs=None, res=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset

Aster Global Digital Elevation Model Dataset.

The Aster Global Digital Elevation Model <https://www.earthdata.nasa.gov/data/catalog/lpcloud-astgtm-003>_ dataset is a Digital Elevation Model (DEM) on a global scale. The dataset can be downloaded from the Earth Data website <https://search.earthdata.nasa.gov/search/>_ after making an account.

Dataset features:

• DEMs at 30 m per pixel spatial resolution (3601x3601 px)
• data collected from the Aster <https://terra.nasa.gov/about/terra-instruments/aster>_ instrument

Dataset format:

• DEMs are single-channel tif files

Initialize a new Dataset instance.

Args: paths: one or more root directories to search or files to load, here the collection of individual zip files for each tile should be found crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS (defaults to the resolution of the first file found) transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling time_series: if True, stack data along the time series dimension

Raises: DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/astergdem.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load,
            here the collection of individual zip files for each tile
            should be found
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension

    Raises:
        DatasetNotFoundError: If dataset is not found.
    """
    self.paths = paths

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/astergdem.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt
    import numpy as np
    from keras import ops

    mask = np.squeeze(ops.convert_to_numpy(sample["mask"]))
    ncols = 1

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = np.squeeze(ops.convert_to_numpy(sample["prediction"]))
        ncols = 2

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(4 * ncols, 4))

    if showing_predictions:
        axs[0].imshow(mask)
        axs[0].axis("off")
        axs[1].imshow(prediction)
        axs[1].axis("off")
        if show_titles:
            axs[0].set_title("Mask")
            axs[1].set_title("Prediction")
    else:
        axs.imshow(mask)
        axs.axis("off")
        if show_titles:
            axs.set_title("Mask")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.benin_cashews.BeninSmallHolderCashews

BeninSmallHolderCashews(root='data', chip_size=256, stride=128, bands=all_bands, transforms=None, download=False)

Bases: NonGeoDataset

Smallholder Cashew Plantations in Benin dataset.

This dataset contains labels for cashew plantations in a 120 km\ :sup:2 area in the center of Benin. Each pixel is classified for Well-managed plantation, Poorly-managed plantation, No plantation and other classes. The labels are generated using a combination of ground data collection with a handheld GPS device, and final corrections based on Airbus Pléiades imagery. See this website <https://source.coop/technoserve/cashews-benin>__ for dataset details.

Specifically, the data consists of Sentinel 2 imagery from a 120 km\ :sup:2 area in the center of Benin over 71 points in time from 11/05/2019 to 10/30/2020 and polygon labels for 6 classes:

1. No data
2. Well-managed plantation
3. Poorly-managed planatation
4. Non-plantation
5. Residential
6. Background
7. Uncertain

If you use this dataset in your research, please cite the following:

• https://source.coop/technoserve/cashews-benin/

.. note::

This dataset requires the following additional library to be installed:

• azcopy <https://github.com/Azure/azure-storage-azcopy>_: to download the dataset from Source Cooperative.

Initialize a new Benin Smallholder Cashew Plantations Dataset instance.

Args: root: root directory where dataset can be found chip_size: size of chips stride: spacing between chips, if less than chip_size, then there will be overlap between chips bands: the subset of bands to load transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory

Raises: AssertionError: If bands is invalid. DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/benin_cashews.py

def __init__(
    self,
    root="data",
    chip_size=256,
    stride=128,
    bands=all_bands,
    transforms=None,
    download=False,
):
    """Initialize a new Benin Smallholder Cashew Plantations Dataset
    instance.

    Args:
        root: root directory where dataset can be found
        chip_size: size of chips
        stride: spacing between chips, if less than chip_size, then
            there will be overlap between chips
        bands: the subset of bands to load
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory

    Raises:
        AssertionError: If *bands* is invalid.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert set(bands) <= set(self.all_bands)

    self.root = root
    self.chip_size = chip_size
    self.stride = stride
    self.bands = bands
    self.transforms = transforms
    self.download = download

    self._verify()

    # Calculate the indices that we will use over all tiles
    self.chips_metadata = []
    for y in [
        *list(range(0, self.tile_height - self.chip_size, stride)),
        self.tile_height - self.chip_size,
    ]:
        for x in [
            *list(range(0, self.tile_width - self.chip_size, stride)),
            self.tile_width - self.chip_size,
        ]:
            self.chips_metadata.append((y, x))

plot

plot(sample, show_titles=True, suptitle=None, time_step=0)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/benin_cashews.py

def plot(self, sample, show_titles=True, suptitle=None, time_step=0):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise RGBBandsMissingError()

    image = ops.convert_to_numpy(sample["image"])[time_step]
    image = np.take(image, rgb_indices, axis=-1)
    image = np.clip(image / 3000, 0, 1)
    mask = ops.convert_to_numpy(sample["mask"])

    num_panels = 2
    showing_predictions = "prediction" in sample
    if showing_predictions:
        predictions = ops.convert_to_numpy(sample["prediction"])
        num_panels += 1

    fig, axs = plt.subplots(ncols=num_panels, figsize=(4 * num_panels, 4))

    axs[0].imshow(image)
    axs[0].axis("off")
    if show_titles:
        axs[0].set_title(f"t={time_step}")

    axs[1].imshow(mask, vmin=0, vmax=6, interpolation="none")
    axs[1].axis("off")
    if show_titles:
        axs[1].set_title("Mask")

    if showing_predictions:
        axs[2].imshow(predictions, vmin=0, vmax=6, interpolation="none")
        axs[2].axis("off")
        if show_titles:
            axs[2].set_title("Predictions")

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.bigearthnet.BigEarthNet

BigEarthNet(root='data', split='train', bands='all', num_classes=19, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

BigEarthNet dataset.

The BigEarthNet <https://bigearth.net/>__ dataset is a dataset for multilabel remote sensing image scene classification.

Dataset features:

• 590,326 patches from 125 Sentinel-1 and Sentinel-2 tiles
• Imagery from tiles in Europe between Jun 2017 - May 2018
• 12 spectral bands with 10-60 m per pixel resolution (base 120x120 px)
• 2 synthetic aperture radar bands (120x120 px)
• 43 or 19 scene classes from the 2018 CORINE Land Cover database (CLC 2018)

Dataset format:

• images are composed of multiple single channel geotiffs
• labels are multiclass, stored in a single json file per image
• mapping of Sentinel-1 to Sentinel-2 patches are within Sentinel-1 json files
• Sentinel-1 bands: (VV, VH)
• Sentinel-2 bands: (B01, B02, B03, B04, B05, B06, B07, B08, B8A, B09, B11, B12)
• All bands: (VV, VH, B01, B02, B03, B04, B05, B06, B07, B08, B8A, B09, B11, B12)
• Sentinel-2 bands are of different spatial resolutions and upsampled to 10m

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1109/IGARSS.2019.8900532

Initialize a new BigEarthNet dataset instance.

Args: root: root directory where dataset can be found split: train/val/test split to load bands: load Sentinel-1 bands, Sentinel-2, or both. one of {s1, s2, all} num_classes: number of classes to load in target. one of {19, 43} transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/bigearthnet.py

def __init__(
    self,
    root="data",
    split="train",
    bands="all",
    num_classes=19,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new BigEarthNet dataset instance.

    Args:
        root: root directory where dataset can be found
        split: train/val/test split to load
        bands: load Sentinel-1 bands, Sentinel-2, or both. one of
            {s1, s2, all}
        num_classes: number of classes to load in target. one of
            {19, 43}
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in self.splits_metadata
    assert bands in ["s1", "s2", "all"]
    assert num_classes in [43, 19]
    self.root = root
    self.split = split
    self.bands = bands
    self.num_classes = num_classes
    self.transforms = transforms
    self.download = download
    self.checksum = checksum
    self.class2idx = {c: i for i, c in enumerate(self.class_sets[43])}
    self._verify()
    self.folders = self._load_folders()

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/bigearthnet.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image_np = ops.convert_to_numpy(sample["image"])
    if self.bands == "s2":
        image = np.take(image_np, [3, 2, 1], axis=-1)
        image = np.clip(image / 2000, 0, 1)
    elif self.bands == "all":
        image = np.take(image_np, [5, 4, 3], axis=-1)
        image = np.clip(image / 2000, 0, 1)
    elif self.bands == "s1":
        image = image_np[..., 0]

    label_mask = ops.convert_to_numpy(sample["label"]).astype(np.bool_)
    labels = self._onehot_labels_to_names(label_mask)

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction_mask = ops.convert_to_numpy(sample["prediction"]).astype(
            np.bool_
        )
        predictions = self._onehot_labels_to_names(prediction_mask)

    fig, ax = plt.subplots(figsize=(4, 4))
    ax.imshow(image)
    ax.axis("off")
    if show_titles:
        title = f"Labels: {', '.join(labels)}"
        if showing_predictions:
            title += f"\nPredictions: {', '.join(predictions)}"
        ax.set_title(title)

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.bigearthnet.BigEarthNetV2

BigEarthNetV2(root='data', split='train', bands='all', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

BigEarthNetV2 dataset.

The BigEarthNet V2 <https://bigearth.net/>__ dataset contains improved labels, improved geospatial data splits and additionally pixel-level labels from CORINE Land Cover (CLC) map of 2018. Additionally, some problematic patches from V1 have been removed.

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2407.03653

Initialize a new BigEarthNet V2 dataset instance.

Args: root: root directory where dataset can be found split: train/val/test split to load bands: load Sentinel-1 bands, Sentinel-2, or both. one of {s1, s2, all} transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False. AssertionError: If split, or bands, are not valid.

Source code in keras_climate/datasets/bigearthnet.py

def __init__(
    self,
    root="data",
    split="train",
    bands="all",
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new BigEarthNet V2 dataset instance.

    Args:
        root: root directory where dataset can be found
        split: train/val/test split to load
        bands: load Sentinel-1 bands, Sentinel-2, or both. one of
            {s1, s2, all}
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
        AssertionError: If *split*, or *bands*, are not valid.
    """
    assert split in self.valid_splits, f"split must be one of {self.valid_splits}"
    assert bands in ["s1", "s2", "all"]
    self.root = root
    self.split = split
    self.bands = bands
    self.transforms = transforms
    self.num_classes = 19
    self.download = download
    self.checksum = checksum
    self.class2idx = {c: i for i, c in enumerate(self.class_set)}
    self._verify()

    self.metadata_df = pd.read_parquet(os.path.join(self.root, "metadata.parquet"))
    self.metadata_df = self.metadata_df[
        self.metadata_df["split"] == self.split_map[self.split]
    ].reset_index(drop=True)

    # Map chosen classes to ordinal numbers, all others mapped to background class
    self.ordinal_map = np.zeros(19, dtype="int64")
    for corine, ordinal in self.clc_codes.items():
        self.ordinal_map[ordinal] = corine

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/bigearthnet.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    fig, axes = plt.subplots(1, 2 if self.bands != "all" else 3, figsize=(12, 4))

    if self.bands in ["s2", "all"]:
        s2_img = ops.convert_to_numpy(
            sample["image_s2" if self.bands == "all" else "image"]
        )
        rgb = np.take(s2_img, [3, 2, 1], axis=-1)
        axes[0].imshow(np.clip(rgb / 2000, 0, 1))
        if show_titles:
            axes[0].set_title("Sentinel-2 RGB")
        axes[0].axis("off")

    if self.bands in ["s1", "all"]:
        idx = 0 if self.bands == "s1" else 1
        s1_img = ops.convert_to_numpy(
            sample["image_s1" if self.bands == "all" else "image"]
        )
        axes[idx].imshow(s1_img[..., 0])
        if show_titles:
            axes[idx].set_title("Sentinel-1 VV")
        axes[idx].axis("off")

    # Handle mask plotting
    mask_idx = 1 if self.bands != "all" else 2
    mask = ops.convert_to_numpy(sample["mask"])[..., 0]

    unique_labels = sorted(np.unique(mask))

    colors = []
    class_names = []
    for label in unique_labels:
        name = self.class_set[label]
        colors.append(self.clc_colors[name])
        class_names.append(name)

    cmap = ListedColormap(colors)
    bounds = [*unique_labels, unique_labels[-1] + 1]
    norm = BoundaryNorm(bounds, len(colors))

    axes[mask_idx].imshow(mask, cmap=cmap, norm=norm)

    legend_elements = [Rectangle((0, 0), 1, 1, facecolor=color) for color in colors]
    wrapped_names = [textwrap.fill(name, width=25) for name in class_names]
    axes[mask_idx].legend(
        legend_elements,
        wrapped_names,
        loc="center left",
        bbox_to_anchor=(1, 0.5),
        fontsize="x-small",
    )
    axes[mask_idx].axis("off")

    if show_titles:
        axes[mask_idx].set_title("Land Cover Map")

    if "label" in sample:
        label_np = ops.convert_to_numpy(sample["label"])
        label_indices = np.nonzero(label_np)[0].tolist()
        label_names = [self.class_set[idx] for idx in label_indices]
        if suptitle:
            suptitle = f"{suptitle}\nLabels: {', '.join(label_names)}"
        else:
            suptitle = f"Labels: {', '.join(label_names)}"

    if suptitle:
        plt.suptitle(suptitle)

    plt.tight_layout()

    return fig

keras_climate.datasets.biomassters.BioMassters

BioMassters(root='data', split='train', sensors=['S1', 'S2'], as_time_series=False)

Bases: NonGeoDataset

BioMassters Dataset for Aboveground Biomass prediction.

Dataset intended for Aboveground Biomass (AGB) prediction over Finnish forests based on Sentinel 1 and 2 data with corresponding target AGB mask values generated by Light Detection and Ranging (LiDAR).

Dataset Format:

• .tif files for Sentinel 1 and 2 data
• .tif file for pixel wise AGB target mask
• .csv files for metadata regarding features and targets

Dataset Features:

• 13,000 target AGB masks of size (256x256px)
• 12 months of data per target mask
• Sentinel 1 and Sentinel 2 data for each location
• Sentinel 1 available for every month
• Sentinel 2 available for almost every month (not available for every month due to ESA acquisition halt over the region during particular periods)

If you use this dataset in your research, please cite the following paper:

• https://nascetti-a.github.io/BioMasster/

.. note:: This dataset can be downloaded from Torchgeo Hugging Face Hub <https://huggingface.co/datasets/torchgeo/biomassters>_.

Initialize a new instance of BioMassters dataset.

If as_time_series=False (the default), each time step becomes its own sample with the target being shared across multiple samples.

Args: root: root directory where dataset can be found split: train or test split sensors: which sensors to consider for the sample, Sentinel 1 and/or Sentinel 2 ('S1', 'S2') as_time_series: whether or not to return all available time-steps or just a single one for a given target location

Raises: AssertionError: if split or sensors is invalid DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/biomassters.py

def __init__(
    self, root="data", split="train", sensors=["S1", "S2"], as_time_series=False
):
    """Initialize a new instance of BioMassters dataset.

    If ``as_time_series=False`` (the default), each time step becomes
    its own sample with the target being shared across multiple
    samples.

    Args:
        root: root directory where dataset can be found
        split: train or test split
        sensors: which sensors to consider for the sample, Sentinel 1
            and/or Sentinel 2 ('S1', 'S2')
        as_time_series: whether or not to return all available
            time-steps or just a single one for a given target location

    Raises:
        AssertionError: if ``split`` or ``sensors`` is invalid
        DatasetNotFoundError: If dataset is not found.
    """
    self.root = root

    assert split in self.valid_splits, (
        f"Please choose one of the valid splits: {self.valid_splits}."
    )
    self.split = split

    assert set(sensors).issubset(set(self.valid_sensors)), (
        f"Please choose a subset of valid sensors: {self.valid_sensors}."
    )
    self.sensors = sensors
    self.as_time_series = as_time_series

    self._verify()

    # open metadata csv files
    self.df = pd.read_csv(os.path.join(self.root, self.metadata_filename))

    # filter sensors
    self.df = self.df[self.df["satellite"].isin(self.sensors)]

    # filter split
    self.df = self.df[self.df["split"] == self.split]

    # generate numerical month from filename since first month is
    # September and has numerical index of 0
    self.df["num_month"] = (
        self.df["filename"]
        .str.split("_", expand=True)[2]
        .str.split(".", expand=True)[0]
        .astype(int)
    )

    # set dataframe index depending on the task for easier indexing
    if self.as_time_series:
        self.df["num_index"] = self.df.groupby(["chip_id"]).ngroup()
    else:
        filter_df = (
            self.df.groupby(["chip_id", "month"])["satellite"].count().reset_index()
        )
        filter_df = filter_df[filter_df["satellite"] == len(self.sensors)].drop(
            "satellite", axis=1
        )
        # guarantee that each sample has corresponding number of images available
        self.df = self.df.merge(filter_df, on=["chip_id", "month"], how="inner")

        self.df["num_index"] = self.df.groupby(["chip_id", "month"]).ngroup()

__getitem__

__getitem__(index)

Return an index within the dataset.

Raises: IndexError: if index is out of range of the dataset

Source code in keras_climate/datasets/biomassters.py

def __getitem__(self, index):
    """Return an index within the dataset.

    Raises:
        IndexError: if index is out of range of the dataset
    """
    sample_df = self.df[self.df["num_index"] == index].copy()

    # sort by satellite and month to return correct order
    sample_df.sort_values(
        by=["satellite", "num_month"], inplace=True, ascending=True
    )

    filepaths = sample_df["filename"].tolist()
    sample = {}
    for sens in self.sensors:
        sens_filepaths = [fp for fp in filepaths if sens in fp]
        sample[f"image_{sens}"] = self._load_input(sens_filepaths)

    if self.split == "train":
        sample["label"] = self._load_target(
            sample_df["corresponding_agbm"].unique()[0]
        )

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/biomassters.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    ncols = len(self.sensors) + 1

    showing_predictions = "prediction" in sample
    if showing_predictions:
        ncols += 1

    fig, axs = plt.subplots(1, ncols=ncols, figsize=(5 * ncols, 10))
    for idx, sens in enumerate(self.sensors):
        img = ops.cast(sample[f"image_{sens}"], "float32")
        if self.as_time_series:
            # plot last time step
            img = img[-1, ...]
        if sens == "S2":
            img = ops.take(img, [2, 1, 0], axis=-1)
            img = quantile_normalization(img)
        else:
            co_polarization = img[..., 0]  # transmit == receive
            cross_polarization = img[..., 1]  # transmit != receive
            ratio = co_polarization / cross_polarization

            # https://gis.stackexchange.com/a/400780/123758
            co_polarization = ops.clip(co_polarization / 0.3, 0, 1)
            cross_polarization = ops.clip(cross_polarization / 0.05, 0, 1)
            ratio = ops.clip(ratio / 25, 0, 1)

            img = ops.stack(
                (co_polarization, cross_polarization, ratio), axis=-1
            )

        axs[idx].imshow(ops.convert_to_numpy(img))
        axs[idx].axis("off")
        if show_titles:
            axs[idx].set_title(sens)

    if showing_predictions:
        pred = axs[ncols - 2].imshow(
            ops.convert_to_numpy(sample["prediction"]), cmap="YlGn"
        )
        plt.colorbar(pred, ax=axs[ncols - 2], fraction=0.046, pad=0.04)
        axs[ncols - 2].axis("off")
        if show_titles:
            axs[ncols - 2].set_title("Prediction")

    # plot target / only available in train set
    if "label" in sample:
        target = axs[-1].imshow(ops.convert_to_numpy(sample["label"]), cmap="YlGn")
        plt.colorbar(target, ax=axs[-1], fraction=0.046, pad=0.04)
        axs[-1].axis("Off")
        if show_titles:
            axs[-1].set_title("Target")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.bright.BRIGHTDFC2025

BRIGHTDFC2025(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

BRIGHT DFC2025 dataset.

The BRIGHT <https://github.com/ChenHongruixuan/BRIGHT>__ dataset consists of bi-temporal high-resolution multimodal images for building damage assessment. The dataset is part of the 2025 IEEE GRSS Data Fusion Contest. The pre-disaster images are optical images and the post-disaster images are SAR images, and targets were manually annotated. The dataset is split into train, val, and test splits, but the test split does not contain targets in this version.

More information can be found at the Challenge website <https://www.grss-ieee.org/technical-committees/image-analysis-and-data-fusion/?tab=data-fusion-contest>__.

Dataset Features:

• Pre-disaster optical images from MAXAR, NAIP, NOAA Digital Coast Raster Datasets, and the National Plan for Aerial Orthophotography Spain
• Post-disaster SAR images from Capella Space and Umbra
• high image resolution of 0.3-1m

Dataset Format:

• Images are in GeoTIFF format with pixel dimensions of 1024x1024
• Pre-disaster are three channel images
• Post-disaster SAR images are single channel but repeated to have 3 channels

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2501.06019

Initialize a new BRIGHT DFC2025 dataset instance.

Args: root: root directory where dataset can be found split: train/val/test split to load transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False. AssertionError: If split is not one of 'train', 'val', or 'test.

Source code in keras_climate/datasets/bright.py

def __init__(
    self, root="data", split="train", transforms=None, download=False, checksum=True
):
    """Initialize a new BRIGHT DFC2025 dataset instance.

    Args:
        root: root directory where dataset can be found
        split: train/val/test split to load
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
        AssertionError: If *split* is not one of 'train', 'val', or
            'test.
    """
    assert split in self.valid_split, f"Split must be one of {self.valid_split}"
    self.root = root
    self.split = split
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    self.sample_paths = self._get_paths()

__getitem__

__getitem__(index)

Return an index within the dataset.

Returns: data and target at that index, pre and post image are stacked along a leading time axis (2, H, W, C)

Source code in keras_climate/datasets/bright.py

def __getitem__(self, index):
    """Return an index within the dataset.

    Returns:
        data and target at that index, pre and post image are stacked
        along a leading time axis (2, H, W, C)
    """
    idx_paths = self.sample_paths[index]

    image_pre = ops.cast(self._load_image(idx_paths["image_pre"]), "float32")
    image_post = ops.cast(self._load_image(idx_paths["image_post"]), "float32")
    # https://github.com/ChenHongruixuan/BRIGHT/blob/11b1ffafa4d30d2df2081189b56864b0de4e3ed7/dfc25_benchmark/dataset/make_data_loader.py#L101
    # post image is repeated to also have 3 channels
    image_post = ops.tile(image_post, (1, 1, 3))

    sample = {"image": ops.stack([image_pre, image_post])}

    if "target" in idx_paths and self.split != "test":
        target = ops.cast(self._load_image(idx_paths["target"]), "int64")
        # single-band mask: squeeze the channel dimension
        sample["mask"] = ops.squeeze(target, axis=-1)

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/bright.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.patches as mpatches
    import matplotlib.pyplot as plt

    ncols = 2
    showing_mask = "mask" in sample
    showing_prediction = "prediction" in sample
    if showing_mask:
        ncols += 1
    if showing_prediction:
        ncols += 1

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(15, 5))

    image = ops.convert_to_numpy(sample["image"])

    axs[0].imshow(image[0] / 255.0)
    axs[0].axis("off")

    axs[1].imshow(image[1] / 255.0)
    axs[1].axis("off")

    cmap = colors.ListedColormap(self.colormap)

    kwargs = {"cmap": cmap, "vmin": 0, "vmax": 3, "interpolation": "none"}
    if showing_mask:
        mask = ops.convert_to_numpy(sample["mask"])
        axs[2].imshow(mask, **kwargs)
        axs[2].axis("off")
        unique_classes = np.unique(mask)
        handles = [
            mpatches.Patch(
                color=cmap(ordinal),
                label="\n".join(
                    textwrap.wrap(self.px_class_values[px_class], width=10)
                ),
            )
            for ordinal, px_class in enumerate(self.px_class_values.keys())
            if ordinal in unique_classes
        ]
        axs[2].legend(handles=handles, loc="upper right", bbox_to_anchor=(1.4, 1))
        if showing_prediction:
            axs[3].imshow(ops.convert_to_numpy(sample["prediction"]), **kwargs)
            axs[3].axis("off")
    elif showing_prediction:
        axs[2].imshow(ops.convert_to_numpy(sample["prediction"]), **kwargs)
        axs[2].axis("off")

    if show_titles:
        axs[0].set_title("Pre-disaster image")
        axs[1].set_title("Post-disaster image")
        if showing_mask:
            axs[2].set_title("Ground truth")
        if showing_prediction:
            axs[-1].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.cabuar.CaBuAr

CaBuAr(root='data', split='train', bands=all_bands, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

CaBuAr dataset.

CaBuAr <https://huggingface.co/datasets/DarthReca/california_burned_areas>__ is a dataset for Change detection for Burned area Delineation and part of the splits are used for the ChaBuD ECML-PKDD 2023 Discovery Challenge.

Dataset features:

• Sentinel-2 multispectral imagery
• binary masks of burned areas
• 12 multispectral bands
• 424 pairs of pre and post images with 20 m per pixel resolution (512x512 px)

Dataset format:

• single hdf5 dataset containing images and masks

Dataset classes:

1. no change
2. burned area

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1109/MGRS.2023.3292467

.. note::

This dataset requires the following additional library to be installed:

• h5py <https://pypi.org/project/h5py/>_ to load the dataset

Initialize a new CaBuAr dataset instance.

Args: root: root directory where dataset can be found split: one of "train", "val", "test" bands: the subset of bands to load transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: If split or bands arguments are invalid. DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/cabuar.py

def __init__(
    self,
    root="data",
    split="train",
    bands=all_bands,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new CaBuAr dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val", "test"
        bands: the subset of bands to load
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: If ``split`` or ``bands`` arguments are invalid.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    lazy_import("h5py")

    assert split in self.folds
    assert set(bands) <= set(self.all_bands)

    # Set the file index based on the split
    file_index = 1 if split == "test" else 0

    self.root = root
    self.split = split
    self.bands = bands
    self.transforms = transforms
    self.download = download
    self.checksum = checksum
    self.filepath = os.path.join(root, self.filenames[file_index])
    self.band_indices = [self.all_bands.index(b) for b in bands]

    self._verify()

    self.uuids = self._load_uuids()

__getitem__

__getitem__(index)

Return an index within the dataset.

Returns a single T x H x W x C image (pre-fire and post-fire stacked along the time axis).

Source code in keras_climate/datasets/cabuar.py

def __getitem__(self, index):
    """Return an index within the dataset.

    Returns a single T x H x W x C image (pre-fire and post-fire stacked
    along the time axis).
    """
    image = self._load_image(index)
    mask = self._load_target(index)

    sample = {"image": image, "mask": mask}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: ValueError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/cabuar.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        ValueError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise ValueError("Dataset doesn't contain some of the RGB bands")

    mask = ops.convert_to_numpy(sample["mask"])[0]
    image_pre = ops.take(sample["image"][0], rgb_indices, axis=-1)
    image_post = ops.take(sample["image"][1], rgb_indices, axis=-1)
    image_pre = ops.convert_to_numpy(quantile_normalization(image_pre))
    image_post = ops.convert_to_numpy(quantile_normalization(image_post))

    ncols = 3

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = ops.convert_to_numpy(sample["prediction"])[0]
        ncols += 1

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 5, 10))

    axs[0].imshow(image_pre)
    axs[0].axis("off")
    axs[1].imshow(image_post)
    axs[1].axis("off")
    axs[2].imshow(mask)
    axs[2].axis("off")

    if showing_predictions:
        axs[3].imshow(prediction)
        axs[3].axis("off")

    if show_titles:
        axs[0].set_title("Image Pre")
        axs[1].set_title("Image Post")
        axs[2].set_title("Mask")
        if showing_predictions:
            axs[3].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.caffe.CaFFe

CaFFe(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

CaFFe (CAlving Fronts and where to Find thEm) dataset.

The CaFFe <https://doi.pangaea.de/10.1594/PANGAEA.940950>__ dataset is a semantic segmentation dataset of marine-terminating glaciers.

Dataset features:

• 13,090 train, 2,241 validation, and 3,761 test images
• varying spatial resolution of 6-20m
• paired binary calving front segmentation masks
• paired multi-class land cover segmentation masks

Dataset format:

• images are single-channel pngs with dimension 512x512
• segmentation masks are single-channel pngs

Dataset classes:

1. N/A
2. rock
3. glacier
4. ocean/ice melange

If you use this dataset in your research, please cite the following paper:

• https://essd.copernicus.org/articles/14/4287/2022/

Initialize a new instance of CaFFe dataset.

Args: root: root directory where dataset can be found split: one of "train", "val", or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: if split argument is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/caffe.py

def __init__(self, root="data", split="train", transforms=None, download=False, checksum=True):
    """Initialize a new instance of CaFFe dataset.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val", or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in self.valid_splits, f"split must be one of {self.valid_splits}"

    self.root = root
    self.split = split
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    self.fpaths = glob.glob(
        os.path.join(
            self.root,
            self.zipfilename.replace(".zip", ""),
            self.mask_dirs[1],
            self.split,
            "*.png",
        )
    )

    self.ordinal_map_zones = np.zeros(
        max(self.px_class_values_zones.keys()) + 1, dtype="int64"
    )
    for ordinal, px_class in enumerate(self.px_class_values_zones.keys()):
        self.ordinal_map_zones[px_class] = ordinal

    self.ordinal_map_fronts = np.zeros(
        max(self.px_class_values_fronts.keys()) + 1, dtype="int64"
    )
    for ordinal, px_class in enumerate(self.px_class_values_fronts.keys()):
        self.ordinal_map_fronts[px_class] = ordinal

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/caffe.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.patches as mpatches
    import matplotlib.pyplot as plt

    if "prediction" in sample:
        ncols = 4
    else:
        ncols = 3
    fig, axs = plt.subplots(1, ncols, figsize=(15, 5))

    image = ops.convert_to_numpy(sample["image"])
    mask_front = ops.convert_to_numpy(sample["mask_front"])
    mask_zones = ops.convert_to_numpy(sample["mask_zones"])

    axs[0].imshow(image[..., 0].astype("uint8"), cmap="gray")
    axs[0].axis("off")

    axs[1].imshow(mask_front, cmap="gray")
    axs[1].axis("off")

    unique_classes = np.unique(mask_zones)
    axs[2].imshow(mask_zones, cmap=self.zone_cmap)
    axs[2].axis("off")

    handles = [
        mpatches.Patch(
            color=self.zone_cmap(ordinal),
            label="\n".join(textwrap.wrap(self.px_class_values_zones[px_class], width=10)),
        )
        for ordinal, px_class in enumerate(self.px_class_values_zones.keys())
        if ordinal in unique_classes
    ]
    axs[2].legend(handles=handles, loc="upper right", bbox_to_anchor=(1.4, 1))

    if show_titles:
        axs[0].set_title("Image")
        axs[1].set_title("Front Mask")
        axs[2].set_title("Zone Mask")

    if "prediction" in sample:
        axs[3].imshow(ops.convert_to_numpy(sample["prediction"]), cmap="gray")
        axs[3].axis("off")
        if show_titles:
            axs[3].set_title("Prediction")

    if suptitle:
        fig.suptitle(suptitle)

    return fig

keras_climate.datasets.cbf.CanadianBuildingFootprints

CanadianBuildingFootprints(paths='data', crs=None, res=(1e-05, 1e-05), transforms=None, download=False, checksum=True)

Bases: VectorDataset

Canadian Building Footprints dataset.

The Canadian Building Footprints <https://github.com/Microsoft/CanadianBuildingFootprints>__ dataset contains 11,842,186 computer generated building footprints in all Canadian provinces and territories in GeoJSON format. This data is freely available for download and use.

Initialize a new Dataset instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS in (xres, yres) format. If a single float is provided, it is used for both the x and y resolution. transforms: a function/transform that takes an input sample and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/cbf.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=(0.00001, 0.00001),
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS in (xres, yres)
            format. If a single float is provided, it is used for both
            the x and y resolution.
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.paths = paths
    self.checksum = checksum

    if download:
        self._download()

    if not self._check_integrity():
        raise DatasetNotFoundError(self)

    super().__init__(paths, crs, res, transforms)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/cbf.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt
    from keras import ops

    image = ops.convert_to_numpy(sample["mask"])
    ncols = 1

    showing_prediction = "prediction" in sample
    if showing_prediction:
        pred = ops.convert_to_numpy(sample["prediction"])
        ncols = 2

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(4, 4))

    if showing_prediction:
        axs[0].imshow(image)
        axs[0].axis("off")
        axs[1].imshow(pred)
        axs[1].axis("off")
        if show_titles:
            axs[0].set_title("Mask")
            axs[1].set_title("Prediction")
    else:
        axs.imshow(image)
        axs.axis("off")
        if show_titles:
            axs.set_title("Mask")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.cdl.CDL

CDL(paths='data', crs=None, res=None, years=None, classes=None, transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: RasterDataset

Cropland Data Layer (CDL) dataset.

The Cropland Data Layer <https://www.nass.usda.gov/Research_and_Science/Cropland/SARS1a.php>__, hosted on CropScape <https://nassgeodata.gmu.edu/CropScape/%3B>_, provides a raster, geo-referenced, crop-specific land cover map for the continental United States. The CDL also includes a crop mask layer and planting frequency layers, as well as boundary, water and road layers. The data is created annually using moderate resolution satellite imagery and extensive agricultural ground truth.

The dataset contains 134 classes, for a description of the classes see the xls file at the top of this page <https://www.nass.usda.gov/Research_and_Science/Cropland/sarsfaqs2.php>_.

If you use this dataset in your research, please cite it using the following format:

• https://www.nass.usda.gov/Research_and_Science/Cropland/sarsfaqs2.php#what.1

Initialize a new Dataset instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS in (xres, yres) format (defaults to the resolution of the first file found) years: list of years for which to use cdl layer classes: list of classes to include, the rest will be mapped to 0 (defaults to all classes) transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow) time_series: if True, stack data along the time series dimension [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

Raises: AssertionError: if years or classes are invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/cdl.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    years=None,
    classes=None,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS in (xres, yres)
            format (defaults to the resolution of the first file found)
        years: list of years for which to use cdl layer
        classes: list of classes to include, the rest will be mapped to 0
            (defaults to all classes)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)
        time_series: if True, stack data along the time series dimension
            [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

    Raises:
        AssertionError: if ``years`` or ``classes`` are invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    if classes is None:
        classes = list(self.valid_classes)
    if years is None:
        years = [2023]
    assert set(years) <= self.md5s.keys(), (
        f"CDL data product only exists for the following years: {list(self.md5s.keys())}."
    )
    assert set(classes) <= set(self.valid_classes), (
        f"Only the following classes are valid: {self.valid_classes}."
    )
    assert 0 in classes, "Classes must include the background class: 0"

    self.paths = paths
    self.years = years
    self.classes = classes
    self.download = download
    self.checksum = checksum
    self.ordinal_map = np.zeros(self.valid_classes[-1] + 1, dtype="int64")
    self.inverse_map = np.zeros(len(classes), dtype="int64")

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

    # Map chosen classes to ordinal numbers, all others mapped to background class
    for v, k in enumerate(self.classes):
        self.ordinal_map[k] = v
        self.inverse_map[v] = k

__getitem__

__getitem__(index)

Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

Source code in keras_climate/datasets/cdl.py

def __getitem__(self, index):
    """Retrieve input, target, and/or metadata indexed by spatiotemporal slice."""
    sample = super().__getitem__(index)
    sample["mask"] = ops.take(
        ops.convert_to_tensor(self.ordinal_map), sample["mask"], axis=0
    )
    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/cdl.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    mask = self.inverse_map[ops.convert_to_numpy(sample["mask"])]
    ncols = 1

    showing_predictions = "prediction" in sample
    if showing_predictions:
        pred = self.inverse_map[ops.convert_to_numpy(sample["prediction"])]
        ncols = 2

    fig, axs = plt.subplots(ncols=ncols, figsize=(ncols * 4, 4), squeeze=False)
    kwargs = {"cmap": self.cmap, "vmin": 0, "vmax": 255, "interpolation": "none"}

    axs[0, 0].imshow(mask, **kwargs)
    axs[0, 0].axis("off")

    if show_titles:
        axs[0, 0].set_title("Mask")

    if showing_predictions:
        axs[0, 1].imshow(pred, **kwargs)
        axs[0, 1].axis("off")
        if show_titles:
            axs[0, 1].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.chabud.ChaBuD

ChaBuD(root='data', split='train', bands=all_bands, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

ChaBuD dataset.

ChaBuD <https://github.com/lccol/chabud-challenge-description>__ is a dataset for Change detection for Burned area Delineation and is used for the ChaBuD ECML-PKDD 2023 Discovery Challenge.

Dataset features:

• Sentinel-2 multispectral imagery
• binary masks of burned areas
• 12 multispectral bands
• 356 pairs of pre and post images with 10 m per pixel resolution (512x512 px)

Dataset format:

• single hdf5 dataset containing images and masks

Dataset classes:

1. no change
2. burned area

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1109/MGRS.2023.3292467

.. note::

This dataset requires the following additional library to be installed:

• h5py <https://pypi.org/project/h5py/>_ to load the dataset

Initialize a new ChaBuD dataset instance.

Args: root: root directory where dataset can be found split: one of "train" or "val" bands: the subset of bands to load transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: AssertionError: If split or bands arguments are invalid. DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/chabud.py

def __init__(
    self,
    root="data",
    split="train",
    bands=all_bands,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new ChaBuD dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "val"
        bands: the subset of bands to load
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: If ``split`` or ``bands`` arguments are invalid.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    lazy_import("h5py")

    assert split in self.folds
    assert set(bands) <= set(self.all_bands)

    self.root = root
    self.split = split
    self.bands = bands
    self.transforms = transforms
    self.download = download
    self.checksum = checksum
    self.filepath = os.path.join(root, self.filename)
    self.band_indices = [self.all_bands.index(b) for b in bands]

    self._verify()

    self.uuids = self._load_uuids()

__getitem__

__getitem__(index)

Return an index within the dataset.

Returns a single T x H x W x C image (pre-fire and post-fire stacked along the time axis).

Source code in keras_climate/datasets/chabud.py

def __getitem__(self, index):
    """Return an index within the dataset.

    Returns a single T x H x W x C image (pre-fire and post-fire stacked
    along the time axis).
    """
    image = self._load_image(index)
    mask = self._load_target(index)

    sample = {"image": image, "mask": mask}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: ValueError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/chabud.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        ValueError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise ValueError("Dataset doesn't contain some of the RGB bands")

    mask = ops.convert_to_numpy(sample["mask"])[0]
    image_pre = ops.take(sample["image"][0], rgb_indices, axis=-1)
    image_post = ops.take(sample["image"][1], rgb_indices, axis=-1)
    image_pre = ops.convert_to_numpy(quantile_normalization(image_pre))
    image_post = ops.convert_to_numpy(quantile_normalization(image_post))

    ncols = 3

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = ops.convert_to_numpy(sample["prediction"])[0]
        ncols += 1

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 5, 10))

    axs[0].imshow(image_pre)
    axs[0].axis("off")
    axs[1].imshow(image_post)
    axs[1].axis("off")
    axs[2].imshow(mask)
    axs[2].axis("off")

    if showing_predictions:
        axs[3].imshow(prediction)
        axs[3].axis("off")

    if show_titles:
        axs[0].set_title("Image Pre")
        axs[1].set_title("Image Post")
        axs[2].set_title("Mask")
        if showing_predictions:
            axs[3].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.chesapeake.Chesapeake

Chesapeake(paths='data', crs=None, res=None, transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: RasterDataset, ABC

Abstract base class for all Chesapeake datasets.

Chesapeake Bay Land Use and Land Cover (LULC) Database 2022 Edition <https://www.chesapeakeconservancy.org/projects/cbp-land-use-land-cover-data-project>_

The Chesapeake Bay Land Use and Land Cover Database (LULC) facilitates characterization of the landscape and land change for and between discrete time periods. The database contains one-meter 13-class Land Cover (LC) and 54-class Land Use/Land Cover (LULC) for all counties within or adjacent to the Chesapeake Bay watershed for 2013/14 and 2017/18, depending on availability of National Agricultural Imagery Program (NAIP) imagery for each state.

If you use this dataset in your research, please cite the following:

• https://doi.org/10.5066/P981GV1L

Initialize a new Chesapeake instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS in (xres, yres) format (defaults to the resolution of the first file found) transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow) time_series: if True, stack data along the time series dimension [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/chesapeake.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new Chesapeake instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS in (xres, yres)
            format (defaults to the resolution of the first file found)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)
        time_series: if True, stack data along the time series dimension
            [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.filename_glob = self.filename_glob.format(state=self.state)
    self.filename_regex = self.filename_regex.format(state=self.state)

    self.paths = paths
    self.download = download
    self.checksum = checksum

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

sha256s abstractmethod property

sha256s

Mapping between data year and zip file sha256.

state property

state

State abbreviation.

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/chesapeake.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    mask = ops.convert_to_numpy(sample["mask"])
    ncols = 1

    showing_predictions = "prediction" in sample
    if showing_predictions:
        pred = ops.convert_to_numpy(sample["prediction"])
        ncols = 2

    fig, axs = plt.subplots(ncols=ncols, squeeze=False, figsize=(4 * ncols, 4))
    kwargs = {"cmap": self.cmap, "vmin": 0, "vmax": 128, "interpolation": "none"}

    axs[0, 0].imshow(mask, **kwargs)
    axs[0, 0].axis("off")
    if show_titles:
        axs[0, 0].set_title("Mask")

    if showing_predictions:
        axs[0, 1].imshow(pred, **kwargs)
        axs[0, 1].axis("off")
        if show_titles:
            axs[0, 1].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.chesapeake.ChesapeakeCVPR

ChesapeakeCVPR(root='data', splits=('de-train',), layers=('naip-new', 'lc'), transforms=None, cache=True, download=False, checksum=True)

Bases: GeoDataset

CVPR 2019 Chesapeake Land Cover dataset.

The CVPR 2019 Chesapeake Land Cover <https://lila.science/datasets/chesapeakelandcover>_ dataset contains two layers of NAIP aerial imagery, Landsat 8 leaf-on and leaf-off imagery, Chesapeake Bay land cover labels, NLCD land cover labels, and Microsoft building footprint labels.

This dataset was organized to accompany the 2019 CVPR paper, "Large Scale High-Resolution Land Cover Mapping with Multi-Resolution Data".

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1109/cvpr.2019.01301

Initialize a new Dataset instance.

Args: root: root directory where dataset can be found splits: a list of strings in the format "{state}-{train,val,test}" indicating the subset of data to use, for example "ny-train" layers: a list containing a subset of "naip-new", "naip-old", "lc", "nlcd", "landsat-leaf-on", "landsat-leaf-off", "buildings", or "prior_from_cooccurrences_101_31_no_osm_no_buildings" indicating which layers to load transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: if splits or layers are not valid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/chesapeake.py

def __init__(
    self,
    root="data",
    splits=("de-train",),
    layers=("naip-new", "lc"),
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
):
    """Initialize a new Dataset instance.

    Args:
        root: root directory where dataset can be found
        splits: a list of strings in the format "{state}-{train,val,test}"
            indicating the subset of data to use, for example "ny-train"
        layers: a list containing a subset of "naip-new", "naip-old",
            "lc", "nlcd", "landsat-leaf-on", "landsat-leaf-off",
            "buildings", or
            "prior_from_cooccurrences_101_31_no_osm_no_buildings"
            indicating which layers to load
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: if ``splits`` or ``layers`` are not valid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    for split in splits:
        assert split in self.splits
    assert all(layer in self.valid_layers for layer in layers)
    self.root = root
    self.layers = layers
    self.transforms = transforms
    self.cache = cache
    self.download = download
    self.checksum = checksum

    if self.prior_layer in layers:
        self.subdatasets = ("base", "prior_extension")
    else:
        self.subdatasets = ("base",)

    self._verify()

    # Add all tiles into the index in epsg:3857 based on the included geojson
    mint = pd.Timestamp.min
    maxt = pd.Timestamp.max
    gdf = gpd.read_file(os.path.join(root, "spatial_index.geojson"))
    gdf = gdf[gdf["split"].isin(splits)]
    gdf["prior_from_cooccurrences_101_31_no_osm_no_buildings"] = gdf["lc"].str.replace(
        "lc.tif", "prior_from_cooccurrences_101_31_no_osm_no_buildings.tif"
    )
    datetimes = [(mint, maxt)] * len(gdf)
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    gdf.set_crs("EPSG:3857", inplace=True)
    gdf.set_index(index, inplace=True)
    self.index = gdf

__getitem__

__getitem__(index)

Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

Source code in keras_climate/datasets/chesapeake.py

def __getitem__(self, index):
    """Retrieve input, target, and/or metadata indexed by spatiotemporal slice."""
    x, y, t = self._disambiguate_slice(index)
    interval = pd.Interval(t.start, t.stop)
    df = self.index.iloc[self.index.index.overlaps(interval)]
    df = df.iloc[:: t.step]
    df = df.cx[x.start : x.stop, y.start : y.stop]

    transform = rasterio.transform.from_origin(x.start, y.stop, x.step, y.step)
    sample = {
        "bounds": self._slice_to_tensor(index),
        "transform": ops.convert_to_tensor(np.array(list(transform))),
    }

    images = []
    masks = []
    if df.empty:
        raise IndexError(
            f"index: {index} not found in dataset with bounds: {self.bounds}"
        )
    elif len(df) == 1:
        filenames = df.iloc[0]
        query_box_transformed = None  # is set by the first layer

        query_box = shapely.geometry.box(x.start, y.start, x.stop, y.stop)

        for layer in self.layers:
            fn = filenames[layer]

            with rasterio.open(os.path.join(self.root, fn)) as f:
                dst_crs = f.crs.to_string().lower()

                if query_box_transformed is None:
                    query_box_transformed = shapely.ops.transform(
                        self.p_transformers[dst_crs].transform, query_box
                    ).envelope

                # Use a boundless windowed read so the returned array
                # always matches the requested patch shape, even when
                # the query extends beyond the raster footprint. The
                # out-of-raster region is filled with nodata (or 0 if
                # the raster has no nodata defined).
                window = rasterio.windows.from_bounds(
                    *query_box_transformed.bounds, transform=f.transform
                )
                out_height = round(window.height)
                out_width = round(window.width)
                fill_value = f.nodata if f.nodata is not None else 0
                data = f.read(
                    window=window,
                    boundless=True,
                    fill_value=fill_value,
                    out_shape=(f.count, out_height, out_width),
                )
                # (C, H, W) -> (H, W, C)
                data = np.transpose(data, (1, 2, 0))

            if layer in ["naip-new", "naip-old", "landsat-leaf-on", "landsat-leaf-off"]:
                images.append(data)
            elif layer in [
                "lc",
                "nlcd",
                "buildings",
                "prior_from_cooccurrences_101_31_no_osm_no_buildings",
            ]:
                masks.append(data)
    else:
        raise IndexError(f"index: {index} spans multiple tiles which is not valid")

    image = np.concatenate(images, axis=-1).astype("float32")
    mask = np.concatenate(masks, axis=-1).astype("int64")
    if mask.shape[-1] == 1:
        mask = mask[..., 0]

    sample["image"] = ops.convert_to_tensor(image)
    sample["mask"] = ops.convert_to_tensor(mask)

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/chesapeake.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"])
    mask = ops.convert_to_numpy(sample["mask"])
    if mask.ndim == 2:
        mask = mask[..., np.newaxis]

    num_panels = len(self.layers)
    showing_predictions = "prediction" in sample
    if showing_predictions:
        predictions = ops.convert_to_numpy(sample["prediction"])
        num_panels += 1

    fig, axs = plt.subplots(1, num_panels, figsize=(num_panels * 4, 5))

    i = 0
    for layer in self.layers:
        if layer == "naip-new" or layer == "naip-old":
            img = image[:, :, :3] / 255
            image = image[:, :, 4:]
            axs[i].axis("off")
            axs[i].imshow(img)
        elif layer == "landsat-leaf-on" or layer == "landsat-leaf-off":
            img = image[:, :, [3, 2, 1]] / 3000
            image = image[:, :, 9:]
            axs[i].axis("off")
            axs[i].imshow(img)
        elif layer == "nlcd":
            from .nlcd import NLCD

            img = mask[:, :, 0]
            mask = mask[:, :, 1:]
            axs[i].imshow(img, vmin=0, vmax=255, cmap=NLCD.cmap, interpolation="none")
            axs[i].axis("off")
        elif layer == "lc":
            img = mask[:, :, 0]
            mask = mask[:, :, 1:]
            axs[i].imshow(
                img, vmin=0, vmax=15, cmap=self.lc_cmap, interpolation="none"
            )
            axs[i].axis("off")
        elif layer == "buildings":
            img = mask[:, :, 0]
            mask = mask[:, :, 1:]
            axs[i].imshow(img, vmin=0, vmax=1, cmap="gray", interpolation="none")
            axs[i].axis("off")
        elif layer == "prior_from_cooccurrences_101_31_no_osm_no_buildings":
            img = (mask[:, :, :4] @ self.prior_color_matrix) / 255
            mask = mask[:, :, 4:]
            axs[i].imshow(img)
            axs[i].axis("off")

        if show_titles:
            if layer == "prior_from_cooccurrences_101_31_no_osm_no_buildings":
                axs[i].set_title("prior")
            else:
                axs[i].set_title(layer)
        i += 1

    if showing_predictions:
        axs[i].imshow(predictions, vmin=0, vmax=15, cmap=self.lc_cmap, interpolation="none")
        axs[i].axis("off")
        if show_titles:
            axs[i].set_title("Predictions")

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.chesapeake.ChesapeakeDC

ChesapeakeDC(paths='data', crs=None, res=None, transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: Chesapeake

This subset of the dataset contains data only for Washington, D.C.

Source code in keras_climate/datasets/chesapeake.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new Chesapeake instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS in (xres, yres)
            format (defaults to the resolution of the first file found)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)
        time_series: if True, stack data along the time series dimension
            [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.filename_glob = self.filename_glob.format(state=self.state)
    self.filename_regex = self.filename_regex.format(state=self.state)

    self.paths = paths
    self.download = download
    self.checksum = checksum

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

keras_climate.datasets.chesapeake.ChesapeakeDE

ChesapeakeDE(paths='data', crs=None, res=None, transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: Chesapeake

This subset of the dataset contains data only for Delaware.

Source code in keras_climate/datasets/chesapeake.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new Chesapeake instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS in (xres, yres)
            format (defaults to the resolution of the first file found)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)
        time_series: if True, stack data along the time series dimension
            [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.filename_glob = self.filename_glob.format(state=self.state)
    self.filename_regex = self.filename_regex.format(state=self.state)

    self.paths = paths
    self.download = download
    self.checksum = checksum

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

keras_climate.datasets.chesapeake.ChesapeakeMD

ChesapeakeMD(paths='data', crs=None, res=None, transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: Chesapeake

This subset of the dataset contains data only for Maryland.

Source code in keras_climate/datasets/chesapeake.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new Chesapeake instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS in (xres, yres)
            format (defaults to the resolution of the first file found)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)
        time_series: if True, stack data along the time series dimension
            [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.filename_glob = self.filename_glob.format(state=self.state)
    self.filename_regex = self.filename_regex.format(state=self.state)

    self.paths = paths
    self.download = download
    self.checksum = checksum

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

keras_climate.datasets.chesapeake.ChesapeakeNY

ChesapeakeNY(paths='data', crs=None, res=None, transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: Chesapeake

This subset of the dataset contains data only for New York.

Source code in keras_climate/datasets/chesapeake.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new Chesapeake instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS in (xres, yres)
            format (defaults to the resolution of the first file found)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)
        time_series: if True, stack data along the time series dimension
            [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.filename_glob = self.filename_glob.format(state=self.state)
    self.filename_regex = self.filename_regex.format(state=self.state)

    self.paths = paths
    self.download = download
    self.checksum = checksum

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

keras_climate.datasets.chesapeake.ChesapeakePA

ChesapeakePA(paths='data', crs=None, res=None, transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: Chesapeake

This subset of the dataset contains data only for Pennsylvania.

Source code in keras_climate/datasets/chesapeake.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new Chesapeake instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS in (xres, yres)
            format (defaults to the resolution of the first file found)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)
        time_series: if True, stack data along the time series dimension
            [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.filename_glob = self.filename_glob.format(state=self.state)
    self.filename_regex = self.filename_regex.format(state=self.state)

    self.paths = paths
    self.download = download
    self.checksum = checksum

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

keras_climate.datasets.chesapeake.ChesapeakeVA

ChesapeakeVA(paths='data', crs=None, res=None, transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: Chesapeake

This subset of the dataset contains data only for Virginia.

Source code in keras_climate/datasets/chesapeake.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new Chesapeake instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS in (xres, yres)
            format (defaults to the resolution of the first file found)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)
        time_series: if True, stack data along the time series dimension
            [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.filename_glob = self.filename_glob.format(state=self.state)
    self.filename_regex = self.filename_regex.format(state=self.state)

    self.paths = paths
    self.download = download
    self.checksum = checksum

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

keras_climate.datasets.chesapeake.ChesapeakeWV

ChesapeakeWV(paths='data', crs=None, res=None, transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: Chesapeake

This subset of the dataset contains data only for West Virginia.

Source code in keras_climate/datasets/chesapeake.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new Chesapeake instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS in (xres, yres)
            format (defaults to the resolution of the first file found)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)
        time_series: if True, stack data along the time series dimension
            [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.filename_glob = self.filename_glob.format(state=self.state)
    self.filename_regex = self.filename_regex.format(state=self.state)

    self.paths = paths
    self.download = download
    self.checksum = checksum

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

keras_climate.datasets.clay.ClayEmbeddings

ClayEmbeddings(root='data', transforms=None)

Bases: NonGeoDataset

Clay Embeddings dataset.

Supports:

• Clay v0 Sentinel embeddings <https://source.coop/clay/clay-model-v0-embeddings>_
• Clay v1.5 NAIP embeddings <https://source.coop/clay/clay-v1-5-naip-2>_
• Clay v1.5 Sentinel embeddings <https://source.coop/clay/lgnd-clay-v1-5-sentinel-2-l2a>_

See https://clay-foundation.github.io/model/ for details.

Initialize a new ClayEmbeddings instance.

Args: root: root directory where dataset can be found transforms: a function/transform that takes input sample and its target as entry and returns a transformed version

Raises: DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/clay.py

def __init__(self, root="data", transforms=None):
    """Initialize a new ClayEmbeddings instance.

    Args:
        root: root directory where dataset can be found
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version

    Raises:
        DatasetNotFoundError: If dataset is not found.
    """
    self.root = root
    self.transforms = transforms

    try:
        self.data = gpd.read_parquet(root)
    except (FileNotFoundError, ValueError):
        raise DatasetNotFoundError(self)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/clay.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    fig, ax = plt.subplots()
    ax.plot(ops.convert_to_numpy(sample["embedding"]))

    if show_titles:
        x = float(ops.convert_to_numpy(sample["x"]))
        y = float(ops.convert_to_numpy(sample["y"]))
        if "t" in sample:
            t = pd.Timestamp.fromtimestamp(float(ops.convert_to_numpy(sample["t"])))
            ax.set_title(rf"{y:0.3f}°N, {x:0.3f}°W, {t}")

    if suptitle is not None:
        plt.suptitle(suptitle)

    fig.tight_layout()
    return fig

keras_climate.datasets.cloud_cover.CloudCoverDetection

CloudCoverDetection(root='data', split='train', bands=all_bands, transforms=None, download=False)

Bases: NonGeoDataset

Sentinel-2 Cloud Cover Segmentation Dataset.

This training dataset was generated as part of a crowdsourcing competition <https://www.drivendata.org/competitions/83/cloud-cover/>_ on DrivenData.org, and later on was validated using a team of expert annotators. See this website <https://source.coop/radiantearth/cloud-cover-detection-challenge>__ for dataset details.

The dataset consists of Sentinel-2 satellite imagery and corresponding cloudy labels stored as GeoTiffs. There are 22,728 chips in the training data, collected between 2018 and 2020.

Each chip has:

• 4 multi-spectral bands from Sentinel-2 L2A product. The four bands are [B02, B03, B04, B08] (refer to Sentinel-2 documentation for more information about the bands).
• Label raster for the corresponding source tile representing a binary classification for if the pixel is a cloud or not.

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.34911/RDNT.HFQ6M7

.. note::

This dataset requires the following additional library to be installed:

• azcopy <https://github.com/Azure/azure-storage-azcopy>_: to download the dataset from Source Cooperative.

Initiatlize a CloudCoverDetection instance.

Args: root: root directory where dataset can be found split: 'train' or 'test' bands: the subset of bands to load transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory

Raises: AssertionError: If split or bands are invalid. DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/cloud_cover.py

def __init__(self, root="data", split="train", bands=all_bands, transforms=None, download=False):
    """Initiatlize a CloudCoverDetection instance.

    Args:
        root: root directory where dataset can be found
        split: 'train' or 'test'
        bands: the subset of bands to load
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory

    Raises:
        AssertionError: If *split* or *bands* are invalid.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in self.splits
    assert set(bands) <= set(self.all_bands)

    self.root = root
    self.split = split
    self.directory = os.path.join(self.root, self.splits[self.split])
    self.bands = bands
    self.transforms = transforms
    self.download = download

    self.csv = os.path.join(self.directory, f"{self.split}_metadata.csv")
    self._verify()

    self.metadata = pd.read_csv(self.csv)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/cloud_cover.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise RGBBandsMissingError()

    if "prediction" in sample:
        prediction = ops.convert_to_numpy(sample["prediction"])
        ncols = 3
    else:
        ncols = 2

    image = ops.convert_to_numpy(ops.take(sample["image"], rgb_indices, axis=-1)) / 3000
    mask = ops.convert_to_numpy(sample["mask"])

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 5, 10))

    axs[0].imshow(image)
    axs[0].axis("off")
    axs[1].imshow(mask)
    axs[1].axis("off")

    if "prediction" in sample:
        axs[2].imshow(prediction)
        axs[2].axis("off")
        if show_titles:
            axs[2].set_title("Prediction")

    if show_titles:
        axs[0].set_title("Image")
        axs[1].set_title("Mask")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.cms_mangrove_canopy.CMSGlobalMangroveCanopy

CMSGlobalMangroveCanopy(paths='data', crs=None, res=None, measurement='agb', country=all_countries[0], transforms=None, cache=True, checksum=True, time_series=False)

Bases: RasterDataset

CMS Global Mangrove Canopy dataset.

The CMS Global Mangrove Canopy dataset <https://www.earthdata.nasa.gov/data/catalog/ornl-cloud-cms-global-map-mangrove-canopy-1665-1.3>_ consists of a single band map at 30m resolution of either aboveground biomass (agb), basal area weighted height (hba95), or maximum canopy height (hmax95).

The dataset needs to be manually downloaded from the above link, where you can make an account and subsequently download the dataset.

Initialize a new Dataset instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS in (xres, yres) format (defaults to the resolution of the first file found) measurement: which of the three measurements, 'agb', 'hba95', or 'hmax95' country: country for which to retrieve data transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling checksum: if True, check the MD5 of the downloaded files (may be slow) time_series: if True, stack data along the time series dimension [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

Raises: AssertionError: if country or measurement arg are not str or invalid DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/cms_mangrove_canopy.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    measurement="agb",
    country=all_countries[0],
    transforms=None,
    cache=True,
    checksum=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS in (xres, yres)
            format (defaults to the resolution of the first file found)
        measurement: which of the three measurements, 'agb', 'hba95', or
            'hmax95'
        country: country for which to retrieve data
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)
        time_series: if True, stack data along the time series dimension
            [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

    Raises:
        AssertionError: if country or measurement arg are not str or
            invalid
        DatasetNotFoundError: If dataset is not found.
    """
    self.paths = paths
    self.checksum = checksum

    assert isinstance(country, str), "Country argument must be a str."
    assert country in self.all_countries, (
        f"You have selected an invalid country, please choose one of {self.all_countries}"
    )
    self.country = country

    assert isinstance(measurement, str), "Measurement must be a string."
    assert measurement in self.measurements, (
        f"You have entered an invalid measurement, please choose one of {self.measurements}."
    )
    self.measurement = measurement

    self.filename_glob = f"**/Mangrove_{self.measurement}_{self.country}*"

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/cms_mangrove_canopy.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    mask = ops.convert_to_numpy(ops.squeeze(sample["mask"]))
    ncols = 1

    showing_predictions = "prediction" in sample
    if showing_predictions:
        pred = ops.convert_to_numpy(ops.squeeze(sample["prediction"]))
        ncols = 2

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 4, 4))

    if showing_predictions:
        axs[0].imshow(mask)
        axs[0].axis("off")
        axs[1].imshow(pred)
        axs[1].axis("off")
        if show_titles:
            axs[0].set_title("Mask")
            axs[1].set_title("Prediction")
    else:
        axs.imshow(mask)
        axs.axis("off")
        if show_titles:
            axs.set_title("Mask")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.cowc.COWC

COWC(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset, ABC

Abstract base class for the COWC dataset.

The Cars Overhead With Context (COWC) <https://gdo152.llnl.gov/cowc/>_ data set is a large set of annotated cars from overhead. It is useful for training a device such as a deep neural network to learn to detect and/or count cars.

The dataset has the following attributes:

1. Data from overhead at 15 cm per pixel resolution at ground (all data is EO).
2. Data from six distinct locations: Toronto, Canada; Selwyn, New Zealand; Potsdam and Vaihingen, Germany; Columbus, Ohio and Utah, United States.
3. 32,716 unique annotated cars. 58,247 unique negative examples.
4. Intentional selection of hard negative examples.
5. Established baseline for detection and counting tasks.
6. Extra testing scenes for use after validation.

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1007/978-3-319-46487-9_48

Initialize a new COWC dataset instance.

Args: root: root directory where dataset can be found split: one of "train" or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: if split argument is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/cowc.py

def __init__(self, root="data", split="train", transforms=None, download=False, checksum=True):
    """Initialize a new COWC dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in ["train", "test"]

    self.root = root
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    if download:
        self._download()

    if not self._check_integrity():
        raise DatasetNotFoundError(self)

    self.images = []
    self.targets = []
    with open(
        os.path.join(self.root, self.filename.format(split)),
        encoding="utf-8-sig",
        newline="",
    ) as f:
        reader = csv.reader(f, delimiter=" ")
        for row in reader:
            self.images.append(row[0])
            self.targets.append(row[1])

base_url abstractmethod property

base_url

Base URL to download dataset from.

filenames abstractmethod property

filenames

List of files to download.

md5s abstractmethod property

md5s

List of MD5 checksums of files to download.

filename abstractmethod property

filename

Filename containing train/test split and target labels.

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/cowc.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"]).astype("uint8")
    label = str(float(ops.convert_to_numpy(sample["label"])))

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = str(float(ops.convert_to_numpy(sample["prediction"])))
    else:
        prediction = None

    fig, ax = plt.subplots(1, 1, figsize=(10, 10))
    ax.imshow(image)
    ax.axis("off")

    if show_titles:
        title = f"Label: {label}"
        if prediction is not None:
            title += f"\nPrediction: {prediction}"
        ax.set_title(title)

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.cowc.COWCCounting

COWCCounting(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: COWC

COWC Dataset for car counting.

Source code in keras_climate/datasets/cowc.py

def __init__(self, root="data", split="train", transforms=None, download=False, checksum=True):
    """Initialize a new COWC dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in ["train", "test"]

    self.root = root
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    if download:
        self._download()

    if not self._check_integrity():
        raise DatasetNotFoundError(self)

    self.images = []
    self.targets = []
    with open(
        os.path.join(self.root, self.filename.format(split)),
        encoding="utf-8-sig",
        newline="",
    ) as f:
        reader = csv.reader(f, delimiter=" ")
        for row in reader:
            self.images.append(row[0])
            self.targets.append(row[1])

keras_climate.datasets.cowc.COWCDetection

COWCDetection(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: COWC

COWC Dataset for car detection.

Source code in keras_climate/datasets/cowc.py

def __init__(self, root="data", split="train", transforms=None, download=False, checksum=True):
    """Initialize a new COWC dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in ["train", "test"]

    self.root = root
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    if download:
        self._download()

    if not self._check_integrity():
        raise DatasetNotFoundError(self)

    self.images = []
    self.targets = []
    with open(
        os.path.join(self.root, self.filename.format(split)),
        encoding="utf-8-sig",
        newline="",
    ) as f:
        reader = csv.reader(f, delimiter=" ")
        for row in reader:
            self.images.append(row[0])
            self.targets.append(row[1])

keras_climate.datasets.cropharvest.CropHarvest

CropHarvest(root='data', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

CropHarvest dataset.

CropHarvest <https://github.com/nasaharvest/cropharvest>__ is a crop classification dataset.

Dataset features:

• single pixel time series with crop-type labels
• 18 bands per image over 12 months

Dataset format:

• arrays are 12x18 with 18 bands over 12 months

Dataset properties:

1. is_crop - whether or not a single pixel contains cropland
2. classification_label - optional field identifying a specific crop type
3. dataset - source dataset for the imagery
4. lat - latitude
5. lon - longitude

If you use this dataset in your research, please cite the following paper:

• https://neurips.cc/virtual/2021/29874

This dataset requires the following additional library to be installed:

• h5py <https://pypi.org/project/h5py/>_ to load the dataset

Initialize a new CropHarvest dataset instance.

Args: root: root directory where dataset can be found transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False. DependencyNotFoundError: If h5py is not installed.

Source code in keras_climate/datasets/cropharvest.py

def __init__(self, root="data", transforms=None, download=False, checksum=True):
    """Initialize a new CropHarvest dataset instance.

    Args:
        root: root directory where dataset can be found
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
        DependencyNotFoundError: If h5py is not installed.
    """
    lazy_import("h5py")

    self.root = root
    self.transforms = transforms
    self.checksum = checksum
    self.download = download

    self._verify()

    self.files = self._load_features(self.root)
    self.labels = self._load_labels(self.root)
    classes = self.labels["properties.label"].unique()
    classes = classes[classes != np.array(None)]
    self.classes = np.insert(classes, 0, ["None", "Other"])

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset using bands for Agriculture RGB composite.

Source code in keras_climate/datasets/cropharvest.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset using bands for Agriculture RGB composite."""
    import matplotlib.pyplot as plt

    fig, axs = plt.subplots()
    bands = [self.all_bands.index(band) for band in self.rgb_bands]
    rgb = ops.convert_to_numpy(sample["array"])[:, bands] / 3000
    axs.imshow(rgb[None, ...])
    if show_titles:
        label = int(ops.convert_to_numpy(sample["label"]))
        axs.set_title(f"Crop type: {self.classes[label]}")
    axs.set_xticks(np.arange(12))
    axs.set_xticklabels(np.arange(12) + 1)
    axs.set_yticks([])
    axs.set_xlabel("Month")
    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.cv4a_kenya_crop_type.CV4AKenyaCropType

CV4AKenyaCropType(root='data', chip_size=256, stride=128, bands=all_bands, transforms=None, download=False)

Bases: NonGeoDataset

CV4A Kenya Crop Type Competition dataset.

The CV4A Kenya Crop Type Competition <https://source.coop/radiantearth/african-crops-kenya-02>__ dataset was produced as part of the Crop Type Detection competition at the Computer Vision for Agriculture (CV4A) Workshop at the ICLR 2020 conference. The objective of the competition was to create a machine learning model to classify fields by crop type from images collected during the growing season by the Sentinel-2 satellites.

Consists of 4 tiles of Sentinel 2 imagery from 13 different points in time.

Each tile has:

• 13 multi-band observations throughout the growing season. Each observation includes 12 bands from Sentinel-2 L2A product, and a cloud probability layer. The twelve bands are [B01, B02, B03, B04, B05, B06, B07, B08, B8A, B09, B11, B12] (refer to Sentinel-2 documentation for more information about the bands). The cloud probability layer is a product of the Sentinel-2 atmospheric correction algorithm (Sen2Cor) and provides an estimated cloud probability (0-100%) per pixel. All of the bands are mapped to a common 10 m spatial resolution grid.
• A raster layer indicating the crop ID for the fields in the training set.
• A raster layer indicating field IDs for the fields (both training and test sets). Fields with a crop ID 0 are the test fields.

There are 3,286 fields in the train set and 1,402 fields in the test set.

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.34911/RDNT.DW605X

.. note::

This dataset requires the following additional library to be installed:

• azcopy <https://github.com/Azure/azure-storage-azcopy>_: to download the dataset from Source Cooperative.

Initialize a new CV4A Kenya Crop Type Dataset instance.

Args: root: root directory where dataset can be found chip_size: size of chips stride: spacing between chips, if less than chip_size, then there will be overlap between chips bands: the subset of bands to load transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory

Raises: AssertionError: If bands are invalid. DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/cv4a_kenya_crop_type.py

def __init__(
    self,
    root="data",
    chip_size=256,
    stride=128,
    bands=all_bands,
    transforms=None,
    download=False,
):
    """Initialize a new CV4A Kenya Crop Type Dataset instance.

    Args:
        root: root directory where dataset can be found
        chip_size: size of chips
        stride: spacing between chips, if less than chip_size, then there
            will be overlap between chips
        bands: the subset of bands to load
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory

    Raises:
        AssertionError: If *bands* are invalid.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert set(bands) <= set(self.all_bands)

    self.root = root
    self.chip_size = chip_size
    self.stride = stride
    self.bands = bands
    self.transforms = transforms
    self.download = download

    self._verify()

    # Calculate the indices that we will use over all tiles
    self.chips_metadata = []
    for tile_index in range(len(self.tiles)):
        for y in [
            *list(range(0, self.tile_height - self.chip_size, stride)),
            self.tile_height - self.chip_size,
        ]:
            for x in [
                *list(range(0, self.tile_width - self.chip_size, stride)),
                self.tile_width - self.chip_size,
            ]:
                self.chips_metadata.append((tile_index, y, x))

plot

plot(sample, show_titles=True, suptitle=None, time_step=0)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/cv4a_kenya_crop_type.py

def plot(self, sample, show_titles=True, suptitle=None, time_step=0):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise RGBBandsMissingError()

    if "prediction" in sample:
        prediction = sample["prediction"]
        ncols = 3
    else:
        ncols = 2

    image, mask = sample["image"], sample["mask"]

    image = ops.take(image[time_step], np.array(rgb_indices), axis=-1)

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 5, 10))

    axs[0].imshow(ops.convert_to_numpy(image))
    axs[0].axis("off")
    axs[1].imshow(ops.convert_to_numpy(mask))
    axs[1].axis("off")

    if "prediction" in sample:
        axs[2].imshow(ops.convert_to_numpy(prediction))
        axs[2].axis("off")
        if show_titles:
            axs[2].set_title("Prediction")

    if show_titles:
        axs[0].set_title("Image")
        axs[1].set_title("Mask")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.cyclone.TropicalCyclone

TropicalCyclone(root='data', split='train', transforms=None, download=False)

Bases: NonGeoDataset

Tropical Cyclone Wind Estimation Competition dataset.

A collection of tropical storms in the Atlantic and East Pacific Oceans from 2000 to 2019 with corresponding maximum sustained surface wind speed. This dataset is split into training and test categories for the purpose of a competition. Read more about the competition here: https://www.drivendata.org/competitions/72/predict-wind-speeds/.

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1109/JSTARS.2020.3011907

.. note::

This dataset requires the following additional library to be installed:

• azcopy <https://github.com/Azure/azure-storage-azcopy>_: to download the dataset from Source Cooperative.

Initialize a new TropicalCyclone instance.

Args: root: root directory where dataset can be found split: one of "train" or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory

Raises: AssertionError: if split argument is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/cyclone.py

def __init__(self, root="data", split="train", transforms=None, download=False):
    """Initialize a new TropicalCyclone instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in {"train", "test"}

    self.root = root
    self.split = split
    self.transforms = transforms
    self.download = download

    self.filename = f"{split}_set"
    if split == "train":
        self.filename = f"{split}ing_set"

    self._verify()

    self.features = pd.read_csv(os.path.join(root, f"{self.filename}_features.csv"))
    self.labels = pd.read_csv(os.path.join(root, f"{self.filename}_labels.csv"))

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/cyclone.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image, label = sample["image"], sample["label"]

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = int(ops.convert_to_numpy(sample["prediction"]))

    fig, ax = plt.subplots(1, 1, figsize=(10, 10))

    ax.imshow(ops.convert_to_numpy(image).astype("uint8"))
    ax.axis("off")

    if show_titles:
        title = f"Label: {int(ops.convert_to_numpy(label))}"
        if showing_predictions:
            title += f"\nPrediction: {prediction}"
        ax.set_title(title)

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.deepglobelandcover.DeepGlobeLandCover

DeepGlobeLandCover(root='data', split='train', transforms=None, checksum=True)

Bases: NonGeoDataset

DeepGlobe Land Cover Classification Challenge dataset.

The DeepGlobe Land Cover Classification Challenge <https://competitions.codalab.org/competitions/18468>__ dataset offers high-resolution sub-meter satellite imagery for the task of semantic segmentation to detect areas of urban, agriculture, rangeland, forest, water, barren, and unknown. It contains 1,146 satellite images of size 2448 x 2448 pixels in total, split into training/validation/test sets, the original dataset can be downloaded from Kaggle <https://www.kaggle.com/datasets/balraj98/deepglobe-land-cover-classification-dataset>. However, only the training dataset with 803 images is used since the original test and valid dataset are not accompanied by labels. The dataset used here with a custom train/test split can be downloaded from Kaggle <https://www.kaggle.com/datasets/geoap96/deepglobe2018-landcover-segmentation-traindataset>.

Dataset format:

• images are RGB data
• masks are RGB image with with unique RGB values representing the class

Dataset classes:

1. Urban land
2. Agriculture land
3. Rangeland
4. Forest land
5. Water
6. Barren land
7. Unknown

File names for satellite images and the corresponding mask image are id_sat.jpg and id_mask.png, where id is an integer assigned to every image.

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/pdf/1805.06561

.. note::

This dataset can be downloaded using:

.. code-block:: console

  $ pip install kaggle  # place api key at ~/.kaggle/kaggle.json
  $ kaggle datasets download -d geoap96/deepglobe2018-landcover-segmentation-traindataset
  $ unzip deepglobe2018-landcover-segmentation-traindataset.zip

Initialize a new DeepGlobeLandCover dataset instance.

Args: root: root directory where dataset can be found split: one of "train" or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/deepglobelandcover.py

def __init__(self, root="data", split="train", transforms=None, checksum=True):
    """Initialize a new DeepGlobeLandCover dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        DatasetNotFoundError: If dataset is not found.
    """
    assert split in self.splits
    self.root = root
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    self._verify()
    if split == "train":
        split_folder = "training_data"
    else:
        split_folder = "test_data"

    self.image_fns = []
    self.mask_fns = []
    for image in sorted(
        os.listdir(os.path.join(root, self.data_root, split_folder, "images"))
    ):
        if image.endswith(".jpg"):
            id = image[:-8]
            image_path = os.path.join(root, self.data_root, split_folder, "images", image)
            mask_path = os.path.join(
                root, self.data_root, split_folder, "masks", str(id) + "_mask.png"
            )

            self.image_fns.append(image_path)
            self.mask_fns.append(mask_path)

plot

plot(sample, show_titles=True, suptitle=None, alpha=0.5)

Plot a sample from the dataset.

Source code in keras_climate/datasets/deepglobelandcover.py

def plot(self, sample, show_titles=True, suptitle=None, alpha=0.5):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    ncols = 1
    image1 = draw_semantic_segmentation_masks(
        sample["image"], sample["mask"], alpha=alpha, colors=list(self.colormap)
    )
    if "prediction" in sample:
        ncols += 1
        image2 = draw_semantic_segmentation_masks(
            sample["image"], sample["prediction"], alpha=alpha, colors=list(self.colormap)
        )

    fig, axs = plt.subplots(ncols=ncols, figsize=(ncols * 10, 10))
    if ncols > 1:
        (ax0, ax1) = axs
    else:
        ax0 = axs

    ax0.imshow(image1)
    ax0.axis("off")
    if ncols > 1:
        ax1.imshow(image2)
        ax1.axis("off")

    if show_titles:
        ax0.set_title("Ground Truth")
        if ncols > 1:
            ax1.set_title("Predictions")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.dfc2022.DFC2022

DFC2022(root='data', split='train', transforms=None, checksum=True)

Bases: NonGeoDataset

DFC2022 dataset.

The DFC2022 <https://www.grss-ieee.org/community/technical-committees/2022-ieee-grss-data-fusion-contest/>__ dataset is used as a benchmark dataset for the 2022 IEEE GRSS Data Fusion Contest and extends the MiniFrance dataset for semi-supervised semantic segmentation. The dataset consists of a train set containing labeled and unlabeled imagery and an unlabeled validation set. The dataset can be downloaded from the IEEEDataPort DFC2022 website <https://ieee-dataport.org/competitions/data-fusion-contest-2022-dfc2022/>_.

Dataset features:

• RGB aerial images at 0.5 m per pixel spatial resolution (~2,000x2,0000 px)
• DEMs at 1 m per pixel spatial resolution (~1,000x1,0000 px)
• Masks at 0.5 m per pixel spatial resolution (~2,000x2,0000 px)
• 16 land use/land cover categories

Dataset format:

• images are three-channel geotiffs
• DEMS are single-channel geotiffs
• masks are single-channel geotiffs with the pixel values represent the class

Dataset classes:

1. No information
2. Urban fabric
3. Industrial, commercial, public, military, private and transport units
4. Mine, dump and construction sites
5. Artificial non-agricultural vegetated areas
6. Arable land (annual crops)
7. Permanent crops
8. Pastures
9. Complex and mixed cultivation patterns
10. Orchards at the fringe of urban classes
11. Forests
12. Herbaceous vegetation associations
13. Open spaces with little or no vegetation
14. Wetlands
15. Water
16. Clouds and Shadows

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1007/s10994-020-05943-y

Initialize a new DFC2022 dataset instance.

Args: root: root directory where dataset can be found split: one of "train" or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: if split is invalid DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/dfc2022.py

def __init__(self, root="data", split="train", transforms=None, checksum=True):
    """Initialize a new DFC2022 dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: if ``split`` is invalid
        DatasetNotFoundError: If dataset is not found.
    """
    assert split in self.metadata
    self.root = root
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    self._verify()

    self.class2idx = {c: i for i, c in enumerate(self.classes)}
    self.files = self._load_files()

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/dfc2022.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt
    from matplotlib import colors

    ncols = 2
    image = ops.convert_to_numpy(sample["image"][:, :, :3]).astype("uint8")

    dem = quantile_normalization(sample["image"][:, :, -1])
    dem = ops.convert_to_numpy(dem)

    showing_mask = "mask" in sample
    showing_prediction = "prediction" in sample

    cmap = colors.ListedColormap(self.colormap)

    if showing_mask:
        mask = ops.convert_to_numpy(sample["mask"])
        ncols += 1
    if showing_prediction:
        pred = ops.convert_to_numpy(sample["prediction"])
        ncols += 1

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 10, 10))

    axs[0].imshow(image)
    axs[0].axis("off")
    axs[1].imshow(dem)
    axs[1].axis("off")
    if showing_mask:
        axs[2].imshow(mask, cmap=cmap, interpolation="none")
        axs[2].axis("off")
        if showing_prediction:
            axs[3].imshow(pred, cmap=cmap, interpolation="none")
            axs[3].axis("off")
    elif showing_prediction:
        axs[2].imshow(pred, cmap=cmap, interpolation="none")
        axs[2].axis("off")

    if show_titles:
        axs[0].set_title("Image")
        axs[1].set_title("DEM")

        if showing_mask:
            axs[2].set_title("Ground Truth")
            if showing_prediction:
                axs[3].set_title("Predictions")
        elif showing_prediction:
            axs[2].set_title("Predictions")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.digital_typhoon.DigitalTyphoon

DigitalTyphoon(root='data', task='regression', features=['wind'], targets=['wind'], sequence_length=3, min_feature_value=None, max_feature_value=None, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

Digital Typhoon Dataset for Analysis Task.

This dataset contains typhoon-centered images, derived from hourly infrared channel images captured by meteorological satellites. It incorporates data from multiple generations of the Himawari weather satellite, dating back to 1978. These images have been transformed into brightness temperatures and adjusted for varying satellite sensor readings, yielding a consistent spatio-temporal dataset that covers over four decades.

See the Digital Typhoon website <https://agora.ex.nii.ac.jp/digital-typhoon/dataset/index.html.en>_ for more information about the dataset.

Dataset features:

• infrared channel images from the Himawari weather satellite (512x512 px) at 5km spatial resolution
• auxiliary features such as wind speed, pressure, and more that can be used for regression or classification tasks
• 1,099 typhoons and 189,364 images

Dataset format:

• hdf5 files containing the infrared channel images
• .csv files containing the metadata for each image

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2311.02665

This dataset requires the following additional library to be installed:

• h5py <https://pypi.org/project/h5py/>_ to load the dataset

Initialize a new Digital Typhoon dataset instance.

Args: root: root directory where dataset can be found task: whether to load 'regression' or 'classification' labels features: which auxiliary features to return targets: which auxiliary features to use as targets sequence_length: length of the sequence to return min_feature_value: minimum value for each feature max_feature_value: maximum value for each feature transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: AssertionError: If any arguments are invalid. DatasetNotFoundError: If dataset is not found and download is False. DependencyNotFoundError: If h5py is not installed.

Source code in keras_climate/datasets/digital_typhoon.py

def __init__(
    self,
    root="data",
    task="regression",
    features=["wind"],
    targets=["wind"],
    sequence_length=3,
    min_feature_value=None,
    max_feature_value=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new Digital Typhoon dataset instance.

    Args:
        root: root directory where dataset can be found
        task: whether to load 'regression' or 'classification' labels
        features: which auxiliary features to return
        targets: which auxiliary features to use as targets
        sequence_length: length of the sequence to return
        min_feature_value: minimum value for each feature
        max_feature_value: maximum value for each feature
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: If any arguments are invalid.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
        DependencyNotFoundError: If h5py is not installed.
    """
    lazy_import("h5py")
    self.root = root
    self.transforms = transforms
    self.download = download
    self.checksum = checksum
    self.sequence_length = sequence_length

    self.min_feature_value = min_feature_value
    self.max_feature_value = max_feature_value

    assert task in self.valid_tasks, f"Please choose one of {self.valid_tasks}, you provided {task}."
    self.task = task

    assert set(features).issubset(set(self.valid_features))
    self.features = features

    assert set(targets).issubset(set(self.valid_features))
    self.targets = targets

    self._verify()

    self.aux_df = pd.read_csv(os.path.join(root, self.data_root, self.aux_file_name))
    self.aux_df["datetime"] = pd.to_datetime(self.aux_df[["year", "month", "day", "hour"]])

    self.aux_df = self.aux_df.sort_values(["year", "month", "day", "hour"])
    self.aux_df["seq_id"] = self.aux_df.groupby(["id"]).cumcount()

    self.aux_df.columns = [str(col) for col in self.aux_df.columns]

    # Compute the hour difference between consecutive images per typhoon id
    self.aux_df["hour_diff_consecutive"] = (
        self.aux_df.sort_values(["id", "datetime"]).groupby("id")["datetime"].diff().dt.total_seconds() / 3600
    )

    # Compute the hour difference between the first and second entry
    self.aux_df["hour_diff_to_next"] = (
        self.aux_df.groupby("id")["datetime"].shift(-1).sub(self.aux_df["datetime"]).abs().dt.total_seconds()
        / 3600
    )

    self.aux_df["hour_diff"] = self.aux_df["hour_diff_consecutive"].combine_first(
        self.aux_df["hour_diff_to_next"]
    )
    self.aux_df.drop(["hour_diff_consecutive", "hour_diff_to_next"], axis=1, inplace=True)

    # 0 hour difference is for the last time step of each typhoon sequence
    # and want to keep only images that have max 1 hour difference
    self.aux_df = self.aux_df[self.aux_df["hour_diff"] <= 1]
    # Filter out all ids that only have less than sequence_length entries
    self.aux_df = self.aux_df.groupby("id").filter(lambda x: len(x) >= self.sequence_length)

    # Filter aux_df according to min_target_value
    if self.min_feature_value is not None:
        for feature, min_value in self.min_feature_value.items():
            self.aux_df = self.aux_df[self.aux_df[feature] >= min_value]

    # Filter aux_df according to max_target_value
    if self.max_feature_value is not None:
        for feature, max_value in self.max_feature_value.items():
            self.aux_df = self.aux_df[self.aux_df[feature] <= max_value]

    # collect target mean and std for each target
    self.target_mean = self.aux_df[self.targets].mean().to_dict()
    self.target_std = self.aux_df[self.targets].std().to_dict()

    def _get_subsequences(df, k):
        """Generate all possible subsequences of length k for a given group."""
        min_seq_id = df["seq_id"].min()
        max_seq_id = df["seq_id"].max()

        # generate possible subsquences of length k for group
        subsequences = [
            {"id": df["id"].iloc[0], "seq_id": list(range(i, i + k))}
            for i in range(min_seq_id, max_seq_id - k + 2)
        ]
        return [subseq for subseq in subsequences if set(subseq["seq_id"]).issubset(df["seq_id"])]

    self.sample_sequences = [
        item
        for sublist in self.aux_df.groupby("id")[["seq_id", "id"]]
        .apply(_get_subsequences, k=self.sequence_length)
        .tolist()
        for item in sublist
    ]

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/digital_typhoon.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image, label = sample["image"], sample["label"]

    image = quantile_normalization(image)
    # take the last frame of the sequence for display
    image = ops.convert_to_numpy(image)[:, :, -1]

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = ops.convert_to_numpy(sample["prediction"])

    fig, ax = plt.subplots(1, 1, figsize=(10, 10))

    ax.imshow(image)
    ax.axis("off")

    if show_titles:
        label_np = ops.convert_to_numpy(label)
        title_dict = {label_name: label_np[idx] for idx, label_name in enumerate(self.targets)}
        title = f"Label: {title_dict}"
        if showing_predictions:
            title_dict = {label_name: prediction[idx] for idx, label_name in enumerate(self.targets)}
            title += f"\nPrediction: {title_dict}"
        ax.set_title(title)

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.dior.DIOR

DIOR(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

DIOR dataset.

DIOR <https://arxiv.org/abs/1909.00133>__ dataset contains horizontal bounding box annotations of Google Earth Aerial RGB imagery. The test split does not contain bounding box annotations and labels.

Dataset features:

• 20 classes
• 192,472 manually annotated bounding box instances

Dataset format:

• Images are three channel .jpg files.
• Annotations are in Pascal VOC XML format <https://roboflow.com/formats/pascal-voc-xml>_

Classes:

1. Airplane
2. Airport
3. Baseball Field
4. Basketball Court
5. Bridge
6. Chimney
7. Dam
8. Expressway Service Area
9. Expressway Toll Station
10. Golf Field
11. Ground Track Field
12. Harbor
13. Overpass
14. Ship
15. Stadium
16. Storage Tank
17. Tennis Court
18. Train Station
19. Vehicle
20. Windmill

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/1909.00133

Initialize a new DIOR dataset instance.

Args: root: root directory where dataset can be found split: split of the dataset to use, one of 'train', 'val', 'test' transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found or corrupted and download is False. AssertionError: If split argument is invalid.

Source code in keras_climate/datasets/dior.py

def __init__(self, root="data", split="train", transforms=None, download=False, checksum=True):
    """Initialize a new DIOR dataset instance.

    Args:
        root: root directory where dataset can be found
        split: split of the dataset to use, one of 'train', 'val', 'test'
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        DatasetNotFoundError: If dataset is not found or corrupted and
            *download* is False.
        AssertionError: If *split* argument is invalid.
    """
    self.root = root
    self.transforms = transforms
    self.checksum = checksum
    self.download = download

    assert split in self.valid_splits, f"Split must be one of {self.valid_splits}."
    self.split = split

    self._verify()

    self.sample_df = pd.read_csv(os.path.join(self.root, "sample_df.csv"))

    self.sample_df = self.sample_df[self.sample_df["split"] == self.split].reset_index(drop=True)

    self.class_to_idx = {c: i for i, c in enumerate(self.classes)}

plot

plot(sample, show_titles=True, suptitle=None, box_alpha=0.7)

Plot a sample from the dataset.

Source code in keras_climate/datasets/dior.py

def plot(self, sample, show_titles=True, suptitle=None, box_alpha=0.7):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt
    from matplotlib import patches

    image = ops.convert_to_numpy(sample["image"]).astype("uint8")
    boxes = ops.convert_to_numpy(sample["bbox_xyxy"])
    labels = ops.convert_to_numpy(sample["label"])

    fig, axs = plt.subplots(ncols=1, figsize=(10, 10))

    axs.imshow(image)
    axs.axis("off")

    cm = plt.get_cmap("gist_rainbow")

    for box, label_idx in zip(boxes, labels):
        color = cm(label_idx / len(self.classes))
        label = self.classes[label_idx]

        # Horizontal box: [xmin, ymin, xmax, ymax]
        x1, y1, x2, y2 = box
        rect = patches.Rectangle(
            (x1, y1),
            x2 - x1,
            y2 - y1,
            linewidth=2,
            alpha=box_alpha,
            linestyle="solid",
            edgecolor=color,
            facecolor="none",
        )
        axs.add_patch(rect)
        # Add label above box
        axs.text(
            x1, y1 - 5, label, color="white", fontsize=8, bbox={"facecolor": color, "alpha": box_alpha}
        )

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.dl4gam.DL4GAMAlps

DL4GAMAlps(root='data', split='train', cv_iter=1, version='small', bands=rgb_nir_swir_bands, extra_features=None, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

A Multi-modal Dataset for Glacier Mapping (Segmentation) in the European Alps.

The dataset consists of Sentinel-2 images from 2015 (mainly), 2016 and 2017, and binary segmentation masks for glaciers, based on an inventory built by glaciology experts (Paul et al. 2020 <https://doi.org/10.1594/PANGAEA.909133>_).

Dataset features:

• Sentinel-2 images (all bands, including cloud and shadow masks which can be used for loss masking)
• glacier mask (0: no glacier, 1: glacier)
• debris mask (0: no debris, 1: debris)
• DEM (Copernicus GLO-30) + five derived features: slope, aspect, terrain ruggedness index, planform and profile curvatures
• dh/dt (surface elevation change) map over 2010-2015
• v (surface velocity) map over 2015

For more details check also: https://huggingface.co/datasets/dcodrut/dl4gam_alps

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.22541/essoar.173557607.70204641/v1

.. note::

This dataset requires the following additional libraries to be
installed:

* `xarray <https://pypi.org/project/xarray/>`_
* `netcdf4 <https://pypi.org/project/netCDF4/>`_
  or `h5netcdf <https://pypi.org/project/h5netcdf/>`_

Initialize the dataset.

Args: root: root directory where dataset can be found split: one of "train", "val", or "test" cv_iter: one of 1, 2, 3, 4, 5 (for the five-fold geographical cross-validation scheme) version: one of "small" or "large" (controls the sampling overlap) bands: the Sentinel-2 bands to use as input (default: RGB + NIR + SWIR) extra_features: additional features to include (default: None; see the class attribute for the available) transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: if any parameters are invalid. DatasetNotFoundError: if dataset is not found and download is False. DependencyNotFoundError: if xarray is not installed.

Source code in keras_climate/datasets/dl4gam.py

def __init__(
    self,
    root="data",
    split="train",
    cv_iter=1,
    version="small",
    bands=rgb_nir_swir_bands,
    extra_features=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize the dataset.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val", or "test"
        cv_iter: one of 1, 2, 3, 4, 5 (for the five-fold geographical
            cross-validation scheme)
        version: one of "small" or "large" (controls the sampling
            overlap)
        bands: the Sentinel-2 bands to use as input (default: RGB + NIR
            + SWIR)
        extra_features: additional features to include (default: None;
            see the class attribute for the available)
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: if any parameters are invalid.
        DatasetNotFoundError: if dataset is not found and *download* is
            False.
        DependencyNotFoundError: if xarray is not installed.
    """
    lazy_import("xarray")

    self.root = pathlib.Path(root)
    self.split = split
    self.cv_iter = cv_iter
    self.version = version
    self.bands = bands
    self.extra_features = extra_features
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    # sanity checks
    assert split in self.valid_splits, f"Split {split} not in: {self.valid_splits}"
    assert cv_iter in self.valid_cv_iters, f"Cross-validation iteration {cv_iter} not in: {self.valid_cv_iters}"
    assert version in self.valid_versions, f"Version {version} not in: {self.valid_versions}"
    for band in bands:
        assert band in self.all_bands, f"Band {band} not in: {self.all_bands}"
    if extra_features:
        for feature in extra_features:
            assert feature in self.valid_extra_features, f"Feature {feature} not in: {self.valid_extra_features}"

    # set the local file paths
    label = f"dataset_{version}"
    self.fp_archive = self.root / f"{label}.tar.gz"
    self.dir_patches = self.root / label
    self.fp_splits_csv = self.root / "splits.csv"

    # get the corresponding urls and checksums
    self.url_dataset = self.download_metadata[label]["url"]
    self.md5_dataset = self.download_metadata[label]["checksum"]
    self.url_csv_splits = self.download_metadata["splits_csv"]["url"]
    self.md5_csv_splits = self.download_metadata["splits_csv"]["checksum"]

    self._verify()
    self._prepare_files()

plot

plot(sample, show_titles=True, suptitle=None, clip_extrema=True)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/dl4gam.py

def plot(self, sample, show_titles=True, suptitle=None, clip_extrema=True):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    # we expect the RGB bands to be present
    if not {"B4", "B3", "B2"}.issubset(set(self.bands)):
        raise RGBBandsMissingError()
    nir_and_swir_present = {"B8", "B11"}.issubset(set(self.bands))

    # prepare the RGB image and the masks
    idx_rgb = [self.bands.index(b) for b in ["B4", "B3", "B2"]]
    rgb_img = ops.take(sample["image"], np.array(idx_rgb), axis=-1)
    images = {
        "RGB Image": rgb_img,
        "Glacier Mask": sample["mask_glacier"],
        "Debris Mask": sample["mask_debris"],
        "Clouds and Shadows Mask": sample["mask_clouds_and_shadows"],
    }

    # add the SWIR-NIR-R image if the bands are present
    if nir_and_swir_present:
        idx_swir_nir_r = [self.bands.index(b) for b in ["B11", "B8", "B4"]]
        swir_nir_r_img = ops.take(sample["image"], np.array(idx_swir_nir_r), axis=-1)
        images["SWIR-NIR-R Image"] = swir_nir_r_img

    # add the extra features if present
    for extra_v, title in (
        ("prediction", "Prediction"),
        ("dem", "DEM"),
        ("slope", "Slope"),
        ("aspect", "Aspect"),
        ("planform_curvature", "Planform Curvature"),
        ("profile_curvature", "Profile Curvature"),
        ("terrain_ruggedness_index", "Terrain Ruggedness Index"),
        ("dhdt", "dh/dt"),
        ("v", "Surface Velocity"),
    ):
        if extra_v in sample:
            images[title] = sample[extra_v]

    cmaps = {
        "RGB Image": None,
        "SWIR-NIR-R Image": None,
        "Glacier Mask": "gray",
        "Prediction": "gray",
        "Debris Mask": "gray",
        "Clouds and Shadows Mask": "gray",
        "DEM": "terrain",
        "Slope": "magma",
        "Aspect": "jet",
        "Planform Curvature": "magma",
        "Profile Curvature": "magma",
        "Terrain Ruggedness Index": "magma",
        "dh/dt": "seismic_r",
        "Surface Velocity": "magma",
    }

    # build the figure
    n_imgs = len(images)
    ncols = 4 if n_imgs <= 8 else 5
    nrows = int(np.ceil(n_imgs / ncols))
    fig, axs = plt.subplots(nrows, ncols, figsize=(5 * ncols, 5 * nrows))

    for ax, k in zip(np.array(axs).flat, images):
        img = ops.convert_to_numpy(images[k])
        cmap = cmaps[k]

        # clip the extrema 5% of the values if needed
        if clip_extrema and k not in ["Glacier Mask", "Prediction", "Debris Mask", "Clouds and Shadows Mask"]:
            q_lim_clip = 0.025
            img = np.clip(img, np.quantile(img, q_lim_clip), np.quantile(img, 1 - q_lim_clip))

        vmin, vmax = np.min(img), np.max(img)
        # scale the images to [0,1]
        if k in ["RGB Image", "SWIR-NIR-R Image"]:
            img = (img - vmin) / (vmax - vmin)

        if k == "dh/dt":  # diverging colormap for the dh/dt, make it symmetric
            max_abs = max(abs(vmin), abs(vmax))
            vmin, vmax = -max_abs, max_abs

        ax.imshow(img, cmap=cmap, interpolation="none", vmin=vmin, vmax=vmax)
        if show_titles:
            ax.set_title(k)

    # disable the axes for all plots, including the empty plots
    for ax in np.array(axs).flat:
        ax.axis("off")

    if suptitle:
        fig.suptitle(suptitle)

    return fig

keras_climate.datasets.dlrsd.DLRSD

DLRSD(root='data', transforms=None, download=False, checksum=True)

Bases: DLRSDBase

DLRSD semantic segmentation dataset.

The :class:DLRSD variant provides pixel-level semantic segmentation annotations for 17 land cover classes.

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.3390/rs10060964

Initialize a new DLRSD dataset instance.

Args: root: root directory where dataset can be found transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the SHA256 of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/dlrsd.py

def __init__(self, root="data", transforms=None, download=False, checksum=True):
    """Initialize a new DLRSD dataset instance.

    Args:
        root: root directory where dataset can be found
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the SHA256 of the downloaded files (may
            be slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    super().__init__(root, transforms, download, checksum)

    images_dir = os.path.join(root, self.directory, "Images")
    labels_dir = os.path.join(root, self.directory, "Labels")
    self.image_fns = sorted(glob.glob(os.path.join(images_dir, "*", "*.tif")))
    self.mask_fns = [
        os.path.join(labels_dir, os.path.basename(os.path.dirname(p)), os.path.basename(p).replace(".tif", ".png"))
        for p in self.image_fns
    ]

plot

plot(sample, show_titles=True, suptitle=None, alpha=0.5)

Plot a sample from the dataset.

Source code in keras_climate/datasets/dlrsd.py

def plot(self, sample, show_titles=True, suptitle=None, alpha=0.5):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"])
    if image.max() > 1:
        image = image.astype("float32") / 255.0

    # Use a uint8 0-255 copy of the image for the segmentation overlay so
    # that user transforms which normalize the tensor to [0, 1] don't
    # produce a near-black overlay (draw_semantic_segmentation_masks
    # casts to uint8).
    overlay_image = sample["image"]
    overlay_np = ops.convert_to_numpy(overlay_image)
    if np.issubdtype(overlay_np.dtype, np.floating):
        if overlay_np.max() <= 1:
            overlay_image = ops.cast(overlay_image * 255, "uint8")
        else:
            overlay_image = ops.cast(overlay_image, "uint8")

    mask_overlay = draw_semantic_segmentation_masks(
        overlay_image, sample["mask"], alpha=alpha, colors=list(self.colormap)
    )

    showing_predictions = "prediction" in sample
    if showing_predictions:
        pred_overlay = draw_semantic_segmentation_masks(
            overlay_image, sample["prediction"], alpha=alpha, colors=list(self.colormap)
        )

    ncols = 3 if showing_predictions else 2
    fig, axs = plt.subplots(ncols=ncols, figsize=(ncols * 10, 10))

    axs[0].imshow(image)
    axs[0].axis("off")
    axs[1].imshow(mask_overlay)
    axs[1].axis("off")
    if showing_predictions:
        axs[2].imshow(pred_overlay)
        axs[2].axis("off")

    if show_titles:
        axs[0].set_title("Image")
        axs[1].set_title("Ground Truth")
        if showing_predictions:
            axs[2].set_title("Predictions")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.dlrsd.DLRSDBase

DLRSDBase(root='data', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

Shared base for DLRSD and DLRSDMultilabel.

The DLRSD <https://sites.google.com/view/zhouwx/dataset>__ dataset is a dataset for dense labeling of remote sensing imagery. It contains 2100 images of size 256x256 pixels across 21 scene categories (100 images per class), and is derived from the UC Merced Land Use Dataset.

Dataset features:

• 2100 images with 0.3m spatial resolution (256x256 px)
• three spectral bands - RGB

Dataset classes:

1. airplane
2. bare soil
3. buildings
4. cars
5. chaparral
6. court
7. dock
8. field
9. grass
10. mobile home
11. pavement
12. sand
13. sea
14. ship
15. tanks
16. trees
17. water

This base class provides shared dataset assets and download/extract/verify plumbing for both the semantic segmentation and multi-label variants of DLRSD.

Initialize a new DLRSD-style dataset instance.

Args: root: root directory where dataset can be found transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the SHA256 of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/dlrsd.py

def __init__(self, root="data", transforms=None, download=False, checksum=True):
    """Initialize a new DLRSD-style dataset instance.

    Args:
        root: root directory where dataset can be found
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the SHA256 of the downloaded files (may
            be slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.root = root
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

keras_climate.datasets.dlrsd.DLRSDMultilabel

DLRSDMultilabel(root='data', transforms=None, download=False, checksum=True)

Bases: DLRSDBase

DLRSD multi-label scene classification dataset.

The :class:DLRSDMultilabel variant provides multi-label scene classification annotations with 17 label classes.

If you use this dataset in your research, please cite the following papers:

• https://doi.org/10.3390/rs10060964
• https://doi.org/10.1109/TGRS.2017.2760909

Initialize a new DLRSDMultilabel dataset instance.

Args: root: root directory where dataset can be found transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the SHA256 of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/dlrsd.py

def __init__(self, root="data", transforms=None, download=False, checksum=True):
    """Initialize a new DLRSDMultilabel dataset instance.

    Args:
        root: root directory where dataset can be found
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the SHA256 of the downloaded files (may
            be slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    super().__init__(root, transforms, download, checksum)

    images_dir = os.path.join(root, self.directory, "Images")
    self.image_fns = sorted(glob.glob(os.path.join(images_dir, "*", "*.tif")))

    self.multilabels = {}
    csv_path = os.path.join(root, self.directory, "multilabels.csv")
    df = pd.read_csv(csv_path)
    for _, row in df.iterrows():
        self.multilabels[row.iloc[0]] = [int(x) for x in row.iloc[1:]]

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/dlrsd.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"])
    if image.max() > 1:
        image = image.astype("float32") / 255.0

    label_mask = ops.convert_to_numpy(sample["label"]).astype(bool)
    labels = self._onehot_labels_to_names(label_mask)

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction_mask = ops.convert_to_numpy(sample["prediction"]).astype(bool)
        predictions = self._onehot_labels_to_names(prediction_mask)

    fig, ax = plt.subplots(figsize=(4, 4))
    ax.imshow(image)
    ax.axis("off")
    if show_titles:
        title = f"Labels: {', '.join(labels)}"
        if showing_predictions:
            title += f"\nPredictions: {', '.join(predictions)}"
        ax.set_title(title)

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.dota.DOTA

DOTA(root='data', split='train', version='2.0', bbox_orientation='oriented', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

DOTA dataset.

DOTA <https://captain-whu.github.io/DOTA/index.html>__ is a large-scale object detection dataset for aerial imagery containing RGB and gray-scale imagery from Google Earth, GF-2 and JL-1 satellites as well as additional aerial imagery from CycloMedia. There are three versions of the dataset: v1.0, v1.5, and v2.0, where v1.0 and v1.5 have the same images but different annotations, and v2.0 extends both the images and annotations with more samples.

Dataset features:

• 1869 samples in v1.0 and v1.5 and 2423 samples in v2.0
• multi-class object detection (15 classes in v1.0 and v1.5 and 18 classes in v2.0)
• horizontal and oriented bounding boxes

Dataset format:

• images are three channel PNGs with various pixel sizes
• annotations are text files with one line per bounding box

Classes:

1. plane
2. ship
3. storage-tank
4. baseball-diamond
5. tennis-court
6. basketball-court
7. ground-track-field
8. harbor
9. bridge
10. large-vehicle
11. small-vehicle
12. helicopter
13. roundabout
14. soccer-ball-field
15. swimming-pool
16. container-crane (v1.5+)
17. airport (v2.0+)
18. helipad (v2.0+)

If you use this work in your research, please cite the following papers:

• https://arxiv.org/abs/2102.12219
• https://arxiv.org/abs/1711.10398

Initialize a new DOTA dataset instance.

Args: root: root directory where dataset can be found split: split of the dataset to use, one of ['train', 'val'] version: version of the dataset to use, one of ['1.0', '1.5', '2.0'] bbox_orientation: bounding box orientation, one of ['horizontal', 'oriented'], where horizontal returns xyxy format and oriented returns x1y1x2y2x3y3x4y4 format transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: AssertionError: if split, version, or bbox_orientation argument are not valid DatasetNotFoundError: if dataset is not found or corrupted, and download is False

Source code in keras_climate/datasets/dota.py

def __init__(
    self,
    root="data",
    split="train",
    version="2.0",
    bbox_orientation="oriented",
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new DOTA dataset instance.

    Args:
        root: root directory where dataset can be found
        split: split of the dataset to use, one of ['train', 'val']
        version: version of the dataset to use, one of ['1.0', '1.5', '2.0']
        bbox_orientation: bounding box orientation, one of
            ['horizontal', 'oriented'], where horizontal returns xyxy
            format and oriented returns x1y1x2y2x3y3x4y4 format
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if *split*, *version*, or *bbox_orientation*
            argument are not valid
        DatasetNotFoundError: if dataset is not found or corrupted, and
            *download* is False
    """
    assert split in self.valid_splits, f"Split '{split}' not supported, use one of {self.valid_splits}"
    assert version in self.valid_versions, f"Version '{version}' not supported, use one of {self.valid_versions}"

    assert bbox_orientation in self.valid_orientations, (
        f"Bounding box orientation must be one of {self.valid_orientations}"
    )

    self.root = root
    self.split = split
    self.version = version
    self.transforms = transforms
    self.download = download
    self.checksum = checksum
    self.bbox_orientation = bbox_orientation

    self._verify()

    self.sample_df = pd.read_csv(os.path.join(self.root, "samples.csv"))
    self.sample_df["version"] = self.sample_df["version"].astype(str)
    self.sample_df = self.sample_df[self.sample_df["split"] == self.split]
    self.sample_df = self.sample_df[self.sample_df["version"] == self.version].reset_index(drop=True)

plot

plot(sample, show_titles=True, suptitle=None, box_alpha=0.7)

Plot a sample from the dataset.

Source code in keras_climate/datasets/dota.py

def plot(self, sample, show_titles=True, suptitle=None, box_alpha=0.7):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt
    from matplotlib import patches

    image = quantile_normalization(sample["image"])
    image = ops.convert_to_numpy(image)
    if self.bbox_orientation == "horizontal":
        boxes = ops.convert_to_numpy(sample["bbox_xyxy"])
    else:
        boxes = ops.convert_to_numpy(sample["bbox"])
    labels = ops.convert_to_numpy(sample["labels"])

    fig, ax = plt.subplots(figsize=(10, 10))
    ax.imshow(image)
    ax.axis("off")

    # Create color map for classes
    cm = plt.get_cmap("gist_rainbow")

    for box, label_idx in zip(boxes, labels):
        color = cm(label_idx / len(self.classes))
        label = self.classes[label_idx]

        if self.bbox_orientation == "horizontal":
            # Horizontal box: [xmin, ymin, xmax, ymax]
            x1, y1, x2, y2 = box
            rect = patches.Rectangle(
                (x1, y1),
                x2 - x1,
                y2 - y1,
                linewidth=2,
                alpha=box_alpha,
                linestyle="solid",
                edgecolor=color,
                facecolor="none",
            )
            ax.add_patch(rect)
            # Add label above box
            ax.text(
                x1, y1 - 5, label, color="white", fontsize=8, bbox={"facecolor": color, "alpha": box_alpha}
            )
        else:
            # Oriented box: [x1,y1,x2,y2,x3,y3,x4,y4]
            vertices = box.reshape(4, 2)
            polygon = patches.Polygon(
                vertices,
                linewidth=2,
                alpha=box_alpha,
                linestyle="solid",
                edgecolor=color,
                facecolor="none",
            )
            ax.add_patch(polygon)
            # Add label at centroid
            centroid_x = vertices[:, 0].mean()
            centroid_y = vertices[:, 1].mean()
            ax.text(
                centroid_x,
                centroid_y,
                label,
                color="white",
                fontsize=8,
                bbox={"facecolor": color, "alpha": box_alpha},
                ha="center",
                va="center",
            )

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.earth_embeddings.EarthEmbeddings

EarthEmbeddings(root='data', transforms=None)

Bases: NonGeoDataset

EarthEmbeddings dataset.

EarthEmbeddings <https://huggingface.co/datasets/ML4Sustain/EarthEmbeddings>__ are pre-computed embeddings of uniformly sampled MajorTOM-Core-S2L2A imagery using SatCLIP, FarSLIP, DINOv2, SigLIP models. These embeddings power the EarthEmbeddingExplorer application, which allows users to search for satellite images using text queries, image uploads, or geographic locations.

Initialize a new EarthEmbeddings instance.

Args: root: root directory (or parquet file) where dataset can be found transforms: a function/transform that takes input sample and its target as entry and returns a transformed version

Raises: DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/earth_embeddings.py

def __init__(self, root="data", transforms=None):
    """Initialize a new EarthEmbeddings instance.

    Args:
        root: root directory (or parquet file) where dataset can be found
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version

    Raises:
        DatasetNotFoundError: If dataset is not found.
    """
    self.root = root
    self.transforms = transforms

    try:
        self.data = pd.read_parquet(root)
    except (FileNotFoundError, ValueError, OSError):
        raise DatasetNotFoundError(self)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/earth_embeddings.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    fig, ax = plt.subplots()
    ax.plot(ops.convert_to_numpy(sample["embedding"]))

    if show_titles:
        x = float(ops.convert_to_numpy(sample["x"]))
        y = float(ops.convert_to_numpy(sample["y"]))
        t = pd.Timestamp.fromtimestamp(float(ops.convert_to_numpy(sample["t"])))
        ax.set_title(rf"{y:0.3f}°N, {x:0.3f}°W, {t}")

    if suptitle is not None:
        plt.suptitle(suptitle)

    fig.tight_layout()
    return fig

keras_climate.datasets.earth_index.EarthIndexEmbeddings

EarthIndexEmbeddings(root='data', transforms=None)

Bases: NonGeoDataset

Earth Index Embeddings dataset.

Earth Index Embeddings <https://source.coop/earthgenome/earthindexembeddings>__ are a global embedding product generated from Earth Index v2 Sentinel-2 mosaics. The embeddings are generated using the SoftCon model from Zhu XLabs and result in an embedding of length 384. Each embedding captures a 320 square meter patch of the Earth, gridded using a MajorTom-based grid. Embeddings, their IDs and centroids are encoded in geoparquet.

Initialize a new EarthIndexEmbeddings instance.

Args: root: root directory (or geoparquet file) where dataset can be found transforms: a function/transform that takes input sample and its target as entry and returns a transformed version

Raises: DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/earth_index.py

def __init__(self, root="data", transforms=None):
    """Initialize a new EarthIndexEmbeddings instance.

    Args:
        root: root directory (or geoparquet file) where dataset can be found
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version

    Raises:
        DatasetNotFoundError: If dataset is not found.
    """
    self.root = root
    self.transforms = transforms

    try:
        self.data = gpd.read_parquet(root)
    except (FileNotFoundError, ValueError, OSError):
        raise DatasetNotFoundError(self)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/earth_index.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    fig, ax = plt.subplots()
    ax.plot(ops.convert_to_numpy(sample["embedding"]))

    if show_titles:
        x = float(ops.convert_to_numpy(sample["x"]))
        y = float(ops.convert_to_numpy(sample["y"]))
        ax.set_title(rf"{y:0.3f}°N, {x:0.3f}°W")

    if suptitle is not None:
        plt.suptitle(suptitle)

    fig.tight_layout()
    return fig

keras_climate.datasets.eddmaps.EDDMapS

EDDMapS(root='data')

Bases: GeoDataset

Dataset for EDDMapS.

EDDMapS <https://www.eddmaps.org/>__, Early Detection and Distribution Mapping System, is a web-based mapping system for documenting invasive species and pest distribution. Launched in 2005 by the Center for Invasive Species and Ecosystem Health at the University of Georgia, it was originally designed as a tool for state Exotic Pest Plant Councils to develop more complete distribution data of invasive species.

EDDMapS query results can be downloaded in CSV, KML, or Shapefile format. This dataset currently only supports CSV files.

If you use an EDDMapS dataset in your research, please cite it like so:

• EDDMapS. YEAR. Early Detection & Distribution Mapping System. The University of Georgia - Center for Invasive Species and Ecosystem Health. Available online at https://www.eddmaps.org/; last accessed DATE.

Initialize a new Dataset instance.

Args: root: root directory where dataset can be found

Raises: DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/eddmaps.py

def __init__(self, root="data"):
    """Initialize a new Dataset instance.

    Args:
        root: root directory where dataset can be found

    Raises:
        DatasetNotFoundError: If dataset is not found.
    """
    self.root = root

    filepath = os.path.join(root, "mappings.csv")
    if not os.path.exists(filepath):
        raise DatasetNotFoundError(self)

    df = pd.read_csv(filepath, usecols=["ObsDate", "Latitude", "Longitude"])
    df = df[df.Latitude.notna()]
    df = df[df.Longitude.notna()]

    func = functools.partial(disambiguate_timestamp, format="%m-%d-%y")
    data = df["ObsDate"].apply(func).to_list()
    index = pd.IntervalIndex.from_tuples(data, closed="both", name="datetime")
    geometry = gpd.points_from_xy(df.Longitude, df.Latitude)
    self.index = GeoDataFrame(index=index, geometry=geometry, crs="EPSG:4326")

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/eddmaps.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    fig, ax = plt.subplots(figsize=(10, 8))
    ax.grid(ls="--")

    keypoints = ops.convert_to_numpy(sample["keypoints"])
    x = keypoints[:, 0]
    y = keypoints[:, 1]

    ax.scatter(x, y)

    ax.set_xlabel("Longitude")
    ax.set_ylabel("Latitude")

    if show_titles:
        ax.set_title("EDDMapS Observation Locations by Date")

    if suptitle is not None:
        fig.suptitle(suptitle)

    fig.tight_layout()
    return fig

keras_climate.datasets.enmap.EnMAP

EnMAP(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset

EnMAP <https://www.enmap.org/>__ dataset.

The Environmental Mapping and Analysis Program (EnMAP) is a German hyperspectral satellite mission that monitors and characterizes Earth's environment on a global scale. EnMAP measures geochemical, biochemical and biophysical variables providing information on the status and evolution of terrestrial and aquatic ecosystems.

Mission Outline:

• Dedicated pushbroom hyperspectral imager
• Broad spectral range from 420 nm to 1000 nm (VNIR) and from 900 nm to 2450 nm (SWIR) with high radiometric resolution and stability
• 30 km swath width at a spatial resolution of 30 x 30 m, nadir revisit time of 27 days and off-nadir (30°) pointing feature for fast target revisit (4 days)

If you use this dataset in your research, please cite the following papers:

• https://doi.org/10.1016/j.rse.2024.114379
• https://doi.org/10.1016/j.rse.2023.113632

Initialize a new EnMAP instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS (defaults to the resolution of the first file found) bands: bands to return (defaults to default_bands, which excludes bands strongly affected by water vapor absorption) transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling time_series: if True, stack data along the time series dimension

Raises: DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/enmap.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new EnMAP instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to ``default_bands``, which
            excludes bands strongly affected by water vapor absorption)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension

    Raises:
        DatasetNotFoundError: If dataset is not found.
    """
    bands = bands or self.default_bands
    super().__init__(paths, crs, res, bands, transforms, cache, time_series)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/enmap.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise RGBBandsMissingError()

    image = ops.take(sample["image"], rgb_indices, axis=-1)
    image = ops.maximum(image, 0)
    image = quantile_normalization(image)

    fig, ax = plt.subplots()
    ax.imshow(ops.convert_to_numpy(image))
    ax.axis("off")

    if show_titles:
        ax.set_title("Image")

    if suptitle:
        fig.suptitle(suptitle)

    return fig

keras_climate.datasets.enviroatlas.EnviroAtlas

EnviroAtlas(root='data', splits=['pittsburgh_pa-2010_1m-train'], layers=['naip', 'prior'], transforms=None, prior_as_input=False, cache=True, download=False, checksum=True)

Bases: GeoDataset

EnviroAtlas dataset covering four cities with prior and weak input data layers.

The EnviroAtlas <https://doi.org/10.5281/zenodo.5778192>__ dataset contains NAIP aerial imagery, NLCD land cover labels, OpenStreetMap roads, water, waterways, and waterbodies, Microsoft building footprint labels, high-resolution land cover labels from the EPA EnviroAtlas dataset, and high-resolution land cover prior layers.

This dataset was organized to accompany the 2022 paper, "Resolving label uncertainty with implicit posterior models" <https://arxiv.org/abs/2202.14000>_. More details can be found at https://github.com/estherrolf/implicit-posterior.

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2202.14000

Initialize a new Dataset instance.

Args: root: root directory where dataset can be found splits: a list of strings in the format "{state}-{train,val,test}" indicating the subset of data to use, for example "ny-train" layers: a list containing a subset of valid_layers indicating which layers to load transforms: a function/transform that takes an input sample and returns a transformed version prior_as_input: whether the prior is used as an input (True) or as supervision (False) cache: if True, cache file handle to speed up repeated sampling download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: if splits or layers are not valid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/enviroatlas.py

def __init__(
    self,
    root="data",
    splits=["pittsburgh_pa-2010_1m-train"],
    layers=["naip", "prior"],
    transforms=None,
    prior_as_input=False,
    cache=True,
    download=False,
    checksum=True,
):
    """Initialize a new Dataset instance.

    Args:
        root: root directory where dataset can be found
        splits: a list of strings in the format "{state}-{train,val,test}"
            indicating the subset of data to use, for example "ny-train"
        layers: a list containing a subset of ``valid_layers`` indicating
            which layers to load
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        prior_as_input: whether the prior is used as an input (True) or
            as supervision (False)
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: if ``splits`` or ``layers`` are not valid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    for split in splits:
        assert split in self.splits
    assert all(layer in self.valid_layers for layer in layers)
    self.root = root
    self.layers = layers
    self.transforms = transforms
    self.cache = cache
    self.download = download
    self.checksum = checksum
    self.prior_as_input = prior_as_input

    self._verify()

    # Add all tiles into the index in epsg:3857 based on the included geojson
    mint = pd.Timestamp.min
    maxt = pd.Timestamp.max
    gdf = gpd.read_file(
        os.path.join(root, "enviroatlas_lotp", "spatial_index.geojson")
    )
    gdf.set_crs(CRS.from_epsg(3857), inplace=True)
    gdf = gdf[gdf["split"].isin(splits)]
    gdf["prior_no_osm_no_buildings"] = gdf["naip"].replace(
        "a_naip", "prior_from_cooccurrences_101_31_no_osm_no_buildings"
    )
    gdf["prior"] = gdf["naip"].replace("a_naip", "prior_from_cooccurrences_101_31")
    datetimes = [(mint, maxt)] * len(gdf)
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    gdf.set_index(index, inplace=True)
    self.index = gdf

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Note: only plots the "naip" and "lc" layers.

Raises: ValueError: if the NAIP layer isn't included in self.layers

Source code in keras_climate/datasets/enviroatlas.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Note: only plots the "naip" and "lc" layers.

    Raises:
        ValueError: if the NAIP layer isn't included in ``self.layers``
    """
    import matplotlib.pyplot as plt

    if "naip" not in self.layers or "lc" not in self.layers:
        raise ValueError("The 'naip' and 'lc' layers must be included for plotting")

    image_layers = []
    mask_layers = []
    for layer in self.layers:
        if layer in [
            "naip",
            "buildings",
            "roads",
            "waterways",
            "waterbodies",
            "water",
        ]:
            image_layers.append(layer)
        elif layer in ["prior", "prior_no_osm_no_buildings"]:
            if self.prior_as_input:
                image_layers.append(layer)
            else:
                mask_layers.append(layer)
        elif layer in ["lc"]:
            mask_layers.append(layer)

    naip_index = image_layers.index("naip")
    lc_index = mask_layers.index("lc")

    full_image = ops.convert_to_numpy(sample["image"])
    full_mask = ops.convert_to_numpy(sample["mask"])

    image = full_image[:, :, naip_index : naip_index + 3]
    mask = full_mask[:, :, lc_index]

    num_panels = 2
    showing_predictions = "prediction" in sample
    if showing_predictions:
        predictions = ops.convert_to_numpy(sample["prediction"])
        num_panels += 1

    fig, axs = plt.subplots(1, num_panels, figsize=(num_panels * 4, 5))
    axs[0].imshow(image)
    axs[0].axis("off")
    axs[1].imshow(
        mask, vmin=0, vmax=10, cmap=self.highres_cmap, interpolation="none"
    )
    axs[1].axis("off")
    if show_titles:
        axs[0].set_title("Image")
        axs[1].set_title("Mask")

    if showing_predictions:
        axs[2].imshow(
            predictions,
            vmin=0,
            vmax=10,
            cmap=self.highres_cmap,
            interpolation="none",
        )
        axs[2].axis("off")
        if show_titles:
            axs[2].set_title("Predictions")

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.esd.ESDQuantizer

ESDQuantizer(levels=(8, 8, 8, 5, 5, 5))

Decode ESD-encoded quantized indices into continuous embedding vectors.

The ESDQuantizer converts integer quantization indices produced by an ESD quantizer into continuous values in the range [-1, 1], representing multi-level embeddings of the original input. This enables downstream tasks, such as visualization, machine learning, or spatial analysis, to operate directly on decoded embeddings without reconstructing the full raw data.

Key points:

• Factorized decoding: Each index is split into multiple levels according to the quantizer configuration.
• Continuous mapping: Level indices are rescaled and centered to [-1, 1], preserving relative distances in embedding space.
• Fully vectorized: The decoding is performed on entire tensors at once.

.. note:: Unlike torchgeo (which moves the decoded level axis to a channels-first position), this port keeps the last axis as the channel axis (matching keras_climate's channels-last convention) and instead moves the raster-band axis (which behaves like a "time" dimension of factorized codes) to the front, producing a (T, H, W, L) tensor, matching the time_series layout already used by :class:~keras_climate.datasets.geo.RasterDataset.

Initialize the quantization levels for the embedding.

Args: levels: sequence of integers specifying the number of quantization levels for each embedding dimension.

Source code in keras_climate/datasets/esd.py

def __init__(self, levels=(8, 8, 8, 5, 5, 5)):
    """Initialize the quantization levels for the embedding.

    Args:
        levels: sequence of integers specifying the number of
            quantization levels for each embedding dimension.
    """
    self._levels = np.array(levels, dtype="int32")
    self._basis = np.cumprod(np.array([1, *levels[:-1]], dtype="int32"))

indices_to_codes

indices_to_codes(indices)

Convert embedding indices to normalized continuous codes.

Args: indices: a tensor of integer indices representing quantized embeddings. Shape can be arbitrary (...,).

Returns: Tensor of shape (..., len(levels)) with values normalized to [-1, 1].

Source code in keras_climate/datasets/esd.py

def indices_to_codes(self, indices):
    """Convert embedding indices to normalized continuous codes.

    Args:
        indices: a tensor of integer indices representing quantized
            embeddings. Shape can be arbitrary (...,).

    Returns:
        Tensor of shape ``(..., len(levels))`` with values normalized to
        [-1, 1].
    """
    indices = ops.cast(ops.convert_to_tensor(indices), "int32")
    indices = ops.expand_dims(indices, axis=-1)

    basis = ops.convert_to_tensor(self._basis)
    levels = ops.convert_to_tensor(self._levels)
    level_indices = ops.mod(ops.floor_divide(indices, basis), levels)

    half = ops.cast(self._levels // 2, "float32")
    codes = (ops.cast(level_indices, "float32") - half) / half

    return codes

quantize

quantize(input)

Quantize the input tensor using predefined levels.

Args: input: a tensor containing integer embedding indices of shape (H, W, T) (channels-last raster bands).

Returns: Tensor of quantized codes with normalized values in [-1, 1] and shape (T, H, W, L), where L is the number of levels.

Source code in keras_climate/datasets/esd.py

def quantize(self, input):
    """Quantize the input tensor using predefined levels.

    Args:
        input: a tensor containing integer embedding indices of shape
            ``(H, W, T)`` (channels-last raster bands).

    Returns:
        Tensor of quantized codes with normalized values in [-1, 1] and
        shape ``(T, H, W, L)``, where ``L`` is the number of levels.
    """
    codes = self.indices_to_codes(input)  # (H, W, T, L)
    return ops.moveaxis(codes, -2, 0)  # (T, H, W, L)

keras_climate.datasets.esd.EmbeddedSeamlessData

EmbeddedSeamlessData(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset

Embedded Seamless Data (ESD).

The Embedded Seamless Data (ESD) <https://arxiv.org/abs/2601.11183>__ is a global, analysis-ready Earth embedding dataset at 30-meter resolution, designed to overcome the computational and storage challenges of planetary-scale Earth system science. By transforming multi-sensor satellite observations into compact, quantized latent vectors, ESD reduces the original data volume (~1 PB for a full year of global land surfaces) to approximately 2.4 TB, enabling decadal-scale analysis on standard workstations.

Key features:

• Longitudinal Consistency: Provides a continuous record from 2000 to 2024, harmonized from Landsat 5, 7, 8, 9, MODIS Terra and NASADEM imagery.
• High Reconstructive Fidelity: Achieves a Mean Absolute Error (MAE) of 0.013 across six spectral bands.
• Semantic Intelligence: Captures complex land surface patterns.
• Implicit Denoising: Filters transient noise such as clouds and shadows via the ESDNet architecture.
• Few-Shot Proficiency: Supports robust learning with minimal labeled data.

If you use this dataset in your research, please refer to:

• Paper: https://arxiv.org/abs/2601.11183
• Code: https://github.com/shuangchencc/ESD
• Dataset: https://data-starcloud.pcl.ac.cn/iearthdata/64

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new RasterDataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a mosaic
            (``[H, W, C]``). For mask-style datasets (``is_image=False``),
            single-band data has the channel dimension squeezed.
    """
    self.paths = paths
    self.bands = bands or self.all_bands
    self.transforms = transforms
    self.cache = cache
    self.time_series = time_series

    if self.all_bands:
        assert set(self.bands) <= set(self.all_bands)

    filename_regex = re.compile(self.filename_regex, re.VERBOSE)
    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        match = re.match(filename_regex, os.path.basename(filepath))
        if match is not None:
            vrt = None
            try:
                vrt = self._load_warp_file(filepath=filepath, crs=crs)
                if self.cmap is None:
                    try:
                        colors = np.array(list(vrt.colormap(1).values())) / 255
                        self.cmap = ListedColormap(colors)
                    except ValueError:
                        pass
                if crs is None:
                    with rasterio.Env(OSR_WKT_FORMAT="WKT2_2018"):
                        crs = PROJ_CRS.from_user_input(vrt.crs)
                footprint = self.footprint_from_datasource(vrt)
                if footprint is None:
                    footprint = shapely.box(*vrt.bounds)
                geometries.append(footprint)
                if res is None:
                    res = vrt.res
            except rasterio.errors.RasterioIOError:
                continue
            else:
                filepaths.append(filepath)
                mint, maxt = self._filepath_to_timestamp(filepath)
                datetimes.append((mint, maxt))
            finally:
                if vrt is not None:
                    vrt.close()

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if not self.separate_files:
        self.band_indexes = None
        if self.bands:
            if self.all_bands:
                self.band_indexes = [
                    self.all_bands.index(i) + 1 for i in self.bands
                ]
            else:
                msg = (
                    f"{self.__class__.__name__} is missing an `all_bands` "
                    "attribute, so `bands` cannot be specified."
                )
                raise AssertionError(msg)

    if res is not None:
        if isinstance(res, (int, float)):
            res = (res, res)

        self._res = res

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/esd.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    vectors = ops.convert_to_numpy(sample["image"])  # (T, H, W, L)
    _months, H, W, _channels = vectors.shape

    # Compute valid mask: any non-zero pixel across channels, combining
    # the first 12 "months" (raster bands).
    valid_mask = ~np.isclose(vectors, 0.0, atol=1e-6)
    valid_mask = valid_mask[:12].any(axis=-1).any(axis=0)

    # Reduce channels to RGB using mean over selected levels.
    R = (vectors[:, :, :, 5].mean(axis=0) + 1) / 2
    G = (vectors[:, :, :, 1].mean(axis=0) + 1) / 2
    B = (vectors[:, :, :, 2].mean(axis=0) + 1) / 2

    disp_img = np.zeros((H, W, 4), dtype=np.uint8)
    disp_img[..., 0] = (np.clip(R, 0, 1) * 255).astype(np.uint8)
    disp_img[..., 1] = (np.clip(G, 0, 1) * 255).astype(np.uint8)
    disp_img[..., 2] = (np.clip(B, 0, 1) * 255).astype(np.uint8)
    disp_img[..., 3] = valid_mask.astype(np.uint8) * 255

    fig, ax = plt.subplots()
    ax.imshow(disp_img)
    ax.axis("off")

    if show_titles:
        ax.set_title("ESD Embedding Visualization")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.esri2020.Esri2020

Esri2020(paths='data', crs=None, res=None, transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: RasterDataset

Esri 2020 Land Cover Dataset.

The Esri 2020 Land Cover dataset <https://www.arcgis.com/home/item.html?id=fc92d38533d440078f17678ebc20e8e2>_ consists of a global single band land use/land cover map derived from ESA Sentinel-2 imagery at 10m resolution with a total of 10 classes. It was published in July 2021 and used the Universal Transverse Mercator (UTM) projection. This dataset only contains labels, no raw satellite imagery.

The 10 classes are:

1. No Data
2. Water
3. Trees
4. Grass
5. Flooded Vegetation
6. Crops
7. Scrub/Shrub
8. Built Area
9. Bare Ground
10. Snow/Ice
11. Clouds

If you use this dataset please cite the following paper:

• https://ieeexplore.ieee.org/document/9553499

Initialize a new Dataset instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS (defaults to the resolution of the first file found) transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow) time_series: if True, stack data along the time series dimension

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/esri2020.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)
        time_series: if True, stack data along the time series dimension

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.paths = paths
    self.download = download
    self.checksum = checksum

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/esri2020.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    mask = ops.convert_to_numpy(sample["mask"]).squeeze()
    ncols = 1

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = ops.convert_to_numpy(sample["prediction"]).squeeze()
        ncols = 2

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(4 * ncols, 4))

    if showing_predictions:
        axs[0].imshow(mask)
        axs[0].axis("off")
        axs[1].imshow(prediction)
        axs[1].axis("off")
        if show_titles:
            axs[0].set_title("Mask")
            axs[1].set_title("Prediction")
    else:
        axs.imshow(mask)
        axs.axis("off")
        if show_titles:
            axs.set_title("Mask")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.etci2021.ETCI2021

ETCI2021(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

ETCI 2021 Flood Detection dataset.

The ETCI2021 <https://nasa-impact.github.io/etci2021/>_ dataset is a dataset for flood detection.

Dataset features:

• 33,405 VV & VH Sentinel-1 Synthetic Aperture Radar (SAR) images
• 2 binary masks per image representing water body & flood, respectively
• 2 polarization band images (VV, VH) of 3 RGB channels per band
• 3 RGB channels per band generated by the Hybrid Pluggable Processing Pipeline (hyp3)
• Images with 5x20m per pixel resolution (256x256) px taken in Interferometric Wide Swath acquisition mode
• Flood events from 5 different regions

Dataset format:

• VV band three-channel png
• VH band three-channel png
• water body mask single-channel png where no water body = 0, water body = 255
• flood mask single-channel png where no flood = 0, flood = 255

Dataset classes:

1. no flood/water
2. flood/water

If you use this dataset in your research, please add the following to your acknowledgements section::

The authors would like to thank the NASA Earth Science Data Systems
Program, NASA Digital Transformation AI/ML thrust, and IEEE GRSS for
organizing the ETCI competition.

Initialize a new ETCI 2021 dataset instance.

Args: root: root directory where dataset can be found split: one of "train", "val", or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: if split argument is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/etci2021.py

def __init__(self, root="data", split="train", transforms=None, download=False, checksum=True):
    """Initialize a new ETCI 2021 dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val", or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in self.metadata

    self.root = root
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    if download:
        self._download()

    if not self._check_integrity():
        raise DatasetNotFoundError(self)

    self.files = self._load_files(self.root, self.split)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/etci2021.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"])
    vv = image[:, :, :3]
    vh = image[:, :, 3:]
    mask = ops.convert_to_numpy(sample["mask"])

    showing_flood_mask = mask.shape[-1] == 2
    showing_predictions = "prediction" in sample
    num_panels = 3
    if showing_flood_mask:
        water_mask = mask[:, :, 0]
        flood_mask = mask[:, :, 1]
        num_panels += 1
    else:
        water_mask = mask[:, :, 0]

    if showing_predictions:
        predictions = ops.convert_to_numpy(sample["prediction"])
        num_panels += 1

    fig, axs = plt.subplots(1, num_panels, figsize=(num_panels * 4, 3))
    axs[0].imshow(vv)
    axs[0].axis("off")
    axs[1].imshow(vh)
    axs[1].axis("off")
    axs[2].imshow(water_mask)
    axs[2].axis("off")
    if show_titles:
        axs[0].set_title("VV")
        axs[1].set_title("VH")
        axs[2].set_title("Water mask")

    idx = 0
    if showing_flood_mask:
        axs[3 + idx].imshow(flood_mask)
        axs[3 + idx].axis("off")
        if show_titles:
            axs[3 + idx].set_title("Flood mask")
        idx += 1

    if showing_predictions:
        axs[3 + idx].imshow(predictions)
        axs[3 + idx].axis("off")
        if show_titles:
            axs[3 + idx].set_title("Predictions")
        idx += 1

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.eudem.EUDEM

EUDEM(paths='data', crs=None, res=None, transforms=None, cache=True, checksum=True, time_series=False)

Bases: RasterDataset

European Digital Elevation Model (EU-DEM) Dataset.

EU-DEM <https://www.eea.europa.eu/en/datahub/datahubitem-view/d08852bc-7b5f-4835-a776-08362e2fbf4b>__ is a Digital Elevation Model of reference for the entire European region.

Dataset features:

• DEMs at 25 m per pixel spatial resolution (~40,000x40,0000 px)
• vertical accuracy of +/- 7 m RMSE
• data fused from ASTER GDEM, SRTM and Russian topomaps

Dataset format:

• DEMs are single-channel tif files

Initialize a new Dataset instance.

Args: paths: one or more root directories to search or files to load, here the collection of individual zip files for each tile should be found crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS (defaults to the resolution of the first file found) transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling checksum: if True, check the MD5 of the downloaded files (may be slow) time_series: if True, stack data along the time series dimension

Raises: DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/eudem.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    transforms=None,
    cache=True,
    checksum=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load,
            here the collection of individual zip files for each tile
            should be found
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)
        time_series: if True, stack data along the time series dimension

    Raises:
        DatasetNotFoundError: If dataset is not found.
    """
    self.paths = paths
    self.checksum = checksum

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/eudem.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    mask = ops.convert_to_numpy(sample["mask"]).squeeze()
    ncols = 1

    showing_predictions = "prediction" in sample
    if showing_predictions:
        pred = ops.convert_to_numpy(sample["prediction"]).squeeze()
        ncols = 2

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 4, 4))

    if showing_predictions:
        axs[0].imshow(mask)
        axs[0].axis("off")
        axs[1].imshow(pred)
        axs[1].axis("off")
        if show_titles:
            axs[0].set_title("Mask")
            axs[1].set_title("Prediction")
    else:
        axs.imshow(mask)
        axs.axis("off")
        if show_titles:
            axs.set_title("Mask")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.eurocrops.EuroCrops

EuroCrops(paths='data', crs=None, res=(1e-05, 1e-05), classes=None, transforms=None, download=False, checksum=True)

Bases: VectorDataset

EuroCrops Dataset (Version 9).

The EuroCrops <https://www.eurocrops.tum.de/index.html>__ dataset combines "all publicly available self-declared crop reporting datasets from countries of the European Union" into a unified format. The dataset is released under CC BY 4.0 Deed.

The dataset consists of shapefiles containing a total of 22M polygons. Each polygon is tagged with an "EC_hcat_n" attribute indicating the harmonized crop name grown within the polygon in the year associated with the shapefile.

If you use this dataset in your research, please follow the citation guidelines at:

• https://github.com/maja601/EuroCrops#reference.

Initialize a new EuroCrops instance.

Args: paths: one or more root directories to search for files to load crs: CRS to warp to (defaults to WGS-84) res: resolution of the dataset in units of CRS (xres, yres) format classes: list of classes to include (specified by their HCAT code), the rest will be mapped to 0 (defaults to all classes) transforms: a function/transform that takes an input sample and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/eurocrops.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=(0.00001, 0.00001),
    classes=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new EuroCrops instance.

    Args:
        paths: one or more root directories to search for files to load
        crs: CRS to warp to (defaults to WGS-84)
        res: resolution of the dataset in units of CRS (xres, yres) format
        classes: list of classes to include (specified by their HCAT
            code), the rest will be mapped to 0 (defaults to all classes)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.paths = paths
    self.checksum = checksum

    if download:
        self._download()

    if not self._check_integrity():
        raise DatasetNotFoundError(self)

    if crs is None:
        crs = CRS.from_epsg(4326)

    self._load_class_map(classes)

    super().__init__(
        paths=paths, crs=crs, res=res, transforms=transforms, label_name=self.label_name
    )

get_label

get_label(feature)

Get label value to use for rendering a feature.

Converts the HCAT code of this feature to its index per self.class_map. We go up the class hierarchy until there is a match (parent code is computed by replacing rightmost non-0 character with 0).

Source code in keras_climate/datasets/eurocrops.py

def get_label(self, feature):
    """Get label value to use for rendering a feature.

    Converts the HCAT code of this feature to its index per
    ``self.class_map``. We go up the class hierarchy until there is a
    match (parent code is computed by replacing rightmost non-0
    character with 0).
    """
    assert self.label_name
    hcat_code = feature[self.label_name]
    if hcat_code is None:
        return 0

    while True:
        if hcat_code in self.class_map:
            return self.class_map[hcat_code]
        hcat_code_list = list(hcat_code)
        if all(c == "0" for c in hcat_code_list):
            break
        for i in range(len(hcat_code_list) - 1, -1, -1):
            if hcat_code_list[i] == "0":
                continue
            hcat_code_list[i] = "0"
            break
        hcat_code = "".join(hcat_code_list)
    return 0

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/eurocrops.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    mask = ops.convert_to_numpy(sample["mask"]).squeeze()
    ncols = 1

    showing_prediction = "prediction" in sample
    if showing_prediction:
        pred = ops.convert_to_numpy(sample["prediction"]).squeeze()
        ncols = 2

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(4, 4))

    def apply_cmap(array):
        """Color 0 as black, applying the default colormap to class indices."""
        cmap = plt.get_cmap("viridis")
        image = cmap(array / len(self.class_map))
        image[array == 0] = 0
        return image

    if showing_prediction:
        axs[0].imshow(apply_cmap(mask), interpolation="none")
        axs[0].axis("off")
        axs[1].imshow(apply_cmap(pred), interpolation="none")
        axs[1].axis("off")
        if show_titles:
            axs[0].set_title("Mask")
            axs[1].set_title("Prediction")
    else:
        axs.imshow(apply_cmap(mask), interpolation="none")
        axs.axis("off")
        if show_titles:
            axs.set_title("Mask")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.eurosat.EuroSAT

EuroSAT(root='data', split='train', bands=BAND_SETS['all'], transforms=None, download=False, checksum=True)

Bases: NonGeoClassificationDataset

EuroSAT dataset.

The EuroSAT <https://github.com/phelber/EuroSAT>__ dataset is based on Sentinel-2 satellite images covering 13 spectral bands and consists of 10 target classes with a total of 27,000 labeled and geo-referenced images.

Dataset format:

• rasters are 13-channel GeoTiffs
• labels are values in the range [0,9]

This dataset uses the train/val/test splits defined in the "In-domain representation learning for remote sensing" paper:

• https://arxiv.org/abs/1911.06721

If you use this dataset in your research, please cite:

• https://ieeexplore.ieee.org/document/8736785
• https://ieeexplore.ieee.org/document/8519248

Initialize a new EuroSAT dataset instance.

Args: root: root directory where dataset can be found split: one of "train", "val", or "test" bands: a sequence of band names to load transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: AssertionError: if split argument is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/eurosat.py

def __init__(
    self,
    root="data",
    split="train",
    bands=BAND_SETS["all"],
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new EuroSAT dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val", or "test"
        bands: a sequence of band names to load
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.root = root
    self.split = split
    self.tg_transforms = transforms
    self.download = download
    self.checksum = checksum

    assert self.split in {"train", "val", "test"}

    self._validate_bands(bands)
    self.bands = bands
    self.band_indices = np.array(
        [self.all_band_names.index(b) for b in bands if b in self.all_band_names],
        dtype="int64",
    )

    self._verify()

    valid_fns = set()
    with open(os.path.join(self.root, self.split_filenames[split])) as f:
        for fn in f:
            valid_fns.add(fn.strip().replace(".jpg", ".tif"))

    def is_in_split(x):
        return os.path.basename(x) in valid_fns

    super().__init__(
        root=os.path.join(root, self.base_dir),
        transforms=transforms,
        loader=rasterio_loader,
        is_valid_file=is_in_split,
    )

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/eurosat.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise RGBBandsMissingError()

    image = np.take(ops.convert_to_numpy(sample["image"]), rgb_indices, axis=-1)
    image = np.clip(image / 3000, 0, 1)

    label = int(ops.convert_to_numpy(sample["label"]))
    label_class = self.classes[label]

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = int(ops.convert_to_numpy(sample["prediction"]))
        prediction_class = self.classes[prediction]

    fig, ax = plt.subplots(figsize=(4, 4))
    ax.imshow(image)
    ax.axis("off")
    if show_titles:
        title = f"Label: {label_class}"
        if showing_predictions:
            title += f"\nPrediction: {prediction_class}"
        ax.set_title(title)

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.eurosat.EuroSAT100

EuroSAT100(root='data', split='train', bands=BAND_SETS['all'], transforms=None, download=False, checksum=True)

Bases: EuroSAT

Subset of EuroSAT containing only 100 images.

Intended for tutorials and demonstrations, not for benchmarking. Maintains the same file structure, classes, and train-val-test split. Each class has 10 images (6 train, 2 val, 2 test), for a total of 100 images.

Source code in keras_climate/datasets/eurosat.py

def __init__(
    self,
    root="data",
    split="train",
    bands=BAND_SETS["all"],
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new EuroSAT dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val", or "test"
        bands: a sequence of band names to load
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.root = root
    self.split = split
    self.tg_transforms = transforms
    self.download = download
    self.checksum = checksum

    assert self.split in {"train", "val", "test"}

    self._validate_bands(bands)
    self.bands = bands
    self.band_indices = np.array(
        [self.all_band_names.index(b) for b in bands if b in self.all_band_names],
        dtype="int64",
    )

    self._verify()

    valid_fns = set()
    with open(os.path.join(self.root, self.split_filenames[split])) as f:
        for fn in f:
            valid_fns.add(fn.strip().replace(".jpg", ".tif"))

    def is_in_split(x):
        return os.path.basename(x) in valid_fns

    super().__init__(
        root=os.path.join(root, self.base_dir),
        transforms=transforms,
        loader=rasterio_loader,
        is_valid_file=is_in_split,
    )

keras_climate.datasets.eurosat.EuroSATSpatial

EuroSATSpatial(root='data', split='train', bands=BAND_SETS['all'], transforms=None, download=False, checksum=True)

Bases: EuroSAT

Overrides the default EuroSAT dataset splits.

Splits the data into training, validation, and test sets based on longitude. The splits are distributed as 60%, 20%, and 20% respectively.

Source code in keras_climate/datasets/eurosat.py

def __init__(
    self,
    root="data",
    split="train",
    bands=BAND_SETS["all"],
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new EuroSAT dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val", or "test"
        bands: a sequence of band names to load
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.root = root
    self.split = split
    self.tg_transforms = transforms
    self.download = download
    self.checksum = checksum

    assert self.split in {"train", "val", "test"}

    self._validate_bands(bands)
    self.bands = bands
    self.band_indices = np.array(
        [self.all_band_names.index(b) for b in bands if b in self.all_band_names],
        dtype="int64",
    )

    self._verify()

    valid_fns = set()
    with open(os.path.join(self.root, self.split_filenames[split])) as f:
        for fn in f:
            valid_fns.add(fn.strip().replace(".jpg", ".tif"))

    def is_in_split(x):
        return os.path.basename(x) in valid_fns

    super().__init__(
        root=os.path.join(root, self.base_dir),
        transforms=transforms,
        loader=rasterio_loader,
        is_valid_file=is_in_split,
    )

keras_climate.datasets.everwatch.EverWatch

EverWatch(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

EverWatch Bird Detection dataset.

The EverWatch Bird Detection <https://zenodo.org/records/11165946>__ dataset contains high-resolution aerial images of birds in the Everglades National Park. Seven bird species have been annotated and classified.

Dataset format:

• images are three-channel pngs
• annotations are csv files

Dataset classes:

1. White Ibis (Eudocimus albus)
2. Great Egret (Ardea alba)
3. Great Blue Heron (Ardea herodias)
4. Snowy Egret (Egretta thula)
5. Wood Stork (Mycteria americana)
6. Roseate Spoonbill (Platalea ajaja)
7. Anhinga (Anhinga anhinga)
8. Unknown White (only present in test split)

If you use this dataset in your research, please cite:

• https://doi.org/10.5281/zenodo.11165946

Initialize a new EverWatch dataset instance.

Args: root: root directory where dataset can be found split: one of {"train", "test"} to specify the dataset split transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False. AssertionError: If split argument is invalid.

Source code in keras_climate/datasets/everwatch.py

def __init__(
    self,
    root="data",
    split="train",
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new EverWatch dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of {"train", "test"} to specify the dataset split
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
        AssertionError: If *split* argument is invalid.
    """
    assert split in self.valid_splits, (
        f"Split '{split}' not supported, please use one of {self.valid_splits}"
    )

    self.root = root
    self.split = split
    self.transforms = transforms
    self.checksum = checksum
    self.download = download

    self._verify()

    self.annot_df = pd.read_csv(os.path.join(self.root, self.dir, f"{self.split}.csv"))

    # remove all entries where xmin == xmax or ymin == ymax
    self.annot_df = self.annot_df[
        (self.annot_df["xmin"] != self.annot_df["xmax"])
        & (self.annot_df["ymin"] != self.annot_df["ymax"])
    ].reset_index(drop=True)

    # group per image path to get all annotations for one sample
    self.annot_df["sample_index"] = pd.factorize(self.annot_df["image_path"])[0]
    self.annot_df = self.annot_df.set_index(["sample_index", self.annot_df.index])

    self.class2idx = {c: i for i, c in enumerate(self.classes)}

__len__

__len__()

Return the number of samples in the dataset.

Source code in keras_climate/datasets/everwatch.py

def __len__(self):
    """Return the number of samples in the dataset."""
    index = self.annot_df.index
    return len(index.levels[0])

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/everwatch.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    sample_df = self.annot_df.loc[index]

    img_path = os.path.join(self.root, self.dir, sample_df["image_path"].iloc[0])

    image = self._load_image(img_path)

    boxes, labels = self._load_target(sample_df)

    sample = {"image": image, "bbox_xyxy": boxes, "label": labels}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None, box_alpha=0.7)

Plot a sample from the dataset.

Source code in keras_climate/datasets/everwatch.py

def plot(self, sample, show_titles=True, suptitle=None, box_alpha=0.7):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt
    from matplotlib import patches

    image = ops.convert_to_numpy(sample["image"])
    boxes = ops.convert_to_numpy(sample["bbox_xyxy"])
    labels = ops.convert_to_numpy(sample["label"])

    fig, axs = plt.subplots(ncols=1, figsize=(10, 10))

    axs.imshow(image)
    axs.axis("off")

    cm = plt.get_cmap("gist_rainbow")

    for box, label_idx in zip(boxes, labels):
        color = cm(label_idx / len(self.classes))
        label = self.classes[label_idx]

        x1, y1, x2, y2 = box
        rect = patches.Rectangle(
            (x1, y1),
            x2 - x1,
            y2 - y1,
            linewidth=2,
            alpha=box_alpha,
            linestyle="solid",
            edgecolor=color,
            facecolor="none",
        )
        axs.add_patch(rect)
        if show_titles:
            axs.text(
                x1,
                y1 - 5,
                label,
                color="white",
                fontsize=8,
                bbox={"facecolor": color, "alpha": box_alpha},
            )

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.fair1m.FAIR1M

FAIR1M(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

FAIR1M dataset.

The FAIR1M <https://www.gaofen-challenge.com/benchmark>__ dataset is a dataset for remote sensing fine-grained oriented object detection.

Dataset features:

• 15,000+ images with 0.3-0.8 m per pixel resolution (1,000-10,000 px)
• 1 million object instances
• 5 object categories, 37 object sub-categories
• three spectral bands - RGB
• images taken by Gaofen satellites and Google Earth

Dataset format:

• images are three-channel tiffs
• labels are xml files with PASCAL VOC like annotations

If you use this dataset in your research, please cite:

• https://doi.org/10.1016/j.isprsjprs.2021.12.004

Initialize a new FAIR1M dataset instance.

Args: root: root directory where dataset can be found split: one of "train", "val", or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: if split argument is invalid DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/fair1m.py

def __init__(
    self,
    root="data",
    split="train",
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new FAIR1M dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val", or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found.
    """
    assert split in self.directories
    self.root = root
    self.split = split
    self.transforms = transforms
    self.download = download
    self.checksum = checksum
    self._verify()
    self.files = sorted(glob.glob(os.path.join(self.root, self.filename_glob[split])))

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/fair1m.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    path = self.files[index]

    image = self._load_image(path)
    sample = {"image": image}

    if self.split != "test":
        label_path = str(path).replace(self.image_root, self.label_root)
        label_path = label_path.replace(".tif", ".xml")
        voc = parse_pascal_voc(label_path)
        boxes, labels = self._load_target(voc["points"], voc["labels"])
        sample = {"image": image, "bbox_xyxy": boxes, "label": labels}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/fair1m.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.files)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/fair1m.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt
    from matplotlib import patches

    image = ops.convert_to_numpy(sample["image"])

    ncols = 1
    if "prediction_bbox_xyxy" in sample:
        ncols += 1

    fig, axs = plt.subplots(ncols=ncols, figsize=(ncols * 10, 10))
    if ncols < 2:
        axs = [axs]

    axs[0].imshow(image)
    axs[0].axis("off")

    if "bbox_xyxy" in sample:
        polygons = [
            patches.Polygon(points, color="r", fill=False)
            for points in ops.convert_to_numpy(sample["bbox_xyxy"])
        ]
        for polygon in polygons:
            axs[0].add_patch(polygon)

    if show_titles:
        axs[0].set_title("Ground Truth")

    if ncols > 1:
        axs[1].imshow(image)
        axs[1].axis("off")
        polygons = [
            patches.Polygon(points, color="r", fill=False)
            for points in ops.convert_to_numpy(sample["prediction_bbox_xyxy"])
        ]
        for polygon in polygons:
            axs[0].add_patch(polygon)

        if show_titles:
            axs[1].set_title("Predictions")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.fire_risk.FireRisk

FireRisk(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: NonGeoClassificationDataset

FireRisk dataset.

The FireRisk <https://github.com/CharmonyShen/FireRisk>__ dataset is a dataset for remote sensing fire risk classification.

Dataset features:

• 91,872 images with 1 m per pixel resolution (320x320 px)
• 70,331 and 21,541 train and val images, respectively
• three spectral bands - RGB
• 7 fire risk classes
• images extracted from NAIP tiles

Dataset format:

• images are three-channel pngs

Dataset classes:

1. high
2. low
3. moderate
4. non-burnable
5. very_high
6. very_low
7. water

If you use this dataset in your research, please cite:

• https://arxiv.org/abs/2303.07035

Initialize a new FireRisk dataset instance.

Args: root: root directory where dataset can be found split: one of "train" or "val" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: AssertionError: if split argument is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/fire_risk.py

def __init__(
    self,
    root="data",
    split="train",
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new FireRisk dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "val"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in self.splits
    self.root = root
    self.split = split
    self.download = download
    self.checksum = checksum
    self._verify()

    super().__init__(root=os.path.join(root, self.directory, self.split), transforms=transforms)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/fire_risk.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"])
    label = int(ops.convert_to_numpy(sample["label"]))
    label_class = self.classes[label]

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = int(ops.convert_to_numpy(sample["prediction"]))
        prediction_class = self.classes[prediction]

    fig, ax = plt.subplots(figsize=(4, 4))
    ax.imshow(image)
    ax.axis("off")
    if show_titles:
        title = f"Label: {label_class}"
        if showing_predictions:
            title += f"\nPrediction: {prediction_class}"
        ax.set_title(title)

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.flair.FLAIRHUB

FLAIRHUB(root='data', split='train', split_column='split_1', transforms=None, download=False, checksum=True, bands=None, dataset_type='land_cover')

Bases: FLAIRHUBBase

FLAIR-HUB Dataset.

Initialize a new FLAIRHUB dataset instance.

Source code in keras_climate/datasets/flair.py

def __init__(
    self,
    root="data",
    split="train",
    split_column="split_1",
    transforms=None,
    download=False,
    checksum=True,
    bands=None,
    dataset_type="land_cover",
):
    """Initialize a new FLAIRHUB dataset instance."""
    super().__init__(
        root=root,
        split=split,
        split_column=split_column,
        transforms=transforms,
        download=download,
        checksum=checksum,
        bands=bands,
        dataset_type=dataset_type,
    )

keras_climate.datasets.flair.FLAIRHUBBase

FLAIRHUBBase(root='data', split='train', split_column='split_1', transforms=None, download=False, checksum=True, bands=None, dataset_type='land_cover')

Bases: NonGeoDataset

Shared base for FLAIRHUB and FLAIRHUBToy.

Large-scale Multimodal Dataset for Land Cover and Crop Mapping dataset.

FLAIR-HUB <https://github.com/IGNF/FLAIR-HUB>__ builds upon and includes the FLAIR#1 and FLAIR#2 datasets, expanding them into a unified, large-scale, multi-sensor land-cover resource with very-high-resolution annotations. Spanning over 2,500 km^2 of diverse French ecoclimatic regions and landscapes, FLAIR-HUB features 63 billion hand-annotated pixels across 19 land-cover and 23 crop type classes.

The dataset integrates complementary sources including aerial imagery, SPOT and Sentinel satellite acquisitions, surface models, and historical aerial photographs.

Dataset structure:

The dataset is organized by domains (geographical areas) and years. Each domain-year combination has multiple modalities available for download.

Available modalities (100% coverage across all domains):

• AERIAL_RGBI: High-resolution aerial imagery (RGB + NIR, 0.2m) -- key: image_aerial_rgbi
• SPOT_RGBI: SPOT satellite imagery (RGB + NIR, 1.5m) -- key: image_spot_rgbi
• DEM_ELEV: Digital Elevation Model (DSM + DTM, 1m) -- key: image_dem_elev
• AERIAL-RLT_PAN: Historical aerial panchromatic (1950s) -- key: image_aerial_rlt_pan
• SENTINEL1-ASC_TS: Sentinel-1 SAR Ascending time series (VV + VH) -- key: image_sentinel1_asc
• SENTINEL1-DESC_TS: Sentinel-1 SAR Descending time series (VV + VH) -- key: image_sentinel1_desc
• SENTINEL2_TS: Sentinel-2 multispectral time series (12 bands, 10m) -- key: image_sentinel2
• SENTINEL2_MSK-SC: Sentinel-2 scene classification mask -- key: mask_sentinel2_snowcloud
• AERIAL_LABEL-COSIA: Land cover labels (19 classes)
• ALL_LABEL-LPIS: Crop type labels (23 classes)

If you use this dataset in your research, please cite:

• https://arxiv.org/abs/2506.07080

Initialize a new FLAIRHUB dataset instance.

The FLAIR-HUB dataset provides multiple complementary data modalities for land cover and crop type mapping. You can selectively load any combination of the available modalities by specifying them in the bands parameter.

Only samples belonging to the specified split (train, val, or test) are loaded, using the official splits from GLOBAL_ALL_MTD_SPLIT.gpkg.

Args: root: Root directory where dataset can be found or will be downloaded. split: One of "train", "val", or "test". split_column: Column name in the official splits GeoPackage (e.g. "split_1", "split_5", "split_flairchallenge"). transforms: Optional transforms to apply to samples. download: If True, download the dataset if it is not found. checksum: If True, check the SHA-256 of downloaded files (may be slow). bands: List of bands/modalities to load. Defaults to None, which enables all bands. See the modality key map above for available options. dataset_type: Type of labels to use. Choose "land_cover" for 19-class COSIA labels or "crop_type" for 23-class LPIS crop classification labels (baseline) or "crop_type_2" for 31-class LPIS crop classification labels or "crop_type_3" for 46-class LPIS crop classification labels.

Raises: AssertionError: If split or split_column is invalid. DatasetNotFoundError: If dataset is not found and download is False. ValueError: If an invalid band name is provided.

Source code in keras_climate/datasets/flair.py

def __init__(
    self,
    root="data",
    split="train",
    split_column="split_1",
    transforms=None,
    download=False,
    checksum=True,
    bands=None,
    dataset_type="land_cover",
):
    """Initialize a new FLAIRHUB dataset instance.

    The FLAIR-HUB dataset provides multiple complementary data modalities
    for land cover and crop type mapping. You can selectively load any
    combination of the available modalities by specifying them in the
    *bands* parameter.

    Only samples belonging to the specified split (train, val, or test)
    are loaded, using the official splits from
    ``GLOBAL_ALL_MTD_SPLIT.gpkg``.

    Args:
        root: Root directory where dataset can be found or will be
            downloaded.
        split: One of "train", "val", or "test".
        split_column: Column name in the official splits GeoPackage
            (e.g. "split_1", "split_5", "split_flairchallenge").
        transforms: Optional transforms to apply to samples.
        download: If True, download the dataset if it is not found.
        checksum: If True, check the SHA-256 of downloaded files (may be
            slow).
        bands: List of bands/modalities to load. Defaults to None, which
            enables all bands. See the modality key map above for
            available options.
        dataset_type: Type of labels to use. Choose "land_cover" for
            19-class COSIA labels or "crop_type" for 23-class LPIS crop
            classification labels (baseline) or "crop_type_2" for
            31-class LPIS crop classification labels or "crop_type_3"
            for 46-class LPIS crop classification labels.

    Raises:
        AssertionError: If ``split`` or ``split_column`` is invalid.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
        ValueError: If an invalid band name is provided.
    """
    self.root = root
    self.split = split
    self.split_column = split_column
    self.transforms = transforms
    self.download = download
    self.checksum = checksum
    self.dataset_type = dataset_type

    if bands is None:
        bands = list(self.modality_key_map.keys())

    for band in bands:
        if band not in self.modality_key_map:
            raise ValueError(f"Invalid band name: {band}")

    self.bands = bands
    self._verify()
    self.files = self._load_files()

__getitem__

__getitem__(index)

Return an index within the dataset.

Returns a dictionary containing tensors for each modality. Keys are the modality names prefixed with image_ or mask_: mask, image_aerial_rgbi, image_spot_rgbi, image_dem_elev, image_aerial_rlt_pan, image_sentinel2, mask_sentinel2_snowcloud, image_sentinel1_asc, image_sentinel1_desc.

Source code in keras_climate/datasets/flair.py

def __getitem__(self, index):
    """Return an index within the dataset.

    Returns a dictionary containing tensors for each modality. Keys are
    the modality names prefixed with ``image_`` or ``mask_``: ``mask``,
    ``image_aerial_rgbi``, ``image_spot_rgbi``, ``image_dem_elev``,
    ``image_aerial_rlt_pan``, ``image_sentinel2``,
    ``mask_sentinel2_snowcloud``, ``image_sentinel1_asc``,
    ``image_sentinel1_desc``.
    """
    file_dict = self.files[index]
    sample = {}

    # Load mask (always present)
    mask_path = file_dict["mask"]
    sample["mask"] = self._load_mask(mask_path)

    # Load requested modalities
    for modality_name in self.bands:
        sample_key = self.modality_key_map[modality_name]
        modality_path = file_dict[modality_name]
        if "TS" in modality_name or "SC" in modality_name:
            if modality_name == "SENTINEL2_TS":
                band_names = self.sentinel2_ts_bands
            elif modality_name == "SENTINEL2_MSK-SC":
                band_names = self.sentinel2_msk_sc_bands
            elif modality_name == "SENTINEL1-ASC_TS":
                band_names = self.sentinel1_asc_ts_bands
            elif modality_name == "SENTINEL1-DESC_TS":
                band_names = self.sentinel1_desc_ts_bands

            sample[sample_key] = self._load_time_series(modality_path, len(band_names))
        else:
            sample[sample_key] = self._load_image(modality_path)

    if self.transforms is not None:
        sample = self.transforms(sample)
    return sample

__len__

__len__()

Return the number of datapoints in the dataset.

Source code in keras_climate/datasets/flair.py

def __len__(self):
    """Return the number of datapoints in the dataset."""
    return len(self.files)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/flair.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    plot_data = {}

    plot_data["mask"] = {"plot_type": "mask", "data": sample["mask"], "title": "mask"}

    for key, (plot_type, title) in _PLOT_KEYS.items():
        if key in sample:
            plot_data[key] = {"plot_type": plot_type, "data": sample[key], "title": title}

    num_plots = len(plot_data)
    ncols = min(4, num_plots)
    nrows = (num_plots + ncols - 1) // ncols

    fig, axs = plt.subplots(nrows, ncols, figsize=(ncols * 4, nrows * 4))
    axs = np.atleast_1d(axs).flatten()
    for ax in axs.flat:
        ax.axis("off")

    for idx, (_, plot_info) in enumerate(plot_data.items()):
        plot_type = plot_info["plot_type"]
        data = plot_info["data"]
        title = plot_info["title"]

        if plot_type == "mask":
            self._plot_mask(data, axs[idx], show_legend=True)
        elif plot_type == "aerial_rgbi":
            self._plot_aerial_rgbi(data, axs[idx], title, show_titles)
        elif plot_type == "dem":
            self._plot_dem(data, axs[idx], title, show_titles)
        elif plot_type == "spot_rgbi":
            self._plot_spot_rgbi(data, axs[idx], title, show_titles)
        elif plot_type == "aerial_rlt_pan":
            self._plot_aerial_rlt_pan(data, axs[idx], title, show_titles)
        elif plot_type == "sentinel2_ts":
            self._plot_sentinel2_ts(data, axs[idx], title, show_titles)
        elif plot_type == "sentinel2_msk_sc":
            self._plot_sentinel2_msk_sc(data, axs[idx], title, show_titles)
        elif plot_type in ("sentinel1_asc_ts", "sentinel1_desc_ts"):
            self._plot_sentinel1_ts(data, axs[idx], title, show_titles)

    if suptitle:
        fig.suptitle(suptitle, fontsize=16)

    plt.tight_layout()
    return fig

keras_climate.datasets.flair.FLAIRHUBToy

FLAIRHUBToy(root='data', split='train', split_column='split_toy', transforms=None, download=False, checksum=True, bands=None, dataset_type='land_cover')

Bases: FLAIRHUBBase

Toy version of the FLAIRHUB dataset.

For further information refer to :class:FLAIRHUB.

Initialize a new FLAIRHUBToy dataset instance.

This is a toy/sample version of the FLAIR-HUB dataset intended for testing and development purposes. It contains a small subset of the full dataset with all available data modalities.

See :class:FLAIRHUB for a description of the arguments.

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/flair.py

def __init__(
    self,
    root="data",
    split="train",
    split_column="split_toy",
    transforms=None,
    download=False,
    checksum=True,
    bands=None,
    dataset_type="land_cover",
):
    """Initialize a new FLAIRHUBToy dataset instance.

    This is a toy/sample version of the FLAIR-HUB dataset intended for
    testing and development purposes. It contains a small subset of the
    full dataset with all available data modalities.

    See :class:`FLAIRHUB` for a description of the arguments.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.root_folder = pathlib.Path(root)
    modified_root = self.root_folder / "FLAIR-HUB_TOY"
    super().__init__(
        root=modified_root,
        split=split,
        split_column=split_column,
        transforms=transforms,
        download=download,
        checksum=checksum,
        bands=bands,
        dataset_type=dataset_type,
    )

keras_climate.datasets.forestdamage.ForestDamage

ForestDamage(root='data', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

Forest Damage dataset.

The ForestDamage <https://lila.science/datasets/forest-damages-larch-casebearer/>_ dataset contains drone imagery that can be used for tree identification, as well as tree damage classification for larch trees.

Dataset features:

• 1543 images
• 101,878 tree annotations
• subset of 840 images contain 44,522 annotations about tree health (Healthy (H), Light Damage (LD), High Damage (HD)), all other images have "other" as damage level

Dataset format:

• images are three-channel jpgs
• annotations are in Pascal VOC XML format

Dataset classes:

1. other
2. healthy
3. light damage
4. high damage

If the download fails or stalls, it is recommended to try azcopy as suggested here <https://lila.science/faq>__. It is expected that the downloaded data file with name Data_Set_Larch_Casebearer can be found in root.

If you use this dataset in your research, please cite:

• Swedish Forest Agency (2021): Forest Damages - Larch Casebearer 1.0. National Forest Data Lab. Dataset.

Initialize a new ForestDamage dataset instance.

Args: root: root directory where dataset can be found transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/forestdamage.py

def __init__(self, root="data", transforms=None, download=False, checksum=True):
    """Initialize a new ForestDamage dataset instance.

    Args:
        root: root directory where dataset can be found
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.root = root
    self.transforms = transforms
    self.checksum = checksum
    self.download = download

    self._verify()

    self.files = self._load_files(self.root)

    self.class_to_idx = {c: i for i, c in enumerate(self.classes)}

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/forestdamage.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    files = self.files[index]
    parsed = parse_pascal_voc(files["annotation"])
    image = self._load_image(files["image"])

    boxes, labels = self._load_target(parsed["bboxes"], parsed["labels"])

    sample = {"image": image, "bbox_xyxy": boxes, "label": labels}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/forestdamage.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.files)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/forestdamage.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt
    from matplotlib import patches

    image = ops.convert_to_numpy(sample["image"])

    ncols = 1
    showing_predictions = "prediction_bbox_xyxy" in sample
    if showing_predictions:
        ncols += 1

    fig, axs = plt.subplots(ncols=ncols, figsize=(ncols * 10, 10))
    if not showing_predictions:
        axs = [axs]

    axs[0].imshow(image)
    axs[0].axis("off")

    bboxes = [
        patches.Rectangle(
            (bbox[0], bbox[1]),
            bbox[2] - bbox[0],
            bbox[3] - bbox[1],
            linewidth=1,
            edgecolor="r",
            facecolor="none",
        )
        for bbox in ops.convert_to_numpy(sample["bbox_xyxy"])
    ]
    for bbox in bboxes:
        axs[0].add_patch(bbox)

    if show_titles:
        axs[0].set_title("Ground Truth")

    if showing_predictions:
        axs[1].imshow(image)
        axs[1].axis("off")

        pred_bboxes = [
            patches.Rectangle(
                (bbox[0], bbox[1]),
                bbox[2] - bbox[0],
                bbox[3] - bbox[1],
                linewidth=1,
                edgecolor="r",
                facecolor="none",
            )
            for bbox in ops.convert_to_numpy(sample["prediction_bbox_xyxy"])
        ]
        for bbox in pred_bboxes:
            axs[1].add_patch(bbox)

        if show_titles:
            axs[1].set_title("Predictions")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.ftw.FieldsOfTheWorld

FieldsOfTheWorld(root='data', split='train', target='2-class', countries=['austria'], transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

Fields Of The World dataset.

The Fields Of The World <https://source.coop/kerner-lab/fields-of-the-world>__ dataset is a semantic and instance segmentation dataset for delineating field boundaries.

Dataset features:

• 70462 patches across 24 countries
• Each country has a train, val, and test split
• Semantic segmentations masks with and without the field boundary class
• Instance segmentation masks

Dataset format:

• images are four-channel GeoTIFFs with dimension 256x256
• segmentation masks (both two and three class) are single-channel GeoTIFFs
• instance masks are single-channel GeoTIFFs

Dataset classes:

1. background
2. field
3. field-boundary (three-class only)
4. unlabeled (kenya, rwanda, brazil and india have presence only labels)

If you use this dataset in your research, please cite:

• https://doi.org/10.48550/arXiv.2409.16252

Initialize a new Fields Of The World dataset instance.

Args: root: root directory where dataset can be found split: one of "train", "val", or "test" target: one of "2-class", "3-class", or "instance" specifying which kind of target mask to load countries: which set of countries to load data from transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: If any arguments are invalid. DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/ftw.py

def __init__(
    self,
    root="data",
    split="train",
    target="2-class",
    countries=["austria"],
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new Fields Of The World dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val", or "test"
        target: one of "2-class", "3-class", or "instance" specifying
            which kind of target mask to load
        countries: which set of countries to load data from
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: If any arguments are invalid.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in self.splits
    assert target in self.targets
    if isinstance(countries, str):
        countries = [countries]
    assert set(countries) <= set(self.valid_countries)

    self.root = root
    self.split = split
    self.target = target
    self.countries = countries
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    self.files = self._load_files()

__getitem__

__getitem__(index)

Return an index within the dataset.

Returns an image of shape (H, W, 8) (window B bands followed by window A bands) and a mask of shape (H, W).

Source code in keras_climate/datasets/ftw.py

def __getitem__(self, index):
    """Return an index within the dataset.

    Returns an image of shape (H, W, 8) (window B bands followed by
    window A bands) and a mask of shape (H, W).
    """
    win_a_fn = self.files[index]["win_a"]
    win_b_fn = self.files[index]["win_b"]
    mask_fn = self.files[index]["mask"]

    win_a = self._load_image(win_a_fn)
    win_b = self._load_image(win_b_fn)
    mask = self._load_target(mask_fn)

    image = ops.concatenate((win_b, win_a), axis=-1)
    sample = {"image": image, "mask": mask}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of datapoints in the dataset.

Source code in keras_climate/datasets/ftw.py

def __len__(self):
    """Return the number of datapoints in the dataset."""
    return len(self.files)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/ftw.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"])
    # Note: as in the original dataset, the channel ranges below index
    # into the concatenated (window B, window A) image; the "Window A"
    # panel therefore shows the window B bands and vice versa.
    win_a = image[..., 0:3]
    win_b = image[..., 4:7]
    mask = ops.convert_to_numpy(sample["mask"])

    win_a = np.clip(win_a / 3000, 0, 1)
    win_b = np.clip(win_b / 3000, 0, 1)

    ncols = 3
    showing_predictions = "prediction" in sample
    if showing_predictions:
        ncols += 1

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(15, 5))

    axs[0].imshow(win_a)
    axs[0].set_title("Window A")
    axs[1].imshow(win_b)
    axs[1].set_title("Window B")
    if self.target == "instance":
        unique_vals = sorted(np.unique(mask))
        for i, val in enumerate(unique_vals):
            mask[mask == val] = i
        bg_mask = mask == 0
        mask = (mask % 9) + 1
        mask[bg_mask] = 0
        axs[2].imshow(mask, vmin=0, vmax=10, cmap="tab10", interpolation="none")
        axs[2].set_title("Instance mask")
    elif self.target == "2-class":
        axs[2].imshow(mask, vmin=0, vmax=2, cmap="gray", interpolation="none")
        axs[2].set_title("2-class mask")
    elif self.target == "3-class":
        axs[2].imshow(mask, vmin=0, vmax=2, cmap="gray", interpolation="none")
        axs[2].set_title("3-class mask")

    if showing_predictions:
        axs[3].imshow(ops.convert_to_numpy(sample["prediction"]))
        axs[3].set_title("Predictions")

    for ax in axs:
        ax.axis("off")

    if not show_titles:
        for ax in axs:
            ax.set_title("")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.gbif.GBIF

GBIF(root='data')

Bases: GeoDataset

Dataset for the Global Biodiversity Information Facility.

GBIF <https://www.gbif.org/>__, the Global Biodiversity Information Facility, is an international network and data infrastructure funded by the world's governments and aimed at providing anyone, anywhere, open access to data about all types of life on Earth.

This dataset is intended for use with GBIF's occurrence records <https://www.gbif.org/occurrence/search>. It may or may not work for other GBIF datasets <https://www.gbif.org/dataset/search>. Data for a particular species or region of interest can be downloaded from the above link.

If you use a GBIF dataset in your research, please cite it according to:

• https://www.gbif.org/citation-guidelines

Initialize a new GBIF dataset instance.

Args: root: root directory where dataset can be found

Raises: DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/gbif.py

def __init__(self, root="data"):
    """Initialize a new GBIF dataset instance.

    Args:
        root: root directory where dataset can be found

    Raises:
        DatasetNotFoundError: If dataset is not found.
    """
    self.root = root

    files = glob.glob(os.path.join(root, "**.csv"))
    if not files:
        raise DatasetNotFoundError(self)

    # Read tab-delimited CSV file
    usecols = ["decimalLatitude", "decimalLongitude", "day", "month", "year"]
    dtype = {"day": str, "month": str, "year": str}
    df = pd.read_table(files[0], usecols=usecols, dtype=dtype)
    df = df[df["decimalLatitude"].notna()]
    df = df[df["decimalLongitude"].notna()]
    df["day"] = df["day"].str.zfill(2)
    df["month"] = df["month"].str.zfill(2)
    date = df["day"] + " " + df["month"] + " " + df["year"]

    # Convert from pandas DataFrame to geopandas GeoDataFrame
    func = functools.partial(disambiguate_timestamp, format="%d %m %Y")
    index = pd.IntervalIndex.from_tuples(
        date.apply(func).to_list(), closed="both", name="datetime"
    )
    geometry = gpd.points_from_xy(df["decimalLongitude"], df["decimalLatitude"])
    self.index = GeoDataFrame(index=index, geometry=geometry, crs="EPSG:4326")

__getitem__

__getitem__(index)

Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

Raises: IndexError: If index is not found in the dataset.

Source code in keras_climate/datasets/gbif.py

def __getitem__(self, index):
    """Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

    Raises:
        IndexError: If *index* is not found in the dataset.
    """
    x, y, t = self._disambiguate_slice(index)
    interval = pd.Interval(t.start, t.stop)
    df = self.index.iloc[self.index.index.overlaps(interval)]
    df = df.iloc[:: t.step]
    df = df.cx[x.start : x.stop, y.start : y.stop]

    if df.empty:
        raise IndexError(f"index: {index} not found in dataset with bounds: {self.bounds}")

    keypoints = ops.convert_to_tensor(df.get_coordinates().values.astype("float32"))
    transform = rasterio.transform.from_origin(x.start, y.stop, x.step, y.step)
    sample = {
        "bounds": self._slice_to_tensor(index),
        "keypoints": keypoints,
        "transform": ops.convert_to_tensor(np.array(list(transform))),
    }

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/gbif.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    fig, ax = plt.subplots(figsize=(10, 8))
    ax.grid(ls="--")

    keypoints = ops.convert_to_numpy(sample["keypoints"])
    x = keypoints[:, 0]
    y = keypoints[:, 1]

    ax.scatter(x, y)

    ax.set_xlabel("Longitude")
    ax.set_ylabel("Latitude")

    if show_titles:
        ax.set_title("GBIF Occurrence Locations by Date")

    if suptitle is not None:
        fig.suptitle(suptitle)

    fig.tight_layout()
    return fig

keras_climate.datasets.gbm.GlobalBuildingMap

GlobalBuildingMap(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset

GlobalBuildingMap dataset.

The GlobalBuildingMap (GBM) dataset provides the highest resolution and highest accuracy building footprint map on a global scale ever created. GBM was generated by training and applying modern deep neural networks on nearly 800,000 satellite images. The dataset is stored in 5 by 5 degree tiles in geotiff format.

The GlobalBuildingMap is generated by applying an ensemble of deep neural networks on nearly 800,000 satellite images of about 3m resolution. The deep neural networks were trained with manually inspected training samples generated from OpenStreetMap.

If you use this dataset in your research, please cite:

• https://arxiv.org/abs/2404.13911

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new RasterDataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a mosaic
            (``[H, W, C]``). For mask-style datasets (``is_image=False``),
            single-band data has the channel dimension squeezed.
    """
    self.paths = paths
    self.bands = bands or self.all_bands
    self.transforms = transforms
    self.cache = cache
    self.time_series = time_series

    if self.all_bands:
        assert set(self.bands) <= set(self.all_bands)

    filename_regex = re.compile(self.filename_regex, re.VERBOSE)
    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        match = re.match(filename_regex, os.path.basename(filepath))
        if match is not None:
            vrt = None
            try:
                vrt = self._load_warp_file(filepath=filepath, crs=crs)
                if self.cmap is None:
                    try:
                        colors = np.array(list(vrt.colormap(1).values())) / 255
                        self.cmap = ListedColormap(colors)
                    except ValueError:
                        pass
                if crs is None:
                    with rasterio.Env(OSR_WKT_FORMAT="WKT2_2018"):
                        crs = PROJ_CRS.from_user_input(vrt.crs)
                footprint = self.footprint_from_datasource(vrt)
                if footprint is None:
                    footprint = shapely.box(*vrt.bounds)
                geometries.append(footprint)
                if res is None:
                    res = vrt.res
            except rasterio.errors.RasterioIOError:
                continue
            else:
                filepaths.append(filepath)
                mint, maxt = self._filepath_to_timestamp(filepath)
                datetimes.append((mint, maxt))
            finally:
                if vrt is not None:
                    vrt.close()

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if not self.separate_files:
        self.band_indexes = None
        if self.bands:
            if self.all_bands:
                self.band_indexes = [
                    self.all_bands.index(i) + 1 for i in self.bands
                ]
            else:
                msg = (
                    f"{self.__class__.__name__} is missing an `all_bands` "
                    "attribute, so `bands` cannot be specified."
                )
                raise AssertionError(msg)

    if res is not None:
        if isinstance(res, (int, float)):
            res = (res, res)

        self._res = res

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/gbm.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    ncols = 2 if "prediction" in sample else 1
    fig, axs = plt.subplots(ncols=ncols, squeeze=False)

    axs[0, 0].imshow(sample["mask"], cmap="gray")
    axs[0, 0].axis("off")
    if show_titles:
        axs[0, 0].set_title("Mask")

    if "prediction" in sample:
        axs[0, 1].imshow(sample["prediction"], cmap="gray")
        axs[0, 1].axis("off")
        if show_titles:
            axs[0, 1].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.geonrw.GeoNRW

GeoNRW(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

GeoNRW dataset.

This dataset contains RGB, DEM and segmentation label data from North Rhine-Westphalia, Germany.

Dataset features:

• 7298 training and 485 test samples
• RGB images, 1000x1000px normalized to [0, 1]
• DEM images, unnormalized
• segmentation labels

Dataset format:

• RGB images are three-channel jp2
• DEM images are single-channel tif
• segmentation labels are single-channel tif

Dataset classes:

1. background
2. forest
3. water
4. agricultural
5. residential,commercial,industrial
6. grassland,swamp,shrubbery
7. railway,trainstation
8. highway,squares
9. airport,shipyard
10. roads
11. buildings

Additional information about the dataset can be found on this site <https://ieee-dataport.org/open-access/geonrw>__.

If you use this dataset in your research, please cite:

• https://ieeexplore.ieee.org/document/9406194

Initialize the GeoNRW dataset.

Args: root: root directory where dataset can be found split: one of "train", or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: AssertionError: if split argument is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/geonrw.py

def __init__(self, root="data", split="train", transforms=None, download=False, checksum=True):
    """Initialize the GeoNRW dataset.

    Args:
        root: root directory where dataset can be found
        split: one of "train", or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in self.splits, f"split must be one of {self.splits}"

    self.root = root
    self.split = split
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self.city_names = self.test_list if split == "test" else self.train_list

    self._verify()

    self.file_list = self._get_file_list()

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/geonrw.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.file_list)

__getitem__

__getitem__(index)

Return an index within the dataset.

Returns an "image" (H, W, 3) normalized to [0, 1], a "dem" (H, W, 1) with unnormalized values, and a "mask" (H, W).

Source code in keras_climate/datasets/geonrw.py

def __getitem__(self, index):
    """Return an index within the dataset.

    Returns an "image" (H, W, 3) normalized to [0, 1], a "dem" (H, W, 1)
    with unnormalized values, and a "mask" (H, W).
    """
    path = self.file_list[index]
    utm_coords = os.path.basename(path).split("_")[:2]
    base_dir = os.path.dirname(path)

    sample = {}
    for modality in self.modalities:
        modality_path = os.path.join(base_dir, self.modality_filenames[modality](utm_coords))
        img = self.readers[modality](modality_path)
        array = np.array(img)

        if modality == "rgb":
            array = array.astype("float32") / 255.0
        elif modality == "dem":
            array = array.astype("float32")
            if array.ndim == 2:
                array = array[..., None]
        elif modality == "seg":
            array = array.astype("int64")

        sample[modality] = ops.convert_to_tensor(array)

    # rename to keras_climate standard keys
    sample["image"] = sample.pop("rgb")
    sample["mask"] = sample.pop("seg")

    if self.transforms:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/geonrw.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    showing_predictions = "prediction" in sample
    ncols = 3
    if showing_predictions:
        prediction = ops.convert_to_numpy(sample["prediction"]).astype("int64")
        ncols += 1

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 5, 10), sharex=True)

    axs[0].imshow(ops.convert_to_numpy(sample["image"]))
    axs[0].axis("off")
    axs[1].imshow(ops.convert_to_numpy(sample["dem"])[..., 0], cmap="gray")
    axs[1].axis("off")
    axs[2].imshow(
        ops.convert_to_numpy(sample["mask"]),
        cmap=self.colormap,
        vmin=0,
        vmax=10,
        interpolation="none",
    )
    axs[2].axis("off")

    if showing_predictions:
        axs[3].imshow(prediction, cmap=self.colormap, vmin=0, vmax=10, interpolation="none")

    # show classes in legend
    if show_titles:
        patches = [matplotlib.patches.Patch(color=c) for c in self.colormap.colors]
        axs[2].legend(patches, self.classes, loc="center left", bbox_to_anchor=(1, 0.5))

    if show_titles:
        axs[0].set_title("RGB Image")
        axs[1].set_title("DEM")
        axs[2].set_title("Labels")

    if suptitle is not None:
        fig.suptitle(suptitle, y=0.8)

    fig.tight_layout()

    return fig

keras_climate.datasets.gid15.GID15

GID15(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

GID-15 dataset.

The GID-15 <https://captain-whu.github.io/GID15/>__ dataset is a dataset for semantic segmentation.

Dataset features:

• images taken by the Gaofen-2 (GF-2) satellite over 60 cities in China
• masks representing 15 semantic categories
• three spectral bands - RGB
• 150 with 3 m per pixel resolution (6800x7200 px)

Dataset format:

• images are three-channel pngs
• masks are single-channel pngs
• colormapped masks are 3 channel tifs

Dataset classes:

1. background
2. industrial_land
3. urban_residential
4. rural_residential
5. traffic_land
6. paddy_field
7. irrigated_land
8. dry_cropland
9. garden_plot
10. arbor_woodland
11. shrub_land
12. natural_grassland
13. artificial_grassland
14. river
15. lake
16. pond

If you use this dataset in your research, please cite:

• https://doi.org/10.1016/j.rse.2019.111322

Initialize a new GID-15 dataset instance.

Args: root: root directory where dataset can be found split: one of "train", "val", or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: if split argument is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/gid15.py

def __init__(
    self,
    root="data",
    split="train",
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new GID-15 dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val", or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in self.splits

    self.root = root
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    if download:
        self._download()

    if not self._check_integrity():
        raise DatasetNotFoundError(self)

    self.files = self._load_files(self.root, self.split)

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/gid15.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    files = self.files[index]
    image = self._load_image(files["image"])

    if self.split != "test":
        mask = self._load_target(files["mask"])
        sample = {"image": image, "mask": mask}
    else:
        sample = {"image": image}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/gid15.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.files)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/gid15.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    if self.split != "test":
        image, mask = sample["image"], sample["mask"]
        ncols = 2
    else:
        image = sample["image"]
        ncols = 1

    if "prediction" in sample:
        ncols += 1
        pred = sample["prediction"]

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 10, 10))

    if self.split != "test":
        axs[0].imshow(ops.convert_to_numpy(image).astype("uint8"))
        axs[0].axis("off")
        axs[1].imshow(ops.convert_to_numpy(mask))
        axs[1].axis("off")
        if show_titles:
            axs[0].set_title("Image")
            axs[1].set_title("Mask")

        if "prediction" in sample:
            axs[2].imshow(ops.convert_to_numpy(pred))
            axs[2].axis("off")
            if show_titles:
                axs[2].set_title("Prediction")
    else:
        if "prediction" in sample:
            axs[0].imshow(ops.convert_to_numpy(image).astype("uint8"))
            axs[0].axis("off")
            axs[1].imshow(ops.convert_to_numpy(pred))
            axs[1].axis("off")
            if show_titles:
                axs[0].set_title("Image")
                axs[1].set_title("Prediction")
        else:
            axs.imshow(ops.convert_to_numpy(image).astype("uint8"))
            axs.axis("off")
            if show_titles:
                axs.set_title("Image")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.globalmangrovewatch.GlobalMangroveWatch

GlobalMangroveWatch(paths='data', crs=None, res=None, years=(2020,), transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: RasterDataset

Global Mangrove Watch (GMW) dataset.

The Global Mangrove Watch <https://www.globalmangrovewatch.org>__ Version 3.0 dataset provides maps of global mangrove extent for eleven epochs between 1996 and 2020, derived from L-band Synthetic Aperture Radar (SAR) and optical (Landsat) satellite data.

Dataset features:

• Binary mangrove extent maps at 25 m resolution (1/4500 degrees)
• 11 epochs: 1996, 2007-2010, 2015-2020
• Global coverage in WGS84 geographic coordinates (EPSG:4326)
• Tiled as 1 x 1 degree GeoTIFF files of 4,500 x 4,500 pixels

Dataset classes:

1. Non-mangrove
2. Mangrove

If you use this dataset in your research, please cite it using the following format:

• https://doi.org/10.5281/zenodo.6894273

Initialize a new Dataset instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS years: years for which to use GMW layers transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling download: if True, download dataset and store it in the root directory checksum: if True, check the SHA256 of downloaded and existing archives time_series: if True, stack data along the time series dimension

Raises: AssertionError: If years are invalid. DatasetNotFoundError: If dataset is not found and download is False. RuntimeError: If an existing archive is corrupted.

Source code in keras_climate/datasets/globalmangrovewatch.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    years=(2020,),
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS
        years: years for which to use GMW layers
        transforms: a function/transform that takes an input sample
            and returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root directory
        checksum: if True, check the SHA256 of downloaded and existing archives
        time_series: if True, stack data along the time series dimension

    Raises:
        AssertionError: If ``years`` are invalid.
        DatasetNotFoundError: If dataset is not found and *download* is False.
        RuntimeError: If an existing archive is corrupted.
    """
    assert set(years) <= set(self.all_years), (
        f"GMW data product only exists for the following years: "
        f"{list(self.all_years)}."
    )

    self.paths = paths
    self.years = tuple(years)
    self.download = download
    self.checksum = checksum

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

files property

files

A list of all files in the dataset, restricted to years.

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/globalmangrovewatch.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    mask = ops.convert_to_numpy(sample["mask"])
    # Masks are [H, W] for a mosaic and [T, H, W] for a time series,
    # both are reshaped to a [T, H, W] stack of frames to plot.
    panels = [("Mask", mask.reshape(-1, *mask.shape[-2:]))]
    if "prediction" in sample:
        pred = ops.convert_to_numpy(sample["prediction"])
        panels.append(("Prediction", pred.reshape(-1, *pred.shape[-2:])))

    nrows = len(panels)
    ncols = panels[0][1].shape[0]

    fig, axs = plt.subplots(
        nrows=nrows, ncols=ncols, figsize=(ncols * 4, nrows * 4), squeeze=False
    )

    kwargs = {"cmap": self.cmap, "vmin": 0, "vmax": 1, "interpolation": "none"}

    for row, (title, frames) in enumerate(panels):
        for col in range(ncols):
            axs[row, col].imshow(frames[col], **kwargs)
            axs[row, col].axis("off")
            if show_titles:
                axs[row, col].set_title(title if ncols == 1 else f"{title} {col}")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.globbiomass.GlobBiomass

GlobBiomass(paths='data', crs=None, res=None, measurement='agb', transforms=None, cache=True, checksum=True, time_series=False)

Bases: RasterDataset

GlobBiomass dataset.

The GlobBiomass <https://doi.pangaea.de/10.1594/PANGAEA.894711>__ dataset consists of global pixelwise aboveground biomass (AGB) and growth stock volume (GSV) maps.

Definitions:

• AGB: the mass, expressed as oven-dry weight of the woody parts (stem, bark, branches and twigs) of all living trees excluding stump and roots.
• GSV: volume of all living trees more than 10 cm in diameter at breast height measured over bark from ground or stump height to a top stem diameter of 0 cm.

Units:

• AGB: m3/ha
• GSV: tons/ha (i.e., Mg/ha)

Dataset features:

• Global estimates of AGB and GSV at ~100 m per pixel resolution (45,000 x 45,000 px)
• Per-pixel uncertainty expressed as standard error

The data can be manually downloaded from this website <https://globbiomass.org/wp-content/uploads/GB_Maps/Globbiomass_global_dataset.html>_.

If you use this dataset in your research, please cite the following dataset:

• https://doi.org/10.1594/PANGAEA.894711

Initialize a new GlobBiomass instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS measurement: use data from 'agb' or 'gsv' measurement transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling checksum: if True, check the MD5 of the downloaded files time_series: if True, stack data along the time series dimension

Raises: AssertionError: If measurement is not valid. DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/globbiomass.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    measurement="agb",
    transforms=None,
    cache=True,
    checksum=True,
    time_series=False,
):
    """Initialize a new GlobBiomass instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS
        measurement: use data from 'agb' or 'gsv' measurement
        transforms: a function/transform that takes an input sample
            and returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        checksum: if True, check the MD5 of the downloaded files
        time_series: if True, stack data along the time series dimension

    Raises:
        AssertionError: If *measurement* is not valid.
        DatasetNotFoundError: If dataset is not found.
    """
    assert measurement in self.measurements

    self.paths = paths
    self.measurement = measurement
    self.checksum = checksum

    self.filename_glob = self.filename_glob.format(measurement)

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

__getitem__

__getitem__(index)

Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

Raises: IndexError: If index is not found in the dataset.

Source code in keras_climate/datasets/globbiomass.py

def __getitem__(self, index):
    """Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

    Raises:
        IndexError: If *index* is not found in the dataset.
    """
    x, y, t = self._disambiguate_slice(index)
    interval = pd.Interval(t.start, t.stop)
    df = self.index.iloc[self.index.index.overlaps(interval)]
    df = df.iloc[:: t.step]
    df = df.cx[x.start : x.stop, y.start : y.stop]

    if df.empty:
        raise IndexError(
            f"index: {index} not found in dataset with bounds: {self.bounds}"
        )

    mask = self._merge_or_stack(df.filepath, index)

    std_error_paths = df.filepath.apply(lambda x: x.replace(".tif", "_err.tif"))
    std_err_mask = self._merge_or_stack(std_error_paths, index)

    mask = ops.concatenate((mask, std_err_mask), axis=-1)

    transform = rasterio.transform.from_origin(x.start, y.stop, x.step, y.step)
    sample = {
        "mask": mask,
        "bounds": self._slice_to_tensor(index),
        "transform": ops.convert_to_tensor(np.array(list(transform))),
    }

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/globbiomass.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    tensor = ops.convert_to_numpy(sample["mask"])
    mask = tensor[..., 0]
    error_mask = tensor[..., 1]

    showing_predictions = "prediction" in sample
    if showing_predictions:
        pred = ops.convert_to_numpy(sample["prediction"])[..., 0]
        ncols = 3
    else:
        ncols = 2

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 4, 4))

    if showing_predictions:
        axs[0].imshow(mask)
        axs[0].axis("off")
        axs[1].imshow(error_mask)
        axs[1].axis("off")
        axs[2].imshow(pred)
        axs[2].axis("off")
        if show_titles:
            axs[0].set_title("Mask")
            axs[1].set_title("Uncertainty Mask")
            axs[2].set_title("Prediction")
    else:
        axs[0].imshow(mask)
        axs[0].axis("off")
        axs[1].imshow(error_mask)
        axs[1].axis("off")
        if show_titles:
            axs[0].set_title("Mask")
            axs[1].set_title("Uncertainty Mask")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.gse.GoogleSatelliteEmbedding

GoogleSatelliteEmbedding(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset

Google Satellite Embedding dataset.

The Google Satellite Embedding dataset <https://developers.google.com/earth-engine/datasets/catalog/GOOGLE_SATELLITE_EMBEDDING_V1_ANNUAL>__ is a global, analysis-ready collection of learned geospatial embeddings. Each 10-meter pixel in this dataset is a 64-dimensional representation, or "embedding vector", that encodes temporal trajectories of surface conditions at and around that pixel as measured by various Earth observation instruments and datasets, over a single calendar year.

The embeddings are unit-length and do not require any additional normalization, and are distributed across the unit sphere, making them well-suited for use with clustering algorithms and tree-based classifiers.

The Satellite Embedding dataset was produced by AlphaEarth Foundations <https://deepmind.google/blog/alphaearth-foundations-helps-map-our-planet-in-unprecedented-detail/>__.

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2507.22291

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new RasterDataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a mosaic
            (``[H, W, C]``). For mask-style datasets (``is_image=False``),
            single-band data has the channel dimension squeezed.
    """
    self.paths = paths
    self.bands = bands or self.all_bands
    self.transforms = transforms
    self.cache = cache
    self.time_series = time_series

    if self.all_bands:
        assert set(self.bands) <= set(self.all_bands)

    filename_regex = re.compile(self.filename_regex, re.VERBOSE)
    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        match = re.match(filename_regex, os.path.basename(filepath))
        if match is not None:
            vrt = None
            try:
                vrt = self._load_warp_file(filepath=filepath, crs=crs)
                if self.cmap is None:
                    try:
                        colors = np.array(list(vrt.colormap(1).values())) / 255
                        self.cmap = ListedColormap(colors)
                    except ValueError:
                        pass
                if crs is None:
                    with rasterio.Env(OSR_WKT_FORMAT="WKT2_2018"):
                        crs = PROJ_CRS.from_user_input(vrt.crs)
                footprint = self.footprint_from_datasource(vrt)
                if footprint is None:
                    footprint = shapely.box(*vrt.bounds)
                geometries.append(footprint)
                if res is None:
                    res = vrt.res
            except rasterio.errors.RasterioIOError:
                continue
            else:
                filepaths.append(filepath)
                mint, maxt = self._filepath_to_timestamp(filepath)
                datetimes.append((mint, maxt))
            finally:
                if vrt is not None:
                    vrt.close()

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if not self.separate_files:
        self.band_indexes = None
        if self.bands:
            if self.all_bands:
                self.band_indexes = [
                    self.all_bands.index(i) + 1 for i in self.bands
                ]
            else:
                msg = (
                    f"{self.__class__.__name__} is missing an `all_bands` "
                    "attribute, so `bands` cannot be specified."
                )
                raise AssertionError(msg)

    if res is not None:
        if isinstance(res, (int, float)):
            res = (res, res)

        self._res = res

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

.. warning:: Visualizations are generated using PCA on each image individually, and are thus not comparable across images. The plot method is provided for visualization purposes only and should not be used to draw conclusions.

Source code in keras_climate/datasets/gse.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    .. warning::
       Visualizations are generated using PCA on each image
       *individually*, and are thus not comparable across images. The
       plot method is provided for visualization purposes only and
       should not be used to draw conclusions.
    """
    import matplotlib.pyplot as plt

    h, w, _ = sample["image"].shape
    image = ops.convert_to_numpy(sample["image"])
    A = image.reshape(h * w, -1).astype("float64")

    # Use PCA (via SVD) to project embeddings from 64D to 3D space
    A_centered = A - A.mean(axis=0, keepdims=True)
    _, _, vt = np.linalg.svd(A_centered, full_matrices=False)
    v = vt[:3].T
    b = A @ v

    b -= b.min(axis=0, keepdims=True)
    b /= b.max(axis=0, keepdims=True)
    image = b.reshape(h, w, 3)

    fig, ax = plt.subplots()
    ax.imshow(image)
    ax.axis("off")

    if show_titles:
        ax.set_title("Embedding")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.hyspecnet.HySpecNet11k

HySpecNet11k(root='data', split='train', strategy='easy', bands=None, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

HySpecNet-11k dataset.

HySpecNet-11k <https://hyspecnet.rsim.berlin/>__ is a large-scale benchmark dataset for hyperspectral image compression and self-supervised learning. It is made up of 11,483 nonoverlapping image patches acquired by the EnMAP satellite <https://www.enmap.org/>_. Each patch is a portion of 128 x 128 pixels with 224 spectral bands and with a ground sample distance of 30 m.

We provide predefined splits obtained by randomly dividing HySpecNet into:

. a training set that includes 70% of the patches,

. a validation set that includes 20% of the patches, and

. a test set that includes 10% of the patches.

Depending on the way that we used for splitting the dataset, we define two different splits:

. an easy split, where patches from the same tile can be present in

different sets (patchwise splitting); and

. a hard split, where all patches from one tile belong to the same set

(tilewise splitting).

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2306.00385

Initialize a new HySpecNet11k instance.

Args: root: Root directory where dataset can be found. split: One of 'train', 'val', or 'test'. strategy: Either 'easy' for patchwise splitting or 'hard' for tilewise splitting. bands: Bands to return. transforms: A function/transform that takes input sample and its target as entry and returns a transformed version. download: If True, download dataset and store it in the root directory. checksum: If True, verify the checksum of the downloaded files.

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/hyspecnet.py

def __init__(
    self,
    root="data",
    split="train",
    strategy="easy",
    bands=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new HySpecNet11k instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', or 'test'.
        strategy: Either 'easy' for patchwise splitting or 'hard' for
            tilewise splitting.
        bands: Bands to return.
        transforms: A function/transform that takes input sample and its
            target as entry and returns a transformed version.
        download: If True, download dataset and store it in the root directory.
        checksum: If True, verify the checksum of the downloaded files.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    self.root = root
    self.split = split
    self.strategy = strategy
    self.bands = bands or self.default_bands
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self.wavelengths = ops.convert_to_tensor(
        np.array([EnMAP.wavelengths[b] for b in self.bands], dtype="float32")
    )
    self.band_indices = [self.all_bands.index(b) + 1 for b in self.bands]

    self._verify()

    path = os.path.join(root, "hyspecnet-11k", "splits", strategy, f"{split}.csv")
    with open(path) as f:
        self.files = f.read().strip().split("\n")

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/hyspecnet.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.files)

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/hyspecnet.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    path = self.files[index].replace("DATA.npy", "SPECTRAL_IMAGE.TIF")
    file = os.path.basename(path)
    match = re.match(EnMAP.filename_regex, file, re.VERBOSE)
    assert match
    mint, maxt = disambiguate_timestamp(match.group("date"), EnMAP.date_format)

    with rio.open(os.path.join(self.root, "hyspecnet-11k", "patches", path)) as src:
        minx, maxx = src.bounds.left, src.bounds.right
        miny, maxy = src.bounds.bottom, src.bounds.top
        array = src.read(self.band_indices).astype("float32")
        array = np.transpose(array, (1, 2, 0))
        sample = {
            "image": ops.convert_to_tensor(array),
            "x": ops.convert_to_tensor(np.array((minx + maxx) / 2, dtype="float64")),
            "y": ops.convert_to_tensor(np.array((miny + maxy) / 2, dtype="float64")),
            "t": ops.convert_to_tensor(
                np.array(
                    (mint.timestamp() + maxt.timestamp()) / 2, dtype="float64"
                )
            ),
            "wavelength": self.wavelengths,
            "res": ops.convert_to_tensor(np.array(30)),
        }

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/hyspecnet.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise RGBBandsMissingError()

    image = ops.take(sample["image"], rgb_indices, axis=-1)
    image = quantile_normalization(image)
    image = ops.convert_to_numpy(image)

    fig, ax = plt.subplots()
    ax.imshow(image)
    ax.axis("off")

    if show_titles:
        ax.set_title("Image")

    if suptitle:
        fig.suptitle(suptitle)

    return fig

keras_climate.datasets.idtrees.IDTReeS

IDTReeS(root='data', split='train', task='task1', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

IDTReeS dataset.

The IDTReeS <https://idtrees.org/competition/>__ dataset is a dataset for tree crown detection.

Dataset features:

• RGB Image, Canopy Height Model (CHM), Hyperspectral Image (HSI), LiDAR Point Cloud
• Remote sensing and field data generated by the National Ecological Observatory Network (NEON)
• 0.1 - 1m resolution imagery
• Task 1 - object detection (tree crown delination)
• Task 2 - object classification (species classification)
• Train set contains 85 images
• Test set (task 1) contains 153 images
• Test set (task 2) contains 353 images and tree crown polygons

Dataset format:

• optical - three-channel RGB 200x200 geotiff
• canopy height model - one-channel 20x20 geotiff
• hyperspectral - 369-channel 20x20 geotiff
• point cloud - Nx3 LAS file (.las), some files contain RGB colors per point
• shapefiles (.shp) containing polygons
• csv file containing species labels and other metadata for each polygon

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1101/2021.08.06.453503

This dataset requires the following additional library to be installed:

• laspy <https://pypi.org/project/laspy/>_ to read lidar point clouds

Initialize a new IDTReeS dataset instance.

Args: root: root directory where dataset can be found split: one of "train" or "test" task: 'task1' for detection, 'task2' for detection + classification (only relevant for split='test') transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files

Raises: DatasetNotFoundError: If dataset is not found and download is False. DependencyNotFoundError: If laspy is not installed.

Source code in keras_climate/datasets/idtrees.py

def __init__(
    self,
    root="data",
    split="train",
    task="task1",
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new IDTReeS dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        task: 'task1' for detection, 'task2' for detection + classification
            (only relevant for split='test')
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root directory
        checksum: if True, check the MD5 of the downloaded files

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
        DependencyNotFoundError: If laspy is not installed.
    """
    lazy_import("laspy")

    assert split in ["train", "test"]
    assert task in ["task1", "task2"]

    self.root = root
    self.split = split
    self.task = task
    self.transforms = transforms
    self.download = download
    self.checksum = checksum
    self.class2idx = {c: i for i, c in enumerate(self.classes)}
    self.idx2class = {i: c for i, c in enumerate(self.classes)}
    self.num_classes = len(self.classes)
    self._verify()
    self.images, self.geometries, self.labels = self._load(root)

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/idtrees.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    path = self.images[index]
    image = ops.cast(self._load_image(path), "uint8")
    hsi = self._load_image(path.replace("RGB", "HSI"))
    chm = self._load_image(path.replace("RGB", "CHM"))
    las = self._load_las(path.replace("RGB", "LAS").replace(".tif", ".las"))
    sample = {"image": image, "hsi": hsi, "chm": chm, "las": las}

    if self.split == "test":
        if self.task == "task2":
            sample["bbox_xyxy"] = self._load_boxes(path)
            h, w = sample["image"].shape[:2]
            sample["bbox_xyxy"], _ = self._filter_boxes(
                image_size=(h, w),
                min_size=1,
                boxes=sample["bbox_xyxy"],
                labels=None,
            )
    else:
        sample["bbox_xyxy"] = self._load_boxes(path)
        sample["label"] = self._load_target(path)

        h, w = sample["image"].shape[:2]
        sample["bbox_xyxy"], sample["label"] = self._filter_boxes(
            image_size=(h, w),
            min_size=1,
            boxes=sample["bbox_xyxy"],
            labels=sample["label"],
        )

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/idtrees.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.images)

plot

plot(sample, show_titles=True, suptitle=None, hsi_indices=(0, 1, 2))

Plot a sample from the dataset.

Args: sample: a sample returned by :meth:__getitem__ show_titles: flag indicating whether to show titles above each panel suptitle: optional string to use as a suptitle hsi_indices: tuple of indices to create HSI false color image

Returns: a matplotlib Figure with the rendered sample

Source code in keras_climate/datasets/idtrees.py

def plot(self, sample, show_titles=True, suptitle=None, hsi_indices=(0, 1, 2)):
    """Plot a sample from the dataset.

    Args:
        sample: a sample returned by :meth:`__getitem__`
        show_titles: flag indicating whether to show titles above each panel
        suptitle: optional string to use as a suptitle
        hsi_indices: tuple of indices to create HSI false color image

    Returns:
        a matplotlib Figure with the rendered sample
    """
    import matplotlib.pyplot as plt
    from matplotlib.patches import Rectangle

    assert len(hsi_indices) == 3

    ncols = 3

    hsi = ops.convert_to_numpy(ops.take(sample["hsi"], hsi_indices, axis=-1))
    hsi = quantile_normalization(hsi)

    chm = ops.convert_to_numpy(sample["chm"])
    chm = quantile_normalization(chm)
    if chm.ndim == 3 and chm.shape[-1] == 1:
        chm = chm[..., 0]

    image = ops.convert_to_numpy(sample["image"])

    boxes, labels = None, None
    if "bbox_xyxy" in sample and len(sample["bbox_xyxy"]):
        boxes = ops.convert_to_numpy(sample["bbox_xyxy"])
        if "label" in sample:
            labels = [
                self.idx2class[int(i)]
                for i in ops.convert_to_numpy(sample["label"])
            ]

    pred_boxes, pred_labels = None, None
    if "prediction_bbox_xyxy" in sample and len(sample["prediction_bbox_xyxy"]):
        ncols += 1
        pred_boxes = ops.convert_to_numpy(sample["prediction_bbox_xyxy"])
        if "prediction_label" in sample:
            pred_labels = [
                self.idx2class[int(i)]
                for i in ops.convert_to_numpy(sample["prediction_label"])
            ]

    fig, axs = plt.subplots(ncols=ncols, figsize=(ncols * 10, 10))
    axs[0].imshow(image)
    axs[0].axis("off")
    if boxes is not None:
        for i, box in enumerate(boxes):
            xmin, ymin, xmax, ymax = box
            axs[0].add_patch(
                Rectangle(
                    (xmin, ymin),
                    xmax - xmin,
                    ymax - ymin,
                    fill=False,
                    edgecolor="red",
                )
            )
            if labels is not None:
                axs[0].text(xmin, ymin, labels[i], color="red", fontsize=8)

    axs[1].imshow(hsi)
    axs[1].axis("off")
    axs[2].imshow(chm)
    axs[2].axis("off")
    if ncols > 3:
        axs[3].imshow(image)
        axs[3].axis("off")
        if pred_boxes is not None:
            for i, box in enumerate(pred_boxes):
                xmin, ymin, xmax, ymax = box
                axs[3].add_patch(
                    Rectangle(
                        (xmin, ymin),
                        xmax - xmin,
                        ymax - ymin,
                        fill=False,
                        edgecolor="blue",
                    )
                )
                if pred_labels is not None:
                    axs[3].text(xmin, ymin, pred_labels[i], color="blue", fontsize=8)

    if show_titles:
        axs[0].set_title("Ground Truth")
        axs[1].set_title("Hyperspectral False Color Image")
        axs[2].set_title("Canopy Height Model")
        if ncols > 3:
            axs[3].set_title("Predictions")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.inaturalist.INaturalist

INaturalist(root='data')

Bases: GeoDataset

Dataset for iNaturalist.

iNaturalist <https://www.inaturalist.org/>__ is a joint initiative of the California Academy of Sciences and the National Geographic Society. It allows citizen scientists to upload observations of organisms that can be downloaded by scientists and researchers.

If you use an iNaturalist dataset in your research, please cite it according to:

• https://help.inaturalist.org/en/support/solutions/articles/151000170344-how-should-i-cite-inaturalist-

Initialize a new Dataset instance.

Args: root: root directory where dataset can be found

Raises: DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/inaturalist.py

def __init__(self, root="data"):
    """Initialize a new Dataset instance.

    Args:
        root: root directory where dataset can be found

    Raises:
        DatasetNotFoundError: If dataset is not found.
    """
    super().__init__()

    self.root = root

    files = glob.glob(os.path.join(root, "**.csv"))
    if not files:
        raise DatasetNotFoundError(self)

    # Read CSV file
    usecols = ["observed_on", "time_observed_at", "latitude", "longitude"]
    df = pd.read_csv(files[0], header=0, usecols=usecols)
    df = df[df.latitude.notna()]
    df = df[df.longitude.notna()]

    # Convert from pandas DataFrame to geopandas GeoDataFrame
    func = functools.partial(disambiguate_timestamp, format="%Y-%m-%d %H:%M:%S %z")
    time = df.time_observed_at.apply(func)
    func = functools.partial(disambiguate_timestamp, format="%Y-%m-%d")
    date = df.observed_on.apply(func)
    time[time.isnull()] = date[time.isnull()]
    times = time.to_list()
    index = pd.IntervalIndex.from_tuples(times, closed="both", name="datetime")
    geometry = gpd.points_from_xy(df.longitude, df.latitude)
    self.index = GeoDataFrame(index=index, geometry=geometry, crs="EPSG:4326")

__getitem__

__getitem__(index)

Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

Raises: IndexError: If index is not found in the dataset.

Source code in keras_climate/datasets/inaturalist.py

def __getitem__(self, index):
    """Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

    Raises:
        IndexError: If *index* is not found in the dataset.
    """
    x, y, t = self._disambiguate_slice(index)
    interval = pd.Interval(t.start, t.stop)
    df = self.index.iloc[self.index.index.overlaps(interval)]
    df = df.iloc[:: t.step]
    df = df.cx[x.start : x.stop, y.start : y.stop]

    if df.empty:
        raise IndexError(
            f"index: {index} not found in dataset with bounds: {self.bounds}"
        )

    keypoints = ops.convert_to_tensor(
        df.get_coordinates().values.astype("float32")
    )
    transform = rasterio.transform.from_origin(x.start, y.stop, x.step, y.step)
    sample = {
        "bounds": self._slice_to_tensor(index),
        "keypoints": keypoints,
        "transform": ops.convert_to_tensor(np.array(list(transform))),
    }

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/inaturalist.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    fig, ax = plt.subplots(figsize=(10, 8))
    ax.grid(ls="--")

    # Extract coordinates
    keypoints = ops.convert_to_numpy(sample["keypoints"])
    x = keypoints[:, 0]
    y = keypoints[:, 1]

    # Plot the points
    ax.scatter(x, y)

    # Set labels
    ax.set_xlabel("Longitude")
    ax.set_ylabel("Latitude")

    # Add titles if requested
    if show_titles:
        ax.set_title("iNaturalist Dataset Plot")

    if suptitle is not None:
        fig.suptitle(suptitle)

    fig.tight_layout()
    return fig

keras_climate.datasets.inria.InriaAerialImageLabeling

InriaAerialImageLabeling(root='data', split='train', transforms=None, checksum=True)

Bases: NonGeoDataset

Inria Aerial Image Labeling Dataset.

The Inria Aerial Image Labeling <https://project.inria.fr/aerialimagelabeling/>__ dataset is a building detection dataset over dissimilar settlements ranging from densely populated areas to alpine towns. Refer to the dataset homepage to download the dataset.

Dataset features:

• Coverage of 810 km2 (405 km2 for training and 405 km2 for testing)
• Aerial orthorectified color imagery with a spatial resolution of 0.3 m
• Number of images: 360 (train: 180, test: 180)
• Train cities: Austin, Chicago, Kitsap, West Tyrol, Vienna
• Test cities: Bellingham, Bloomington, Innsbruck, San Francisco, East Tyrol

Dataset format:

• Imagery - RGB aerial GeoTIFFs of shape 5000 x 5000
• Labels - RGB aerial GeoTIFFs of shape 5000 x 5000

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1109/IGARSS.2017.8127684

Initialize a new InriaAerialImageLabeling Dataset instance.

Args: root: root directory where dataset can be found split: train/val/test split transforms: a function/transform that takes input sample and its target as entry and returns a transformed version. checksum: if True, check the MD5 of the downloaded files

Raises: AssertionError: if split is invalid DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/inria.py

def __init__(self, root="data", split="train", transforms=None, checksum=True):
    """Initialize a new InriaAerialImageLabeling Dataset instance.

    Args:
        root: root directory where dataset can be found
        split: train/val/test split
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version.
        checksum: if True, check the MD5 of the downloaded files

    Raises:
        AssertionError: if ``split`` is invalid
        DatasetNotFoundError: If dataset is not found.
    """
    self.root = root
    assert split in {"train", "val", "test"}
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    self._verify()
    self.files = self._load_files(root)

__len__

__len__()

Return the number of samples in the dataset.

Source code in keras_climate/datasets/inria.py

def __len__(self):
    """Return the number of samples in the dataset."""
    return len(self.files)

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/inria.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    files = self.files[index]
    img = self._load_image(files["image"])
    sample = {"image": img}
    if files.get("label"):
        mask = self._load_target(files["label"])
        sample["mask"] = mask

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/inria.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"])[..., :3]
    image = quantile_normalization(image)
    image = ops.convert_to_numpy(image)

    ncols = 1
    show_mask = "mask" in sample
    show_predictions = "prediction" in sample

    if show_mask:
        mask = ops.convert_to_numpy(sample["mask"])
        ncols += 1

    if show_predictions:
        prediction = ops.convert_to_numpy(sample["prediction"])
        ncols += 1

    fig, axs = plt.subplots(ncols=ncols, figsize=(ncols * 8, 8))
    if not isinstance(axs, np.ndarray):
        axs = [axs]
    axs[0].imshow(image)
    axs[0].axis("off")
    if show_titles:
        axs[0].set_title("Image")

    if show_mask:
        axs[1].imshow(mask, interpolation="none")
        axs[1].axis("off")
        if show_titles:
            axs[1].set_title("Label")

    if show_predictions:
        axs[2].imshow(prediction, interpolation="none")
        axs[2].axis("off")
        if show_titles:
            axs[2].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.iobench.IOBench

IOBench(root='data', split='preprocessed', crs=None, res=None, bands=None, classes=None, transforms=None, cache=True, download=False, checksum=True)

Bases: IntersectionDataset

I/O Bench dataset.

I/O Bench is a dataset designed to benchmark the I/O performance of keras_climate/torchgeo. It contains a single Landsat 9 scene and CDL file from 2023, and consists of the following splits

• original: the original files as downloaded from USGS Earth Explorer and USDA CropScape
• raw: the same files with compression and with CDL clipped to the bounds of the Landsat scene
• preprocessed: the same files with compression, reprojected to the same CRS, as COGs, with TAP

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1145/3557915.3560953

Initialize a new IOBench instance.

Args: root: Root directory where dataset can be found. split: One of 'original', 'raw', or 'preprocessed'. crs: CRS to warp to (defaults to the CRS of the first file found) res: Resolution of the dataset in units of CRS. bands: Bands to return (defaults to all bands). classes: List of classes to include, the rest will be mapped to 0. transforms: A function/transform that takes an input sample and returns a transformed version. cache: If True, cache file handle to speed up repeated sampling. download: If True, download dataset and store it in the root directory. checksum: If True, verify the checksum of the downloaded files.

Raises: AssertionError: If split argument is invalid. DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/iobench.py

def __init__(
    self,
    root="data",
    split="preprocessed",
    crs=None,
    res=None,
    bands=None,
    classes=None,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
):
    """Initialize a new IOBench instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'original', 'raw', or 'preprocessed'.
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: Resolution of the dataset in units of CRS.
        bands: Bands to return (defaults to all bands).
        classes: List of classes to include, the rest will be mapped to 0.
        transforms: A function/transform that takes an input sample
            and returns a transformed version.
        cache: If True, cache file handle to speed up repeated sampling.
        download: If True, download dataset and store it in the root directory.
        checksum: If True, verify the checksum of the downloaded files.

    Raises:
        AssertionError: If *split* argument is invalid.
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    if bands is None:
        bands = [*Landsat9.default_bands, "SR_QA_AEROSOL"]
    if classes is None:
        classes = list(CDL.valid_classes)
    assert split in self.sha256s

    self.root = root
    self.split = split
    self.download = download
    self.checksum = checksum

    self._verify()

    root = os.path.join(root, split)
    self.landsat = Landsat9(root, crs, res, bands, transforms, cache)
    self.cdl = CDL(root, crs, res, [2023], classes, transforms, cache)

    super().__init__(self.landsat, self.cdl)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/iobench.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.landsat.rgb_bands:
        if band in self.landsat.bands:
            rgb_indices.append(self.landsat.bands.index(band))
        else:
            raise RGBBandsMissingError()

    image = ops.take(sample["image"], rgb_indices, axis=-1)
    image = ops.cast(image, "float32")
    image = quantile_normalization(image)
    image = ops.convert_to_numpy(image)

    mask = ops.convert_to_numpy(ops.squeeze(sample["mask"])).astype("int64")
    mask = self.cdl.inverse_map[mask]

    fig, axes = plt.subplots(1, 2, figsize=(8, 4))
    kwargs = {"cmap": self.cdl.cmap, "vmin": 0, "vmax": 255, "interpolation": "none"}

    axes[0].imshow(image)
    axes[1].imshow(mask, **kwargs)

    axes[0].axis("off")
    axes[1].axis("off")

    if show_titles:
        axes[0].set_title("Image")
        axes[1].set_title("Mask")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.l7irish.L7Irish

L7Irish(paths='data', crs=None, res=None, bands=L7IrishImage.all_bands, transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: IntersectionDataset

L7 Irish dataset.

The L7 Irish <https://landsat.usgs.gov/landsat-7-cloud-cover-assessment-validation-data>__ dataset is based on Landsat 7 Enhanced Thematic Mapper Plus (ETM+) Level-1G scenes. Manually generated cloud masks are used to train and validate cloud cover assessment algorithms, which in turn are intended to compute the percentage of cloud cover in each scene.

Dataset features:

• Images divided between 9 unique biomes
• 206 scenes from Landsat 7 ETM+ sensor
• Imagery from global tiles between June 2000--December 2001
• 9 Level-1 spectral bands with 30 m per pixel resolution

Dataset classes:

1. Fill
2. Cloud Shadow
3. Clear
4. Thin Cloud
5. Cloud

If you use this dataset in your research, please cite the following:

• https://doi.org/10.5066/F7XD0ZWC
• https://doi.org/10.1109/TGRS.2011.2164087
• https://www.sciencebase.gov/catalog/item/573ccf18e4b0dae0d5e4b109

Initialize a new L7Irish instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to EPSG:3857) res: resolution of the dataset in units of CRS bands: bands to return (defaults to all bands) transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files time_series: if True, stack data along the time series dimension

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/l7irish.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=L7IrishImage.all_bands,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new L7Irish instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to EPSG:3857)
        res: resolution of the dataset in units of CRS
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample
            and returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root directory
        checksum: if True, verify the checksum of the downloaded files
        time_series: if True, stack data along the time series dimension

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    from pyproj import CRS as PROJ_CRS

    self.paths = paths
    self.download = download
    self.checksum = checksum

    self._verify()

    if crs is None:
        crs = PROJ_CRS.from_epsg(3857)

    self.image = L7IrishImage(
        paths, crs, res, bands, transforms, cache, time_series
    )
    self.mask = L7IrishMask(paths, crs, res, None, transforms, cache, time_series)

    # Mask filename does not include the date, grab it from the image filename
    self.mask.index.index = self.image.index.index

    super().__init__(self.image, self.mask)

    # Ignore unintentional partial overlap
    self.index = self.image.index

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/l7irish.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.image.rgb_bands:
        if band in self.image.bands:
            rgb_indices.append(self.image.bands.index(band))
        else:
            raise RGBBandsMissingError()

    image = ops.take(sample["image"], rgb_indices, axis=-1)
    image = quantile_normalization(image)
    image = ops.convert_to_numpy(image)

    mask = ops.convert_to_numpy(sample["mask"]).astype("uint8").squeeze()

    num_panels = 2
    showing_predictions = "prediction" in sample
    if showing_predictions:
        predictions = (
            ops.convert_to_numpy(sample["prediction"]).astype("uint8").squeeze()
        )
        num_panels += 1

    kwargs = {"cmap": "gray", "vmin": 0, "vmax": 4, "interpolation": "none"}
    fig, axs = plt.subplots(1, num_panels, figsize=(num_panels * 4, 5))
    axs[0].imshow(image)
    axs[0].axis("off")
    axs[1].imshow(mask, **kwargs)
    axs[1].axis("off")
    if show_titles:
        axs[0].set_title("Image")
        axs[1].set_title("Mask")

    if showing_predictions:
        axs[2].imshow(predictions, **kwargs)
        axs[2].axis("off")
        if show_titles:
            axs[2].set_title("Predictions")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.l7irish.L7IrishImage

L7IrishImage(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset

Images from the L7 Irish dataset.

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new RasterDataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a mosaic
            (``[H, W, C]``). For mask-style datasets (``is_image=False``),
            single-band data has the channel dimension squeezed.
    """
    self.paths = paths
    self.bands = bands or self.all_bands
    self.transforms = transforms
    self.cache = cache
    self.time_series = time_series

    if self.all_bands:
        assert set(self.bands) <= set(self.all_bands)

    filename_regex = re.compile(self.filename_regex, re.VERBOSE)
    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        match = re.match(filename_regex, os.path.basename(filepath))
        if match is not None:
            vrt = None
            try:
                vrt = self._load_warp_file(filepath=filepath, crs=crs)
                if self.cmap is None:
                    try:
                        colors = np.array(list(vrt.colormap(1).values())) / 255
                        self.cmap = ListedColormap(colors)
                    except ValueError:
                        pass
                if crs is None:
                    with rasterio.Env(OSR_WKT_FORMAT="WKT2_2018"):
                        crs = PROJ_CRS.from_user_input(vrt.crs)
                footprint = self.footprint_from_datasource(vrt)
                if footprint is None:
                    footprint = shapely.box(*vrt.bounds)
                geometries.append(footprint)
                if res is None:
                    res = vrt.res
            except rasterio.errors.RasterioIOError:
                continue
            else:
                filepaths.append(filepath)
                mint, maxt = self._filepath_to_timestamp(filepath)
                datetimes.append((mint, maxt))
            finally:
                if vrt is not None:
                    vrt.close()

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if not self.separate_files:
        self.band_indexes = None
        if self.bands:
            if self.all_bands:
                self.band_indexes = [
                    self.all_bands.index(i) + 1 for i in self.bands
                ]
            else:
                msg = (
                    f"{self.__class__.__name__} is missing an `all_bands` "
                    "attribute, so `bands` cannot be specified."
                )
                raise AssertionError(msg)

    if res is not None:
        if isinstance(res, (int, float)):
            res = (res, res)

        self._res = res

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

keras_climate.datasets.l7irish.L7IrishMask

L7IrishMask(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset

Masks from the L7 Irish dataset.

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new RasterDataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a mosaic
            (``[H, W, C]``). For mask-style datasets (``is_image=False``),
            single-band data has the channel dimension squeezed.
    """
    self.paths = paths
    self.bands = bands or self.all_bands
    self.transforms = transforms
    self.cache = cache
    self.time_series = time_series

    if self.all_bands:
        assert set(self.bands) <= set(self.all_bands)

    filename_regex = re.compile(self.filename_regex, re.VERBOSE)
    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        match = re.match(filename_regex, os.path.basename(filepath))
        if match is not None:
            vrt = None
            try:
                vrt = self._load_warp_file(filepath=filepath, crs=crs)
                if self.cmap is None:
                    try:
                        colors = np.array(list(vrt.colormap(1).values())) / 255
                        self.cmap = ListedColormap(colors)
                    except ValueError:
                        pass
                if crs is None:
                    with rasterio.Env(OSR_WKT_FORMAT="WKT2_2018"):
                        crs = PROJ_CRS.from_user_input(vrt.crs)
                footprint = self.footprint_from_datasource(vrt)
                if footprint is None:
                    footprint = shapely.box(*vrt.bounds)
                geometries.append(footprint)
                if res is None:
                    res = vrt.res
            except rasterio.errors.RasterioIOError:
                continue
            else:
                filepaths.append(filepath)
                mint, maxt = self._filepath_to_timestamp(filepath)
                datetimes.append((mint, maxt))
            finally:
                if vrt is not None:
                    vrt.close()

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if not self.separate_files:
        self.band_indexes = None
        if self.bands:
            if self.all_bands:
                self.band_indexes = [
                    self.all_bands.index(i) + 1 for i in self.bands
                ]
            else:
                msg = (
                    f"{self.__class__.__name__} is missing an `all_bands` "
                    "attribute, so `bands` cannot be specified."
                )
                raise AssertionError(msg)

    if res is not None:
        if isinstance(res, (int, float)):
            res = (res, res)

        self._res = res

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

__getitem__

__getitem__(index)

Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

Raises: IndexError: If index is not found in the dataset.

Source code in keras_climate/datasets/l7irish.py

def __getitem__(self, index):
    """Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

    Raises:
        IndexError: If *index* is not found in the dataset.
    """
    sample = super().__getitem__(index)
    mask = ops.convert_to_numpy(sample["mask"]).astype("int64")
    sample["mask"] = ops.convert_to_tensor(self.ordinal_map[mask])
    return sample

keras_climate.datasets.l8biome.L8Biome

L8Biome(paths, crs=None, res=None, bands=L8BiomeImage.all_bands, transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: IntersectionDataset

L8 Biome dataset.

The L8 Biome <https://landsat.usgs.gov/landsat-8-cloud-cover-assessment-validation-data>__ dataset is a validation dataset for cloud cover assessment algorithms, consisting of Pre-Collection Landsat 8 Operational Land Imager (OLI) Thermal Infrared Sensor (TIRS) terrain-corrected (Level-1T) scenes.

Dataset features:

• Images evenly divided between 8 unique biomes
• 96 scenes from Landsat 8 OLI/TIRS sensors
• Imagery from global tiles between April 2013--October 2014
• 11 Level-1 spectral bands with 30 m per pixel resolution

Dataset classes:

1. Fill
2. Cloud Shadow
3. Clear
4. Thin Cloud
5. Cloud

If you use this dataset in your research, please cite the following:

• https://doi.org/10.5066/F7251GDH
• https://doi.org/10.1016/j.rse.2017.03.026

Initialize a new L8Biome instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to EPSG:3857) res: resolution of the dataset in units of CRS bands: bands to return (defaults to all bands) transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files time_series: if True, stack data along the time series dimension

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/l8biome.py

def __init__(
    self,
    paths,
    crs=None,
    res=None,
    bands=L8BiomeImage.all_bands,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new L8Biome instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to EPSG:3857)
        res: resolution of the dataset in units of CRS
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample
            and returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root directory
        checksum: if True, verify the checksum of the downloaded files
        time_series: if True, stack data along the time series dimension

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    from pyproj import CRS as PROJ_CRS

    self.paths = paths
    self.download = download
    self.checksum = checksum

    self._verify()

    if crs is None:
        crs = PROJ_CRS.from_epsg(3857)

    self.image = L8BiomeImage(
        paths, crs, res, bands, transforms, cache, time_series
    )
    self.mask = L8BiomeMask(paths, crs, res, None, transforms, cache, time_series)

    super().__init__(self.image, self.mask)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/l8biome.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.image.rgb_bands:
        if band in self.image.bands:
            rgb_indices.append(self.image.bands.index(band))
        else:
            raise RGBBandsMissingError()

    image = ops.take(sample["image"], rgb_indices, axis=-1)
    image = quantile_normalization(image)
    image = ops.convert_to_numpy(image)

    mask = ops.convert_to_numpy(sample["mask"]).astype("uint8").squeeze()

    num_panels = 2
    showing_predictions = "prediction" in sample
    if showing_predictions:
        predictions = (
            ops.convert_to_numpy(sample["prediction"]).astype("uint8").squeeze()
        )
        num_panels += 1

    kwargs = {"cmap": "gray", "vmin": 0, "vmax": 4, "interpolation": "none"}
    fig, axs = plt.subplots(1, num_panels, figsize=(num_panels * 4, 5))
    axs[0].imshow(image)
    axs[0].axis("off")
    axs[1].imshow(mask, **kwargs)
    axs[1].axis("off")
    if show_titles:
        axs[0].set_title("Image")
        axs[1].set_title("Mask")

    if showing_predictions:
        axs[2].imshow(predictions, **kwargs)
        axs[2].axis("off")
        if show_titles:
            axs[2].set_title("Predictions")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.l8biome.L8BiomeImage

L8BiomeImage(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset

Images from the L8 Biome dataset.

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new RasterDataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a mosaic
            (``[H, W, C]``). For mask-style datasets (``is_image=False``),
            single-band data has the channel dimension squeezed.
    """
    self.paths = paths
    self.bands = bands or self.all_bands
    self.transforms = transforms
    self.cache = cache
    self.time_series = time_series

    if self.all_bands:
        assert set(self.bands) <= set(self.all_bands)

    filename_regex = re.compile(self.filename_regex, re.VERBOSE)
    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        match = re.match(filename_regex, os.path.basename(filepath))
        if match is not None:
            vrt = None
            try:
                vrt = self._load_warp_file(filepath=filepath, crs=crs)
                if self.cmap is None:
                    try:
                        colors = np.array(list(vrt.colormap(1).values())) / 255
                        self.cmap = ListedColormap(colors)
                    except ValueError:
                        pass
                if crs is None:
                    with rasterio.Env(OSR_WKT_FORMAT="WKT2_2018"):
                        crs = PROJ_CRS.from_user_input(vrt.crs)
                footprint = self.footprint_from_datasource(vrt)
                if footprint is None:
                    footprint = shapely.box(*vrt.bounds)
                geometries.append(footprint)
                if res is None:
                    res = vrt.res
            except rasterio.errors.RasterioIOError:
                continue
            else:
                filepaths.append(filepath)
                mint, maxt = self._filepath_to_timestamp(filepath)
                datetimes.append((mint, maxt))
            finally:
                if vrt is not None:
                    vrt.close()

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if not self.separate_files:
        self.band_indexes = None
        if self.bands:
            if self.all_bands:
                self.band_indexes = [
                    self.all_bands.index(i) + 1 for i in self.bands
                ]
            else:
                msg = (
                    f"{self.__class__.__name__} is missing an `all_bands` "
                    "attribute, so `bands` cannot be specified."
                )
                raise AssertionError(msg)

    if res is not None:
        if isinstance(res, (int, float)):
            res = (res, res)

        self._res = res

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

keras_climate.datasets.l8biome.L8BiomeMask

L8BiomeMask(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset

Masks from the L8 Biome dataset.

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new RasterDataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a mosaic
            (``[H, W, C]``). For mask-style datasets (``is_image=False``),
            single-band data has the channel dimension squeezed.
    """
    self.paths = paths
    self.bands = bands or self.all_bands
    self.transforms = transforms
    self.cache = cache
    self.time_series = time_series

    if self.all_bands:
        assert set(self.bands) <= set(self.all_bands)

    filename_regex = re.compile(self.filename_regex, re.VERBOSE)
    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        match = re.match(filename_regex, os.path.basename(filepath))
        if match is not None:
            vrt = None
            try:
                vrt = self._load_warp_file(filepath=filepath, crs=crs)
                if self.cmap is None:
                    try:
                        colors = np.array(list(vrt.colormap(1).values())) / 255
                        self.cmap = ListedColormap(colors)
                    except ValueError:
                        pass
                if crs is None:
                    with rasterio.Env(OSR_WKT_FORMAT="WKT2_2018"):
                        crs = PROJ_CRS.from_user_input(vrt.crs)
                footprint = self.footprint_from_datasource(vrt)
                if footprint is None:
                    footprint = shapely.box(*vrt.bounds)
                geometries.append(footprint)
                if res is None:
                    res = vrt.res
            except rasterio.errors.RasterioIOError:
                continue
            else:
                filepaths.append(filepath)
                mint, maxt = self._filepath_to_timestamp(filepath)
                datetimes.append((mint, maxt))
            finally:
                if vrt is not None:
                    vrt.close()

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if not self.separate_files:
        self.band_indexes = None
        if self.bands:
            if self.all_bands:
                self.band_indexes = [
                    self.all_bands.index(i) + 1 for i in self.bands
                ]
            else:
                msg = (
                    f"{self.__class__.__name__} is missing an `all_bands` "
                    "attribute, so `bands` cannot be specified."
                )
                raise AssertionError(msg)

    if res is not None:
        if isinstance(res, (int, float)):
            res = (res, res)

        self._res = res

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

__getitem__

__getitem__(index)

Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

Raises: IndexError: If index is not found in the dataset.

Source code in keras_climate/datasets/l8biome.py

def __getitem__(self, index):
    """Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

    Raises:
        IndexError: If *index* is not found in the dataset.
    """
    sample = super().__getitem__(index)
    mask = ops.convert_to_numpy(sample["mask"]).astype("int64")
    sample["mask"] = ops.convert_to_tensor(self.ordinal_map[mask])
    return sample

keras_climate.datasets.landcoverai.LandCoverAI

LandCoverAI(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: LandCoverAIBase, NonGeoDataset

LandCover.ai dataset.

See the abstract LandCoverAIBase class to find out more.

.. note::

This dataset uses a pre-chipped version of the data available on HuggingFace. The pre-chipped dataset contains the output/ directory with 512x512 image chips and corresponding masks in JPG/PNG format.

Initialize a new LandCover.ai dataset instance.

Args: root: root directory where dataset can be found split: one of "train", "val", or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: AssertionError: if split argument is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/landcoverai.py

def __init__(
    self, root="data", split="train", transforms=None, download=False, checksum=True
):
    """Initialize a new LandCover.ai dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val", or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    assert split in ["train", "val", "test"]

    super().__init__(root, download, checksum)

    self.transforms = transforms
    self.split = split
    with open(os.path.join(self.root, split + ".txt")) as f:
        self.ids = f.readlines()

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/landcoverai.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    id_ = self.ids[index].rstrip()
    sample = {"image": self._load_image(id_), "mask": self._load_target(id_)}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/landcoverai.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.ids)

keras_climate.datasets.landcoverai.LandCoverAI100

LandCoverAI100(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: LandCoverAI

Subset of LandCoverAI containing only 100 images.

Intended for tutorials and demonstrations, not for benchmarking. Maintains the same file structure, classes, and train-val-test split.

Source code in keras_climate/datasets/landcoverai.py

def __init__(
    self, root="data", split="train", transforms=None, download=False, checksum=True
):
    """Initialize a new LandCover.ai dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val", or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    assert split in ["train", "val", "test"]

    super().__init__(root, download, checksum)

    self.transforms = transforms
    self.split = split
    with open(os.path.join(self.root, split + ".txt")) as f:
        self.ids = f.readlines()

keras_climate.datasets.landcoverai.LandCoverAIBase

LandCoverAIBase(root='data', download=False, checksum=True)

Bases: ABC

Abstract base class for LandCover.ai Geo and NonGeo datasets.

The LandCover.ai <https://landcover.ai.linuxpolska.com/>__ (Land Cover from Aerial Imagery) dataset is a dataset for automatic mapping of buildings, woodlands, water and roads from aerial images. This implementation is specifically for Version 1 of LandCover.ai.

Dataset classes:

1. background
2. building
3. woodland
4. water
5. road

If you use this dataset in your research, please cite:

• https://arxiv.org/abs/2005.02264v4

Initialize a new LandCover.ai dataset instance.

Args: root: root directory where dataset can be found download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/landcoverai.py

def __init__(self, root="data", download=False, checksum=True):
    """Initialize a new LandCover.ai dataset instance.

    Args:
        root: root directory where dataset can be found
        download: if True, download dataset and store it in the root directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    self.root = root
    self.download = download
    self.checksum = checksum

    self._verify()

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/landcoverai.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"]).astype("uint8").squeeze()
    mask = ops.convert_to_numpy(sample["mask"]).astype("uint8").squeeze()

    num_panels = 2
    showing_predictions = "prediction" in sample
    if showing_predictions:
        predictions = ops.convert_to_numpy(sample["prediction"])
        num_panels += 1

    fig, axs = plt.subplots(1, num_panels, figsize=(num_panels * 4, 5))
    axs[0].imshow(image)
    axs[0].axis("off")
    axs[1].imshow(mask, vmin=0, vmax=4, cmap=self.cmap, interpolation="none")
    axs[1].axis("off")
    if show_titles:
        axs[0].set_title("Image")
        axs[1].set_title("Mask")

    if showing_predictions:
        axs[2].imshow(
            predictions, vmin=0, vmax=4, cmap=self.cmap, interpolation="none"
        )
        axs[2].axis("off")
        if show_titles:
            axs[2].set_title("Predictions")

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.landcoverai.LandCoverAIGeo

LandCoverAIGeo(root='data', crs=None, res=None, transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: LandCoverAIBase, RasterDataset

LandCover.ai Geo dataset.

See the abstract LandCoverAIBase class to find out more.

Initialize a new LandCover.ai Geo dataset instance.

Args: root: root directory where dataset can be found crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS (defaults to the resolution of the first file found) transforms: a function/transform that takes input sample and its target as entry and returns a transformed version cache: if True, cache file handle to speed up repeated sampling download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow) time_series: if True, stack data along the time series dimension ([T, H, W, C]). If False, merge data into a ([H, W, C]) mosaic.

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/landcoverai.py

def __init__(
    self,
    root="data",
    crs=None,
    res=None,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new LandCover.ai Geo dataset instance.

    Args:
        root: root directory where dataset can be found
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a
            (``[H, W, C]``) mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    LandCoverAIBase.__init__(self, root, download, checksum)
    RasterDataset.__init__(
        self,
        root,
        crs,
        res,
        transforms=transforms,
        cache=cache,
        time_series=time_series,
    )

__getitem__

__getitem__(index)

Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

Raises: IndexError: If index is not found in the dataset.

Source code in keras_climate/datasets/landcoverai.py

def __getitem__(self, index):
    """Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

    Raises:
        IndexError: If *index* is not found in the dataset.
    """
    x, y, t = self._disambiguate_slice(index)
    interval = pd.Interval(t.start, t.stop)
    df = self.index.iloc[self.index.index.overlaps(interval)]
    df = df.iloc[:: t.step]
    df = df.cx[x.start : x.stop, y.start : y.stop]

    img_filepaths = df.filepath
    mask_filepaths = img_filepaths.apply(lambda x: x.replace("images", "masks"))

    if df.empty:
        raise IndexError(
            f"index: {index} not found in dataset with bounds: {self.bounds}"
        )

    img = self._merge_or_stack(img_filepaths, index, self.band_indexes)
    mask = self._merge_or_stack(mask_filepaths, index, self.band_indexes)
    transform = rasterio.transform.from_origin(x.start, y.stop, x.step, y.step)
    sample = {
        "bounds": self._slice_to_tensor(index),
        "image": ops.cast(img, "float32"),
        "mask": ops.cast(ops.squeeze(mask, axis=-1), "int64"),
        "transform": ops.convert_to_tensor(np.array(list(transform))),
    }

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

keras_climate.datasets.landsat.Landsat

Landsat(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset, ABC

Abstract base class for all Landsat datasets.

Landsat <https://science.nasa.gov/mission/landsat/>__ is a joint NASA/USGS program, providing the longest continuous space-based record of Earth's land in existence.

If you use this dataset in your research, please cite it using the following format:

• https://www.usgs.gov/centers/eros/data-citation

Initialize a new Dataset instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS (defaults to the resolution of the first file found) bands: bands to return (defaults to all bands) transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling time_series: if True, stack data along the time series dimension ([T, H, W, C]). If False, merge data into a ([H, W, C]) mosaic.

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/landsat.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a
            (``[H, W, C]``) mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    bands = bands or self.default_bands
    self.filename_glob = self.filename_glob.format(bands[0])

    super().__init__(paths, crs, res, bands, transforms, cache, time_series)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/landsat.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise RGBBandsMissingError()

    image = ops.take(sample["image"], rgb_indices, axis=-1)
    image = ops.cast(image, "float32")
    image = quantile_normalization(image)

    fig, ax = plt.subplots(1, 1, figsize=(4, 4))

    ax.imshow(ops.convert_to_numpy(image))
    ax.axis("off")

    if show_titles:
        ax.set_title("Image")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.landsat.Landsat1

Landsat1(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: Landsat

Landsat 1 Multispectral Scanner (MSS).

Source code in keras_climate/datasets/landsat.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a
            (``[H, W, C]``) mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    bands = bands or self.default_bands
    self.filename_glob = self.filename_glob.format(bands[0])

    super().__init__(paths, crs, res, bands, transforms, cache, time_series)

keras_climate.datasets.landsat.Landsat2

Landsat2(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: Landsat1

Landsat 2 Multispectral Scanner (MSS).

Source code in keras_climate/datasets/landsat.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a
            (``[H, W, C]``) mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    bands = bands or self.default_bands
    self.filename_glob = self.filename_glob.format(bands[0])

    super().__init__(paths, crs, res, bands, transforms, cache, time_series)

keras_climate.datasets.landsat.Landsat3

Landsat3(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: Landsat1

Landsat 3 Multispectral Scanner (MSS).

Source code in keras_climate/datasets/landsat.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a
            (``[H, W, C]``) mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    bands = bands or self.default_bands
    self.filename_glob = self.filename_glob.format(bands[0])

    super().__init__(paths, crs, res, bands, transforms, cache, time_series)

keras_climate.datasets.landsat.Landsat4MSS

Landsat4MSS(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: Landsat

Landsat 4 Multispectral Scanner (MSS).

Source code in keras_climate/datasets/landsat.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a
            (``[H, W, C]``) mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    bands = bands or self.default_bands
    self.filename_glob = self.filename_glob.format(bands[0])

    super().__init__(paths, crs, res, bands, transforms, cache, time_series)

keras_climate.datasets.landsat.Landsat4TM

Landsat4TM(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: Landsat

Landsat 4 Thematic Mapper (TM).

Source code in keras_climate/datasets/landsat.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a
            (``[H, W, C]``) mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    bands = bands or self.default_bands
    self.filename_glob = self.filename_glob.format(bands[0])

    super().__init__(paths, crs, res, bands, transforms, cache, time_series)

keras_climate.datasets.landsat.Landsat5MSS

Landsat5MSS(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: Landsat4MSS

Landsat 4 Multispectral Scanner (MSS).

Source code in keras_climate/datasets/landsat.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a
            (``[H, W, C]``) mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    bands = bands or self.default_bands
    self.filename_glob = self.filename_glob.format(bands[0])

    super().__init__(paths, crs, res, bands, transforms, cache, time_series)

keras_climate.datasets.landsat.Landsat5TM

Landsat5TM(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: Landsat4TM

Landsat 5 Thematic Mapper (TM).

Source code in keras_climate/datasets/landsat.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a
            (``[H, W, C]``) mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    bands = bands or self.default_bands
    self.filename_glob = self.filename_glob.format(bands[0])

    super().__init__(paths, crs, res, bands, transforms, cache, time_series)

keras_climate.datasets.landsat.Landsat7

Landsat7(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: Landsat

Landsat 7 Enhanced Thematic Mapper Plus (ETM+).

Source code in keras_climate/datasets/landsat.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a
            (``[H, W, C]``) mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    bands = bands or self.default_bands
    self.filename_glob = self.filename_glob.format(bands[0])

    super().__init__(paths, crs, res, bands, transforms, cache, time_series)

keras_climate.datasets.landsat.Landsat8

Landsat8(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: Landsat

Landsat 8 Operational Land Imager (OLI) and Thermal Infrared Sensor (TIRS).

Source code in keras_climate/datasets/landsat.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a
            (``[H, W, C]``) mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    bands = bands or self.default_bands
    self.filename_glob = self.filename_glob.format(bands[0])

    super().__init__(paths, crs, res, bands, transforms, cache, time_series)

keras_climate.datasets.landsat.Landsat9

Landsat9(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: Landsat8

Landsat 9 Operational Land Imager (OLI-2) and Thermal Infrared Sensor (TIRS-2).

Source code in keras_climate/datasets/landsat.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a
            (``[H, W, C]``) mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    bands = bands or self.default_bands
    self.filename_glob = self.filename_glob.format(bands[0])

    super().__init__(paths, crs, res, bands, transforms, cache, time_series)

keras_climate.datasets.levircd.LEVIRCD

LEVIRCD(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: LEVIRCDBase

LEVIR-CD dataset.

The LEVIR-CD <https://github.com/justchenhao/STANet>__ dataset is a dataset for building change detection.

Dataset format:

• images are three-channel pngs
• masks are single-channel pngs where no change = 0, change = 255

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.3390/rs12101662

Source code in keras_climate/datasets/levircd.py

def __init__(
    self, root="data", split="train", transforms=None, download=False, checksum=True
):
    """Initialize a new LEVIR-CD base dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root directory
        checksum: if True, check the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    assert split in self.splits

    self.root = root
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    if download:
        self._download()

    if not self._check_integrity():
        raise DatasetNotFoundError(self)

    self.files = self._load_files(self.root, self.split)

keras_climate.datasets.levircd.LEVIRCDBase

LEVIRCDBase(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset, ABC

Abstract base class for the LEVIRCD datasets.

Initialize a new LEVIR-CD base dataset instance.

Args: root: root directory where dataset can be found split: one of "train" or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the checksum of the downloaded files (may be slow)

Raises: AssertionError: if split argument is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/levircd.py

def __init__(
    self, root="data", split="train", transforms=None, download=False, checksum=True
):
    """Initialize a new LEVIR-CD base dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root directory
        checksum: if True, check the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    assert split in self.splits

    self.root = root
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    if download:
        self._download()

    if not self._check_integrity():
        raise DatasetNotFoundError(self)

    self.files = self._load_files(self.root, self.split)

__getitem__

__getitem__(index)

Return an index within the dataset.

The returned image has shape [T, H, W, C] where T=2 (before, after). The mask has shape [H, W].

Source code in keras_climate/datasets/levircd.py

def __getitem__(self, index):
    """Return an index within the dataset.

    The returned image has shape ``[T, H, W, C]`` where ``T=2`` (before,
    after). The mask has shape ``[H, W]``.
    """
    files = self.files[index]
    image1 = self._load_image(files["image1"])
    image2 = self._load_image(files["image2"])
    mask = self._load_target(files["mask"])
    sample = {"image": ops.stack([image1, image2]), "mask": mask}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/levircd.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.files)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/levircd.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    ncols = 3

    image1 = quantile_normalization(sample["image"][0])
    image2 = quantile_normalization(sample["image"][1])

    if "prediction" in sample:
        ncols += 1

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 5, 10))

    axs[0].imshow(ops.convert_to_numpy(image1))
    axs[0].axis("off")
    axs[1].imshow(ops.convert_to_numpy(image2))
    axs[1].axis("off")
    axs[2].imshow(ops.convert_to_numpy(sample["mask"]), cmap="gray", interpolation="none")
    axs[2].axis("off")

    if "prediction" in sample:
        axs[3].imshow(
            ops.convert_to_numpy(sample["prediction"]), cmap="gray", interpolation="none"
        )
        axs[3].axis("off")
        if show_titles:
            axs[3].set_title("Prediction")

    if show_titles:
        axs[0].set_title("Image 1")
        axs[1].set_title("Image 2")
        axs[2].set_title("Mask")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.levircd.LEVIRCDPlus

LEVIRCDPlus(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: LEVIRCDBase

LEVIR-CD+ dataset.

The LEVIR-CD+ <https://github.com/S2Looking/Dataset>__ dataset extends LEVIR-CD to 985 image pairs and is designed to be easier due to its urban locations and near-nadir angles.

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2107.09244

Source code in keras_climate/datasets/levircd.py

def __init__(
    self, root="data", split="train", transforms=None, download=False, checksum=True
):
    """Initialize a new LEVIR-CD base dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root directory
        checksum: if True, check the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    assert split in self.splits

    self.root = root
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    if download:
        self._download()

    if not self._check_integrity():
        raise DatasetNotFoundError(self)

    self.files = self._load_files(self.root, self.split)

keras_climate.datasets.loveda.LoveDA

LoveDA(root='data', split='train', scene=['urban', 'rural'], transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

LoveDA dataset.

The LoveDA <https://github.com/Junjue-Wang/LoveDA>__ dataset is a semantic segmentation dataset.

Dataset features:

• 2713 urban scene and 3274 rural scene HSR images, spatial resolution of 0.3m
• dataset comes with predefined train, validation, and test set
• dataset differentiates between 'rural' and 'urban' images

Dataset classes:

1. background
2. building
3. road
4. water
5. barren
6. forest
7. agriculture

No-data regions assigned with 0 and should be ignored.

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2110.08733

Initialize a new LoveDA dataset instance.

Args: root: root directory where dataset can be found split: one of "train", "val", or "test" scene: specify whether to load only 'urban', only 'rural' or both transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the checksum of the downloaded files (may be slow)

Raises: AssertionError: if split or scene arguments are invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/loveda.py

def __init__(
    self,
    root="data",
    split="train",
    scene=["urban", "rural"],
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new LoveDA dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val", or "test"
        scene: specify whether to load only 'urban', only 'rural' or both
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root directory
        checksum: if True, check the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if ``split`` or ``scene`` arguments are invalid
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    assert split in self.splits
    assert set(scene).intersection(set(self.scenes)), (
        "The possible scenes are 'rural' and/or 'urban'"
    )
    assert len(scene) <= 2, "There are no other scenes than 'rural' or 'urban'"

    self.root = root
    self.split = split
    self.scene = scene
    self.transforms = transforms
    self.checksum = checksum

    self.url = self.info_dict[self.split]["url"]
    self.filename = self.info_dict[self.split]["filename"]
    self.md5 = self.info_dict[self.split]["md5"]

    self.directory = os.path.join(self.root, split.capitalize())
    self.scene_paths = [
        os.path.join(self.directory, s.capitalize()) for s in self.scene
    ]

    if download:
        self._download()

    if not self._check_integrity():
        raise DatasetNotFoundError(self)

    self.files = self._load_files(self.scene_paths, self.split)

__getitem__

__getitem__(index)

Return an index within the dataset.

Returns: image and mask at that index with image of shape [H, W, 3] and mask of shape [H, W].

Source code in keras_climate/datasets/loveda.py

def __getitem__(self, index):
    """Return an index within the dataset.

    Returns:
        image and mask at that index with image of shape ``[H, W, 3]``
        and mask of shape ``[H, W]``.
    """
    files = self.files[index]
    image = self._load_image(files["image"])

    if self.split != "test":
        mask = self._load_target(files["mask"])
        sample = {"image": image, "mask": mask}
    else:
        sample = {"image": image}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of datapoints in the dataset.

Source code in keras_climate/datasets/loveda.py

def __len__(self):
    """Return the number of datapoints in the dataset."""
    return len(self.files)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/loveda.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    if self.split != "test":
        image, mask = sample["image"], sample["mask"]
        ncols = 2
    else:
        image = sample["image"]
        ncols = 1

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 10, 10))

    image_np = ops.convert_to_numpy(image).astype("uint8")

    if self.split != "test":
        axs[0].imshow(image_np)
        axs[0].axis("off")
        axs[1].imshow(ops.convert_to_numpy(mask))
        axs[1].axis("off")
        if show_titles:
            axs[0].set_title("Image")
            axs[1].set_title("Mask")
    else:
        axs.imshow(image_np)
        axs.axis("off")
        if show_titles:
            axs.set_title("Image")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.major_tom.MajorTOMEmbeddings

MajorTOMEmbeddings(root='data', transforms=None)

Bases: NonGeoDataset

Major TOM Embeddings dataset.

Major TOM <https://huggingface.co/Major-TOM>__ (Terrestrial Observation Metaset) is a standard for curating, sharing and combining large-scale EO datasets. This data loader provides access to the official embedding datasets created using Major TOM Core and several existing foundation models.

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2412.05600

Initialize a new MajorTOMEmbeddings instance.

Args: root: root directory where dataset can be found transforms: a function/transform that takes input sample and its target as entry and returns a transformed version

Raises: DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/major_tom.py

def __init__(self, root="data", transforms=None):
    """Initialize a new MajorTOMEmbeddings instance.

    Args:
        root: root directory where dataset can be found
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version

    Raises:
        DatasetNotFoundError: If dataset is not found.
    """
    self.root = root
    self.transforms = transforms

    try:
        self.data = gpd.read_parquet(root)
    except (FileNotFoundError, ValueError):
        raise DatasetNotFoundError(self)

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/major_tom.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.data)

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/major_tom.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    row = self.data.iloc[index]
    t = pd.Timestamp(row["timestamp"])

    sample = {
        "embedding": ops.convert_to_tensor(np.asarray(row["embedding"])),
        "x": ops.convert_to_tensor(np.asarray(row["centre_lon"])),
        "y": ops.convert_to_tensor(np.asarray(row["centre_lat"])),
        "t": ops.convert_to_tensor(np.asarray(t.timestamp())),
    }

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/major_tom.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    fig, ax = plt.subplots()
    ax.plot(ops.convert_to_numpy(sample["embedding"]))

    if show_titles:
        x = float(ops.convert_to_numpy(sample["x"]))
        y = float(ops.convert_to_numpy(sample["y"]))
        t = pd.Timestamp.fromtimestamp(float(ops.convert_to_numpy(sample["t"])))
        ax.set_title(rf"{y:0.3f}°N, {x:0.3f}°W, {t}")

    if suptitle is not None:
        plt.suptitle(suptitle)

    fig.tight_layout()
    return fig

keras_climate.datasets.mapinwild.MapInWild

MapInWild(root='data', modality=None, split='train', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

MapInWild dataset.

The MapInWild <https://ieeexplore.ieee.org/document/10089830>__ dataset is curated for the task of wilderness mapping on a pixel-level. MapInWild is a multi-modal dataset and comprises various geodata acquired and formed from different RS sensors over 1018 locations: dual-pol Sentinel-1, four-season Sentinel-2 with 10 bands, ESA WorldCover map, and Visible Infrared Imaging Radiometer Suite NightTime Day/Night band. The dataset consists of 8144 images with the shape of 1920 x 1920 pixels. The images are weakly annotated from the World Database of Protected Areas (WDPA).

If you use this dataset in your research, please cite the following paper:

• https://ieeexplore.ieee.org/document/10089830

Initialize a new MapInWild dataset instance.

Args: root: root directory where dataset can be found modality: the modality to download. Choose from: "mask", "esa_wc", "viirs", "s1", "s2_temporal_subset", "s2_[season]". split: one of "train", "validation", or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: AssertionError: if split argument is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/mapinwild.py

def __init__(
    self,
    root="data",
    modality=None,
    split="train",
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new MapInWild dataset instance.

    Args:
        root: root directory where dataset can be found
        modality: the modality to download. Choose from: "mask", "esa_wc",
            "viirs", "s1", "s2_temporal_subset", "s2_[season]".
        split: one of "train", "validation", or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    if modality is None:
        modality = ["mask", "esa_wc", "viirs", "s2_summer"]
    assert split in ["train", "validation", "test"]

    self.checksum = checksum
    self.root = root
    self.transforms = transforms
    self.modality = modality
    self.download = download

    modality.append("split_IDs")
    for mode in modality:
        for modality_link in self.modality_urls[mode]:
            modality_url = os.path.join(self.url, modality_link)
            self._verify(
                url=modality_url, sha256=self.sha256s[os.path.split(modality_link)[-1]]
            )

        if (
            download
            and mode not in os.listdir(self.root)
            and len(self.modality_urls[mode]) == 2
        ):
            self._merge_parts(mode)

    # Masks will be loaded separately in __getitem__
    if "mask" in self.modality:
        self.modality.remove("mask")

    if "split_IDs" in self.modality:
        self.modality.remove("split_IDs")

    if os.path.exists(os.path.join(self.root, "split_IDs.csv")):
        split_dataframe = pd.read_csv(os.path.join(self.root, "split_IDs.csv"))
        ids = split_dataframe[split].dropna().values.tolist()
        self.ids = list(map(int, ids))

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/mapinwild.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    list_modalities = []
    id = self.ids[index]

    mask = self._load_raster(id, "mask")
    mask = np.where(mask != 0, 1, mask)

    for mode in self.modality:
        mode = mode.upper() if mode in ["esa_wc", "viirs"] else mode
        data = self._load_raster(id, mode)
        list_modalities.append(data)

    image = np.concatenate(list_modalities, axis=-1)

    sample = {
        "image": ops.convert_to_tensor(image),
        "mask": ops.cast(ops.convert_to_tensor(np.squeeze(mask, axis=-1)), "int64"),
    }

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/mapinwild.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.ids)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/mapinwild.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    modality_channels = defaultdict(lambda: 10, {"viirs": 1, "esa_wc": 1, "s1": 2})

    start_idx = 0
    split_images = {}

    image = ops.convert_to_numpy(sample["image"])
    for modality in self.modality:
        end_idx = start_idx + modality_channels[modality]
        split_images[modality] = image[:, :, start_idx:end_idx]
        start_idx = end_idx

    mask = ops.convert_to_numpy(sample["mask"]).squeeze()
    color_mask = self._convert_to_color(mask, cmap=self.mask_cmap)

    num_subplots = len(split_images) + 1
    showing_predictions = "prediction" in sample
    if showing_predictions:
        num_subplots += 1

    fig, axs = plt.subplots(1, num_subplots, figsize=(num_subplots * 4, 5))

    for i, (modality, img) in enumerate(split_images.items()):
        ax = axs[i]
        img = img.squeeze()
        if modality.startswith("s2"):
            img = img[:, :, [4, 3, 2]]
        if modality == "esa_wc":
            img = self._convert_to_color(img, cmap=self.wc_cmap)
        if modality == "s1":
            img = img[:, :, 0]

        if modality != "esa_wc":
            img = ops.convert_to_numpy(quantile_normalization(img))

        ax.imshow(img)
        if show_titles:
            ax.set_title(modality)
        ax.axis("off")

    axs[len(split_images)].imshow(color_mask)
    if show_titles:
        axs[len(split_images)].set_title("Annotation")
    axs[len(split_images)].axis("off")

    if showing_predictions:
        prediction = ops.convert_to_numpy(sample["prediction"]).squeeze()
        color_predictions = self._convert_to_color(prediction, cmap=self.mask_cmap)
        axs[-1].imshow(color_predictions, vmin=0, vmax=1, interpolation="none")
        if show_titles:
            axs[-1].set_title("Prediction")
        axs[-1].axis("off")

    plt.tight_layout()
    return fig

keras_climate.datasets.mdas.MDAS

MDAS(root='data', subareas=None, modalities=None, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

MDAS dataset.

The MDAS <https://essd.copernicus.org/articles/15/113/2023/>__ multimodal dataset is a comprehensive dataset for the city of Augsburg, Germany, collected on 7th May 2018. It includes SAR, multispectral, hyperspectral, DSM, and GIS data, providing comprehensive options for data fusion research.

Dataset format:

• 3K_RGB.tif (Shape: (15000, 18000, 4)px, Data Type: uint8)
• 3K_dsm.tif (Shape: (10000, 12000, 1)px, Data Type: float32)
• HySpex.tif (Shape: (1364, 1636, 368)px, Data Type: int16)
• EeteS_EnMAP_2dot2m.tif (Shape: (1364, 1636, 242)px, Data Type: float32)
• EeteS_EnMAP_10m.tif (Shape: (300, 360, 242)px, Data Type: uint16)
• EeteS_EnMAP_30m.tif (Shape: (100, 120, 242)px, Data Type: uint16)
• EeteS_Sentinel_2_10m.tif (Shape: (300, 360, 4)px, Data Type: uint16)
• Sentinel_2.tif (Shape: (300, 360, 12)px, Data Type: uint16)
• Sentinel_1.tif (Shape: (300, 360, 2)px, Data Type: float32)
• osm_buildings.tif (Shape: (1364, 1636, 1)px, Data Type: uint8)
• osm_landuse.tif (Shape: (1364, 1636, 1)px, Data Type: float64)
• osm_water.tif (Shape: (1364, 1636, 1)px, Data Type: float64)

If you use this dataset in your research, please cite the following paper:

• https://essd.copernicus.org/articles/15/113/2023/

Initialize a new MDAS dataset instance.

Args: root: root directory where the dataset should be stored. subareas: the subareas to load. Options are 'sub_area_1', 'sub_area_2', 'sub_area_3'. modalities: the modalities to load. transforms: a function/transform that takes in a dictionary and returns a transformed version. download: if True, download dataset and store it in the root directory checksum: if True, check the integrity of the dataset after download.

Raises: AssertionError: If the subareas or modalities are not valid. DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/mdas.py

def __init__(
    self,
    root="data",
    subareas=None,
    modalities=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new MDAS dataset instance.

    Args:
        root: root directory where the dataset should be stored.
        subareas: the subareas to load. Options are 'sub_area_1',
            'sub_area_2', 'sub_area_3'.
        modalities: the modalities to load.
        transforms: a function/transform that takes in a dictionary and
            returns a transformed version.
        download: if True, download dataset and store it in the root directory
        checksum: if True, check the integrity of the dataset after download.

    Raises:
        AssertionError: If the subareas or modalities are not valid.
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    if modalities is None:
        modalities = ["3K_RGB", "HySpex", "Sentinel_2"]
    if subareas is None:
        subareas = ["sub_area_1"]
    self.root = root
    self.download = download
    assert all(sub in self.valid_subareas for sub in subareas), (
        f"Subareas must be one of {self.valid_subareas}"
    )
    self.subareas = subareas
    assert all(mod in self.valid_modalities for mod in modalities), (
        f"Modalities must be one of {self.valid_modalities}"
    )
    self.modalities = modalities
    self.transforms = transforms
    self.checksum = checksum

    self._verify()
    self.files = self._load_files()

__len__

__len__()

Return the number of samples in the dataset.

Source code in keras_climate/datasets/mdas.py

def __len__(self):
    """Return the number of samples in the dataset."""
    return len(self.files)

__getitem__

__getitem__(index)

Return the dataset sample at the given index.

Source code in keras_climate/datasets/mdas.py

def __getitem__(self, index):
    """Return the dataset sample at the given index."""
    sample_files = self.files[index]
    sample = {}
    for modality, path in sample_files.items():
        if "osm" in modality:
            sample[f"{modality}_mask"] = ops.cast(self._load_image(path), "int64")
        else:
            sample[f"{modality}_image"] = self._load_image(path)

    if self.transforms:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/mdas.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    ncols = len(sample)
    fig, axs = plt.subplots(1, ncols, figsize=(5 * ncols, 5))

    if ncols == 1:
        axs = [axs]

    for idx, (key, data) in enumerate(sample.items()):
        data = ops.convert_to_numpy(data)
        match key:
            case "3K_RGB_image":
                img = data[:, :, :3] / 255.0
                axs[idx].imshow(img)
            case "3K_DSM_image":
                img = data.squeeze(-1)
                axs[idx].imshow(img, cmap="gray")
            case "EeteS_EnMAP_10m_image" | "EeteS_EnMAP_30m_image":
                img = data[:, :, self.enmap_rgb_band_idx] / 10000.0
                axs[idx].imshow(img)
            case "EeteS_Sentinel_2_10m_image":
                img = data[:, :, self.sentinel_2_rgb_band_idx] / 10000.0
                axs[idx].imshow(img)
            case "Sentinel_1_image":
                img = data[:, :, 0].clip(0, 1)
                axs[idx].imshow(img)
            case "Sentinel_2_image":
                img = data[:, :, self.sentinel_2_rgb_band_idx] / 10000.0
                axs[idx].imshow(img)
            case "HySpex_image":
                img = data[:, :, self.hyspex_rgb_band_idx] / 15000.0
                axs[idx].imshow(img)
            case "osm_landuse_mask":
                img = data.squeeze(-1)
                cmap = ListedColormap([plt.get_cmap("tab20")(i) for i in range(20)])
                im = axs[idx].imshow(img, cmap=cmap)
                cbar = plt.colorbar(im, ax=axs[idx], ticks=range(19))
                cbar.ax.set_yticklabels(
                    [self.landuse_class_names[i] for i in range(19)]
                )
            case "osm_buildings_mask":
                img = data.squeeze(-1)
                axs[idx].imshow(img, cmap="gray")
            case "osm_water_mask":
                img = data.squeeze(-1)
                axs[idx].imshow(img, cmap="Blues")

        axs[idx].axis("off")
        if show_titles:
            axs[idx].set_title(key)

    if suptitle:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.meta_chm.MetaCHM

MetaCHM(transforms=None, cache=True)

Bases: RasterDataset

Meta Canopy Height Map (CHM) v2 dataset.

The Meta CHMv2 (DINOv3) global canopy height map <https://ai.meta.com/ai-for-good/datasets/canopy-height-maps/>__ is a global, ~1.19 m/pixel estimate of tree canopy height derived from high-resolution satellite imagery.

Dataset features:

• canopy height in meters at ~1.19 m/pixel (uint8, 0 = no canopy or no data)
• 213,109 tiles on a zoom-10 Web Mercator (EPSG:3857) grid

Dataset format:

• a STAC GeoParquet index with each tile's geometry, acquisition date, and COG URL
• single-channel uint8 Cloud-Optimized GeoTIFFs whose pixel values are the canopy height in meters (0 = no canopy or no data)

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2603.06382

Initialize a new MetaCHM instance.

Args: transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling

Source code in keras_climate/datasets/meta_chm.py

def __init__(self, transforms=None, cache=True):
    """Initialize a new MetaCHM instance.

    Args:
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
    """
    self.paths = self.url
    self.transforms = transforms
    self.cache = cache
    self.bands = self.all_bands
    self.band_indexes = None
    self.time_series = False

    request = urllib.request.Request(self.url, headers={"User-Agent": "keras_climate"})
    with urllib.request.urlopen(request) as response:
        buffer = io.BytesIO(response.read())
    columns = ["geometry", "assets", "datetime"]
    gdf = gpd.read_parquet(buffer, columns=columns)
    gdf.to_crs("EPSG:3857", inplace=True)
    filepaths = (
        gdf["assets"]
        .map(lambda asset: asset["chm"]["href"])
        .str.replace(
            "s3://dataforgood-fb-data/",
            "https://dataforgood-fb-data.s3.amazonaws.com/",
            regex=False,
        )
    )
    datetimes = gdf["datetime"].dt.tz_localize(None)
    index = pd.IntervalIndex.from_arrays(
        datetimes, datetimes + pd.Timedelta(days=1), closed="both", name="datetime"
    )
    self.index = gpd.GeoDataFrame(
        {"filepath": filepaths.to_numpy()},
        index=index,
        geometry=gdf.geometry.to_numpy(),
        crs=gdf.crs,
    )

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/meta_chm.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    mask = ops.convert_to_numpy(sample["mask"])
    ncols = 1

    showing_prediction = "prediction" in sample
    if showing_prediction:
        ncols = 2

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 4, 4))
    axs = [axs] if ncols == 1 else axs

    im = axs[0].imshow(mask, cmap="YlGn", vmin=0, vmax=40)
    axs[0].axis("off")
    fig.colorbar(im, ax=axs[0], fraction=0.046, pad=0.04, label="Canopy height (m)")
    if show_titles:
        axs[0].set_title("Canopy Height")

    if showing_prediction:
        pred = ops.convert_to_numpy(sample["prediction"])
        im = axs[1].imshow(pred, cmap="YlGn", vmin=0, vmax=40)
        axs[1].axis("off")
        fig.colorbar(
            im, ax=axs[1], fraction=0.046, pad=0.04, label="Canopy height (m)"
        )
        if show_titles:
            axs[1].set_title("Prediction")

    if suptitle is not None:
        fig.suptitle(suptitle)
    return fig

keras_climate.datasets.millionaid.MillionAID

MillionAID(root='data', task='multi-class', split='train', transforms=None, checksum=True)

Bases: NonGeoDataset

Million-AID Dataset.

The MillionAID <https://captain-whu.github.io/DiRS/>_ dataset consists of one million aerial images from Google Earth Engine that offers either a multi-class learning task with 51 classes or a multi-label learning task with 73 different possible labels. For more details please consult the accompanying paper <https://ieeexplore.ieee.org/document/9393553>_.

Dataset format:

• images are three-channel jpg

If you use this dataset in your research, please cite the following paper:

• https://ieeexplore.ieee.org/document/9393553

Initialize a new MillionAID dataset instance.

Args: root: root directory where dataset can be found task: type of task, either "multi-class" or "multi-label" split: train or test split transforms: a function/transform that takes input sample and its target as entry and returns a transformed version checksum: if True, check the checksum of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/millionaid.py

def __init__(
    self,
    root="data",
    task="multi-class",
    split="train",
    transforms=None,
    checksum=True,
):
    """Initialize a new MillionAID dataset instance.

    Args:
        root: root directory where dataset can be found
        task: type of task, either "multi-class" or "multi-label"
        split: train or test split
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        checksum: if True, check the checksum of the downloaded files
            (may be slow)

    Raises:
        DatasetNotFoundError: If dataset is not found.
    """
    self.root = root
    self.transforms = transforms
    self.checksum = checksum
    assert task in self.tasks
    assert split in self.splits
    self.task = task
    self.split = split

    self._verify()

    self.files = self._load_files(self.root)

    self.classes = sorted({cls for f in self.files for cls in f["label"]})
    self.class_to_idx = {c: i for i, c in enumerate(self.classes)}

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/millionaid.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.files)

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/millionaid.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    files = self.files[index]
    image = self._load_image(files["image"])
    cls_label = [self.class_to_idx[label] for label in files["label"]]
    label = ops.convert_to_tensor(np.array(cls_label, dtype="int64"))
    sample = {"image": image, "label": label}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/millionaid.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"]).astype("uint8")
    labels = [self.classes[int(label)] for label in ops.convert_to_numpy(sample["label"])]

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction_labels = [
            self.classes[int(label)] for label in ops.convert_to_numpy(sample["prediction"])
        ]

    fig, ax = plt.subplots(figsize=(4, 4))
    ax.imshow(image)
    ax.axis("off")
    if show_titles:
        title = f"Label: {labels}"
        if showing_predictions:
            title += f"\nPrediction: {prediction_labels}"
        ax.set_title(title)

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.mmearth.MMEarth

MMEarth(root='data', subset='MMEarth', modalities=all_modalities, modality_bands=None, normalization_mode='z-score', transforms=None)

Bases: NonGeoDataset

MMEarth dataset.

There are three different versions of the dataset, that vary in image size and the number of tiles:

• MMEarth: 128x128 px, 1.2M tiles, 579 GB
• MMEarth64: 64x64 px, 1.2M tiles, 162 GB
• MMEarth100k: 128x128 px, 100K tiles, 48 GB

The dataset consists of 12 modalities: Aster, Biome, ETH Canopy Height, Dynamic World, Ecoregion, ERA5, ESA World Cover, Sentinel-1, Sentinel-2, Geolocation, and Date. Additionally, there are three masks available as modalities: Sentinel-2 Cloudmask, Sentinel-2 Cloud probability, and Sentinel-2 SCL.

Dataset format:

• Dataset in single HDF5 file
• JSON files for band statistics, splits, and tile information

For additional information, as well as bash scripts to download the data, please refer to the official repository <https://github.com/vishalned/MMEarth-data?tab=readme-ov-file#data-download>_.

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2405.02771

.. note::

This dataset requires the following additional library to be installed:

• h5py <https://pypi.org/project/h5py/>_ to load the dataset

Initialize the MMEarth dataset.

Args: root: root directory where dataset can be found subset: one of "MMEarth", "MMEarth64", or "MMEarth100k" modalities: list of modalities to load modality_bands: dictionary of modality bands normalization_mode: one of "z-score" or "min-max" transforms: a function/transform that takes input sample dictionary and returns a transformed version

Raises: AssertionError: if normalization_mode or subset is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/mmearth.py

def __init__(
    self,
    root="data",
    subset="MMEarth",
    modalities=all_modalities,
    modality_bands=None,
    normalization_mode="z-score",
    transforms=None,
):
    """Initialize the MMEarth dataset.

    Args:
        root: root directory where dataset can be found
        subset: one of "MMEarth", "MMEarth64", or "MMEarth100k"
        modalities: list of modalities to load
        modality_bands: dictionary of modality bands
        normalization_mode: one of "z-score" or "min-max"
        transforms: a function/transform that takes input sample
            dictionary and returns a transformed version

    Raises:
        AssertionError: if *normalization_mode* or *subset* is invalid
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    lazy_import("h5py")

    assert normalization_mode in self.norm_modes, (
        f"Invalid normalization mode: {normalization_mode}, please choose "
        f"from {self.norm_modes}"
    )
    assert subset in self.subsets, (
        f"Invalid dataset version: {subset}, please choose from {self.subsets}"
    )

    self._validate_modalities(modalities)
    self.modalities = modalities
    if modality_bands is None:
        modality_bands = {
            modality: self.all_modality_bands[modality] for modality in modalities
        }
    self._validate_modality_bands(modality_bands)
    self.modality_bands = modality_bands

    self.root = root
    self.subset = subset
    self.normalization_mode = normalization_mode
    self.split = "train"
    self.transforms = transforms

    self.dataset_filename = f"{self.filenames[subset]}.h5"
    self.band_stats_filename = f"{self.filenames[subset]}_band_stats.json"
    self.splits_filename = f"{self.filenames[subset]}_splits.json"
    self.tile_info_filename = f"{self.filenames[subset]}_tile_info.json"

    self._verify()

    self.indices = self._load_indices()
    self.band_stats = self._load_normalization_stats()
    self.tile_info = self._load_tile_info()

__getitem__

__getitem__(index)

Return a sample from the dataset.

Normalization is applied to the data with chosen normalization_mode. In addition to the modalities, the sample contains the following raw metadata: lat, lon, date, tile_id.

Source code in keras_climate/datasets/mmearth.py

def __getitem__(self, index):
    """Return a sample from the dataset.

    Normalization is applied to the data with chosen ``normalization_mode``.
    In addition to the modalities, the sample contains the following raw
    metadata: lat, lon, date, tile_id.
    """
    ds_index = self.indices[index]

    sample = self._retrieve_sample(ds_index)

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the length of the dataset.

Source code in keras_climate/datasets/mmearth.py

def __len__(self):
    """Return the length of the dataset."""
    return len(self.indices)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset as shown in fig. 2 from https://arxiv.org/pdf/2405.02771.

Source code in keras_climate/datasets/mmearth.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset as shown in fig. 2 from https://arxiv.org/pdf/2405.02771."""
    import matplotlib.pyplot as plt

    color_map = {
        "esa_worldcover": {
            0: [0, 100, 0],
            1: [255, 187, 34],
            2: [255, 255, 76],
            3: [240, 150, 255],
            4: [250, 0, 0],
            5: [180, 180, 180],
            6: [240, 240, 240],
            7: [0, 100, 200],
            8: [0, 150, 160],
            9: [0, 207, 117],
            10: [250, 230, 160],
            255: [0, 0, 0],
        },
        "dynamic_world": {
            0: [65, 155, 223],
            1: [57, 125, 73],
            2: [136, 176, 83],
            3: [122, 135, 198],
            4: [228, 150, 53],
            5: [223, 195, 90],
            6: [196, 40, 27],
            7: [165, 155, 143],
            8: [179, 159, 225],
        },
    }

    images = []
    titles = []

    keys_to_plot = [
        "image_sentinel2",
        "image_sentinel1_asc",
        "image_aster",
        "mask_esa_worldcover",
        "mask_dynamic_world",
        "image_canopy_height_eth",
    ]

    for key in keys_to_plot:
        if key not in sample:
            continue
        val = ops.convert_to_numpy(sample[key])
        modalities_name = key.split("_", 1)[1]
        match modalities_name:
            case "sentinel2":
                norm_img = ops.convert_to_numpy(
                    quantile_normalization(val[:, :, [3, 2, 1]])
                )
                images.append(norm_img)
                titles.append("Sentinel-2 RGB")
            case "esa_worldcover":
                tensor = val.squeeze(-1)
                rgb_image = np.zeros((*tensor.shape, 3), dtype=np.uint8)
                for value, color in color_map[modalities_name].items():
                    rgb_image[tensor == value] = color

                images.append(rgb_image)
                titles.append(modalities_name.replace("_", " ").title())
            case "dynamic_world":
                tensor = val.squeeze(-1)
                rgb_image = np.zeros((*tensor.shape, 3), dtype=np.uint8)
                for value, color in color_map[modalities_name].items():
                    rgb_image[tensor == value] = color

                images.append(rgb_image)
                titles.append(modalities_name.replace("_", " ").title())
            case _:
                band_val = val[:, :, 0]
                norm_img = ops.convert_to_numpy(quantile_normalization(band_val))
                images.append(norm_img)

                modalities_name = key.split("_", 1)[1]
                titles.append(modalities_name.replace("_", " ").title())

    fig, ax = plt.subplots(1, len(images) or 1, figsize=(12, 4))
    if len(images) == 1:
        ax = [ax]

    for i, (image, title) in enumerate(zip(images, titles)):
        ax[i].imshow(image)
        ax[i].axis("off")

        if show_titles:
            title_words = title.split(" ")
            title_word_len = len(title_words)
            if title_word_len > 2:
                title = (
                    str.join(" ", title_words[:2]) + "\n" + str.join(" ", title_words[2:])
                )
            ax[i].set_title(title)

    if suptitle is not None:
        plt.suptitle(suptitle)

    plt.tight_layout()

    return fig

keras_climate.datasets.mmflood.MMFlood

MMFlood(root='data', crs=None, res=None, split='train', include_dem=False, include_hydro=False, transforms=None, download=False, checksum=True, cache=False, time_series=False)

Bases: IntersectionDataset

MMFlood dataset.

MMFlood <https://huggingface.co/datasets/links-ads/mmflood>__ dataset is a multimodal flood delineation dataset. Sentinel-1 data is matched with masks and DEM data for all available tiles. If hydrography maps are loaded, only a subset of the dataset is loaded, since only 1,012 Sentinel-1 tiles have a corresponding hydrography map. Some Sentinel-1 tiles have missing data, which are automatically set to 0. Corresponding pixels in masks are set to 255 and should be ignored in performance computation.

Dataset features:

• 1,748 Sentinel-1 tiles of varying pixel dimensions
• multimodal dataset
• 95 flood events from 42 different countries
• includes DEMs
• includes hydrography maps (available for 1,012 tiles out of 1,748)
• flood delineation maps (ground truth) is obtained from Copernicus EMS

Dataset classes:

1. no flood
2. flood

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1109/ACCESS.2022.3205419

Initialize a new MMFlood dataset instance.

Args: root: root directory where dataset can be found crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS in (xres, yres) format split: train/val/test split to load include_dem: If True, DEM data is concatenated after Sentinel-1 bands. include_hydro: If True, hydrography data is concatenated as last channel. Only a smaller subset of the original dataset is loaded in this case. transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow) cache: if True, cache file handle to speed up repeated sampling time_series: if True, stack data along the time series dimension

Raises: DatasetNotFoundError: If dataset is not found and download is False. AssertionError: If split is invalid.

Source code in keras_climate/datasets/mmflood.py

def __init__(
    self,
    root="data",
    crs=None,
    res=None,
    split="train",
    include_dem=False,
    include_hydro=False,
    transforms=None,
    download=False,
    checksum=True,
    cache=False,
    time_series=False,
):
    """Initialize a new MMFlood dataset instance.

    Args:
        root: root directory where dataset can be found
        crs: CRS to warp to (defaults to the CRS of the first file
            found)
        res: resolution of the dataset in units of CRS in (xres, yres)
            format
        split: train/val/test split to load
        include_dem: If True, DEM data is concatenated after Sentinel-1
            bands.
        include_hydro: If True, hydrography data is concatenated as last
            channel. Only a smaller subset of the original dataset is
            loaded in this case.
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
        AssertionError: If *split* is invalid.
    """
    assert split in self._splits

    self.root = root
    self.split = split
    self.include_dem = include_dem
    self.include_hydro = include_hydro
    self.transforms = transforms
    self.download = download
    self.checksum = checksum
    # Verify integrity of the dataset
    self._verify()
    self.metadata_df = pd.read_json(
        os.path.join(self.root, self.metadata["metadata_file"])
    ).transpose()

    split_subfolders = self.metadata_df[
        self.metadata_df["subset"] == self.split
    ].index.tolist()
    self.image = MMFloodComponent(
        split_subfolders,
        "s1_raw",
        root,
        crs,
        res,
        cache=cache,
        time_series=time_series,
    )
    if include_dem:
        dem = MMFloodComponent(
            split_subfolders,
            "DEM",
            root,
            crs,
            res,
            cache=cache,
            time_series=time_series,
        )
        self.image = IntersectionDataset(
            self.image, dem, collate_fn=_concat_channels
        )
        self.image.index = dem.index
    if include_hydro:
        hydro = MMFloodComponent(
            split_subfolders,
            "hydro",
            root,
            crs,
            res,
            cache=cache,
            time_series=time_series,
        )
        self.image = IntersectionDataset(
            self.image, hydro, collate_fn=_concat_channels
        )
        self.image.index = hydro.index
    self.mask = MMFloodComponent(
        split_subfolders,
        "mask",
        root,
        crs,
        res,
        cache=cache,
        time_series=time_series,
    )

    super().__init__(
        self.image, self.mask, collate_fn=_concat_channels, transforms=transforms
    )
    self.index = self.image.index

__getitem__

__getitem__(index)

Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

Source code in keras_climate/datasets/mmflood.py

def __getitem__(self, index):
    """Retrieve input, target, and/or metadata indexed by spatiotemporal slice."""
    data = super().__getitem__(index)

    image_np = ops.convert_to_numpy(data["image"])
    missing_data = np.isnan(image_np).any(axis=-1)
    image_np = np.where(missing_data[..., None], 0, image_np)

    mask_np = ops.convert_to_numpy(data["mask"]).astype("int64")
    mask_np = np.where(missing_data, self._ignore_index, mask_np)

    data["image"] = ops.convert_to_tensor(image_np.astype("float32"))
    data["mask"] = ops.convert_to_tensor(mask_np)
    return data

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/mmflood.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    show_mask = "mask" in sample
    image = ops.convert_to_numpy(sample["image"])[..., [0, 1]]
    ncols = 1
    show_predictions = "prediction" in sample
    axs_idx = 1
    if self.include_dem:
        dem_idx = -2 if self.include_hydro else -1
        dem = ops.convert_to_numpy(sample["image"])[..., dem_idx]
        ncols += 1
    if self.include_hydro:
        hydro = ops.convert_to_numpy(sample["image"])[..., -1]
        ncols += 1
    if show_mask:
        mask = ops.convert_to_numpy(sample["mask"]).copy()
        # Set ignore_index values to 0
        mask[mask == self._ignore_index] = 0
        ncols += 1
    if show_predictions:
        pred = ops.convert_to_numpy(sample["prediction"])
        ncols += 1

    # Compute False Color image, from Sentinel1 plot function
    co_polarization = image[..., 0]  # transmit == receive
    cross_polarization = image[..., 1]  # transmit != receive
    ratio = co_polarization / cross_polarization

    # https://gis.stackexchange.com/a/400780/123758
    co_polarization = np.clip(co_polarization / 0.3, a_min=0, a_max=1)
    cross_polarization = np.clip(cross_polarization / 0.05, a_min=0, a_max=1)
    ratio = np.clip(ratio / 25, a_min=0, a_max=1)

    image = np.stack((co_polarization, cross_polarization, ratio), axis=-1)

    # Generate the figure
    fig, axs = plt.subplots(ncols=ncols, figsize=(4 * ncols, 4))
    axs_list = axs if ncols > 1 else [axs]
    axs_list[0].imshow(image)
    axs_list[0].axis("off")
    axs_idx = 1
    if self.include_dem:
        axs_list[axs_idx].imshow(dem, cmap="gray")
        axs_list[axs_idx].axis("off")
        axs_idx += 1
    if self.include_hydro:
        axs_list[axs_idx].imshow(hydro, cmap="gray")
        axs_list[axs_idx].axis("off")
        axs_idx += 1
    if show_mask:
        axs_list[axs_idx].imshow(mask, cmap="gray")
        axs_list[axs_idx].axis("off")
        axs_idx += 1
    if show_predictions:
        axs_list[axs_idx].imshow(pred, cmap="gray")
        axs_list[axs_idx].axis("off")

    if show_titles:
        axs_list[0].set_title("Image")
        axs_idx = 1
        if self.include_dem:
            axs_list[axs_idx].set_title("DEM")
            axs_idx += 1
        if self.include_hydro:
            axs_list[axs_idx].set_title("Hydrography Map")
            axs_idx += 1
        if show_mask:
            axs_list[axs_idx].set_title("Mask")
            axs_idx += 1
        if show_predictions:
            axs_list[axs_idx].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.mmflood.MMFloodComponent

MMFloodComponent(subfolders, content, root='data', crs=None, res=None, transforms=None, cache=False, time_series=False)

Bases: RasterDataset

Base component for MMFlood dataset.

Initialize MMFloodComponent dataset instance.

Args: subfolders: list of directories to be loaded content: specifies which component to load, one of "s1_raw", "DEM", "hydro", "mask" root: root directory where dataset can be found crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS in (xres, yres) format transforms: a function/transform that takes input sample and its target as entry and returns a transformed version cache: if True, cache file handle to speed up repeated sampling time_series: if True, stack data along the time series dimension

Source code in keras_climate/datasets/mmflood.py

def __init__(
    self,
    subfolders,
    content,
    root="data",
    crs=None,
    res=None,
    transforms=None,
    cache=False,
    time_series=False,
):
    """Initialize MMFloodComponent dataset instance.

    Args:
        subfolders: list of directories to be loaded
        content: specifies which component to load, one of "s1_raw",
            "DEM", "hydro", "mask"
        root: root directory where dataset can be found
        crs: CRS to warp to (defaults to the CRS of the first file
            found)
        res: resolution of the dataset in units of CRS in (xres, yres)
            format
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series
            dimension
    """
    self.content = content
    self.is_image = content != "mask"
    paths = []
    for s in subfolders:
        paths += glob(os.path.join(root, "**", f"{s}*-*", self.content, "*.tif"))
    paths = sorted(paths)
    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

keras_climate.datasets.naip.NAIP

NAIP(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset

National Agriculture Imagery Program (NAIP) dataset.

The National Agriculture Imagery Program (NAIP) <https://catalog.data.gov/dataset/national-agriculture-imagery-program-naip>_ acquires aerial imagery during the agricultural growing seasons in the continental U.S. A primary goal of the NAIP program is to make digital ortho photography available to governmental agencies and the public within a year of acquisition.

NAIP is administered by the USDA's Farm Service Agency (FSA) through the Aerial Photography Field Office in Salt Lake City. This "leaf-on" imagery is used as a base layer for GIS programs in FSA's County Service Centers, and is used to maintain the Common Land Unit (CLU) boundaries.

If you use this dataset in your research, please cite it using the following format:

• https://www.fisheries.noaa.gov/inport/item/49508/citation

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new RasterDataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a mosaic
            (``[H, W, C]``). For mask-style datasets (``is_image=False``),
            single-band data has the channel dimension squeezed.
    """
    self.paths = paths
    self.bands = bands or self.all_bands
    self.transforms = transforms
    self.cache = cache
    self.time_series = time_series

    if self.all_bands:
        assert set(self.bands) <= set(self.all_bands)

    filename_regex = re.compile(self.filename_regex, re.VERBOSE)
    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        match = re.match(filename_regex, os.path.basename(filepath))
        if match is not None:
            vrt = None
            try:
                vrt = self._load_warp_file(filepath=filepath, crs=crs)
                if self.cmap is None:
                    try:
                        colors = np.array(list(vrt.colormap(1).values())) / 255
                        self.cmap = ListedColormap(colors)
                    except ValueError:
                        pass
                if crs is None:
                    with rasterio.Env(OSR_WKT_FORMAT="WKT2_2018"):
                        crs = PROJ_CRS.from_user_input(vrt.crs)
                footprint = self.footprint_from_datasource(vrt)
                if footprint is None:
                    footprint = shapely.box(*vrt.bounds)
                geometries.append(footprint)
                if res is None:
                    res = vrt.res
            except rasterio.errors.RasterioIOError:
                continue
            else:
                filepaths.append(filepath)
                mint, maxt = self._filepath_to_timestamp(filepath)
                datetimes.append((mint, maxt))
            finally:
                if vrt is not None:
                    vrt.close()

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if not self.separate_files:
        self.band_indexes = None
        if self.bands:
            if self.all_bands:
                self.band_indexes = [
                    self.all_bands.index(i) + 1 for i in self.bands
                ]
            else:
                msg = (
                    f"{self.__class__.__name__} is missing an `all_bands` "
                    "attribute, so `bands` cannot be specified."
                )
                raise AssertionError(msg)

    if res is not None:
        if isinstance(res, (int, float)):
            res = (res, res)

        self._res = res

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/naip.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"])[..., 0:3]

    fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(4, 4))

    ax.imshow(image)
    ax.axis("off")
    if show_titles:
        ax.set_title("Image")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.nasa_marine_debris.NASAMarineDebris

NASAMarineDebris(root='data', transforms=None, download=False)

Bases: NonGeoDataset

NASA Marine Debris dataset.

The NASA Marine Debris <https://source.coop/nasa/marine-debris>__ dataset is a dataset for detection of floating marine debris in satellite imagery.

Dataset features:

• 707 patches with 3 m per pixel resolution (256x256 px)
• three spectral bands - RGB
• 1 object class: marine_debris
• images taken by Planet Labs PlanetScope satellites
• imagery taken from 2016-2019 from coasts of Greece, Honduras, and Ghana

Dataset format:

• images are three-channel geotiffs in uint8 format
• labels are numpy files (.npy) containing bounding box (xyxy) coordinates
• additional: images in jpg format and labels in geojson format

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.34911/rdnt.9r6ekg

.. note::

This dataset requires the following additional library to be installed:

• azcopy <https://github.com/Azure/azure-storage-azcopy>_: to download the dataset from Source Cooperative.

Initialize a new NASA Marine Debris Dataset instance.

Args: root: root directory where dataset can be found transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/nasa_marine_debris.py

def __init__(self, root="data", transforms=None, download=False):
    """Initialize a new NASA Marine Debris Dataset instance.

    Args:
        root: root directory where dataset can be found
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.root = root
    self.transforms = transforms
    self.download = download

    self._verify()

    self.source = sorted(glob.glob(os.path.join(self.root, "source", "*.tif")))
    self.labels = sorted(glob.glob(os.path.join(self.root, "labels", "*.npy")))

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/nasa_marine_debris.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    with rasterio.open(self.source[index]) as source:
        image = source.read()

    # (C, H, W) -> (H, W, C)
    image = np.transpose(image, (1, 2, 0)).astype("float32")
    image = ops.convert_to_tensor(image)

    labels = np.load(self.labels[index])

    # Boxes contain unnecessary value of 1 after xyxy coords
    boxes = labels[:, :4].astype("float32")

    # Filter invalid boxes
    w_check = (boxes[:, 2] - boxes[:, 0]) > 0
    h_check = (boxes[:, 3] - boxes[:, 1]) > 0
    keep = w_check & h_check
    boxes = boxes[keep]

    sample = {"image": image, "bbox_xyxy": ops.convert_to_tensor(boxes)}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/nasa_marine_debris.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.source)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/nasa_marine_debris.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.patches as patches
    import matplotlib.pyplot as plt

    ncols = 1

    image = ops.convert_to_numpy(sample["image"]).astype("uint8")
    boxes = ops.convert_to_numpy(sample.get("bbox_xyxy", []))

    showing_predictions = "prediction_bbox_xyxy" in sample
    if showing_predictions:
        ncols += 1
        pred_boxes = ops.convert_to_numpy(sample["prediction_bbox_xyxy"])

    fig, axs = plt.subplots(ncols=ncols, figsize=(ncols * 10, 10))
    axs_list = axs if ncols > 1 else [axs]

    axs_list[0].imshow(image)
    axs_list[0].axis("off")
    for box in boxes:
        x1, y1, x2, y2 = box
        rect = patches.Rectangle(
            (x1, y1), x2 - x1, y2 - y1, linewidth=2, edgecolor="r", facecolor="none"
        )
        axs_list[0].add_patch(rect)
    if show_titles:
        axs_list[0].set_title("Ground Truth")

    if showing_predictions:
        axs_list[1].imshow(image)
        axs_list[1].axis("off")
        for box in pred_boxes:
            x1, y1, x2, y2 = box
            rect = patches.Rectangle(
                (x1, y1),
                x2 - x1,
                y2 - y1,
                linewidth=2,
                edgecolor="r",
                facecolor="none",
            )
            axs_list[1].add_patch(rect)
        if show_titles:
            axs_list[1].set_title("Predictions")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.nccm.NCCM

NCCM(paths='data', crs=None, res=None, years=None, transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: RasterDataset

The Northeastern China Crop Map Dataset.

Link: https://www.nature.com/articles/s41597-021-00827-9

This dataset produced annual 10-m crop maps of the major crops (maize, soybean, and rice) in Northeast China from 2017 to 2019, using hierarchial mapping strategies, random forest classifiers, interpolated and smoothed 10-day Sentinel-2 time series data and optimized features from spectral, temporal and textural characteristics of the land surface. The resultant maps have high overall accuracies (OA) based on ground truth data. The dataset contains information specific to three years: 2017, 2018, 2019.

The dataset contains 5 classes:

1. paddy rice
2. maize
3. soybean
4. others crops and lands
5. nodata

Dataset format:

• Three .TIF files containing the labels
• JavaScript code to download images from the dataset.

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1038/s41597-021-00827-9

Initialize a new dataset.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS in (xres, yres) format (defaults to the resolution of the first file found) years: list of years for which to use nccm layers transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow) time_series: if True, stack data along the time series dimension [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/nccm.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    years=None,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new dataset.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS in (xres, yres)
            format (defaults to the resolution of the first file found)
        years: list of years for which to use nccm layers
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)
        time_series: if True, stack data along the time series dimension
            [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    if years is None:
        years = [2019]
    assert set(years) <= self.md5s.keys(), (
        f"NCCM data product only exists for the following years: {list(self.md5s.keys())}."
    )
    self.paths = paths
    self.years = years
    self.download = download
    self.checksum = checksum

    self._verify()
    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

__getitem__

__getitem__(index)

Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

Source code in keras_climate/datasets/nccm.py

def __getitem__(self, index):
    """Retrieve input, target, and/or metadata indexed by spatiotemporal slice."""
    sample = super().__getitem__(index)

    # Convert nodata class (15) to 4 so there are no gaps in our ordinal mapping
    mask = ops.convert_to_numpy(sample["mask"])
    mask = np.where(mask == 15, 4, mask)
    sample["mask"] = ops.convert_to_tensor(mask)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/nccm.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    mask = ops.convert_to_numpy(sample["mask"])
    ncols = 1

    showing_predictions = "prediction" in sample
    if showing_predictions:
        pred = ops.convert_to_numpy(sample["prediction"])
        ncols = 2

    fig, axs = plt.subplots(ncols=ncols, figsize=(ncols * 4, 4), squeeze=False)
    kwargs = {"cmap": self.cmap, "vmin": 0, "vmax": 4, "interpolation": "none"}

    axs[0, 0].imshow(mask, **kwargs)
    axs[0, 0].axis("off")

    if show_titles:
        axs[0, 0].set_title("Mask")

    if showing_predictions:
        axs[0, 1].imshow(pred, **kwargs)
        axs[0, 1].axis("off")
        if show_titles:
            axs[0, 1].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.nlcd.NLCD

NLCD(paths='data', crs=None, res=None, years=None, classes=None, transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: RasterDataset

Annual National Land Cover Database (NLCD) dataset.

The Annual NLCD products <https://www.usgs.gov/centers/eros/science/annual-national-land-cover-database>_ is an annual land cover product for the conterminous U.S. covering the period from 1985 to 2024. The product is a joint effort between the United States Geological Survey (USGS <https://www.usgs.gov/>) and the Multi-Resolution Land Characteristics Consortium (MRLC <https://www.mrlc.gov/>).

The dataset contains the following 17 classes:

1. Open Water

. Perennial Ice/Snow

. Developed, Open Space

. Developed, Low Intensity

. Developed, Medium Intensity

. Developed, High Intensity

. Barren Land (Rock/Sand/Clay)

. Deciduous Forest

. Evergreen Forest

. Mixed Forest

. Shrub/Scrub

. Grassland/Herbaceous

. Pasture/Hay

. Cultivated Crops

. Woody Wetlands

. Emergent Herbaceous Wetlands

. Background

Detailed descriptions of the classes can be found here <https://www.mrlc.gov/data/legends/national-land-cover-database-class-legend-and-description>__.

Dataset format:

• single channel .img file with integer class labels

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.5066/P94UXNTS

Initialize a new Dataset instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS in (xres, yres) format (defaults to the resolution of the first file found) years: list of years for which to use nlcd layer classes: list of classes to include, the rest will be mapped to 0 (defaults to all classes) transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 after downloading files (may be slow) time_series: if True, stack data along the time series dimension [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

Raises: AssertionError: if years or classes are invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/nlcd.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    years=None,
    classes=None,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS in (xres, yres)
            format (defaults to the resolution of the first file found)
        years: list of years for which to use nlcd layer
        classes: list of classes to include, the rest will be mapped to 0
            (defaults to all classes)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 after downloading files (may be
            slow)
        time_series: if True, stack data along the time series dimension
            [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

    Raises:
        AssertionError: if ``years`` or ``classes`` are invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    if classes is None:
        classes = list(self.valid_classes)
    if years is None:
        years = [2024]
    assert set(years) <= self.md5s.keys(), (
        f"NLCD data product only exists for the following years: {list(self.md5s.keys())}."
    )
    assert set(classes) <= set(self.valid_classes), (
        f"Only the following classes are valid: {self.valid_classes}."
    )
    assert 250 in classes, "Classes must include the background class: 250"

    self.paths = paths
    self.years = years
    self.classes = classes
    self.download = download
    self.checksum = checksum
    self.ordinal_map = np.zeros(self.valid_classes[-1] + 1, dtype="int64")
    self.inverse_map = np.zeros(len(classes), dtype="int64")

    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

    # Map chosen classes to ordinal numbers, all others mapped to background class
    for v, k in enumerate(self.classes):
        self.ordinal_map[k] = v
        self.inverse_map[v] = k

__getitem__

__getitem__(index)

Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

Source code in keras_climate/datasets/nlcd.py

def __getitem__(self, index):
    """Retrieve input, target, and/or metadata indexed by spatiotemporal slice."""
    sample = super().__getitem__(index)
    sample["mask"] = ops.take(
        ops.convert_to_tensor(self.ordinal_map), sample["mask"], axis=0
    )
    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/nlcd.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    mask = self.inverse_map[ops.convert_to_numpy(sample["mask"])]
    ncols = 1

    showing_predictions = "prediction" in sample
    if showing_predictions:
        pred = self.inverse_map[ops.convert_to_numpy(sample["prediction"])]
        ncols = 2

    fig, axs = plt.subplots(ncols=ncols, figsize=(ncols * 4, 4), squeeze=False)
    kwargs = {"cmap": self.cmap, "vmin": 0, "vmax": 255, "interpolation": "none"}

    axs[0, 0].imshow(mask, **kwargs)
    axs[0, 0].axis("off")

    if show_titles:
        axs[0, 0].set_title("Mask")

    if showing_predictions:
        axs[0, 1].imshow(pred, **kwargs)
        axs[0, 1].axis("off")
        if show_titles:
            axs[0, 1].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.openaerialmap.OpenAerialMap

OpenAerialMap(paths='data', crs=None, res=None, bbox=None, zoom=19, max_items=1, transforms=None, cache=True, download=False, image_id=None, tile_size=256)

Bases: RasterDataset

OpenAerialMap dataset.

The OpenAerialMap (OAM) <https://openaerialmap.org/>__ is an open service for accessing and sharing aerial imagery. The dataset provides access to crowd-sourced aerial imagery from various sources including drones, satellites, and aircraft.

This implementation uses the STAC API <https://api.imagery.hotosm.org/stac>__ to query imagery and download tiles via TMS endpoints. The STAC API returns imagery sorted by most recent first.

Dataset features:

• Aerial imagery from various sources (drones, satellites, aircraft)
• Global coverage with varying resolution
• STAC-based querying (most recent imagery first)
• Tile naming following mercantile standard: OAM-{x}-{y}-{z}.tif
• Automatic georeferencing using rasterio
• RGB imagery (3-band)

Usage::

dataset = OpenAerialMap(
    paths='data/openaerial',
    bbox=(lon_min, lat_min, lon_max, lat_max),
    zoom=19,
    download=True,
)

If you use this dataset in your research, please cite OpenAerialMap:

• https://openaerialmap.org/

Initialize a new OpenAerialMap dataset instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to EPSG:4326) res: resolution of the dataset in units of CRS (defaults to resolution of first file found) bbox: bounding box for STAC query as (xmin, ymin, xmax, ymax) in EPSG:4326 zoom: zoom level for tiles (15-23), only used when download=True max_items: maximum number of STAC items to query transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling download: if True, download imagery from STAC API based on bbox image_id: optional STAC item ID to download specific imagery tile_size: size of the tiles to download (256, 512, 768, 1024)

Raises: DatasetNotFoundError: If dataset is not found and download=False. ValueError: If download=True but neither bbox nor image_id is provided.

Source code in keras_climate/datasets/openaerialmap.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bbox=None,
    zoom=19,
    max_items=1,
    transforms=None,
    cache=True,
    download=False,
    image_id=None,
    tile_size=256,
):
    """Initialize a new OpenAerialMap dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to EPSG:4326)
        res: resolution of the dataset in units of CRS (defaults to
            resolution of first file found)
        bbox: bounding box for STAC query as (xmin, ymin, xmax, ymax) in
            EPSG:4326
        zoom: zoom level for tiles (15-23), only used when download=True
        max_items: maximum number of STAC items to query
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download imagery from STAC API based on bbox
        image_id: optional STAC item ID to download specific imagery
        tile_size: size of the tiles to download (256, 512, 768, 1024)

    Raises:
        DatasetNotFoundError: If dataset is not found and download=False.
        ValueError: If download=True but neither bbox nor image_id is
            provided.
    """
    self.paths = paths
    self.bbox = bbox
    self.zoom = zoom
    self.max_items = max_items
    self.download = download
    self.image_id = image_id
    self.tile_size = tile_size

    if download:
        if bbox is None:
            raise ValueError("bbox must be provided when download=True")
        if not 15 <= zoom <= 23:
            raise ValueError(f"zoom must be between 15 and 23, got {zoom}")
        if self._download():
            print("Download complete.")

    if crs is None:
        crs = PROJ_CRS.from_epsg(4326)

    super().__init__(paths, crs, res, transforms=transforms, cache=cache)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/openaerialmap.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt
    import numpy as np

    rgb = ops.convert_to_numpy(sample["image"])[..., 0:3]

    if np.issubdtype(rgb.dtype, np.floating) and rgb.max() > 1:
        rgb = np.clip(rgb / 255.0, 0, 1)

    fig, ax = plt.subplots(nrows=1, ncols=1, figsize=(4, 4))
    ax.imshow(rgb)
    ax.axis("off")

    if show_titles:
        ax.set_title("Image")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.openaerialmap.TileUtils

Web Mercator tile utilities for XYZ tile calculations.

Implements standard Web Mercator (EPSG:3857) tile math for converting between geographic coordinates and tile indices.

References: * OpenStreetMap Wiki - Slippy map tilenames: https://wiki.openstreetmap.org/wiki/Slippy_map_tilenames * Web Mercator / Pseudo-Mercator (EPSG:3857): https://epsg.io/3857 * OSGeo Tile Map Service Specification: https://wiki.osgeo.org/wiki/Tile_Map_Service_Specification

tile classmethod

tile(lng, lat, zoom, truncate=False)

Get tile coordinates containing a geographic point.

Source code in keras_climate/datasets/openaerialmap.py

@classmethod
def tile(cls, lng, lat, zoom, truncate=False):
    """Get tile coordinates containing a geographic point."""
    if truncate:
        lng = max(-180.0, min(180.0, lng))
        lat = max(-90.0, min(90.0, lat))

    x_frac = (lng + 180.0) / 360.0
    lat_rad = math.radians(lat)
    y_frac = (
        1.0 - math.log(math.tan(lat_rad) + (1.0 / math.cos(lat_rad))) / math.pi
    ) / 2.0

    n = 2**zoom
    x = int(min(n - 1, max(0, math.floor(x_frac * n))))
    y = int(min(n - 1, max(0, math.floor(y_frac * n))))

    return cls.Tile(x, y, zoom)

bounds classmethod

bounds(t)

Get geographic bounds of a tile.

Source code in keras_climate/datasets/openaerialmap.py

@classmethod
def bounds(cls, t):
    """Get geographic bounds of a tile."""
    n = 2**t.z
    west = t.x / n * 360.0 - 180.0
    east = (t.x + 1) / n * 360.0 - 180.0

    north_rad = math.atan(math.sinh(math.pi * (1 - 2 * t.y / n)))
    south_rad = math.atan(math.sinh(math.pi * (1 - 2 * (t.y + 1) / n)))
    north = math.degrees(north_rad)
    south = math.degrees(south_rad)

    return cls.LngLatBbox(west, south, east, north)

tiles classmethod

tiles(west, south, east, north, zoom, truncate=False)

Generate tiles covering a bounding box.

Source code in keras_climate/datasets/openaerialmap.py

@classmethod
def tiles(cls, west, south, east, north, zoom, truncate=False):
    """Generate tiles covering a bounding box."""
    if truncate:
        west = max(-180.0, min(180.0, west))
        south = max(-90.0, min(90.0, south))
        east = max(-180.0, min(180.0, east))
        north = max(-90.0, min(90.0, north))

    if west > east:
        bboxes = [(-180.0, south, east, north), (west, south, 180.0, north)]
    else:
        bboxes = [(west, south, east, north)]

    for w, s, e, n in bboxes:
        w = max(-180.0, w)
        s = max(-85.051129, s)
        e = min(180.0, e)
        n = min(85.051129, n)

        ul_tile = cls.tile(w, n, zoom)
        lr_tile = cls.tile(e, s, zoom)

        for x in range(ul_tile.x, lr_tile.x + 1):
            for y in range(ul_tile.y, lr_tile.y + 1):
                yield cls.Tile(x, y, zoom)

keras_climate.datasets.openbuildings.OpenBuildings

OpenBuildings(paths='data', crs=None, res=0.0001, transforms=None, checksum=True)

Bases: VectorDataset

Open Buildings dataset.

The Open Buildings <https://sites.research.google/gr/open-buildings/>__ dataset consists of computer generated building detections across the African continent.

Dataset features:

• 516M building detections as polygons with centroid lat/long
• covering area of 19.4M km\ :sup:2\ (64% of the African continent)
• confidence score and Plus Code <https://maps.google.com/pluscodes/>_

Dataset format:

• csv files containing building detections compressed as csv.gz
• meta data geojson file

The data can be downloaded from here <https://sites.research.google/gr/open-buildings/#open-buildings-download>__. Additionally, the meta data geometry file <https://openbuildings-public-dot-gweb-research.uw.r.appspot.com/public/tiles.geojson>_ also needs to be placed in root as tiles.geojson.

If you use this dataset in your research, please cite the following technical report:

• https://arxiv.org/abs/2107.12283

Initialize a new Dataset instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS in (xres, yres) format. If a single float is provided, it is used for both the x and y resolution. transforms: a function/transform that takes input sample and its target as entry and returns a transformed version checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/openbuildings.py

def __init__(self, paths="data", crs=None, res=0.0001, transforms=None, checksum=True):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS in (xres, yres)
            format. If a single float is provided, it is used for both
            the x and y resolution.
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        DatasetNotFoundError: If dataset is not found.
    """
    self.paths = paths
    if isinstance(res, (int, float)):
        res = (res, res)
    self.res = res
    self.checksum = checksum
    self.transforms = transforms

    self._verify()

    polygon_files = glob.glob(os.path.join(paths, self.zipfile_glob))
    polygon_filenames = [f.split(os.sep)[-1] for f in polygon_files]

    filename = os.path.join(paths, self.meta_data_filename)
    gdf = gpd.read_file(filename)
    if gdf.crs is None:
        gdf = gdf.set_crs(self._source_crs)

    # Filter to only include desired polygon files
    gdf["filepath"] = gdf["tile_url"].str.split("/").str[-1]
    gdf = gdf[gdf["filepath"].isin(polygon_filenames)]

    # Convert geometries to bounding boxes
    geometries = gdf.bounds.apply(
        lambda row: shapely.box(row["minx"], row["miny"], row["maxx"], row["maxy"]),
        axis=1,
    )
    filepaths = [os.path.join(paths, filepath) for filepath in gdf["filepath"]]
    datetimes = [(pd.Timestamp.min, pd.Timestamp.max)] * len(filepaths)

    if not len(filepaths):
        raise DatasetNotFoundError(self)

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(
        data, index=index, geometry=list(geometries), crs=self._source_crs
    )
    if crs is not None and crs != self._source_crs:
        self.index.to_crs(crs, inplace=True)

__getitem__

__getitem__(index)

Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

Source code in keras_climate/datasets/openbuildings.py

def __getitem__(self, index):
    """Retrieve input, target, and/or metadata indexed by spatiotemporal slice."""
    x, y, t = self._disambiguate_slice(index)
    interval = pd.Interval(t.start, t.stop)
    df = self.index.iloc[self.index.index.overlaps(interval)]
    df = df.iloc[:: t.step]
    df = df.cx[x.start : x.stop, y.start : y.stop]

    if df.empty:
        raise IndexError(
            f"index: {index} not found in dataset with bounds: {self.bounds}"
        )

    shapes = self._filter_geometries(index, df.filepath)

    # Rasterize geometries
    width = (x.stop - x.start) / x.step
    height = (y.stop - y.start) / y.step
    transform = rasterio.transform.from_bounds(
        x.start, y.start, x.stop, y.stop, width, height
    )
    if shapes:
        masks = rasterio.features.rasterize(
            shapes, out_shape=(round(height), round(width)), transform=transform
        )
        masks = np.expand_dims(masks, axis=-1)
    else:
        masks = np.zeros((round(height), round(width), 1), dtype="uint8")

    transform = rasterio.transform.from_origin(x.start, y.stop, x.step, y.step)
    sample = {
        "mask": ops.convert_to_tensor(masks),
        "bounds": self._slice_to_tensor(index),
        "transform": ops.convert_to_tensor(np.array(list(transform))),
    }

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/openbuildings.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    mask = ops.convert_to_numpy(sample["mask"])

    showing_predictions = "prediction" in sample
    if showing_predictions:
        pred = ops.convert_to_numpy(sample["prediction"])
        ncols = 2
    else:
        ncols = 1

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 4, 4))
    axs_list = axs if ncols > 1 else [axs]

    axs_list[0].imshow(mask)
    axs_list[0].axis("off")
    if show_titles:
        axs_list[0].set_title("Mask")

    if showing_predictions:
        axs_list[1].imshow(pred)
        axs_list[1].axis("off")
        if show_titles:
            axs_list[1].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.openstreetmap.OpenStreetMap

OpenStreetMap(bbox, classes, paths='data', res=(0.0001, 0.0001), transforms=None, download=False)

Bases: VectorDataset

OpenStreetMap dataset.

The OpenStreetMap <https://www.openstreetmap.org/>__ dataset provides access to crowd-sourced geographic data. This implementation uses the Overpass API <https://wiki.openstreetmap.org/wiki/Overpass_API>__ to query and download OSM data for a specified geographic bounding box at initialization, then allows spatial querying of the cached data.

Dataset features:

• Vector data (points, lines, polygons) for various geographic features
• Flexible querying by class configuration (buildings, highways, amenities, etc.)
• Data fetched once at initialization and cached locally
• Class-based labeling with priority-based assignment

Class priority and label assignment: classes is a list of dicts defining feature classes. Each has name (str) and selector (list of OSM tag filters). Features are assigned labels based on the order of classes in this list:

• First class gets label=1, second gets label=2, etc.
• If a feature matches multiple classes, it receives the label of the first matching class
• Features that don't match any class get label=0 (background)

Example::

classes = [
    {"name": "buildings", "selector": [{"building": "*"}]},  # label=1
    {"name": "roads", "selector": [{"highway": "*"}]},  # label=2
]

If you use this dataset in your research, please cite the following source:

• https://www.openstreetmap.org/copyright

Initialize a new OpenStreetMap dataset instance.

Args: bbox: bounding box for initial data fetch as (xmin, ymin, xmax, ymax) in EPSG:4326 classes: list of dicts defining feature classes. Each dict must have "name" (str) and "selector" (list of OSM tag filters). Features get labels 1-N based on class order, with first match taking priority. paths: paths directory where dataset will be stored res: resolution of the dataset in units of EPSG:4326 (degrees) transforms: a function/transform that takes input sample and returns a transformed version download: if True, download dataset and store it in the paths directory

Raises: DatasetNotFoundError: if dataset is not found and download is False TypeError: if invalid class configuration ValueError: if invalid class configuration

Source code in keras_climate/datasets/openstreetmap.py

def __init__(
    self,
    bbox,
    classes,
    paths="data",
    res=(0.0001, 0.0001),
    transforms=None,
    download=False,
):
    """Initialize a new OpenStreetMap dataset instance.

    Args:
        bbox: bounding box for initial data fetch as (xmin, ymin, xmax,
            ymax) in EPSG:4326
        classes: list of dicts defining feature classes. Each dict must
            have "name" (str) and "selector" (list of OSM tag filters).
            Features get labels 1-N based on class order, with first
            match taking priority.
        paths: paths directory where dataset will be stored
        res: resolution of the dataset in units of EPSG:4326 (degrees)
        transforms: a function/transform that takes input sample and
            returns a transformed version
        download: if True, download dataset and store it in the paths
            directory

    Raises:
        DatasetNotFoundError: if dataset is not found and download is
            False
        TypeError: if invalid class configuration
        ValueError: if invalid class configuration
    """
    self._validate_classes(classes)

    self.bbox = bbox
    self.classes = classes
    self.root = pathlib.Path(paths)
    self.root.mkdir(parents=True, exist_ok=True)

    if download:
        self._download_data()

    if not self._check_integrity():
        raise DatasetNotFoundError(self)

    data_file = self._get_data_filename()

    super().__init__(
        paths=data_file,
        crs=CRS.from_epsg(4326),  # Always use WGS84 for OSM data
        res=res,
        transforms=transforms,
        label_name="label",
    )

    # Check for empty classes after initialization
    self._check_empty_classes()

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/openstreetmap.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.patches as mpatches
    import matplotlib.pyplot as plt

    mask = np.squeeze(ops.convert_to_numpy(sample["mask"]))
    ncols = 1

    showing_prediction = "prediction" in sample
    if showing_prediction:
        pred = np.squeeze(ops.convert_to_numpy(sample["prediction"]))
        ncols = 2

    colors = [
        "#000000",  # Background (label=0)
        "#FF6B6B",
        "#4ECDC4",
        "#45B7D1",
        "#96CEB4",
        "#FECA57",
        "#FF9FF3",
        "#54A0FF",
        "#5F27CD",
        "#00D2D3",
        "#FF3838",
        "#FF9500",
        "#7bed9f",
    ]

    cmap = ListedColormap(colors)

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 5, 5))
    axs_list = axs if ncols > 1 else [axs]

    legend_handles = []
    unique_labels = np.unique(mask)
    for label in unique_labels:
        if label == 0:
            continue
        class_idx = int(label - 1)
        if class_idx < len(self.classes):
            class_name = self.classes[class_idx]["name"]
            color = colors[int(label) % len(colors)]
            legend_handles.append(mpatches.Patch(color=color, label=class_name.title()))

    axs_list[0].imshow(mask, cmap=cmap, vmin=0, vmax=len(colors) - 1, interpolation="none")
    axs_list[0].axis("off")
    if show_titles:
        axs_list[0].set_title("Mask")
    if legend_handles:
        axs_list[0].legend(handles=legend_handles, loc="upper right")

    if showing_prediction:
        axs_list[1].imshow(
            pred, cmap=cmap, vmin=0, vmax=len(colors) - 1, interpolation="none"
        )
        axs_list[1].axis("off")
        if show_titles:
            axs_list[1].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.oscd.OSCD

OSCD(root='data', split='train', bands=all_bands, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

OSCD dataset.

The Onera Satellite Change Detection <https://rcdaudt.github.io/oscd/>_ dataset addresses the issue of detecting changes between satellite images from different dates. Imagery comes from Sentinel-2 which contains varying resolutions per band.

Dataset format:

• images are 13-channel tifs
• masks are single-channel pngs where no change = 0, change = 255

Dataset classes:

1. no change
2. change

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1109/IGARSS.2018.8518015

Initialize a new OSCD dataset instance.

Args: root: root directory where dataset can be found split: one of "train" or "test" bands: bands to return (defaults to all bands) transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: AssertionError: if split argument is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/oscd.py

def __init__(
    self,
    root="data",
    split="train",
    bands=all_bands,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new OSCD dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in self.splits
    assert set(bands) <= set(self.all_bands)
    self.bands = bands
    self.all_band_indices = [self.all_bands.index(b) for b in self.bands]

    self.root = root
    self.split = split
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    self.files = self._load_files()

__getitem__

__getitem__(index)

Return an index within the dataset.

.. versionchanged:: Now returns a single T x H x W x C image.

Source code in keras_climate/datasets/oscd.py

def __getitem__(self, index):
    """Return an index within the dataset.

    .. versionchanged::
       Now returns a single T x H x W x C image.
    """
    files = self.files[index]
    image1 = self._load_image(files["images1"])
    image2 = self._load_image(files["images2"])
    mask = self._load_target(files["mask"])
    image = ops.stack([image1, image2], axis=0)
    sample = {"image": image, "mask": mask}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/oscd.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.files)

plot

plot(sample, show_titles=True, suptitle=None, alpha=None)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/oscd.py

def plot(self, sample, show_titles=True, suptitle=None, alpha=None):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    if alpha is not None:
        warnings.warn(
            "The alpha parameter is deprecated and has no effect.",
            DeprecationWarning,
            stacklevel=2,
        )

    import matplotlib.pyplot as plt

    try:
        rgb_indices = [self.bands.index(band) for band in self.rgb_bands]
    except ValueError as e:
        raise RGBBandsMissingError() from e

    def to_rgb(img):
        rgb = ops.take(img, np.array(rgb_indices), axis=-1)
        return quantile_normalization(rgb)

    ncols = 3
    if "prediction" in sample:
        ncols = 4

    image1 = ops.convert_to_numpy(to_rgb(sample["image"][0]))
    image2 = ops.convert_to_numpy(to_rgb(sample["image"][1]))
    mask = ops.convert_to_numpy(sample["mask"][0])

    h, w = image1.shape[:2]
    fig, axs = plt.subplots(
        1, ncols, figsize=(ncols * 5, 5 * h / w), layout="constrained"
    )
    axs[0].imshow(image1)
    axs[1].imshow(image2)
    axs[2].imshow(mask, cmap="gray", interpolation="none", vmin=0, vmax=1)
    if ncols == 4:
        axs[3].imshow(
            ops.convert_to_numpy(sample["prediction"][0]),
            cmap="gray",
            interpolation="none",
            vmin=0,
            vmax=1,
        )

    for ax in axs:
        ax.axis("off")

    if show_titles:
        axs[0].set_title("Pre-change (T1)")
        axs[1].set_title("Post-change (T2)")
        axs[2].set_title("Ground Truth")
        if ncols == 4:
            axs[3].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.oscd.OSCD100

OSCD100(root='data', split='train', bands=all_bands, transforms=None, download=False, checksum=True)

Bases: OSCD

Subset of OSCD with 100 pre-cropped image pairs at 256x256 resolution.

Intended for tutorials and demonstrations, not benchmarking.

Maintains the same file structure and all 13 Sentinel-2 bands as OSCD, but with 100 pre-cropped 256x256 patches. Adds a validation split (train/val/test).

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1109/IGARSS.2018.8518015

Source code in keras_climate/datasets/oscd.py

def __init__(
    self,
    root="data",
    split="train",
    bands=all_bands,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new OSCD dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in self.splits
    assert set(bands) <= set(self.all_bands)
    self.bands = bands
    self.all_band_indices = [self.all_bands.index(b) for b in self.bands]

    self.root = root
    self.split = split
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    self.files = self._load_files()

keras_climate.datasets.pastis.PASTIS

PASTIS(root='data', folds=(1, 2, 3, 4, 5), bands=s2_bands, mode='semantic', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

PASTIS dataset.

The PASTIS <https://github.com/VSainteuf/pastis-benchmark>__ dataset is a dataset for time-series panoptic segmentation of agricultural parcels.

Dataset features:

• support for the original PASTIS and PASTIS-R versions of the dataset
• 2,433 time-series with 10 m per pixel resolution (128x128 px)
• 18 crop categories, 1 background category, 1 void category
• semantic and instance annotations
• 3 Sentinel-1 Ascending bands
• 3 Sentinel-1 Descending bands
• 10 Sentinel-2 L2A multispectral bands

Dataset format:

• time-series and annotations are in numpy format (.npy)

Dataset classes:

1. Background
2. Meadow
3. Soft Winter Wheat
4. Corn
5. Winter Barley
6. Winter Rapeseed
7. Spring Barley
8. Sunflower
9. Grapevine
10. Beet
11. Winter Triticale
12. Winter Durum Wheat
13. Fruits Vegetables Flowers
14. Potatoes
15. Leguminous Fodder
16. Soybeans
17. Orchard
18. Mixed Cereal
19. Sorghum
20. Void Label

If you use this dataset in your research, please cite the following papers:

• https://doi.org/10.1109/ICCV48922.2021.00483
• https://doi.org/10.1016/j.isprsjprs.2022.03.012

Initialize a new PASTIS dataset instance.

Args: root: root directory where dataset can be found folds: a sequence of integers from 1 to 5 specifying which of the five dataset folds to include bands: sequence of band names to load. Must be a non-empty subset of s2_bands, s1a_bands, or s1d_bands. All bands must come from the same sensor. Defaults to all S2 bands. mode: load semantic ("semantic") or instance ("instance") annotations transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/pastis.py

def __init__(
    self,
    root="data",
    folds=(1, 2, 3, 4, 5),
    bands=s2_bands,
    mode="semantic",
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new PASTIS dataset instance.

    Args:
        root: root directory where dataset can be found
        folds: a sequence of integers from 1 to 5 specifying which of
            the five dataset folds to include
        bands: sequence of band names to load. Must be a non-empty
            subset of ``s2_bands``, ``s1a_bands``, or ``s1d_bands``. All
            bands must come from the same sensor. Defaults to all S2
            bands.
        mode: load semantic ("semantic") or instance ("instance")
            annotations
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    for fold in folds:
        assert 1 <= fold <= 5
    assert mode in ["semantic", "instance"]

    bands_set = set(bands)
    if not bands_set:
        raise ValueError("bands must not be empty")
    if bands_set <= set(self.s2_bands):
        self.image_key = "s2"
        all_bands = self.s2_bands
    elif bands_set <= set(self.s1a_bands):
        self.image_key = "s1a"
        all_bands = self.s1a_bands
    elif bands_set <= set(self.s1d_bands):
        self.image_key = "s1d"
        all_bands = self.s1d_bands
    else:
        raise ValueError(
            f"bands must be a subset of s2_bands, s1a_bands, or s1d_bands; got {bands}"
        )
    self.bands = tuple(bands)
    self.band_indices = [all_bands.index(b) for b in self.bands]

    self.root = root
    self.folds = folds
    self.mode = mode
    self.transforms = transforms
    self.download = download
    self.checksum = checksum
    self._verify()
    self.files = self._load_files()

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/pastis.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    image = self._load_image(index)
    if self.mode == "semantic":
        mask = self._load_semantic_targets(index)
        sample = {"image": image, "mask": mask}
    elif self.mode == "instance":
        mask, boxes, labels = self._load_instance_targets(index)
        sample = {"image": image, "mask": mask, "bbox_xyxy": boxes, "label": labels}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/pastis.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.idxs)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/pastis.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    # Keep the RGB bands and quantile-normalize each frame independently.
    rgb_indices = np.array([2, 1, 0])
    rgb_frames = ops.take(sample["image"], rgb_indices, axis=-1)
    images = np.stack(
        [
            ops.convert_to_numpy(quantile_normalization(frame))
            for frame in rgb_frames
        ]
    )
    mask = ops.convert_to_numpy(sample["mask"])

    if self.mode == "instance":
        label = ops.convert_to_numpy(sample["label"])
        mask = label[mask.argmax(axis=0)]

    num_panels = 3
    showing_predictions = "prediction" in sample
    if showing_predictions:
        predictions = ops.convert_to_numpy(sample["prediction"])
        num_panels += 1
        if self.mode == "instance":
            predictions = predictions.argmax(axis=0)
            pred_label = ops.convert_to_numpy(sample["prediction_labels"])
            predictions = pred_label[predictions]

    fig, axs = plt.subplots(1, num_panels, figsize=(num_panels * 4, 4))
    axs[0].imshow(images[0])
    axs[1].imshow(images[1])
    axs[2].imshow(mask, vmin=0, vmax=19, cmap=self.cmap, interpolation="none")
    axs[0].axis("off")
    axs[1].axis("off")
    axs[2].axis("off")
    if showing_predictions:
        axs[3].imshow(predictions, vmin=0, vmax=19, cmap=self.cmap, interpolation="none")
        axs[3].axis("off")

    if show_titles:
        axs[0].set_title("Image 0")
        axs[1].set_title("Image 1")
        axs[2].set_title("Mask")
        if showing_predictions:
            axs[3].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.pastis.PASTIS100

PASTIS100(root='data', folds=(1, 2, 3, 4, 5), bands=s2_bands, mode='semantic', transforms=None, download=False, checksum=True)

Bases: PASTIS

Subset of PASTIS-R containing only 100 time-series.

Intended for tutorials and demonstrations, not for benchmarking.

Source code in keras_climate/datasets/pastis.py

def __init__(
    self,
    root="data",
    folds=(1, 2, 3, 4, 5),
    bands=s2_bands,
    mode="semantic",
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new PASTIS dataset instance.

    Args:
        root: root directory where dataset can be found
        folds: a sequence of integers from 1 to 5 specifying which of
            the five dataset folds to include
        bands: sequence of band names to load. Must be a non-empty
            subset of ``s2_bands``, ``s1a_bands``, or ``s1d_bands``. All
            bands must come from the same sensor. Defaults to all S2
            bands.
        mode: load semantic ("semantic") or instance ("instance")
            annotations
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    for fold in folds:
        assert 1 <= fold <= 5
    assert mode in ["semantic", "instance"]

    bands_set = set(bands)
    if not bands_set:
        raise ValueError("bands must not be empty")
    if bands_set <= set(self.s2_bands):
        self.image_key = "s2"
        all_bands = self.s2_bands
    elif bands_set <= set(self.s1a_bands):
        self.image_key = "s1a"
        all_bands = self.s1a_bands
    elif bands_set <= set(self.s1d_bands):
        self.image_key = "s1d"
        all_bands = self.s1d_bands
    else:
        raise ValueError(
            f"bands must be a subset of s2_bands, s1a_bands, or s1d_bands; got {bands}"
        )
    self.bands = tuple(bands)
    self.band_indices = [all_bands.index(b) for b in self.bands]

    self.root = root
    self.folds = folds
    self.mode = mode
    self.transforms = transforms
    self.download = download
    self.checksum = checksum
    self._verify()
    self.files = self._load_files()

keras_climate.datasets.patternnet.PatternNet

PatternNet(root='data', transforms=None, download=False, checksum=True)

Bases: NonGeoClassificationDataset

PatternNet dataset.

The PatternNet <https://sites.google.com/view/zhouwx/dataset>__ dataset is a dataset for remote sensing scene classification and image retrieval.

Dataset features:

• 30,400 images with 6-50 cm per pixel resolution (256x256 px)
• three spectral bands - RGB
• 38 scene classes, 800 images per class

Dataset format:

• images are three-channel jpgs

If you use this dataset in your research, please cite:

• https://doi.org/10.1016/j.isprsjprs.2018.01.004

Initialize a new PatternNet dataset instance.

Args: root: root directory where dataset can be found transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/patternnet.py

def __init__(self, root="data", transforms=None, download=False, checksum=True):
    """Initialize a new PatternNet dataset instance.

    Args:
        root: root directory where dataset can be found
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.root = root
    self.download = download
    self.checksum = checksum
    self._verify()
    super().__init__(root=os.path.join(root, self.directory), transforms=transforms)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/patternnet.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt
    from keras import ops

    image = ops.convert_to_numpy(sample["image"]).astype("uint8")
    label = int(ops.convert_to_numpy(sample["label"]))

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = int(ops.convert_to_numpy(sample["prediction"]))

    fig, ax = plt.subplots(1, 1, figsize=(10, 10))

    ax.imshow(image)
    ax.axis("off")

    if show_titles:
        title = f"Label: {self.classes[label]}"
        if showing_predictions:
            title += f"\nPrediction: {self.classes[prediction]}"
        ax.set_title(title)

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.potsdam.Potsdam2D

Potsdam2D(root='data', split='train', transforms=None, checksum=True)

Bases: NonGeoDataset

Potsdam 2D Semantic Segmentation dataset.

The Potsdam <https://www.isprs.org/resources/datasets/benchmarks/UrbanSemLab/2d-sem-label-potsdam.aspx>__ dataset is a dataset for urban semantic segmentation used in the 2D Semantic Labeling Contest - Potsdam. This dataset uses the "4_Ortho_RGBIR.zip" and "5_Labels_all.zip" files to create the train/test sets used in the challenge. The dataset can be requested at the challenge homepage.

Dataset format:

• images are 4-channel geotiffs
• masks are 3-channel geotiffs with unique RGB values representing the class

Dataset classes:

1. Clutter/background
2. Impervious surfaces
3. Building
4. Low Vegetation
5. Tree
6. Car

If you use this dataset in your research, please cite:

• https://doi.org/10.5194/isprsannals-I-3-293-2012

Initialize a new Potsdam dataset instance.

Args: root: root directory where dataset can be found split: one of "train" or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: If split is invalid. DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/potsdam.py

def __init__(self, root="data", split="train", transforms=None, checksum=True):
    """Initialize a new Potsdam dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: If *split* is invalid.
        DatasetNotFoundError: If dataset is not found.
    """
    assert split in self.splits
    self.root = root
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    self._verify()

    self.files = []
    for name in self.splits[split]:
        image = os.path.join(root, self.image_root, name) + "_RGBIR.tif"
        mask = os.path.join(root, name) + "_label.tif"
        if os.path.exists(image) and os.path.exists(mask):
            self.files.append({"image": image, "mask": mask})

plot

plot(sample, show_titles=True, suptitle=None, alpha=0.5)

Plot a sample from the dataset.

Source code in keras_climate/datasets/potsdam.py

def plot(self, sample, show_titles=True, suptitle=None, alpha=0.5):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    ncols = 1
    image = ops.convert_to_numpy(sample["image"])[..., :3].astype("uint8")
    mask = ops.convert_to_numpy(sample["mask"])
    image1 = draw_semantic_segmentation_masks(
        ops.convert_to_tensor(image), mask, alpha=alpha, colors=list(self.colormap)
    )
    if "prediction" in sample:
        ncols += 1
        prediction = ops.convert_to_numpy(sample["prediction"])
        image2 = draw_semantic_segmentation_masks(
            ops.convert_to_tensor(image),
            prediction,
            alpha=alpha,
            colors=list(self.colormap),
        )

    fig, axs = plt.subplots(ncols=ncols, figsize=(ncols * 10, 10))
    if ncols > 1:
        (ax0, ax1) = axs
    else:
        ax0 = axs

    ax0.imshow(image1)
    ax0.axis("off")
    if ncols > 1:
        ax1.imshow(image2)
        ax1.axis("off")

    if show_titles:
        ax0.set_title("Ground Truth")
        if ncols > 1:
            ax1.set_title("Predictions")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.presto.PrestoEmbeddings

PrestoEmbeddings(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset

Presto Embeddings dataset.

Geospatial embeddings <https://nasaharvest.github.io/presto-embeddings/>__ for Togo generated using the Presto geospatial foundation model <https://arxiv.org/abs/2304.14065>__.

Presto geospatial embeddings provide a compressed representation of Earth Observation data, enabling more efficient mapping and analysis. Embeddings are generated by using the Presto encoder to compress location information, optical imagery (Sentinel-2), radar imagery (Sentinel-1), climatology data (ERA5), and elevation data (SRTM) over the course of a year (March 2019 - March 2020). Each embedding contains 128 features representing a single 10 m2 pixel on Earth.

The dataset can be downloaded from one of two sources:

• Hugging Face <https://huggingface.co/datasets/izvonkov/Togo_Presto_Embeddings>__
• Google Earth Engine <https://code.earthengine.google.com/?asset=users/izvonkov/Togo/Presto_embeddings_v2025_06_19>__

If you use this dataset in your research, please cite:

• https://arxiv.org/abs/2511.02923

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new RasterDataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a mosaic
            (``[H, W, C]``). For mask-style datasets (``is_image=False``),
            single-band data has the channel dimension squeezed.
    """
    self.paths = paths
    self.bands = bands or self.all_bands
    self.transforms = transforms
    self.cache = cache
    self.time_series = time_series

    if self.all_bands:
        assert set(self.bands) <= set(self.all_bands)

    filename_regex = re.compile(self.filename_regex, re.VERBOSE)
    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        match = re.match(filename_regex, os.path.basename(filepath))
        if match is not None:
            vrt = None
            try:
                vrt = self._load_warp_file(filepath=filepath, crs=crs)
                if self.cmap is None:
                    try:
                        colors = np.array(list(vrt.colormap(1).values())) / 255
                        self.cmap = ListedColormap(colors)
                    except ValueError:
                        pass
                if crs is None:
                    with rasterio.Env(OSR_WKT_FORMAT="WKT2_2018"):
                        crs = PROJ_CRS.from_user_input(vrt.crs)
                footprint = self.footprint_from_datasource(vrt)
                if footprint is None:
                    footprint = shapely.box(*vrt.bounds)
                geometries.append(footprint)
                if res is None:
                    res = vrt.res
            except rasterio.errors.RasterioIOError:
                continue
            else:
                filepaths.append(filepath)
                mint, maxt = self._filepath_to_timestamp(filepath)
                datetimes.append((mint, maxt))
            finally:
                if vrt is not None:
                    vrt.close()

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if not self.separate_files:
        self.band_indexes = None
        if self.bands:
            if self.all_bands:
                self.band_indexes = [
                    self.all_bands.index(i) + 1 for i in self.bands
                ]
            else:
                msg = (
                    f"{self.__class__.__name__} is missing an `all_bands` "
                    "attribute, so `bands` cannot be specified."
                )
                raise AssertionError(msg)

    if res is not None:
        if isinstance(res, (int, float)):
            res = (res, res)

        self._res = res

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

.. warning:: Visualizations are generated using PCA on each image individually, and are thus not comparable across images. The plot method is provided for visualization purposes only and should not be used to draw conclusions.

Source code in keras_climate/datasets/presto.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    .. warning::
       Visualizations are generated using PCA on each image
       *individually*, and are thus not comparable across images. The
       plot method is provided for visualization purposes only and
       should not be used to draw conclusions.
    """
    import matplotlib.pyplot as plt
    from keras import ops

    image = ops.convert_to_numpy(sample["image"])
    h, w, c = image.shape
    A = image.reshape(h * w, c)

    # Use PCA to project embeddings from 128D to 3D space
    valid = A.sum(axis=1) != 0
    invalid = A.sum(axis=1) == 0

    A_centered = A[valid] - A[valid].mean(axis=0, keepdims=True)
    _, _, Vt = np.linalg.svd(A_centered, full_matrices=False)
    V = Vt[:3].T
    B = A @ V

    B -= B[valid].min(axis=0, keepdims=True)
    B /= B[valid].max(axis=0, keepdims=True)
    B[invalid] = 1
    image = B.reshape(h, w, 3)

    fig, ax = plt.subplots()
    ax.imshow(image)
    ax.axis("off")

    if show_titles:
        ax.set_title("Embedding")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.prisma.PRISMA

PRISMA(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset

PRISMA dataset.

Hyperspectral Precursor and Application Mission PRISMA <https://www.eoportal.org/satellite-missions/prisma-hyperspectral>__ (PRecursore IperSpettrale della Missione Applicativa) is a medium-resolution hyperspectral imaging satellite, developed, owned, and operated by the Italian Space Agency ASI.

PRISMA carries two sensor instruments, the HYC (Hyperspectral Camera) module and the PAN (Panchromatic Camera) module. The HYC sensor is a prism spectrometer for two bands, VIS/NIR and NIR/SWIR, with a total of 237 channels across both bands, at 30 m spatial resolution.

If you use this dataset in your research, please cite:

• https://doi.org/10.1109/IGARSS.2018.8517785

.. note:: PRISMA imagery is distributed as HDF5 files. This loader requires you to first convert all files from HDF5 to GeoTIFF using something like this script <https://gist.github.com/adamjstewart/7f5324a6a339c20e778f39536402fb4a>__.

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new RasterDataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a mosaic
            (``[H, W, C]``). For mask-style datasets (``is_image=False``),
            single-band data has the channel dimension squeezed.
    """
    self.paths = paths
    self.bands = bands or self.all_bands
    self.transforms = transforms
    self.cache = cache
    self.time_series = time_series

    if self.all_bands:
        assert set(self.bands) <= set(self.all_bands)

    filename_regex = re.compile(self.filename_regex, re.VERBOSE)
    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        match = re.match(filename_regex, os.path.basename(filepath))
        if match is not None:
            vrt = None
            try:
                vrt = self._load_warp_file(filepath=filepath, crs=crs)
                if self.cmap is None:
                    try:
                        colors = np.array(list(vrt.colormap(1).values())) / 255
                        self.cmap = ListedColormap(colors)
                    except ValueError:
                        pass
                if crs is None:
                    with rasterio.Env(OSR_WKT_FORMAT="WKT2_2018"):
                        crs = PROJ_CRS.from_user_input(vrt.crs)
                footprint = self.footprint_from_datasource(vrt)
                if footprint is None:
                    footprint = shapely.box(*vrt.bounds)
                geometries.append(footprint)
                if res is None:
                    res = vrt.res
            except rasterio.errors.RasterioIOError:
                continue
            else:
                filepaths.append(filepath)
                mint, maxt = self._filepath_to_timestamp(filepath)
                datetimes.append((mint, maxt))
            finally:
                if vrt is not None:
                    vrt.close()

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if not self.separate_files:
        self.band_indexes = None
        if self.bands:
            if self.all_bands:
                self.band_indexes = [
                    self.all_bands.index(i) + 1 for i in self.bands
                ]
            else:
                msg = (
                    f"{self.__class__.__name__} is missing an `all_bands` "
                    "attribute, so `bands` cannot be specified."
                )
                raise AssertionError(msg)

    if res is not None:
        if isinstance(res, (int, float)):
            res = (res, res)

        self._res = res

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/prisma.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt
    from keras import ops

    # RGB band indices based on https://doi.org/10.3390/rs14164080
    rgb_indices = [34, 23, 11]
    image = ops.take(sample["image"], rgb_indices, axis=-1)
    image = ops.cast(image, "float32")
    image = quantile_normalization(image)
    image = ops.convert_to_numpy(image)

    fig, ax = plt.subplots(1, 1, figsize=(4, 4))
    ax.imshow(image)
    ax.axis("off")

    if show_titles:
        ax.set_title("Image")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.quakeset.QuakeSet

QuakeSet(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

QuakeSet dataset.

QuakeSet <https://huggingface.co/datasets/DarthReca/quakeset>__ is a dataset for Earthquake Change Detection and Magnitude Estimation and is used for the Seismic Monitoring and Analysis (SMAC) ECML-PKDD 2024 Discovery Challenge.

Dataset features:

• Sentinel-1 SAR imagery
• before/pre/post imagery of areas affected by earthquakes
• 2 SAR bands (VV/VH)
• 3,327 pairs of pre and post images with 5 m per pixel resolution (512x512 px)
• 2 classification labels (unaffected / affected by earthquake)
• pre/post image pairs represent earthquake affected areas
• before/pre image pairs represent hard negative unaffected areas
• earthquake magnitudes for each sample

Dataset format:

• single hdf5 dataset containing images, magnitudes, hypercenters, and splits

Dataset classes:

1. unaffected area
2. earthquake affected area

If you use this dataset in your research, please cite:

• https://arxiv.org/abs/2403.18116

.. note::

This dataset requires the following additional library to be installed:

• h5py <https://pypi.org/project/h5py/>_ to load the dataset

Initialize a new QuakeSet dataset instance.

Args: root: root directory where dataset can be found split: one of "train", "val", or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: AssertionError: If split argument is invalid. DatasetNotFoundError: If dataset is not found and download is False. DependencyNotFoundError: If h5py is not installed.

Source code in keras_climate/datasets/quakeset.py

def __init__(
    self, root="data", split="train", transforms=None, download=False, checksum=True
):
    """Initialize a new QuakeSet dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val", or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: If ``split`` argument is invalid.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
        DependencyNotFoundError: If h5py is not installed.
    """
    lazy_import("h5py")

    assert split in self.splits

    self.root = root
    self.split = split
    self.transforms = transforms
    self.download = download
    self.checksum = checksum
    self.filepath = os.path.join(root, self.filename)
    self._verify()
    self.data = self._load_data()

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/quakeset.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image = sample["image"]
    label = int(ops.convert_to_numpy(sample["label"]))
    label_class = self.classes[label]

    # Create false color image for image1
    vv = quantile_normalization(image[..., 0]) + 1e-16
    vh = quantile_normalization(image[..., 1]) + 1e-16
    fci1 = ops.clip(ops.stack([vv, vh, vv / vh], axis=-1), 0, 1)
    fci1 = ops.convert_to_numpy(fci1)

    # Create false color image for image2
    vv = quantile_normalization(image[..., 2]) + 1e-16
    vh = quantile_normalization(image[..., 3]) + 1e-16
    fci2 = ops.clip(ops.stack([vv, vh, vv / vh], axis=-1), 0, 1)
    fci2 = ops.convert_to_numpy(fci2)

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = int(ops.convert_to_numpy(sample["prediction"]))
        prediction_class = self.classes[prediction]

    ncols = 2
    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 5, 10), sharex=True)

    axs[0].imshow(fci1)
    axs[0].axis("off")
    axs[0].set_title("Image Pre")
    axs[1].imshow(fci2)
    axs[1].axis("off")
    axs[1].set_title("Image Post")

    if show_titles:
        title = f"Label: {label_class}"
        if "magnitude" in sample:
            magnitude = float(ops.convert_to_numpy(sample["magnitude"]))
            title += f" | Magnitude: {magnitude:.2f}"
        if showing_predictions:
            title += f"\nPrediction: {prediction_class}"
        fig.supxlabel(title, y=0.22)

    if suptitle is not None:
        fig.suptitle(suptitle, y=0.8)

    fig.tight_layout()

    return fig

keras_climate.datasets.reforestree.ReforesTree

ReforesTree(root='data', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

ReforesTree dataset.

The ReforesTree <https://github.com/gyrrei/ReforesTree>__ dataset contains drone imagery that can be used for tree crown detection, tree species classification and Aboveground Biomass (AGB) estimation.

Dataset features:

• 100 high resolution RGB drone images at 2 cm/pixel of size 4,000 x 4,000 px
• more than 4,600 tree crown box annotations
• tree crown matched with field measurements of diameter at breast height (DBH), and computed AGB and carbon values

Dataset format:

• images are three-channel pngs
• annotations are csv file

Dataset classes:

1. other
2. banana
3. cacao
4. citrus
5. fruit
6. timber

If you use this dataset in your research, please cite:

• https://arxiv.org/abs/2201.11192

Initialize a new ReforesTree dataset instance.

Args: root: root directory where dataset can be found transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/reforestree.py

def __init__(self, root="data", transforms=None, download=False, checksum=True):
    """Initialize a new ReforesTree dataset instance.

    Args:
        root: root directory where dataset can be found
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.root = root
    self.transforms = transforms
    self.checksum = checksum
    self.download = download

    self._verify()

    self.files = self._load_files(self.root)

    self.annot_df = pd.read_csv(os.path.join(root, "mapping", "final_dataset.csv"))

    self.class2idx = {c: i for i, c in enumerate(self.classes)}

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/reforestree.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt
    from matplotlib import patches

    image = ops.convert_to_numpy(sample["image"]).astype("uint8")
    ncols = 1
    showing_predictions = "prediction_bbox_xyxy" in sample
    if showing_predictions:
        ncols += 1

    fig, axs = plt.subplots(ncols=ncols, figsize=(ncols * 10, 10))
    if not showing_predictions:
        axs = [axs]

    axs[0].imshow(image)
    axs[0].axis("off")

    bboxes = [
        patches.Rectangle(
            (bbox[0], bbox[1]),
            bbox[2] - bbox[0],
            bbox[3] - bbox[1],
            linewidth=1,
            edgecolor="r",
            facecolor="none",
        )
        for bbox in ops.convert_to_numpy(sample["bbox_xyxy"])
    ]
    for bbox in bboxes:
        axs[0].add_patch(bbox)

    if show_titles:
        axs[0].set_title("Ground Truth")

    if showing_predictions:
        axs[1].imshow(image)
        axs[1].axis("off")

        pred_bboxes = [
            patches.Rectangle(
                (bbox[0], bbox[1]),
                bbox[2] - bbox[0],
                bbox[3] - bbox[1],
                linewidth=1,
                edgecolor="r",
                facecolor="none",
            )
            for bbox in ops.convert_to_numpy(sample["prediction_bbox_xyxy"])
        ]
        for bbox in pred_bboxes:
            axs[1].add_patch(bbox)

        if show_titles:
            axs[1].set_title("Predictions")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.resisc45.RESISC45

RESISC45(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: NonGeoClassificationDataset

NWPU-RESISC45 dataset.

The RESISC45 <https://doi.org/10.1109/jproc.2017.2675998>__ dataset is a dataset for remote sensing image scene classification.

Dataset features:

• 31,500 images with 0.2-30 m per pixel resolution (256x256 px)
• three spectral bands - RGB
• 45 scene classes, 700 images per class

Dataset format:

• images are three-channel jpgs

This dataset uses the train/val/test splits defined in the "In-domain representation learning for remote sensing" paper:

• https://arxiv.org/abs/1911.06721

If you use this dataset in your research, please cite:

• https://doi.org/10.1109/jproc.2017.2675998

Initialize a new RESISC45 dataset instance.

Args: root: root directory where dataset can be found split: one of "train", "val", or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: AssertionError: if split argument is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/resisc45.py

def __init__(
    self, root="data", split="train", transforms=None, download=False, checksum=True
):
    """Initialize a new RESISC45 dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val", or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in self.splits
    self.root = root
    self.download = download
    self.checksum = checksum
    self._verify()

    valid_fns = set()
    with open(os.path.join(self.root, f"resisc45-{split}.txt")) as f:
        for fn in f:
            valid_fns.add(fn.strip())

    def is_in_split(x):
        return os.path.basename(x) in valid_fns

    super().__init__(
        root=os.path.join(root, self.directory),
        transforms=transforms,
        is_valid_file=is_in_split,
    )

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/resisc45.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt
    from keras import ops

    image = ops.convert_to_numpy(sample["image"]).astype("uint8")
    label = int(ops.convert_to_numpy(sample["label"]))
    label_class = self.classes[label]

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = int(ops.convert_to_numpy(sample["prediction"]))
        prediction_class = self.classes[prediction]

    fig, ax = plt.subplots(figsize=(4, 4))
    ax.imshow(image)
    ax.axis("off")
    if show_titles:
        title = f"Label: {label_class}"
        if showing_predictions:
            title += f"\nPrediction: {prediction_class}"
        ax.set_title(title)

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.rwanda_field_boundary.RwandaFieldBoundary

RwandaFieldBoundary(root='data', split='train', bands=all_bands, transforms=None, download=False)

Bases: NonGeoDataset

Rwanda Field Boundary Competition dataset.

This dataset contains field boundaries for smallholder farms in eastern Rwanda. The Nasa Harvest program funded a team of annotators from TaQadam to label Planet imagery for the 2021 growing season for the purpose of conducting the Rwanda Field boundary detection Challenge. The dataset includes rasterized labeled field boundaries and time series satellite imagery from Planet's NICFI program. Planet's basemap imagery is provided for six months (March, April, August, October, November and December). Note: only fields that were big enough to be differentiated on the Planetscope imagery were labeled, only fields that were fully contained within the chips were labeled. The paired dataset is provided in 256x256 chips for a total of 70 tiles covering 1532 individual fields.

The labels are provided as binary semantic segmentation labels:

1. No field-boundary
2. Field-boundary

If you use this dataset in your research, please cite:

• https://doi.org/10.34911/RDNT.G580WW

.. note::

This dataset requires the following additional library to be installed:

• azcopy <https://github.com/Azure/azure-storage-azcopy>_: to download the dataset from Source Cooperative.

Initialize a new RwandaFieldBoundary instance.

Args: root: root directory where dataset can be found split: one of "train" or "test" bands: the subset of bands to load transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory

Raises: AssertionError: If split or bands are invalid. DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/rwanda_field_boundary.py

def __init__(
    self, root="data", split="train", bands=all_bands, transforms=None, download=False
):
    """Initialize a new RwandaFieldBoundary instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        bands: the subset of bands to load
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory

    Raises:
        AssertionError: If *split* or *bands* are invalid.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in self.splits
    assert set(bands) <= set(self.all_bands)

    self.root = root
    self.split = split
    self.bands = bands
    self.transforms = transforms
    self.download = download

    self._verify()

plot

plot(sample, show_titles=True, suptitle=None, time_step=0)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/rwanda_field_boundary.py

def plot(self, sample, show_titles=True, suptitle=None, time_step=0):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise RGBBandsMissingError()

    ncols = 1
    for key in ("mask", "prediction"):
        if key in sample:
            ncols += 1

    fig, axs = plt.subplots(ncols=ncols, squeeze=False)

    image = ops.convert_to_numpy(sample["image"])[time_step]
    image = np.take(image, rgb_indices, axis=-1)
    image = np.clip(image / 2000, 0, 1)
    axs[0, 0].imshow(image)
    axs[0, 0].axis("off")
    if show_titles:
        axs[0, 0].set_title(f"t={time_step}")

    if "mask" in sample:
        axs[0, 1].imshow(ops.convert_to_numpy(sample["mask"]))
        axs[0, 1].axis("off")
        if show_titles:
            axs[0, 1].set_title("Mask")

    if "prediction" in sample:
        axs[0, 2].imshow(ops.convert_to_numpy(sample["prediction"]))
        axs[0, 2].axis("off")
        if show_titles:
            axs[0, 2].set_title("Prediction")

    if suptitle is not None:
        fig.suptitle(suptitle)

    return fig

keras_climate.datasets.s2_100k.S2100k

S2100k(root='data', mode='both', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

S2-100K dataset.

The S2-100k dataset <https://hf.co/datasets/kklmmr/s2-100k>__ contains 100,000 256x256 patches of 12 band Sentinel imagery sampled randomly from Sentinel 2 scenes on the Microsoft Planetary Computer that have <20% cloud cover, intersect land, and were captured between 2021-01-01 and 2023-05-17.

If you use this dataset in your research, please cite:

• https://doi.org/10.1609/aaai.v39i4.32457

Initialize a new S2100K dataset instance.

Args: root: root directory where dataset can be found mode: which data to return (options are "both" or "points"), useful for embedding locations without loading images transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: If mode argument is invalid. DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/s2_100k.py

def __init__(
    self, root="data", mode="both", transforms=None, download=False, checksum=True
):
    """Initialize a new S2100K dataset instance.

    Args:
        root: root directory where dataset can be found
        mode: which data to return (options are "both" or "points"),
            useful for embedding locations without loading images
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: If *mode* argument is invalid.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert mode in {"both", "points"}

    self.root = pathlib.Path(root)
    self.transforms = transforms
    self.mode = mode
    self.download = download
    self.checksum = checksum

    self._verify()

    self.index = pd.read_csv(self.root / "index.csv")

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/s2_100k.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"])
    image = np.take(image, [3, 2, 1], axis=-1)
    image = np.clip(image / 4000, 0, 1)

    fig, ax = plt.subplots(figsize=(4, 4))

    ax.imshow(image)
    ax.axis("off")

    if show_titles:
        point = ops.convert_to_numpy(sample["point"])
        ax.set_title(f"({point[0]:0.4f}, {point[1]:0.4f})")

    if suptitle is not None:
        fig.suptitle(suptitle)

    return fig

keras_climate.datasets.satlas.SatlasPretrain

SatlasPretrain(root='data', split='train_lowres', good_images='good_images_lowres_all', image_times='image_times', images=('sentinel1', 'sentinel2', 'landsat'), labels=('land_cover',), transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

SatlasPretrain dataset.

SatlasPretrain <https://satlas-pretrain.allen.ai/>__ is a large-scale pre-training dataset for tasks that involve understanding satellite images. Regularly-updated satellite data is publicly available for much of the Earth through sources such as Sentinel-2 and NAIP, and can inform numerous applications from tackling illegal deforestation to monitoring marine infrastructure. By combining over 30 TB of satellite images with 137 label categories, SatlasPretrain serves as an effective pre-training dataset that greatly reduces the effort needed to develop robust models for downstream satellite image applications.

Reference implementation:

• https://github.com/allenai/satlas/blob/main/satlas/model/dataset.py

If you use this dataset in your research, please cite:

• https://doi.org/10.48550/arXiv.2211.15660

.. note:: This dataset requires the following additional library to be installed:

• AWS CLI <https://aws.amazon.com/cli/>_: to download the dataset from AWS.

Initialize a new SatlasPretrain instance.

Args: root: root directory where dataset can be found split: metadata split to load good_images: metadata mapping between col/row and directory image_times: metadata mapping between directory and ISO time images: list of image products labels: list of label products transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: If images is invalid. DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/satlas.py

def __init__(
    self,
    root="data",
    split="train_lowres",
    good_images="good_images_lowres_all",
    image_times="image_times",
    images=("sentinel1", "sentinel2", "landsat"),
    labels=("land_cover",),
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new SatlasPretrain instance.

    Args:
        root: root directory where dataset can be found
        split: metadata split to load
        good_images: metadata mapping between col/row and directory
        image_times: metadata mapping between directory and ISO time
        images: list of image products
        labels: list of label products
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: If *images* is invalid.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert set(images) <= set(self.bands.keys())

    self.root = root
    self.images = images
    self.labels = labels
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    # Read metadata files
    self.split = pd.read_json(
        os.path.join(root, "metadata", f"{split}.json"), typ="frame"
    )
    good_images_df = pd.read_json(
        os.path.join(root, "metadata", f"{good_images}.json"), typ="frame"
    )
    self.image_times = pd.read_json(
        os.path.join(root, "metadata", f"{image_times}.json"), typ="series"
    )

    self.split.columns = ["col", "row"]
    good_images_df.columns = ["col", "row", "directory"]
    self.good_images = good_images_df.groupby(["col", "row"])

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/satlas.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    images = []
    titles = []
    for key, value in sample.items():
        value = ops.convert_to_numpy(value)
        parts = key.split("_", 1)
        match parts:
            case ["image", "landsat"]:
                images.append(np.take(value, [3, 2, 1], axis=-1) / 255)
                titles.append("Landsat 8/9")
            case ["image", "naip"]:
                images.append(value[..., :3] / 255)
                titles.append("NAIP")
            case ["image", "sentinel1"]:
                images.extend([value[..., 0] / 255, value[..., 1] / 255])
                titles.extend(["Sentinel-1 VH", "Sentinel-1 VV"])
            case ["image", "sentinel2"]:
                images.append(value[..., :3] / 255)
                titles.append("Sentinel-2")
            case [("mask" | "prediction"), label]:
                cmap = np.array(TASKS[label]["colors"])
                images.append(cmap[value])
                titles.append(label.replace("_", " ").capitalize())

    fig, ax = plt.subplots(ncols=len(images), squeeze=False)
    for i, (image, title) in enumerate(zip(images, titles)):
        ax[0, i].imshow(image)
        ax[0, i].axis("off")

        if show_titles:
            ax[0, i].set_title(title)

    if suptitle is not None:
        fig.suptitle(suptitle)

    return fig

keras_climate.datasets.seasonet.SeasoNet

SeasoNet(root='data', split='train', seasons=all_seasons, bands=all_bands, grids=[1, 2], concat_seasons=1, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

SeasoNet Semantic Segmentation dataset.

The SeasoNet <https://doi.org/10.5281/zenodo.5850306>__ dataset consists of 1,759,830 multi-spectral Sentinel-2 image patches, taken from 519,547 unique locations, covering the whole surface area of Germany. Annotations are provided in the form of pixel-level land cover and land usage segmentation masks from the German land cover model LBM-DE2018 with land cover classes based on the CORINE Land Cover database (CLC) 2018. The set is split into two overlapping grids, consisting of roughly 880,000 samples each, which are shifted by half the patch size in both dimensions. The images in each of the both grids themselves do not overlap.

Dataset format:

• images are 16-bit GeoTiffs, split into separate files based on resolution
• images include 12 spectral bands with 10, 20 and 60 m per pixel resolutions
• masks are single-channel 8-bit GeoTiffs

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1109/IGARSS46834.2022.9884079

Initialize a new SeasoNet dataset instance.

Args: root: root directory where dataset can be found split: one of "train", "val" or "test" seasons: list of seasons to load bands: list of bands to load grids: which of the overlapping grids to load concat_seasons: number of seasonal images to return per sample. if 1, each seasonal image is returned as its own sample, otherwise seasonal images are randomly picked from the seasons specified in seasons and returned stacked along the channel axis transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/seasonet.py

def __init__(
    self,
    root="data",
    split="train",
    seasons=all_seasons,
    bands=all_bands,
    grids=[1, 2],
    concat_seasons=1,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new SeasoNet dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val" or "test"
        seasons: list of seasons to load
        bands: list of bands to load
        grids: which of the overlapping grids to load
        concat_seasons: number of seasonal images to return per sample.
            if 1, each seasonal image is returned as its own sample,
            otherwise seasonal images are randomly picked from the seasons
            specified in ``seasons`` and returned stacked along the
            channel axis
        transforms: a function/transform that takes input sample and its target as
            entry and returns a transformed version
        download: if True, download dataset and store it in the root directory
        checksum: if True, check the MD5 of the downloaded files (may be slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    assert split in self.splits
    assert set(seasons) <= self.all_seasons
    assert set(bands) <= set(self.all_bands)
    assert set(grids) <= {1, 2}
    assert concat_seasons in range(1, len(seasons) + 1)

    self.root = root
    self.bands = bands
    self.concat_seasons = concat_seasons
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    self.channels = 0
    for b in bands:
        self.channels += self.band_nums[b]

    csv = pd.read_csv(os.path.join(self.root, "meta.csv"), index_col="Index")

    if split is not None:
        # Filter entries by split
        split_csv = pd.read_csv(
            os.path.join(self.root, f"splits/{split}.csv"), header=None
        )[0]
        csv = csv.iloc[split_csv]

    # Filter entries by grids and seasons
    csv = csv[csv["Grid"].isin(grids)]
    csv = csv[csv["Season"].isin(seasons)]

    # Replace relative data paths with absolute paths
    csv["Path"] = csv["Path"].apply(
        lambda p: [os.path.join(self.root, p, os.path.basename(p))]
    )

    if self.concat_seasons > 1:
        # Group entries by location
        files = csv.groupby(["Latitude", "Longitude"])
        self.files = files["Path"].agg("sum")

        # Remove entries with less than concat_seasons available seasons
        self.files = self.files[
            self.files.apply(lambda d: len(d) >= self.concat_seasons)
        ]
    else:
        self.files = csv["Path"]

__getitem__

__getitem__(index)

Return an index within the dataset.

Returns: sample containing the image with shape T x H x W x C and the mask with shape H x W, where T = self.concat_seasons

Source code in keras_climate/datasets/seasonet.py

def __getitem__(self, index):
    """Return an index within the dataset.

    Returns:
        sample containing the image with shape T x H x W x C and the mask
        with shape H x W, where ``T = self.concat_seasons``
    """
    image = self._load_image(index)
    mask = self._load_target(index)
    sample = {"image": image, "mask": mask}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None, show_legend=True)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/seasonet.py

def plot(self, sample, show_titles=True, suptitle=None, show_legend=True):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.patches as mpatches
    import matplotlib.pyplot as plt

    if "10m_RGB" not in self.bands:
        raise RGBBandsMissingError()

    ncols = self.concat_seasons + 1

    images, mask = sample["image"], sample["mask"]
    mask = ops.convert_to_numpy(mask)
    show_predictions = "prediction" in sample
    if show_predictions:
        prediction = ops.convert_to_numpy(sample["prediction"])
        ncols += 1

    start = 0
    for b in self.bands:
        if b == "10m_RGB":
            break
        start += self.band_nums[b]

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 4.5, 5))
    fig.subplots_adjust(wspace=0.05)
    for t in range(self.concat_seasons):
        image = images[t, :, :, start : start + 3]
        image = quantile_normalization(image)
        axs[t].imshow(ops.convert_to_numpy(image))
        axs[t].axis("off")
        if show_titles:
            axs[t].set_title(f"Image {t + 1}")

    axs[self.concat_seasons].imshow(
        mask, vmin=0, vmax=32, cmap=self.cmap, interpolation="none"
    )
    axs[self.concat_seasons].axis("off")
    if show_titles:
        axs[self.concat_seasons].set_title("Mask")

    if show_predictions:
        axs[self.concat_seasons + 1].imshow(
            prediction, vmin=0, vmax=32, cmap=self.cmap, interpolation="none"
        )
        axs[self.concat_seasons + 1].axis("off")
        if show_titles:
            axs[self.concat_seasons + 1].set_title("Prediction")

    if show_legend:
        lgd = np.unique(mask)

        if show_predictions:
            lgd = np.union1d(lgd, np.unique(prediction))
        patches = [
            mpatches.Patch(color=self.cmap(i), label=self.classes[i]) for i in lgd
        ]
        plt.legend(
            handles=patches, bbox_to_anchor=(1.05, 1), borderaxespad=0, loc=2
        )

    if suptitle is not None:
        plt.suptitle(suptitle, size="xx-large")

    return fig

keras_climate.datasets.seco.SeasonalContrastS2

SeasonalContrastS2(root='data', version='100k', seasons=1, bands=rgb_bands, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

Sentinel 2 imagery from the Seasonal Contrast paper.

The Seasonal Contrast imagery <https://github.com/ServiceNow/seasonal-contrast>_ dataset contains Sentinel 2 imagery patches sampled from different points in time around the 10k most populated cities on Earth.

Dataset features:

• Two versions: 100K and 1M patches
• 12 band Sentinel 2 imagery from 5 points in time at each location

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/pdf/2103.16607

Initialize a new SeasonalContrastS2 instance.

Args: root: root directory where dataset can be found version: one of "100k" or "1m" for the version of the dataset to use seasons: number of seasonal patches to sample per location, 1--5 bands: list of bands to load transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: if version argument is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/seco.py

def __init__(
    self,
    root="data",
    version="100k",
    seasons=1,
    bands=rgb_bands,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new SeasonalContrastS2 instance.

    Args:
        root: root directory where dataset can be found
        version: one of "100k" or "1m" for the version of the dataset to use
        seasons: number of seasonal patches to sample per location, 1--5
        bands: list of bands to load
        transforms: a function/transform that takes input sample and its target as
            entry and returns a transformed version
        download: if True, download dataset and store it in the root directory
        checksum: if True, check the MD5 of the downloaded files (may be slow)

    Raises:
        AssertionError: if ``version`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    assert version in self.metadata
    assert seasons in range(5)
    for band in bands:
        assert band in self.all_bands

    self.root = root
    self.version = version
    self.seasons = seasons
    self.bands = bands
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

__getitem__

__getitem__(index)

Return an index within the dataset.

Returns: sample with an "image" in H x W x SC format where S is the number of seasons

Source code in keras_climate/datasets/seco.py

def __getitem__(self, index):
    """Return an index within the dataset.

    Returns:
        sample with an "image" in H x W x SC format where S is the number of
        seasons
    """
    root = os.path.join(
        self.root, self.metadata[self.version]["directory"], f"{index:06}"
    )
    subdirs = [f for f in os.listdir(root) if os.path.isdir(os.path.join(root, f))]
    subdirs = random.sample(subdirs, self.seasons)

    images = [self._load_patch(root, subdir) for subdir in subdirs]

    sample = {"image": ops.concatenate(images, axis=-1)}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands. ValueError: if sample contains a "prediction" key

Source code in keras_climate/datasets/seco.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
        ValueError: if sample contains a "prediction" key
    """
    import matplotlib.pyplot as plt

    if "prediction" in sample:
        raise ValueError("This dataset doesn't support plotting predictions")

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise RGBBandsMissingError()

    fig, axes = plt.subplots(ncols=self.seasons, figsize=(20, 4))
    if self.seasons == 1:
        axes = [axes]

    for i in range(self.seasons):
        indices = [idx + i * len(self.bands) for idx in rgb_indices]
        image = ops.take(sample["image"], indices, axis=-1)
        image = quantile_normalization(image)

        axes[i].imshow(ops.convert_to_numpy(image))
        axes[i].axis("off")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.sen12ms.SEN12MS

SEN12MS(root='data', split='train', bands=BAND_SETS['all'], transforms=None, checksum=True)

Bases: NonGeoDataset

SEN12MS dataset.

The SEN12MS <https://doi.org/10.14459/2019mp1474000>__ dataset contains 180,662 patch triplets of corresponding Sentinel-1 dual-pol SAR data, Sentinel-2 multi-spectral images, and MODIS-derived land cover maps. The patches are distributed across the land masses of the Earth and spread over all four meteorological seasons. This is reflected by the dataset structure. All patches are provided in the form of 16-bit GeoTiffs containing the following specific information:

• Sentinel-1 SAR: 2 channels corresponding to sigma nought backscatter values in dB scale for VV and VH polarization.
• Sentinel-2 Multi-Spectral: 13 channels corresponding to the 13 spectral bands (B1, B2, B3, B4, B5, B6, B7, B8, B8a, B9, B10, B11, B12).
• MODIS Land Cover: 4 channels corresponding to IGBP, LCCS Land Cover, LCCS Land Use, and LCCS Surface Hydrology layers.

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.5194/isprs-annals-IV-2-W7-153-2019

.. note::

This dataset must be manually downloaded from https://dataserv.ub.tum.de/s/m1474000 and https://github.com/schmitt-muc/SEN12MS/tree/master/splits.

Initialize a new SEN12MS dataset instance.

The bands argument allows for the subsetting of bands returned by the dataset. Names in bands index into a stack of Sentinel 1 and Sentinel 2 imagery, where band_names[0:2] correspond to the Sentinel 1 imagery and band_names[2:15] correspond to the Sentinel 2 imagery.

Args: root: root directory where dataset can be found split: one of "train" or "test" bands: a sequence of band names to use transforms: a function/transform that takes input sample and its target as entry and returns a transformed version checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: if split argument is invalid DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/sen12ms.py

def __init__(
    self, root="data", split="train", bands=BAND_SETS["all"], transforms=None, checksum=True
):
    """Initialize a new SEN12MS dataset instance.

    The ``bands`` argument allows for the subsetting of bands returned by the
    dataset. Names in ``bands`` index into a stack of Sentinel 1 and Sentinel 2
    imagery, where ``band_names[0:2]`` correspond to the Sentinel 1 imagery and
    ``band_names[2:15]`` correspond to the Sentinel 2 imagery.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        bands: a sequence of band names to use
        transforms: a function/transform that takes input sample and its target as
            entry and returns a transformed version
        checksum: if True, check the MD5 of the downloaded files (may be slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found.
    """
    assert split in ["train", "test"]

    self._validate_bands(bands)
    self.band_indices = np.array(
        [self.band_names.index(b) for b in bands], dtype="int64"
    )
    self.bands = bands

    self.root = root
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    if (
        checksum and not self._check_integrity()
    ) or not self._check_integrity_light():
        raise DatasetNotFoundError(self)

    with open(os.path.join(self.root, split + "_list.txt")) as f:
        self.ids = [line.rstrip() for line in f]

__getitem__

__getitem__(index)

Return an index within the dataset.

Returns: data and label at that index

Source code in keras_climate/datasets/sen12ms.py

def __getitem__(self, index):
    """Return an index within the dataset.

    Returns:
        data and label at that index
    """
    filename = self.ids[index]

    lc = self._load_raster(filename, "lc")
    s1 = self._load_raster(filename, "s1")
    s2 = self._load_raster(filename, "s2")

    image = ops.concatenate([s1, s2], axis=-1)
    image = ops.take(image, self.band_indices, axis=-1)

    sample = {"image": image, "mask": ops.cast(lc[..., 0], "int64")}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/sen12ms.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise RGBBandsMissingError()

    image = ops.take(sample["image"], rgb_indices, axis=-1)
    image = quantile_normalization(image)
    mask = ops.convert_to_numpy(sample["mask"])
    ncols = 2

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = ops.convert_to_numpy(sample["prediction"])
        ncols += 1

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 5, 10))

    axs[0].imshow(ops.convert_to_numpy(image))
    axs[0].axis("off")
    axs[1].imshow(mask)
    axs[1].axis("off")

    if showing_predictions:
        axs[2].imshow(prediction)
        axs[2].axis("off")

    if show_titles:
        axs[0].set_title("Image")
        axs[1].set_title("Mask")
        if showing_predictions:
            axs[2].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.sentinel.Sentinel

Sentinel(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset

Abstract base class for all Sentinel datasets.

Sentinel <https://sentinels.copernicus.eu/>__ is a family of satellites launched by the European Space Agency (ESA) <https://www.esa.int/>_ under the Copernicus Programme <https://www.copernicus.eu/en>_.

If you use this dataset in your research, please cite it using the following format:

• https://sentinels.copernicus.eu/documents/247904/690755/Sentinel_Data_Legal_Notice

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new RasterDataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a mosaic
            (``[H, W, C]``). For mask-style datasets (``is_image=False``),
            single-band data has the channel dimension squeezed.
    """
    self.paths = paths
    self.bands = bands or self.all_bands
    self.transforms = transforms
    self.cache = cache
    self.time_series = time_series

    if self.all_bands:
        assert set(self.bands) <= set(self.all_bands)

    filename_regex = re.compile(self.filename_regex, re.VERBOSE)
    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        match = re.match(filename_regex, os.path.basename(filepath))
        if match is not None:
            vrt = None
            try:
                vrt = self._load_warp_file(filepath=filepath, crs=crs)
                if self.cmap is None:
                    try:
                        colors = np.array(list(vrt.colormap(1).values())) / 255
                        self.cmap = ListedColormap(colors)
                    except ValueError:
                        pass
                if crs is None:
                    with rasterio.Env(OSR_WKT_FORMAT="WKT2_2018"):
                        crs = PROJ_CRS.from_user_input(vrt.crs)
                footprint = self.footprint_from_datasource(vrt)
                if footprint is None:
                    footprint = shapely.box(*vrt.bounds)
                geometries.append(footprint)
                if res is None:
                    res = vrt.res
            except rasterio.errors.RasterioIOError:
                continue
            else:
                filepaths.append(filepath)
                mint, maxt = self._filepath_to_timestamp(filepath)
                datetimes.append((mint, maxt))
            finally:
                if vrt is not None:
                    vrt.close()

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if not self.separate_files:
        self.band_indexes = None
        if self.bands:
            if self.all_bands:
                self.band_indexes = [
                    self.all_bands.index(i) + 1 for i in self.bands
                ]
            else:
                msg = (
                    f"{self.__class__.__name__} is missing an `all_bands` "
                    "attribute, so `bands` cannot be specified."
                )
                raise AssertionError(msg)

    if res is not None:
        if isinstance(res, (int, float)):
            res = (res, res)

        self._res = res

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

keras_climate.datasets.sentinel.Sentinel1

Sentinel1(paths='data', crs=None, res=(10, 10), bands=['VV', 'VH'], transforms=None, cache=True, time_series=False)

Bases: Sentinel

Sentinel-1 dataset.

The Sentinel-1 mission <https://sentiwiki.copernicus.eu/web/s1-mission>_ comprises a constellation of two polar-orbiting satellites, operating day and night performing C-band synthetic aperture radar imaging, enabling them to acquire imagery regardless of the weather.

Data can be downloaded from:

• Copernicus Data Space Ecosystem <https://dataspace.copernicus.eu/>_
• Alaska Satellite Facility (ASF) Distributed Active Archive Center (DAAC) <https://asf.alaska.edu/>_
• Microsoft's Planetary Computer <https://planetarycomputer.microsoft.com/dataset/sentinel-1-rtc>_

Polarizations:

• HH: horizontal transmit, horizontal receive
• HV: horizontal transmit, vertical receive
• VV: vertical transmit, vertical receive
• VH: vertical transmit, horizontal receive

.. note:: At the moment, this dataset only supports the GRD product type. Data must be radiometrically terrain corrected (RTC). This can be done manually using a DEM, or you can download an On Demand RTC product from ASF DAAC.

Initialize a new Dataset instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS in (xres, yres) format. If a single float is provided, it is used for both the x and y resolution. (defaults to the resolution of the first file found) bands: bands to return (defaults to ["VV", "VH"]) transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling time_series: if True, stack data along the time series dimension [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

Raises: AssertionError: if bands is invalid DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/sentinel.py

    def __init__(
        self,
        paths="data",
        crs=None,
        res=(10, 10),
        bands=["VV", "VH"],
        transforms=None,
        cache=True,
        time_series=False,
    ):
        """Initialize a new Dataset instance.

        Args:
            paths: one or more root directories to search or files to load
            crs: CRS to warp to (defaults to the CRS of the first file found)
            res: resolution of the dataset in units of CRS in (xres, yres) format. If a
                single float is provided, it is used for both the x and y resolution.
                (defaults to the resolution of the first file found)
            bands: bands to return (defaults to ["VV", "VH"])
            transforms: a function/transform that takes an input sample
                and returns a transformed version
            cache: if True, cache file handle to speed up repeated sampling
            time_series: if True, stack data along the time series dimension
                [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

        Raises:
            AssertionError: if ``bands`` is invalid
            DatasetNotFoundError: If dataset is not found.
        """
        assert len(bands) > 0, "'bands' cannot be an empty list"
        assert len(bands) == len(set(bands)), "'bands' contains duplicate bands"

        for band in bands:
            assert band in self.all_bands, f"invalid band '{band}'"

        # Sentinel-1 uses dual polarization, it can only transmit either
        # horizontal or vertical at a single time
        msg = """
'bands' cannot contain both horizontal and vertical transmit at the same time.
To create a dataset containing both, use:

    horizontal = Sentinel1(root, bands=["HH", "HV"])
    vertical = Sentinel1(root, bands=["VV", "VH"])
    dataset = horizontal | vertical
"""
        assert len({band[0] for band in bands}) == 1, msg

        self.filename_glob = self.filename_glob.format(bands[0])

        super().__init__(paths, crs, res, bands, transforms, cache, time_series)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/sentinel.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    bands = self.bands

    if len(bands) == 1:
        # Only horizontal or vertical receive, plot as grayscale
        image = sample["image"][..., 0]
        image = ops.clip(image, 0, 1)

        title = f"({bands[0]})"
    else:
        # Both horizontal and vertical receive, plot as RGB

        # Deal with reverse order
        if bands in [["HV", "HH"], ["VH", "VV"]]:
            bands = bands[::-1]
            sample["image"] = ops.flip(sample["image"], axis=-1)

        co_polarization = sample["image"][..., 0]  # transmit == receive
        cross_polarization = sample["image"][..., 1]  # transmit != receive
        ratio = co_polarization / cross_polarization

        # https://gis.stackexchange.com/a/400780/123758
        co_polarization = ops.clip(co_polarization / 0.3, 0, 1)
        cross_polarization = ops.clip(cross_polarization / 0.05, 0, 1)
        ratio = ops.clip(ratio / 25, 0, 1)

        image = ops.stack((co_polarization, cross_polarization, ratio), axis=-1)

        title = "({0}, {1}, {0}/{1})".format(*bands)

    fig, ax = plt.subplots(1, 1, figsize=(4, 4))

    ax.imshow(ops.convert_to_numpy(image))
    ax.axis("off")

    if show_titles:
        ax.set_title(title)

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.sentinel.Sentinel2

Sentinel2(paths='data', crs=None, res=10, bands=None, transforms=None, cache=True, time_series=False)

Bases: Sentinel

Sentinel-2 dataset.

The Copernicus Sentinel-2 mission <https://sentiwiki.copernicus.eu/web/s2-mission>_ comprises a constellation of two polar-orbiting satellites placed in the same sun-synchronous orbit, phased at 180° to each other. It aims at monitoring variability in land surface conditions, and its wide swath width (290 km) and high revisit time will support monitoring of Earth's surface changes.

Initialize a new Dataset instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS in (xres, yres) format. If a single float is provided, it is used for both the x and y resolution. (defaults to the resolution of the first file found) bands: bands to return (defaults to all bands) transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling time_series: if True, stack data along the time series dimension [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

Raises: DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/sentinel.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=10,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS in (xres, yres) format. If a
            single float is provided, it is used for both the x and y resolution.
            (defaults to the resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample
            and returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found.
    """
    bands = bands or self.all_bands
    self.filename_glob = self.filename_glob.format(bands[0])
    self.filename_regex = self.filename_regex.format(self.resolutions[bands[0]])
    super().__init__(paths, crs, res, bands, transforms, cache, time_series)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/sentinel.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise RGBBandsMissingError()

    image = ops.take(sample["image"], rgb_indices, axis=-1)
    # DN = 10000 * REFLECTANCE
    # https://docs.sentinel-hub.com/api/latest/data/sentinel-2-l2a/
    image = ops.clip(image / 10000, 0, 1)

    fig, ax = plt.subplots(1, 1, figsize=(4, 4))

    ax.imshow(ops.convert_to_numpy(image))
    ax.axis("off")

    if show_titles:
        ax.set_title("Image")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

footprint_from_datasource

footprint_from_datasource(datasource)

Extract the true geometric footprint from a Sentinel-2 datasource.

This method reads the original footprint geometry from the associated Sentinel-2 metadata XML file (MTD_MSIL1C.xml) and transforms it from geographic coordinates (EPSG:4326) into the CRS of the given dataset.

Returns: The true footprint in the dataset's CRS, or None if the metadata file is not found (falling back to the raster's bounding box).

Source code in keras_climate/datasets/sentinel.py

def footprint_from_datasource(self, datasource):
    """Extract the true geometric footprint from a Sentinel-2 datasource.

    This method reads the original footprint geometry from the associated
    Sentinel-2 metadata XML file (`MTD_MSIL1C.xml`) and transforms it from
    geographic coordinates (EPSG:4326) into the CRS of the given dataset.

    Returns:
        The true footprint in the dataset's CRS, or ``None`` if the metadata
        file is not found (falling back to the raster's bounding box).
    """
    if hasattr(datasource, "src_dataset"):
        # When dataset is a WarpedVRT
        filepath = datasource.src_dataset.name
    else:
        filepath = datasource.name
    metadata_path = filepath.split("GRANULE")[0] + "MTD_MSIL1C.xml"
    if not os.path.exists(metadata_path):
        return None
    with warnings.catch_warnings():
        # XML file does not have a CRS
        warnings.simplefilter("ignore")
        with rasterio.open(metadata_path) as metadata_src:
            tags = metadata_src.tags()
    # The FOOTPRINT tag in MTD_MSIL1C.xml is always stored in EPSG:4326.
    footprint = gpd.GeoSeries.from_wkt([tags["FOOTPRINT"]], crs="EPSG:4326")
    return footprint.to_crs(datasource.crs).iloc[0]

keras_climate.datasets.skippd.SKIPPD

SKIPPD(root='data', split='trainval', task='nowcast', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

SKy Images and Photovoltaic Power Dataset (SKIPP'D).

The SKIPP'D dataset <https://purl.stanford.edu/dj417rh1007>_ contains ground-based fish-eye photos of the sky for solar forecasting tasks.

Dataset Format:

• .hdf5 file containing images and labels

Dataset Features:

• fish-eye RGB images (64x64px)
• power output measurements from 30-kW rooftop PV array
• 1-min interval across 3 years (2017-2019)

Nowcast task:

• 349,372 images under the split key trainval
• 14,003 images under the split key test

Forecast task:

• 130,412 images under the split key trainval
• 2,462 images under the split key test
• consists of a concatenated RGB time-series of 16 time-steps

If you use this dataset in your research, please cite:

• https://doi.org/10.48550/arXiv.2207.00913

.. note::

This dataset requires the following additional library to be installed:

• <https://pypi.org/project/h5py/>_ to load the dataset

Initialize a new Dataset instance.

Args: root: root directory where dataset can be found split: one of "trainval", or "test" task: one of "nowcast", or "forecast" transforms: a function/transform that takes an input sample and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum after downloading files (may be slow)

Raises: AssertionError: if task or split is invalid DatasetNotFoundError: If dataset is not found and download is False. DependencyNotFoundError: If h5py is not installed.

Source code in keras_climate/datasets/skippd.py

def __init__(
    self,
    root="data",
    split="trainval",
    task="nowcast",
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new Dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "trainval", or "test"
        task: one of "nowcast", or "forecast"
        transforms: a function/transform that takes an input sample
            and returns a transformed version
        download: if True, download dataset and store it in the root directory
        checksum: if True, verify the checksum after downloading files (may be slow)

    Raises:
        AssertionError: if ``task`` or ``split`` is invalid
        DatasetNotFoundError: If dataset is not found and *download* is False.
        DependencyNotFoundError: If h5py is not installed.
    """
    lazy_import("h5py")

    assert split in self.valid_splits, (
        f"Please choose one of these valid data splits {self.valid_splits}."
    )
    self.split = split

    assert task in self.valid_tasks, (
        f"Please choose one of these valid tasks {self.valid_tasks}."
    )
    self.task = task

    self.root = root
    self.transforms = transforms
    self.download = download
    self.checksum = checksum
    self._verify()

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/skippd.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    sample = {"image": self._load_image(index)}
    sample.update(self._load_features(index))

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

In the forecast task the latest image is plotted.

Source code in keras_climate/datasets/skippd.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    In the ``forecast`` task the latest image is plotted.
    """
    import matplotlib.pyplot as plt

    if self.task == "nowcast":
        image = ops.convert_to_numpy(sample["image"])
        label = float(ops.convert_to_numpy(sample["label"]))
    else:
        image_np = ops.convert_to_numpy(sample["image"]).reshape(64, 64, 16, 3)
        image = image_np[:, :, -1, :]
        label = float(ops.convert_to_numpy(sample["label"])[-1])

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = ops.convert_to_numpy(sample["prediction"])
        prediction = float(
            prediction[-1] if prediction.ndim > 0 else prediction
        )

    fig, ax = plt.subplots(1, 1, figsize=(10, 10))

    ax.imshow(image / 255)
    ax.axis("off")

    if show_titles:
        title = f"Label: {label:.3f}"
        if showing_predictions:
            title += f"\nPrediction: {prediction:.3f}"
        ax.set_title(title)

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.skyscript.SkyScript

SkyScript(root='data', split='train', transforms=None, download=False, checksum=True, tokenizer=None)

Bases: NonGeoDataset

SkyScript dataset.

SkyScript <https://github.com/wangzhecheng/SkyScript>__ is a large and semantically diverse image-text dataset for remote sensing images. It contains 5.2 million remote sensing image-text pairs in total, covering more than 29K distinct semantic tags.

If you use this dataset in your research, please cite it using the following format:

• https://arxiv.org/abs/2312.12856

.. note:: This dataset requires the following additional library to be installed:

• tokenizers <https://pypi.org/project/tokenizers/>_ to tokenize the captions

Initialize a new SkyScript instance.

Args: root: Root directory where dataset can be found. split: One of 'train', 'val', 'test'. transforms: A function/transform that takes input sample and its target as entry and returns a transformed version. download: If True, download dataset and store it in the root directory. checksum: If True, check the MD5 of the downloaded files (may be slow). tokenizer: A pre-trained tokenizer (defaults to a BPE tokenizer trained on the training split's captions).

Raises: AssertionError: If split is invalid. DatasetNotFoundError: If dataset is not found and download is False. DependencyNotFoundError: If tokenizers is not installed.

Source code in keras_climate/datasets/skyscript.py

def __init__(
    self,
    root="data",
    split="train",
    transforms=None,
    download=False,
    checksum=True,
    tokenizer=None,
):
    """Initialize a new SkyScript instance.

    Args:
        root: Root directory where dataset can be found.
        split: One of 'train', 'val', 'test'.
        transforms: A function/transform that takes input sample and its target as
            entry and returns a transformed version.
        download: If True, download dataset and store it in the root directory.
        checksum: If True, check the MD5 of the downloaded files (may be slow).
        tokenizer: A pre-trained tokenizer (defaults to a BPE tokenizer trained
            on the training split's captions).

    Raises:
        AssertionError: If *split* is invalid.
        DatasetNotFoundError: If dataset is not found and *download* is False.
        DependencyNotFoundError: If tokenizers is not installed.
    """
    assert split in self.caption_files

    self.root = pathlib.Path(root)
    self.split = split
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    self.captions = pd.read_csv(self.root / self.caption_files[split])

    if tokenizer is None:
        tokenizers = lazy_import("tokenizers")
        self.tokenizer = tokenizers.Tokenizer(tokenizers.models.BPE())
        trainer = tokenizers.trainers.BpeTrainer()
        train_captions = pd.read_csv(self.root / self.caption_files["train"])
        self.tokenizer.train_from_iterator(train_captions["title"], trainer)
    else:
        self.tokenizer = tokenizer

__len__

__len__()

Return the number of images in the dataset.

Source code in keras_climate/datasets/skyscript.py

def __len__(self):
    """Return the number of images in the dataset."""
    return len(self.captions)

__getitem__

__getitem__(index)

Return an index within the dataset.

Returns: A dict containing image and tokenized caption at index.

Source code in keras_climate/datasets/skyscript.py

def __getitem__(self, index):
    """Return an index within the dataset.

    Returns:
        A dict containing image and tokenized caption at index.
    """
    filepath, title = self.captions.iloc[index][:2]
    output = self.tokenizer.encode(title)

    with Image.open(self.root / filepath) as img:
        array = np.array(img, dtype=np.float32)

    sample = {
        "image": ops.convert_to_tensor(array),
        "caption": ops.convert_to_tensor(np.array(output.ids, dtype="int64")),
    }

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/skyscript.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    fig, ax = plt.subplots()

    image = ops.convert_to_numpy(sample["image"]) / 255
    ax.imshow(image)
    ax.axis("off")

    if show_titles:
        caption = ops.convert_to_numpy(sample["caption"]).tolist()
        title = textwrap.wrap(self.tokenizer.decode(caption))
        if "prediction" in sample:
            caption = ops.convert_to_numpy(sample["prediction"]).tolist()
            title += textwrap.wrap(self.tokenizer.decode(caption))
        ax.set_title("\n".join(title))

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.so2sat.So2Sat

So2Sat(root='data', version='2', split='train', bands=BAND_SETS['all'], transforms=None, checksum=True)

Bases: NonGeoDataset

So2Sat dataset.

The So2Sat <https://doi.org/10.1109/MGRS.2020.2964708>__ dataset consists of corresponding synthetic aperture radar and multispectral optical image data acquired by the Sentinel-1 and Sentinel-2 remote sensing satellites, and a corresponding local climate zones (LCZ) label. The dataset is distributed over 42 cities across different continents and cultural regions of the world, and comes with a variety of different splits.

This implementation covers the 2nd and 3rd versions of the dataset as described in the author's github repository: https://github.com/zhu-xlab/So2Sat-LCZ42.

The different versions are as follows:

Version 2: This version contains imagery from 52 cities and is split into train/val/test as follows:

• Training: 42 cities around the world
• Validation: western half of 10 other cities covering 10 cultural zones
• Testing: eastern half of the 10 other cities

Version 3: A version of the dataset with 3 different train/test splits, as follows:

• Random split: every city 80% training / 20% testing (randomly sampled)
• Block split: every city is split in a geospatial 80%/20%-manner
• Cultural 10: 10 cities from different cultural zones are held back for testing

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1109/MGRS.2020.2964708

.. note::

This dataset requires the following additional library to be installed:

• <https://pypi.org/project/h5py/>_ to load the dataset

Initialize a new So2Sat dataset instance.

Args: root: root directory where dataset can be found version: one of "2", "3_random", "3_block", or "3_culture_10" split: one of "train", "validation", or "test" bands: a sequence of band names to use where the indices correspond to the array index of combined Sentinel 1 and Sentinel 2 transforms: a function/transform that takes input sample and its target as entry and returns a transformed version checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: if split argument is invalid DatasetNotFoundError: If dataset is not found. DependencyNotFoundError: If h5py is not installed.

Source code in keras_climate/datasets/so2sat.py

def __init__(
    self,
    root="data",
    version="2",
    split="train",
    bands=BAND_SETS["all"],
    transforms=None,
    checksum=True,
):
    """Initialize a new So2Sat dataset instance.

    Args:
        root: root directory where dataset can be found
        version: one of "2", "3_random", "3_block", or "3_culture_10"
        split: one of "train", "validation", or "test"
        bands: a sequence of band names to use where the indices correspond to the
            array index of combined Sentinel 1 and Sentinel 2
        transforms: a function/transform that takes input sample and its target as
            entry and returns a transformed version
        checksum: if True, check the MD5 of the downloaded files (may be slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found.
        DependencyNotFoundError: If h5py is not installed.
    """
    lazy_import("h5py")

    assert version in self.versions
    assert split in self.filenames_by_version[version]

    self._validate_bands(bands)
    self.s1_band_indices = np.array(
        [
            self.all_s1_band_names.index(b)
            for b in bands
            if b in self.all_s1_band_names
        ]
    ).astype(int)

    self.s1_band_names = [self.all_s1_band_names[i] for i in self.s1_band_indices]

    self.s2_band_indices = np.array(
        [
            self.all_s2_band_names.index(b)
            for b in bands
            if b in self.all_s2_band_names
        ]
    ).astype(int)

    self.s2_band_names = [self.all_s2_band_names[i] for i in self.s2_band_indices]

    self.bands = bands

    self.root = root
    self.version = version
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    self.fn = os.path.join(self.root, self.filenames_by_version[version][split])

    if not self._check_integrity():
        raise DatasetNotFoundError(self)

    h5py = lazy_import("h5py")
    with h5py.File(self.fn, "r") as f:
        self.size = f["label"].shape[0]

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/so2sat.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    h5py = lazy_import("h5py")
    with h5py.File(self.fn, "r") as f:
        s1 = f["sen1"][index].astype(np.float64)  # convert from <f8 to float64
        s1 = np.take(s1, indices=self.s1_band_indices, axis=-1)
        s2 = f["sen2"][index].astype(np.float64)  # convert from <f8 to float64
        s2 = np.take(s2, indices=self.s2_band_indices, axis=-1)

        # convert one-hot encoding to int64
        label = ops.convert_to_tensor(np.array(f["label"][index].argmax()))

    image = np.concatenate([s1, s2], axis=-1).astype("float32")
    sample = {"image": ops.convert_to_tensor(image), "label": label}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/so2sat.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.s2_band_names:
            idx = self.s2_band_names.index(band) + len(self.s1_band_names)
            rgb_indices.append(idx)
        else:
            raise RGBBandsMissingError()

    image = ops.take(sample["image"], rgb_indices, axis=-1)
    image = quantile_normalization(image)

    label = int(ops.convert_to_numpy(sample["label"]))
    label_class = self.classes[label]

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = int(ops.convert_to_numpy(sample["prediction"]))
        prediction_class = self.classes[prediction]

    fig, ax = plt.subplots(figsize=(4, 4))
    ax.imshow(ops.convert_to_numpy(image))
    ax.axis("off")
    if show_titles:
        title = f"Label: {label_class}"
        if showing_predictions:
            title += f"\nPrediction: {prediction_class}"
        ax.set_title(title)

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.soda.SODAA

SODAA(root='data', split='train', bbox_orientation='horizontal', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

SODA-A dataset.

The SODA-A <https://shaunyuan22.github.io/SODA/>_ dataset is a high resolution aerial imagery dataset for small object detection.

Dataset features:

• 2513 images
• 872,069 annotations with oriented bounding boxes
• 9 classes

Dataset format:

• Images are three channel .jpg files.
• Annotations are in json files

Classes:

1. Airplane
2. Helicopter
3. Small vehicle
4. Large vehicle
5. Ship
6. Container
7. Storage tank
8. Swimming-pool
9. Windmill
10. Other

If you use this dataset in your research, please cite the following paper:

• https://ieeexplore.ieee.org/document/10168277

Initialize a new instance of SODA-A dataset.

Args: root: root directory where dataset can be found split: one of "train", "val", or "test" bbox_orientation: one of "oriented" or "horizontal" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: AssertionError: if split or bbox_orientation argument is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/soda.py

def __init__(
    self,
    root="data",
    split="train",
    bbox_orientation="horizontal",
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new instance of SODA-A dataset.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val", or "test"
        bbox_orientation: one of "oriented" or "horizontal"
        transforms: a function/transform that takes input sample and its target as
            entry and returns a transformed version
        download: if True, download dataset and store it in the root directory
        checksum: if True, verify the checksum of the downloaded files (may be slow)

    Raises:
        AssertionError: if *split* or *bbox_orientation* argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    assert split in self.valid_splits, f"split must be one of {self.valid_splits}"

    assert bbox_orientation in self.valid_orientations, (
        f"bbox_orientation must be one of {self.valid_orientations}"
    )

    self.root = root
    self.split = split
    self.bbox_orientation = bbox_orientation
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    self.sample_df = pd.read_csv(os.path.join(self.root, "sample_df.csv"))
    self.sample_df = self.sample_df[
        self.sample_df["split"] == self.split
    ].reset_index(drop=True)

__len__

__len__()

Return the number of samples in the dataset.

Source code in keras_climate/datasets/soda.py

def __len__(self):
    """Return the number of samples in the dataset."""
    return len(self.sample_df)

__getitem__

__getitem__(index)

Return the sample at the given index.

Source code in keras_climate/datasets/soda.py

def __getitem__(self, index):
    """Return the sample at the given index."""
    row = self.sample_df.iloc[index]

    image = self._load_image(os.path.join(self.root, row["image_path"]))
    boxes, labels = self._load_labels(os.path.join(self.root, row["label_path"]))

    sample = {"image": image, "label": labels}

    if self.bbox_orientation == "oriented":
        sample["bbox"] = boxes
    else:
        sample["bbox_xyxy"] = boxes

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None, box_alpha=0.7)

Plot a sample from the dataset with legend.

Source code in keras_climate/datasets/soda.py

def plot(self, sample, show_titles=True, suptitle=None, box_alpha=0.7):
    """Plot a sample from the dataset with legend."""
    import matplotlib.pyplot as plt
    from matplotlib import patches

    image = ops.convert_to_numpy(sample["image"])
    if self.bbox_orientation == "horizontal":
        boxes = ops.convert_to_numpy(sample["bbox_xyxy"])
    else:
        boxes = ops.convert_to_numpy(sample["bbox"])
    labels = ops.convert_to_numpy(sample["label"])

    fig, ax = plt.subplots(ncols=1, figsize=(10, 10))

    ax.imshow(image)
    ax.axis("off")

    cm = plt.get_cmap("gist_rainbow")

    unique_labels = set()
    legend_elements = []

    for box, label_idx in zip(boxes, labels):
        color = cm(label_idx / len(self.classes))
        label = self.classes[label_idx]

        if self.bbox_orientation == "horizontal":
            # Horizontal box: [xmin, ymin, xmax, ymax]
            x1, y1, x2, y2 = box
            rect = patches.Rectangle(
                (x1, y1),
                x2 - x1,
                y2 - y1,
                linewidth=2,
                alpha=box_alpha,
                linestyle="solid",
                edgecolor=color,
                facecolor="none",
            )
            ax.add_patch(rect)
        else:
            # Oriented box: [x1,y1,x2,y2,x3,y3,x4,y4]
            vertices = box.reshape(4, 2)
            polygon = patches.Polygon(
                vertices,
                linewidth=2,
                alpha=box_alpha,
                linestyle="solid",
                edgecolor=color,
                facecolor="none",
            )
            ax.add_patch(polygon)

        if label not in unique_labels:
            legend_elements.append(
                patches.Patch(facecolor=color, alpha=box_alpha, label=label)
            )
            unique_labels.add(label)

    ax.legend(
        handles=legend_elements,
        loc="lower center",
        ncol=len(legend_elements),
        mode="expand",
    )

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.solar_plants_brazil.SolarPlantsBrazil

SolarPlantsBrazil(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

Solar Plants Brazil dataset (semantic segmentation for photovoltaic detection).

The Solar Plants Brazil <https://huggingface.co/datasets/FederCO23/solar-plants-brazil>__ dataset provides satellite imagery and pixel-level annotations for detecting photovoltaic solar power stations.

Dataset features:

• 272 RGB+NIR GeoTIFF images (256x256 pixels)
• Binary masks indicating presence of solar panels (1 = panel, 0 = background)
• Organized into train, val, and test splits
• Float32 GeoTIFF files for both input and mask images
• Spatial metadata included (CRS, bounding box), but not used directly for training

Folder structure:

.. code-block:: text

root/train/input/img(123).tif
root/train/labels/target(123).tif

Access:

• Dataset is hosted on Hugging Face: https://huggingface.co/datasets/FederCO23/solar-plants-brazil
• Code and preprocessing steps available at: https://github.com/FederCO23/UCSD_MLBootcamp_Capstone

Initialize a SolarPlantsBrazil dataset split.

Args: root: root directory where dataset can be found split: dataset split to use, one of "train", "val", or "test" transforms: a function/transform that takes an input sample and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/solar_plants_brazil.py

def __init__(
    self, root="data", split="train", transforms=None, download=False, checksum=True
):
    """Initialize a SolarPlantsBrazil dataset split.

    Args:
        root: root directory where dataset can be found
        split: dataset split to use, one of "train", "val", or "test"
        transforms: a function/transform that takes an input sample
            and returns a transformed version
        download: if True, download dataset and store it in the root directory
        checksum: if True, check the MD5 of the downloaded files (may be slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    self.root = root
    self.transforms = transforms
    self.dataset_path = os.path.join(self.root, split)
    self.split = split
    self.download = download
    self.checksum = checksum

    self._verify()

    input_dir = os.path.join(self.dataset_path, "input")
    labels_dir = os.path.join(self.dataset_path, "labels")

    self.image_paths = sorted(glob.glob(os.path.join(input_dir, "img(*).tif")))
    self.mask_paths = sorted(glob.glob(os.path.join(labels_dir, "target(*).tif")))

__getitem__

__getitem__(index)

Return the image and mask at the given index.

Source code in keras_climate/datasets/solar_plants_brazil.py

def __getitem__(self, index):
    """Return the image and mask at the given index."""
    image = self._load_image(self.image_paths[index])
    mask = self._load_mask(self.mask_paths[index])
    sample = {"image": image, "mask": mask}
    if self.transforms:
        sample = self.transforms(sample)
    return sample

__len__

__len__()

Return the number of samples in the dataset.

Source code in keras_climate/datasets/solar_plants_brazil.py

def __len__(self):
    """Return the number of samples in the dataset."""
    return len(self.image_paths)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the SolarPlantsBrazil dataset.

Source code in keras_climate/datasets/solar_plants_brazil.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the SolarPlantsBrazil dataset."""
    import matplotlib.pyplot as plt

    image = sample["image"]
    mask = sample["mask"]
    show_prediction = "prediction" in sample

    # Use RGB only
    if image.shape[-1] == 4:
        image = image[..., :3]

    # Normalize for display
    image_np = ops.convert_to_numpy(image)
    max_val = np.max(image_np)
    if max_val > 0:
        image_np = image_np / max_val

    mask_np = np.squeeze(ops.convert_to_numpy(mask))

    ncols = 3 if show_prediction else 2
    fig, axs = plt.subplots(1, ncols, figsize=(5 * ncols, 5))
    axs[0].imshow(image_np)
    if show_titles:
        axs[0].set_title("RGB Image")
    axs[0].axis("off")

    axs[1].imshow(mask_np, cmap="gray")
    if show_titles:
        axs[1].set_title("Mask")
    axs[1].axis("off")

    if show_prediction:
        pred_np = np.squeeze(ops.convert_to_numpy(sample["prediction"]))
        axs[2].imshow(pred_np, cmap="gray")
        if show_titles:
            axs[2].set_title("Prediction")
        axs[2].axis("off")

    if suptitle is not None:
        plt.suptitle(suptitle)

    plt.tight_layout()
    return fig

keras_climate.datasets.south_africa_crop_type.SouthAfricaCropType

SouthAfricaCropType(paths='data', crs=None, classes=list(range(10)), bands=s2_bands, transforms=None, download=False, time_series=False)

Bases: RasterDataset

South Africa Crop Type Challenge dataset.

The South Africa Crop Type Challenge <https://source.coop/radiantearth/south-africa-crops-competition>__ dataset includes satellite imagery from Sentinel-1 and Sentinel-2 and labels for crop type that were collected by aerial and vehicle survey from May 2017 to March 2018. Data was provided by the Western Cape Department of Agriculture and is available via the Radiant Earth Foundation. For each field id the dataset contains time series imagery and a single label mask. Since only the first available imagery in July is returned for each field, the dates for S1 and S2 imagery for a given field are not guaranteed to be the same. Due to this date mismatch only S1 or S2 bands may be queried at a time, a mix of both is not supported. Each pixel in the label contains an integer field number and crop type class.

Dataset format:

• images are 2-band Sentinel 1 and 12-band Sentinel-2 data with a cloud mask
• masks are tiff images with unique values representing the class and field id.

Dataset classes:

1. No Data
2. Lucerne/Medics
3. Planted pastures (perennial)
4. Fallow
5. Wine grapes
6. Weeds
7. Small grain grazing
8. Wheat
9. Canola
10. Rooibos

If you use this dataset in your research, please cite the following dataset:

• Western Cape Department of Agriculture, Radiant Earth Foundation (2021) "Crop Type Classification Dataset for Western Cape, South Africa", Version 1.0, Radiant MLHub, https://doi.org/10.34911/rdnt.j0co8q

.. note:: This dataset requires the following additional library to be installed:

• azcopy <https://github.com/Azure/azure-storage-azcopy>_: to download the dataset from Source Cooperative.

Initialize a new South Africa Crop Type dataset instance.

Args: paths: paths directory where dataset can be found crs: coordinate reference system to be used classes: crop type classes to be included bands: the subset of bands to load transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory time_series: if True, stack data along the time series dimension [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/south_africa_crop_type.py

def __init__(
    self,
    paths="data",
    crs=None,
    classes=list(range(10)),
    bands=s2_bands,
    transforms=None,
    download=False,
    time_series=False,
):
    """Initialize a new South Africa Crop Type dataset instance.

    Args:
        paths: paths directory where dataset can be found
        crs: coordinate reference system to be used
        classes: crop type classes to be included
        bands: the subset of bands to load
        transforms: a function/transform that takes input sample and its target as
            entry and returns a transformed version
        download: if True, download dataset and store it in the root directory
        time_series: if True, stack data along the time series dimension
            [T, H, W, C]. If False, merge data into a [H, W, C] mosaic.

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is False.
    """
    assert set(classes) <= set(range(10)), (
        "Only the following classes are valid: [0-9]."
    )
    assert 0 in classes, "Classes must include the background class: 0"

    self.paths = paths
    self.download = download
    self.filename_glob = self.filename_glob.format(bands[0])

    self._verify()

    super().__init__(
        paths=paths,
        crs=crs,
        bands=bands,
        transforms=transforms,
        time_series=time_series,
    )

    # Map chosen classes to ordinal numbers, all others mapped to background class
    self.ordinal_map = np.zeros(10, dtype=self.dtype)
    self.inverse_map = np.zeros(len(classes), dtype=self.dtype)
    for v, k in enumerate(classes):
        self.ordinal_map[k] = v
        self.inverse_map[v] = k

__getitem__

__getitem__(index)

Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

Raises: IndexError: If index is not found in the dataset.

Source code in keras_climate/datasets/south_africa_crop_type.py

def __getitem__(self, index):
    """Retrieve input, target, and/or metadata indexed by spatiotemporal slice.

    Raises:
        IndexError: If *index* is not found in the dataset.
    """
    x, y, t = self._disambiguate_slice(index)
    interval = pd.Interval(t.start, t.stop)
    df = self.index.iloc[self.index.index.overlaps(interval)]
    df = df.iloc[:: t.step]
    df = df.cx[x.start : x.stop, y.start : y.stop]

    if df.empty:
        raise IndexError(
            f"index: {index} not found in dataset with bounds: {self.bounds}"
        )

    data_list = []
    filename_regex = re.compile(self.filename_regex, re.VERBOSE)

    # Loop through matched filepaths and find all unique field ids
    field_ids = []
    # Store date in July for s1 and s2 we want to use for each sample
    imagery_dates = {}

    for filepath in df.filepath:
        filename = os.path.basename(filepath)
        match = re.match(filename_regex, filename)
        if match:
            field_id = match.group("field_id")
            date = match.group("date")
            band = match.group("band")
            band_type = "s1" if band in self.s1_bands else "s2"
            if field_id not in field_ids:
                field_ids.append(field_id)
                imagery_dates[field_id] = {"s1": "", "s2": ""}
            if (
                date.split("_")[1] == "07"
                and not imagery_dates[field_id][band_type]
            ):
                imagery_dates[field_id][band_type] = date

    # Create Tensors for each band using stored dates
    paths = self.paths
    for band in self.bands:
        band_type = "s1" if band in self.s1_bands else "s2"
        band_filepaths = []
        for field_id in field_ids:
            date = imagery_dates[field_id][band_type]
            filepath = os.path.join(
                paths,
                "train",
                "imagery",
                band_type,
                field_id,
                date,
                f"{field_id}_{date}_{band}_10m.tif",
            )
            band_filepaths.append(filepath)
        data_list.append(self._merge_or_stack(band_filepaths, index))
    image = ops.concatenate(data_list, axis=-1)

    # Add labels for each field
    mask_filepaths = []
    for field_id in field_ids:
        file_path = filepath = os.path.join(
            paths, "train", "labels", f"{field_id}.tif"
        )
        mask_filepaths.append(file_path)

    mask = ops.squeeze(self._merge_or_stack(mask_filepaths, index), axis=-1)

    transform = rasterio.transform.from_origin(x.start, y.stop, x.step, y.step)
    sample = {
        "bounds": self._slice_to_tensor(index),
        "image": ops.cast(image, "float32"),
        "mask": ops.cast(mask, "int64"),
        "transform": ops.convert_to_tensor(np.array(list(transform))),
    }

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/south_africa_crop_type.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise RGBBandsMissingError()

    image = ops.take(sample["image"], rgb_indices, axis=-1)
    image = quantile_normalization(image)

    mask_np = ops.convert_to_numpy(sample["mask"]).astype("int64")
    mask = self.inverse_map[mask_np]
    ncols = 2

    showing_prediction = "prediction" in sample
    if showing_prediction:
        pred_np = ops.convert_to_numpy(sample["prediction"]).astype("int64")
        pred = self.inverse_map[pred_np]
        ncols += 1

    fig, axs = plt.subplots(nrows=1, ncols=ncols, figsize=(ncols * 4, 4))
    kwargs = {"cmap": self.cmap, "vmin": 0, "vmax": 9, "interpolation": "none"}
    axs[0].imshow(ops.convert_to_numpy(image))
    axs[0].axis("off")
    axs[1].imshow(mask, **kwargs)
    axs[1].axis("off")
    if show_titles:
        axs[0].set_title("Image")
        axs[1].set_title("Mask")

    if showing_prediction:
        axs[2].imshow(pred, **kwargs)
        axs[2].axis("off")
        if show_titles:
            axs[2].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.south_america_soybean.SouthAmericaSoybean

SouthAmericaSoybean(paths='data', crs=None, res=None, years=None, transforms=None, cache=True, download=False, checksum=True, time_series=False)

Bases: RasterDataset

South America Soybean Dataset.

This dataset produced annual 30-m soybean maps of South America from 2001 to 2021.

Link: https://www.nature.com/articles/s41893-021-00729-z

Dataset contains 2 classes:

1. other
2. soybean

Dataset Format:

• 21 .tif files

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1038/s41893-021-00729-z

Initialize a new Dataset instance.

Args: paths: one or more root directories to search or files to load crs: CRS to warp to (defaults to the CRS of the first file found) res: resolution of the dataset in units of CRS (defaults to the resolution of the first file found) years: list of years for which to use the South America Soybean layer transforms: a function/transform that takes an input sample and returns a transformed version cache: if True, cache file handle to speed up repeated sampling download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 after downloading files (may be slow) time_series: if True, stack data along the time series dimension ([T, H, W, C]). If False, merge data into a mosaic ([H, W, C]).

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/south_america_soybean.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    years=None,
    transforms=None,
    cache=True,
    download=False,
    checksum=True,
    time_series=False,
):
    """Initialize a new Dataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        years: list of years for which to use the South America Soybean layer
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 after downloading files (may be
            slow)
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a mosaic
            (``[H, W, C]``).

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    if years is None:
        years = [2021]
    self.paths = paths
    self.download = download
    self.checksum = checksum
    self.years = years
    self._verify()

    super().__init__(
        paths, crs, res, transforms=transforms, cache=cache, time_series=time_series
    )

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/south_america_soybean.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    mask = ops.convert_to_numpy(sample["mask"])
    ncols = 1

    showing_predictions = "prediction" in sample
    if showing_predictions:
        pred = ops.convert_to_numpy(sample["prediction"])
        ncols = 2

    fig, axs = plt.subplots(
        nrows=1, ncols=ncols, figsize=(ncols * 4, 4), squeeze=False
    )

    axs[0, 0].imshow(mask, interpolation="none")
    axs[0, 0].axis("off")

    if show_titles:
        axs[0, 0].set_title("Mask")

    if showing_predictions:
        axs[0, 1].imshow(pred, interpolation="none")
        axs[0, 1].axis("off")
        if show_titles:
            axs[0, 1].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.ssl4eo.SSL4EO

Bases: NonGeoDataset

Base class for all SSL4EO datasets.

Self-Supervised Learning for Earth Observation (SSL4EO) is a collection of large-scale multimodal multitemporal datasets for unsupervised/self-supervised pre-training in Earth observation.

keras_climate.datasets.ssl4eo.SSL4EOL

SSL4EOL(root='data', split='oli_sr', seasons=1, transforms=None, download=False, checksum=True)

Bases: SSL4EO

SSL4EO-L dataset.

Landsat version of SSL4EO.

The dataset consists of a parallel corpus (same locations and dates for SR/TOA) for the following sensors: tm_toa (Landsat 4-5 TM), etm_toa (Landsat 7 ETM+ TOA), etm_sr (Landsat 7 ETM+ SR), oli_tirs_toa (Landsat 8-9 OLI+TIRS TOA), oli_sr (Landsat 8-9 OLI SR).

Each patch has the following properties:

• 264 x 264 pixels
• Resampled to 30 m resolution (7920 x 7920 m)
• 4 seasonal timestamps
• Single multispectral GeoTIFF file

If you use this dataset in your research, please cite the following paper:

• https://proceedings.neurips.cc/paper_files/paper/2023/hash/bbf7ee04e2aefec136ecf60e346c2e61-Abstract-Datasets_and_Benchmarks.html

Initialize a new SSL4EOL instance.

Args: root: root directory where dataset can be found split: one of ['tm_toa', 'etm_toa', 'etm_sr', 'oli_tirs_toa', 'oli_sr'] seasons: number of seasonal patches to sample per location, 1--4 transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum after downloading files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/ssl4eo.py

def __init__(
    self,
    root="data",
    split="oli_sr",
    seasons=1,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new SSL4EOL instance.

    Args:
        root: root directory where dataset can be found
        split: one of ['tm_toa', 'etm_toa', 'etm_sr', 'oli_tirs_toa', 'oli_sr']
        seasons: number of seasonal patches to sample per location, 1--4
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum after downloading files
            (may be slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.root = root
    self.subdir = os.path.join(root, f"ssl4eo_l_{split}")
    self.split = split
    self.seasons = seasons
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    if split.startswith("tm"):
        base = Landsat5TM
    elif split.startswith("etm"):
        base = Landsat7
    else:
        base = Landsat8

    self.wavelengths = []
    for band in self.metadata[split]["all_bands"]:
        self.wavelengths.append(base.wavelengths[band])

    self.scenes = sorted(os.listdir(self.subdir))

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/ssl4eo.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    root = os.path.join(self.subdir, self.scenes[index])
    subdirs = os.listdir(root)
    subdirs = random.sample(subdirs, self.seasons)

    images = []
    xs = []
    ys = []
    ts = []
    wavelengths = []
    for subdir in subdirs:
        mint, maxt = disambiguate_timestamp(subdir[-8:], Landsat.date_format)
        directory = os.path.join(root, subdir)
        filename = os.path.join(directory, "all_bands.tif")
        with rasterio.open(filename) as f:
            minx, maxx = f.bounds.left, f.bounds.right
            miny, maxy = f.bounds.bottom, f.bounds.top
            image = f.read().astype("float32")
            # (C, H, W) -> (H, W, C)
            images.append(np.transpose(image, (1, 2, 0)))
            xs.append((minx + maxx) / 2)
            ys.append((miny + maxy) / 2)
            ts.append((mint.timestamp() + maxt.timestamp()) / 2)
            wavelengths.extend(self.wavelengths)

    sample = {
        "image": ops.convert_to_tensor(np.concatenate(images, axis=-1)),
        "x": ops.convert_to_tensor(np.array(xs, dtype="float32")),
        "y": ops.convert_to_tensor(np.array(ys, dtype="float32")),
        "t": ops.convert_to_tensor(np.array(ts, dtype="float64")),
        "wavelength": ops.convert_to_tensor(np.array(wavelengths, dtype="float32")),
        "res": ops.convert_to_tensor(np.array(30)),
    }

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/ssl4eo.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.scenes)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/ssl4eo.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    fig, axes = plt.subplots(
        ncols=self.seasons, squeeze=False, figsize=(4 * self.seasons, 4)
    )
    num_bands = len(self.metadata[self.split]["all_bands"])
    rgb_bands = self.metadata[self.split]["rgb_bands"]

    image = ops.convert_to_numpy(sample["image"])
    for i in range(self.seasons):
        season = image[:, :, i * num_bands : (i + 1) * num_bands].astype("uint8")

        season = np.take(season, rgb_bands, axis=-1)
        axes[0, i].imshow(season)
        axes[0, i].axis("off")

        if show_titles:
            axes[0, i].set_title(f"Split {self.split}, Season {i + 1}")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.ssl4eo.SSL4EOS12

SSL4EOS12(root='data', split='s2c', seasons=1, transforms=None, download=False, checksum=True)

Bases: SSL4EO

SSL4EO-S12 dataset.

Sentinel-1/2 <https://github.com/zhu-xlab/SSL4EO-S12>_ version of SSL4EO.

The dataset consists of a parallel corpus (same locations and dates) for the following satellites: s1 (Sentinel-1 GRD), s2c (Sentinel-2 L1C TOA), s2a (Sentinel-2 L2A SR).

Each patch has the following properties:

• 264 x 264 pixels
• Resampled to 10 m resolution (2640 x 2640 m)
• 4 seasonal timestamps

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2211.07044

Initialize a new SSL4EOS12 instance.

Args: root: root directory where dataset can be found split: one of "s1" (Sentinel-1 GRD dual-pol SAR), "s2c" (Sentinel-2 Level-1C top-of-atmosphere reflectance), or "s2a" (Sentinel-2 Level-2A surface reflectance) seasons: number of seasonal patches to sample per location, 1--4 transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/ssl4eo.py

def __init__(
    self,
    root="data",
    split="s2c",
    seasons=1,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new SSL4EOS12 instance.

    Args:
        root: root directory where dataset can be found
        split: one of "s1" (Sentinel-1 GRD dual-pol SAR), "s2c"
            (Sentinel-2 Level-1C top-of-atmosphere reflectance), or
            "s2a" (Sentinel-2 Level-2A surface reflectance)
        seasons: number of seasonal patches to sample per location, 1--4
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.root = root
    self.split = split
    self.seasons = seasons
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    self.bands = self.metadata[self.split]["bands"]

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/ssl4eo.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    root = os.path.join(self.root, self.split, f"{index:07}")
    subdirs = os.listdir(root)
    subdirs = random.sample(subdirs, self.seasons)
    filename_regex = self.metadata[self.split]["filename_regex"]

    images = []
    xs = []
    ys = []
    ts = []
    wavelengths = []
    for subdir in subdirs:
        directory = os.path.join(root, subdir)
        match = re.match(filename_regex, subdir)
        if match:
            date_str = match.group("date")
            if self.split == "s1":
                date_format = Sentinel1.date_format
            else:
                date_format = Sentinel2.date_format
            mint, maxt = disambiguate_timestamp(date_str, date_format)
            band_arrays = []
            for band in self.bands:
                if self.split == "s1":
                    wavelengths.append(Sentinel1.wavelength)
                else:
                    wavelengths.append(Sentinel2.wavelengths[band])

                filename = os.path.join(directory, f"{band}.tif")
                with rasterio.open(filename) as f:
                    minx, maxx = f.bounds.left, f.bounds.right
                    miny, maxy = f.bounds.bottom, f.bounds.top
                    image = f.read(out_shape=(1, self.size, self.size))
                    band_arrays.append(image.astype("float32")[0])
            # (H, W, num_bands)
            images.append(np.stack(band_arrays, axis=-1))
            xs.append((minx + maxx) / 2)
            ys.append((miny + maxy) / 2)
            ts.append((mint.timestamp() + maxt.timestamp()) / 2)

    sample = {
        "image": ops.convert_to_tensor(np.concatenate(images, axis=-1)),
        "x": ops.convert_to_tensor(np.array(xs, dtype="float32")),
        "y": ops.convert_to_tensor(np.array(ys, dtype="float32")),
        "t": ops.convert_to_tensor(np.array(ts, dtype="float64")),
        "wavelength": ops.convert_to_tensor(np.array(wavelengths, dtype="float32")),
        "res": ops.convert_to_tensor(np.array(10)),
    }

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/ssl4eo.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return 251079

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/ssl4eo.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    nrows = 2 if self.split == "s1" else 1
    fig, axes = plt.subplots(
        nrows=nrows,
        ncols=self.seasons,
        squeeze=False,
        figsize=(4 * self.seasons, 4 * nrows),
    )

    image = ops.convert_to_numpy(sample["image"])
    for i in range(self.seasons):
        season = image[:, :, i * len(self.bands) : (i + 1) * len(self.bands)]

        if self.split == "s1":
            axes[0, i].imshow(season[:, :, 0])
            axes[1, i].imshow(season[:, :, 1])
        else:
            season = np.take(season, [3, 2, 1], axis=-1)
            season = np.clip(season / 3000, 0, 1)
            axes[0, i].imshow(season)

        axes[0, i].axis("off")

        if show_titles:
            axes[0, i].set_title(f"Split {self.split}, Season {i + 1}")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.ssl4eo_benchmark.SSL4EOLBenchmark

SSL4EOLBenchmark(root='data', sensor='oli_sr', product='cdl', split='train', classes=None, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

SSL4EO Landsat Benchmark Evaluation Dataset.

Dataset is intended to be used for evaluation of SSL techniques. Each benchmark dataset consists of 25,000 images with corresponding land cover classification masks.

Dataset format:

• Input landsat image and single channel mask
• 25,000 total samples split into train, val, test (70%, 15%, 15%)
• NLCD dataset version has 17 classes
• CDL dataset version has 134 classes

Each patch has the following properties:

• 264 x 264 pixels
• Resampled to 30 m resolution (7920 x 7920 m)
• Single multispectral GeoTIFF file

If you use this dataset in your research, please cite the following paper:

• https://proceedings.neurips.cc/paper_files/paper/2023/hash/bbf7ee04e2aefec136ecf60e346c2e61-Abstract-Datasets_and_Benchmarks.html

Initialize a new SSL4EO Landsat Benchmark instance.

Args: root: root directory where dataset can be found sensor: one of ['etm_toa', 'etm_sr', 'oli_tirs_toa, 'oli_sr'] product: mask target, one of ['cdl', 'nlcd'] split: dataset split, one of ['train', 'val', 'test'] classes: list of classes to include, the rest will be mapped to 0 (defaults to all classes for the chosen product) transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum after downloading files (may be slow)

Raises: AssertionError: if any arguments are invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/ssl4eo_benchmark.py

def __init__(
    self,
    root="data",
    sensor="oli_sr",
    product="cdl",
    split="train",
    classes=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new SSL4EO Landsat Benchmark instance.

    Args:
        root: root directory where dataset can be found
        sensor: one of ['etm_toa', 'etm_sr', 'oli_tirs_toa, 'oli_sr']
        product: mask target, one of ['cdl', 'nlcd']
        split: dataset split, one of ['train', 'val', 'test']
        classes: list of classes to include, the rest will be mapped to
            0 (defaults to all classes for the chosen product)
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum after downloading files
            (may be slow)

    Raises:
        AssertionError: if any arguments are invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert sensor in self.valid_sensors, (
        f"Only supports one of {self.valid_sensors}, but found {sensor}."
    )
    self.sensor = sensor
    assert product in self.valid_products, (
        f"Only supports one of {self.valid_products}, but found {product}."
    )
    self.product = product
    assert split in self.valid_splits, (
        f"Only supports one of {self.valid_splits}, but found {split}."
    )
    self.split = split

    self.cmap = self.cmaps[product]
    if classes is None:
        self.classes = self.valid_classes[product]
    else:
        self.classes = classes

    assert set(self.classes) <= set(self.valid_classes[product]), (
        f"Only the following classes are valid: {self.valid_classes[product]}."
    )

    self.root = root
    self.transforms = transforms
    self.download = download
    self.checksum = checksum
    self.ordinal_map = np.zeros(self.valid_classes[product][-1] + 1, dtype="int64")
    self.inverse_map = np.zeros(len(self.classes), dtype="int64")
    self.img_dir_name = self.image_root.format(self.sensor)
    self.mask_dir_name = self.mask_dir_dict[self.sensor].format(self.product)

    self._verify()

    self.sample_collection = self.retrieve_sample_collection()

    # train, val, test split
    np.random.seed(0)
    sizes = (np.array(self.split_percentages) * len(self.sample_collection)).astype(
        int
    )
    cutoffs = np.cumsum(sizes)[:-1]
    sample_indices = np.arange(len(self.sample_collection))
    np.random.shuffle(sample_indices)
    groups = np.split(sample_indices, cutoffs)
    split_indices = {"train": groups[0], "val": groups[1], "test": groups[2]}[
        self.split
    ]

    self.sample_collection = [self.sample_collection[idx] for idx in split_indices]

    # Map chosen classes to ordinal numbers, all others mapped to background class
    for v, k in enumerate(self.classes):
        self.ordinal_map[k] = v
        self.inverse_map[v] = k

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/ssl4eo_benchmark.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    img_path, mask_path = self.sample_collection[index]

    sample = {
        "image": self._load_image(img_path),
        "mask": self._load_mask(mask_path),
    }

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/ssl4eo_benchmark.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.sample_collection)

retrieve_sample_collection

retrieve_sample_collection()

Retrieve paths to samples in data directory.

Source code in keras_climate/datasets/ssl4eo_benchmark.py

def retrieve_sample_collection(self):
    """Retrieve paths to samples in data directory."""
    img_paths = glob.glob(
        os.path.join(self.root, self.img_dir_name, "**", "all_bands.tif"),
        recursive=True,
    )
    img_paths = sorted(img_paths)
    sample_collection = []
    for img_path in img_paths:
        mask_path = img_path.replace(self.img_dir_name, self.mask_dir_name).replace(
            "all_bands.tif", f"{self.product}_{self.year_dict[self.sensor]}.tif"
        )
        sample_collection.append((img_path, mask_path))
    return sample_collection

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/ssl4eo_benchmark.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"])
    image = np.take(image, self.rgb_indices[self.sensor], axis=-1)
    image = image / 255

    mask = self.inverse_map[ops.convert_to_numpy(sample["mask"])]

    showing_predictions = "prediction" in sample
    ncols = 2
    if showing_predictions:
        pred = self.inverse_map[ops.convert_to_numpy(sample["prediction"])]
        ncols = 3

    fig, ax = plt.subplots(ncols=ncols, figsize=(4 * ncols, 4))
    kwargs = {"cmap": self.cmap, "vmin": 0, "vmax": 255, "interpolation": "none"}

    ax[0].imshow(image)
    ax[0].axis("off")
    ax[1].imshow(mask, **kwargs)
    ax[1].axis("off")
    if show_titles:
        ax[0].set_title("Image")
        ax[1].set_title("Mask")

    if showing_predictions:
        ax[2].imshow(pred, **kwargs)
        if show_titles:
            ax[2].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.substation.Substation

Substation(root='data', bands=all_bands, mask_2d=True, num_of_timepoints=4, timepoint_aggregation=None, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

Substation dataset.

The Substation <https://github.com/Lindsay-Lab/substation-seg>__ dataset is curated by TransitionZero and sourced from publicly available data repositories, including OpenSreetMap (OSM) and Copernicus Sentinel data. The dataset consists of Sentinel-2 images from 27k+ locations; the task is to segment power-substations, which appear in the majority of locations in the dataset. Most locations have 4-5 images taken at different timepoints (i.e., revisits).

Dataset Format:

• .npz file for each datapoint

Dataset Features:

• 26,522 image-mask pairs stored as numpy files.
• Data from 5 revisits for most locations.
• Multi-temporal, multi-spectral images (13 channels) paired with masks, with a spatial resolution of 228x228 pixels. When timepoint_aggregation is None, images are returned as T x H x W x C tensors.

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.48550/arXiv.2409.17363

Initialize the Substation dataset.

Args: root: path to the directory containing the dataset bands: channels to use from the image mask_2d: whether to use a 2D (one-hot) mask num_of_timepoints: number of timepoints to use for each image timepoint_aggregation: one of "concat", "median", "first", "random", or None. If None, returns the time-series as T x H x W x C. transforms: a transform takes input sample and returns a transformed version download: whether to download the dataset if it is not found checksum: whether to verify the dataset after downloading

Source code in keras_climate/datasets/substation.py

def __init__(
    self,
    root="data",
    bands=all_bands,
    mask_2d=True,
    num_of_timepoints=4,
    timepoint_aggregation=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize the Substation dataset.

    Args:
        root: path to the directory containing the dataset
        bands: channels to use from the image
        mask_2d: whether to use a 2D (one-hot) mask
        num_of_timepoints: number of timepoints to use for each image
        timepoint_aggregation: one of "concat", "median", "first",
            "random", or None. If None, returns the time-series as
            ``T x H x W x C``.
        transforms: a transform takes input sample and returns a
            transformed version
        download: whether to download the dataset if it is not found
        checksum: whether to verify the dataset after downloading
    """
    self.root = root
    self.bands = bands
    self.mask_2d = mask_2d
    self.num_of_timepoints = num_of_timepoints
    self.timepoint_aggregation = timepoint_aggregation
    self.transforms = transforms
    self.download = download
    self.checksum = checksum
    self.image_dir = os.path.join(root, "image_stack")
    self.mask_dir = os.path.join(root, "mask")
    self._verify()
    self.image_filenames = pd.Series(sorted(os.listdir(self.image_dir)))

__getitem__

__getitem__(index)

Get an item from the dataset by index.

Source code in keras_climate/datasets/substation.py

def __getitem__(self, index):
    """Get an item from the dataset by index."""
    image_filename = self.image_filenames[index]
    image_path = os.path.join(self.image_dir, image_filename)
    mask_path = os.path.join(self.mask_dir, image_filename)

    # (T, C, H, W)
    image = np.load(image_path)["arr_0"]

    # selecting channels
    indices = [self.all_bands.index(band) for band in self.bands]
    image = image[:, indices, :, :]

    # handling multiple images across timepoints
    if image.shape[0] < self.num_of_timepoints:
        # Padding: cycle through existing timepoints
        padded_images = []
        for i in range(self.num_of_timepoints):
            padded_images.append(image[i % image.shape[0]])
        image = np.stack(padded_images)
    elif image.shape[0] > self.num_of_timepoints:
        # Removal: take the most recent timepoints
        image = image[-self.num_of_timepoints :]

    if self.timepoint_aggregation == "concat":
        # (num_of_timepoints*channels, h, w)
        image = np.reshape(image, (-1, image.shape[2], image.shape[3]))
    elif self.timepoint_aggregation == "median":
        image = np.median(image, axis=0)
    elif self.timepoint_aggregation == "first":
        image = image[0]
    elif self.timepoint_aggregation == "random":
        image = image[np.random.randint(image.shape[0])]

    # Move channel axis to the end: (..., C, H, W) -> (..., H, W, C)
    image = np.moveaxis(image, -3, -1)

    mask = np.load(mask_path)["arr_0"]
    mask = np.where(mask != 3, 0, 1).astype("int64")

    if self.mask_2d:
        mask_0 = 1 - mask
        mask = np.stack([mask_0, mask], axis=-1)

    image = ops.convert_to_tensor(image.astype("float32"))
    mask = ops.convert_to_tensor(mask)

    sample = {"image": image, "mask": mask}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of items in the dataset.

Source code in keras_climate/datasets/substation.py

def __len__(self):
    """Return the number of items in the dataset."""
    return len(self.image_filenames)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

When the image is 4D (T x H x W x C), the first two timepoints are plotted.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/substation.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    When the image is 4D (``T x H x W x C``), the first two timepoints
    are plotted.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"])
    is_time_series = image.ndim == 4

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise RGBBandsMissingError()

    if is_time_series:
        images = np.clip(np.take(image, rgb_indices, axis=-1) / 4000, 0, 1)
        num_images = min(len(images), 2)
        ncols = num_images + 1
    else:
        image_rgb = np.clip(np.take(image, rgb_indices, axis=-1) / 4000, 0, 1)
        ncols = 2

    mask = ops.convert_to_numpy(sample["mask"])
    if self.mask_2d:
        mask = mask[..., 1]

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = ops.convert_to_numpy(sample["prediction"])
        if self.mask_2d:
            prediction = prediction[..., 1]
        ncols += 1

    fig, axs = plt.subplots(ncols=ncols, figsize=(4 * ncols, 4))

    if is_time_series:
        for i in range(num_images):
            axs[i].imshow(images[i])
            axs[i].axis("off")
            if show_titles:
                axs[i].set_title(f"Image {i}")
        axs[num_images].imshow(mask, cmap="gray", interpolation="none")
        axs[num_images].axis("off")
        if show_titles:
            axs[num_images].set_title("Mask")
        if showing_predictions:
            axs[num_images + 1].imshow(prediction, cmap="gray", interpolation="none")
            axs[num_images + 1].axis("off")
            if show_titles:
                axs[num_images + 1].set_title("Prediction")
    else:
        axs[0].imshow(image_rgb)
        axs[0].axis("off")
        axs[1].imshow(mask, cmap="gray", interpolation="none")
        axs[1].axis("off")
        if show_titles:
            axs[0].set_title("Image")
            axs[1].set_title("Mask")
        if showing_predictions:
            axs[2].imshow(prediction, cmap="gray", interpolation="none")
            axs[2].axis("off")
            if show_titles:
                axs[2].set_title("Prediction")

    if suptitle:
        fig.suptitle(suptitle)

    return fig

keras_climate.datasets.sustainbench_crop_yield.SustainBenchCropYield

SustainBenchCropYield(root='data', split='train', countries=None, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

SustainBench Crop Yield Dataset.

This dataset contains MODIS band histograms and soybean yield estimates for selected counties in the USA, Argentina and Brazil. The dataset is part of the SustainBench <https://sustainlab-group.github.io/sustainbench/docs/datasets/sdg2/crop_yield.html>_ datasets for tackling the UN Sustainable Development Goals (SDGs).

Dataset Format:

• .npz files of stacked samples

Dataset Features:

• input histogram of 7 surface reflectance and 2 surface temperature bands from MODIS pixel values in 32 ranges across 32 timesteps resulting in 32x32x9 input images
• regression target value of soybean yield in metric tonnes per harvested hectare

If you use this dataset in your research, please cite:

• https://doi.org/10.1145/3209811.3212707
• https://doi.org/10.1609/aaai.v31i1.11172

Initialize a new Dataset instance.

Args: root: root directory where dataset can be found split: one of "train", "dev", or "test" countries: which countries to include in the dataset transforms: a function/transform that takes an input sample and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum after downloading files (may be slow)

Raises: AssertionError: if countries contains invalid countries or if split is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/sustainbench_crop_yield.py

def __init__(
    self,
    root="data",
    split="train",
    countries=None,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new Dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "dev", or "test"
        countries: which countries to include in the dataset
        transforms: a function/transform that takes an input sample
            and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum after downloading files
            (may be slow)

    Raises:
        AssertionError: if ``countries`` contains invalid countries or
            if ``split`` is invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    if countries is None:
        countries = ["usa"]
    assert set(countries).issubset(self.valid_countries), (
        f"Please choose a subset of these valid countried: {self.valid_countries}."
    )
    self.countries = countries

    assert split in self.valid_splits, (
        f"Pleas choose one of these valid data splits {self.valid_splits}."
    )
    self.split = split

    self.root = root
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    self.images = []
    self.features = []

    for country in self.countries:
        image_file_path = os.path.join(
            self.root, self.dir, country, f"{self.split}_hists.npz"
        )
        target_file_path = image_file_path.replace("_hists", "_yields")
        years_file_path = image_file_path.replace("_hists", "_years")
        ndvi_file_path = image_file_path.replace("_hists", "_ndvi")

        npz_file = np.load(image_file_path)["data"]
        target_npz_file = np.load(target_file_path)["data"]
        year_npz_file = np.load(years_file_path)["data"]
        ndvi_npz_file = np.load(ndvi_file_path)["data"]
        num_data_points = npz_file.shape[0]
        for idx in range(num_data_points):
            sample = npz_file[idx].astype("float32")
            self.images.append(sample)

            target = target_npz_file[idx]
            year = year_npz_file[idx]
            ndvi = ndvi_npz_file[idx]

            features = {
                "label": np.array(target, dtype="float32"),
                "year": np.array(int(year), dtype="int64"),
                "ndvi": ndvi.astype("float32"),
            }
            self.features.append(features)

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/sustainbench_crop_yield.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.images)

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/sustainbench_crop_yield.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    sample = {"image": ops.convert_to_tensor(self.images[index])}
    for key, value in self.features[index].items():
        sample[key] = ops.convert_to_tensor(value)

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None, band_idx=0)

Plot a sample from the dataset.

Args: sample: a sample return by :meth:__getitem__ show_titles: flag indicating whether to show titles above each panel suptitle: optional suptitle to use for figure band_idx: which of the nine histograms to index

Source code in keras_climate/datasets/sustainbench_crop_yield.py

def plot(self, sample, show_titles=True, suptitle=None, band_idx=0):
    """Plot a sample from the dataset.

    Args:
        sample: a sample return by :meth:`__getitem__`
        show_titles: flag indicating whether to show titles above each panel
        suptitle: optional suptitle to use for figure
        band_idx: which of the nine histograms to index
    """
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"])
    label = float(ops.convert_to_numpy(sample["label"]))

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = float(ops.convert_to_numpy(sample["prediction"]))

    fig, ax = plt.subplots(1, 1, figsize=(10, 10))

    ax.imshow(image[:, :, band_idx])
    ax.axis("off")

    if show_titles:
        title = f"Label: {label:.3f}"
        if showing_predictions:
            title += f"\nPrediction: {prediction:.3f}"
        ax.set_title(title)

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.tessera.TesseraEmbeddings

TesseraEmbeddings(paths='data', crs=None, res=None, bands=None, transforms=None, cache=True, time_series=False)

Bases: RasterDataset

Tessera embeddings dataset.

This is a data loader for geospatial embeddings from the Tessera foundation model <https://github.com/ucam-eo/tessera>__, which processes Sentinel-1 and Sentinel-2 satellite imagery to generate 128-channel representation maps at 10m resolution. These embeddings compress a full year of temporal-spectral features into dense representations optimized for downstream geospatial analysis tasks.

If you use this dataset in your research, please cite the following paper:

• https://arxiv.org/abs/2506.20380

.. note:: The dataset can be downloaded using the geotessera <https://github.com/ucam-eo/geotessera>__ library. Be sure to use --format tiff to download GeoTIFF files compatible with keras_climate.

Source code in keras_climate/datasets/geo.py

def __init__(
    self,
    paths="data",
    crs=None,
    res=None,
    bands=None,
    transforms=None,
    cache=True,
    time_series=False,
):
    """Initialize a new RasterDataset instance.

    Args:
        paths: one or more root directories to search or files to load
        crs: CRS to warp to (defaults to the CRS of the first file found)
        res: resolution of the dataset in units of CRS (defaults to the
            resolution of the first file found)
        bands: bands to return (defaults to all bands)
        transforms: a function/transform that takes an input sample and
            returns a transformed version
        cache: if True, cache file handle to speed up repeated sampling
        time_series: if True, stack data along the time series dimension
            (``[T, H, W, C]``). If False, merge data into a mosaic
            (``[H, W, C]``). For mask-style datasets (``is_image=False``),
            single-band data has the channel dimension squeezed.
    """
    self.paths = paths
    self.bands = bands or self.all_bands
    self.transforms = transforms
    self.cache = cache
    self.time_series = time_series

    if self.all_bands:
        assert set(self.bands) <= set(self.all_bands)

    filename_regex = re.compile(self.filename_regex, re.VERBOSE)
    filepaths = []
    datetimes = []
    geometries = []
    for filepath in self.files:
        match = re.match(filename_regex, os.path.basename(filepath))
        if match is not None:
            vrt = None
            try:
                vrt = self._load_warp_file(filepath=filepath, crs=crs)
                if self.cmap is None:
                    try:
                        colors = np.array(list(vrt.colormap(1).values())) / 255
                        self.cmap = ListedColormap(colors)
                    except ValueError:
                        pass
                if crs is None:
                    with rasterio.Env(OSR_WKT_FORMAT="WKT2_2018"):
                        crs = PROJ_CRS.from_user_input(vrt.crs)
                footprint = self.footprint_from_datasource(vrt)
                if footprint is None:
                    footprint = shapely.box(*vrt.bounds)
                geometries.append(footprint)
                if res is None:
                    res = vrt.res
            except rasterio.errors.RasterioIOError:
                continue
            else:
                filepaths.append(filepath)
                mint, maxt = self._filepath_to_timestamp(filepath)
                datetimes.append((mint, maxt))
            finally:
                if vrt is not None:
                    vrt.close()

    if len(filepaths) == 0:
        raise DatasetNotFoundError(self)

    if not self.separate_files:
        self.band_indexes = None
        if self.bands:
            if self.all_bands:
                self.band_indexes = [
                    self.all_bands.index(i) + 1 for i in self.bands
                ]
            else:
                msg = (
                    f"{self.__class__.__name__} is missing an `all_bands` "
                    "attribute, so `bands` cannot be specified."
                )
                raise AssertionError(msg)

    if res is not None:
        if isinstance(res, (int, float)):
            res = (res, res)

        self._res = res

    data = {"filepath": filepaths}
    index = pd.IntervalIndex.from_tuples(datetimes, closed="both", name="datetime")
    self.index = GeoDataFrame(data, index=index, geometry=geometries, crs=crs)

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

.. warning:: Visualizations are generated using PCA on each image individually, and are thus not comparable across images. The plot method is provided for visualization purposes only and should not be used to draw conclusions.

Source code in keras_climate/datasets/tessera.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset.

    .. warning::
       Visualizations are generated using PCA on each image
       *individually*, and are thus not comparable across images. The
       plot method is provided for visualization purposes only and
       should not be used to draw conclusions.
    """
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"])
    h, w, c = image.shape
    A = image.reshape(h * w, c)

    # Use PCA to project embeddings from 128D to 3D space
    A_centered = A - A.mean(axis=0, keepdims=True)
    _, _, Vt = np.linalg.svd(A_centered, full_matrices=False)
    V = Vt[:3].T
    B = A @ V

    B -= B.min(axis=0, keepdims=True)
    B /= B.max(axis=0, keepdims=True)
    image = B.reshape(h, w, 3)

    fig, ax = plt.subplots()
    ax.imshow(image)
    ax.axis("off")

    if show_titles:
        ax.set_title("Embedding")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.treesatai.TreeSatAI

TreeSatAI(root='data', split='train', sensors=all_sensors, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

TreeSatAI Benchmark Archive.

TreeSatAI Benchmark Archive <https://zenodo.org/records/6780578>_ is a multi-sensor, multi-label dataset for tree species classification in remote sensing. It was created by combining labels from the federal forest inventory of Lower Saxony, Germany with 20 cm Color-Infrared (CIR) and 10 m Sentinel imagery.

The TreeSatAI Benchmark Archive contains:

• 50,381 image triplets (aerial, Sentinel-1, Sentinel-2)
• synchronized time steps and locations
• all original spectral bands/polarizations from the sensors
• 20 species classes (single labels)
• 12 age classes (single labels)
• 15 genus classes (multi labels)
• 60 m and 200 m patches
• fixed split for train (90%) and test (10%) data
• additional single labels such as English species name, genus, forest stand type, foliage type, land cover

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.5194/essd-15-681-2023

Initialize a new TreeSatAI instance.

Args: root: root directory where dataset can be found split: either "train" or "test" sensors: one or more of "aerial", "s1", and/or "s2" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: If invalid sensors are chosen. DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/treesatai.py

def __init__(
    self,
    root="data",
    split="train",
    sensors=all_sensors,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new TreeSatAI instance.

    Args:
        root: root directory where dataset can be found
        split: either "train" or "test"
        sensors: one or more of "aerial", "s1", and/or "s2"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: If invalid *sensors* are chosen.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert set(sensors) <= set(self.all_sensors)

    self.root = root
    self.split = split
    self.sensors = sensors
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    path = os.path.join(self.root, f"{split}_filenames.lst")
    with open(path) as f:
        self.files = f.read().strip().split("\n")

    path = os.path.join(self.root, "labels", "TreeSatBA_v9_60m_multi_labels.json")
    with open(path) as f:
        self.labels = json.load(f)

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/treesatai.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.files)

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/treesatai.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    file = self.files[index]
    label = np.zeros(len(self.classes), dtype="int64")
    for genus, _ in self.labels[file]:
        i = self.classes.index(genus)
        label[i] = 1

    sample = {"label": ops.convert_to_tensor(label)}
    for directory in self.sensors:
        with rio.open(os.path.join(self.root, directory, "60m", file)) as f:
            array = f.read().astype("float32")
            # (C, H, W) -> (H, W, C)
            array = np.transpose(array, (1, 2, 0))
            sample[f"image_{directory}"] = ops.convert_to_tensor(array)

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/treesatai.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    fig, ax = plt.subplots(ncols=len(self.sensors), squeeze=False)

    for i, sensor in enumerate(self.sensors):
        image = ops.convert_to_numpy(sample[f"image_{sensor}"])
        bands = [
            self.all_bands_dict[sensor].index(b)
            for b in self.rgb_bands_dict[sensor]
        ]
        image = np.take(image, bands, axis=-1)
        image = ops.convert_to_numpy(quantile_normalization(image))
        ax[0, i].imshow(image)
        ax[0, i].axis("off")

        if show_titles:
            ax[0, i].set_title(sensor)

    suptitle_str = ""

    if suptitle is not None:
        suptitle_str = suptitle

    if show_titles:
        label = self._multilabel_to_string(sample["label"])
        if suptitle_str:
            suptitle_str += f"\nLabel: ({label})"
        else:
            suptitle_str = f"Label: ({label})"

        if "prediction" in sample:
            prediction = self._multilabel_to_string(sample["prediction"])
            suptitle_str += f"\nPrediction: ({prediction})"

    if suptitle_str:
        plt.suptitle(suptitle_str)

    fig.tight_layout()
    return fig

keras_climate.datasets.ucmerced.UCMerced

UCMerced(root='data', split='train', transforms=None, download=False, checksum=True)

Bases: NonGeoClassificationDataset

UC Merced Land Use dataset.

The UC Merced Land Use <https://www.kaggle.com/datasets/abdulhasibuddin/uc-merced-land-use-dataset>_ dataset is a land use classification dataset of 2.1k 256x256 1ft resolution RGB images of urban locations around the U.S. extracted from the USGS National Map Urban Area Imagery collection with 21 land use classes (100 images per class).

Dataset features:

• land use class labels from around the U.S.
• three spectral bands - RGB
• 21 classes

This dataset uses the train/val/test splits defined in the "In-domain representation learning for remote sensing" paper:

• https://arxiv.org/abs/1911.06721

If you use this dataset in your research, please cite the following paper:

• https://dl.acm.org/doi/10.1145/1869790.1869829

Initialize a new UC Merced dataset instance.

Args: root: root directory where dataset can be found split: one of "train", "val", or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/ucmerced.py

def __init__(
    self,
    root="data",
    split="train",
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new UC Merced dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train", "val", or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in self.splits
    self.root = root
    self.tg_transforms = transforms
    self.download = download
    self.checksum = checksum
    self._verify()

    valid_fns = set()
    with open(os.path.join(self.root, self.split_filenames[split])) as f:
        for fn in f:
            valid_fns.add(fn.strip())

    def is_in_split(x):
        return os.path.basename(x) in valid_fns

    super().__init__(
        root=os.path.join(root, self.base_dir),
        transforms=transforms,
        is_valid_file=is_in_split,
    )

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/ucmerced.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    image = ops.convert_to_numpy(sample["image"])

    # Normalize the image if the max value is greater than 1
    if image.max() > 1:
        image = image.astype(np.float32) / 255.0  # Scale to [0, 1]

    label = int(ops.convert_to_numpy(sample["label"]))
    label_class = self.classes[label]

    showing_predictions = "prediction" in sample
    if showing_predictions:
        prediction = int(ops.convert_to_numpy(sample["prediction"]))
        prediction_class = self.classes[prediction]

    fig, ax = plt.subplots(figsize=(4, 4))
    ax.imshow(image)
    ax.axis("off")
    if show_titles:
        title = f"Label: {label_class}"
        if showing_predictions:
            title += f"\nPrediction: {prediction_class}"
        ax.set_title(title)

    if suptitle is not None:
        plt.suptitle(suptitle)
    return fig

keras_climate.datasets.usavars.USAVars

USAVars(root='data', split='train', labels=ALL_LABELS, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

USAVars dataset.

The USAVars dataset is reproduction of the dataset used in the paper "A generalizable and accessible approach to machine learning with global satellite imagery <https://doi.org/10.1038/s41467-021-24638-z>_". Specifically, this dataset includes 1 sq km. crops of NAIP imagery resampled to 4m/px cenetered on ~100k points that are sampled randomly from the contiguous states in the USA. Each point contains three continuous valued labels (taken from the dataset released in the paper): tree cover percentage, elevation, and population density.

Dataset format:

• images are 4-channel GeoTIFFs
• labels are singular float values

Dataset labels:

• tree cover
• elevation
• population density

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.1038/s41467-021-24638-z

Initialize a new USAVars dataset instance.

Args: root: root directory where dataset can be found split: train/val/test split to load labels: list of labels to include transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: AssertionError: if invalid labels are provided DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/usavars.py

def __init__(
    self,
    root="data",
    split="train",
    labels=ALL_LABELS,
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize a new USAVars dataset instance.

    Args:
        root: root directory where dataset can be found
        split: train/val/test split to load
        labels: list of labels to include
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if invalid labels are provided
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    self.root = root

    assert split in self.split_metadata
    self.split = split

    for lab in labels:
        assert lab in self.ALL_LABELS

    self.labels = labels
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    self.files = self._load_files()

    self.label_dfs = {
        lab: pd.read_csv(os.path.join(self.root, lab + ".csv"), index_col="ID")
        for lab in self.labels
    }

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/usavars.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    tif_file = self.files[index]
    id_ = tif_file[5:-4]

    sample = {
        "labels": ops.convert_to_tensor(
            np.array(
                [self.label_dfs[lab].loc[id_][lab] for lab in self.labels],
                dtype="float32",
            )
        ),
        "image": self._load_image(os.path.join(self.root, "uar", tif_file)),
        "centroid_lat": ops.convert_to_tensor(
            np.array([self.label_dfs[self.labels[0]].loc[id_]["lat"]], dtype="float32")
        ),
        "centroid_lon": ops.convert_to_tensor(
            np.array([self.label_dfs[self.labels[0]].loc[id_]["lon"]], dtype="float32")
        ),
    }

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/usavars.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.files)

plot

plot(sample, show_titles=True, suptitle=None, show_labels=None)

Plot a sample from the dataset.

.. versionchanged:: Renamed show_labels to show_titles - keep show_labels as deprecated alias

Source code in keras_climate/datasets/usavars.py

def plot(self, sample, show_titles=True, suptitle=None, show_labels=None):
    """Plot a sample from the dataset.

    .. versionchanged::
        Renamed show_labels to show_titles - keep show_labels as
        deprecated alias
    """
    import matplotlib.pyplot as plt

    if show_labels is not None:
        warnings.warn(
            "The show_labels parameter is deprecated, use show_titles instead.",
            DeprecationWarning,
        )
        show_titles = show_labels

    image = ops.convert_to_numpy(sample["image"])[:, :, :3]  # get RGB inds

    fig, axs = plt.subplots(figsize=(10, 10))
    axs.imshow(image)
    axs.axis("off")

    if show_titles:
        labels = [(lab, val) for lab, val in sample.items() if lab != "image"]
        label_string = ""
        for lab, val in labels:
            val_np = ops.convert_to_numpy(val)
            label_string += f"{lab}={round(float(val_np.reshape(-1)[0]), 2)} "
        axs.set_title(label_string)

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.vaihingen.Vaihingen2D

Vaihingen2D(root='data', split='train', transforms=None, checksum=True)

Bases: NonGeoDataset

Vaihingen 2D Semantic Segmentation dataset.

The Vaihingen <https://www.isprs.org/resources/datasets/benchmarks/UrbanSemLab/2d-sem-label-vaihingen.aspx>__ dataset is a dataset for urban semantic segmentation used in the 2D Semantic Labeling Contest - Vaihingen. This dataset uses the "ISPRS_semantic_labeling_Vaihingen.zip" and "ISPRS_semantic_labeling_Vaihingen_ground_truth_COMPLETE.zip" files to create the train/test sets used in the challenge. The dataset can be downloaded from here <https://www.isprs.org/resources/datasets/benchmarks/UrbanSemLab/default.aspx>__. Note, the server contains additional data for 3D Semantic Labeling which are currently not supported.

Dataset format:

• images are 3-channel RGB geotiffs
• masks are 3-channel geotiffs with unique RGB values representing the class

Dataset classes:

1. Clutter/background
2. Impervious surfaces
3. Building
4. Low Vegetation
5. Tree
6. Car

If you use this dataset in your research, please cite the following paper:

• https://doi.org/10.5194/isprsannals-I-3-293-2012

Initialize a new Vaihingen2D dataset instance.

Args: root: root directory where dataset can be found split: one of "train" or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version checksum: if True, check the MD5 of the downloaded files (may be slow)

Raises: AssertionError: If split is invalid. DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/vaihingen.py

def __init__(self, root="data", split="train", transforms=None, checksum=True):
    """Initialize a new Vaihingen2D dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        checksum: if True, check the MD5 of the downloaded files (may be
            slow)

    Raises:
        AssertionError: If *split* is invalid.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in self.splits
    self.root = root
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    self._verify()

    self.files = []
    for name in self.splits[split]:
        image = os.path.join(root, self.image_root, name)
        mask = os.path.join(root, name)
        if os.path.exists(image) and os.path.exists(mask):
            self.files.append({"image": image, "mask": mask})

__getitem__

__getitem__(index)

Return an index within the dataset.

Source code in keras_climate/datasets/vaihingen.py

def __getitem__(self, index):
    """Return an index within the dataset."""
    image = self._load_image(index)
    mask = self._load_target(index)
    sample = {"image": image, "mask": mask}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

__len__

__len__()

Return the number of data points in the dataset.

Source code in keras_climate/datasets/vaihingen.py

def __len__(self):
    """Return the number of data points in the dataset."""
    return len(self.files)

plot

plot(sample, show_titles=True, suptitle=None, alpha=0.5)

Plot a sample from the dataset.

Source code in keras_climate/datasets/vaihingen.py

def plot(self, sample, show_titles=True, suptitle=None, alpha=0.5):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    ncols = 1
    image1 = draw_semantic_segmentation_masks(
        sample["image"][:, :, :3], sample["mask"], alpha=alpha, colors=list(self.colormap)
    )
    if "prediction" in sample:
        ncols += 1
        image2 = draw_semantic_segmentation_masks(
            sample["image"][:, :, :3],
            sample["prediction"],
            alpha=alpha,
            colors=list(self.colormap),
        )

    fig, axs = plt.subplots(ncols=ncols, figsize=(ncols * 10, 10))
    if ncols > 1:
        (ax0, ax1) = axs
    else:
        ax0 = axs

    ax0.imshow(image1)
    ax0.axis("off")
    if ncols > 1:
        ax1.imshow(image2)
        ax1.axis("off")

    if show_titles:
        ax0.set_title("Ground Truth")
        if ncols > 1:
            ax1.set_title("Predictions")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.vhr10.VHR10

VHR10(root='data', split='positive', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

NWPU VHR-10 dataset.

Northwestern Polytechnical University (NWPU) very-high-resolution ten-class (VHR-10) remote sensing image dataset.

Consists of 800 VHR optical remote sensing images, where 715 color images were acquired from Google Earth with the spatial resolution ranging from 0.5 to 2 m, and 85 pansharpened color infrared (CIR) images were acquired from Vaihingen data with a spatial resolution of 0.08 m.

The data set is divided into two sets:

• Positive image set (650 images) which contains at least one target
• Negative image set (150 images) does not contain any targets

The positive image set consists of objects from ten classes: airplane, ship, storage tank, baseball diamond, tennis court, basketball court, ground track field, harbor, bridge, and vehicle.

Includes object detection bounding boxes from the original paper and instance segmentation masks from follow-up publications. If you use this dataset in your research, please cite:

• https://doi.org/10.1016/j.isprsjprs.2014.10.002
• https://doi.org/10.1109/IGARSS.2019.8898573
• https://doi.org/10.3390/rs12060989

Initialize a new VHR-10 dataset instance.

Args: root: root directory where dataset can be found split: one of "positive" or "negative" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: AssertionError: if split argument is invalid DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/vhr10.py

def __init__(self, root="data", split="positive", transforms=None, download=False, checksum=True):
    """Initialize a new VHR-10 dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "positive" or "negative"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: if ``split`` argument is invalid
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert split in {"positive", "negative"}

    self.root = root
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    if download:
        self._download()

    if not self._check_integrity():
        raise DatasetNotFoundError(self)

    directory = os.path.join(self.root, "NWPU VHR-10 dataset", f"{self.split} image set")
    self.files = sorted(glob.glob(os.path.join(directory, "*.jpg")))

    if not self.files:
        raise DatasetNotFoundError(self)

    if split == "positive":
        path = os.path.join(self.root, "NWPU VHR-10 dataset", "annotations.json")
        with open(path) as f:
            annotations = json.load(f)

            # Gather image shapes
            out_shapes = []
            image_ids = {}
            for image in annotations["images"]:
                out_shapes.append((image["height"], image["width"]))
                image_ids[image["file_name"]] = image["id"]
            self.ids = [image_ids[os.path.basename(file)] for file in self.files]

            self.labels = defaultdict(list)
            self.boxes = defaultdict(list)
            self.masks = defaultdict(list)
            for annotation in annotations["annotations"]:
                i = annotation["image_id"]
                self.labels[i].append(annotation["category_id"])

                # Convert box format
                x1, y1, w, h = annotation["bbox"]
                self.boxes[i].append([x1, y1, x1 + w, y1 + h])

                # Rasterize segmentation mask
                segmentation = annotation["segmentation"]  # [[x1, y1, x2, y2, ...]]
                xs = segmentation[0][::2]  # [x1, x2, ...]
                ys = segmentation[0][1::2]  # [y1, y2, ...]
                coords = list(zip(xs, ys))  # [(x1, y1), (x2, y2), ...]
                shapes = [(Polygon(coords), 1)]
                mask = rasterize(shapes, out_shapes[i], dtype=np.uint8)
                self.masks[i].append(mask)

plot

plot(sample, show_titles=True, suptitle=None, show_feats='both', box_alpha=0.7, mask_alpha=0.7)

Plot a sample from the dataset.

Raises: AssertionError: if show_feats argument is invalid

Source code in keras_climate/datasets/vhr10.py

def plot(self, sample, show_titles=True, suptitle=None, show_feats="both", box_alpha=0.7, mask_alpha=0.7):
    """Plot a sample from the dataset.

    Raises:
        AssertionError: if ``show_feats`` argument is invalid
    """
    assert show_feats in {"boxes", "masks", "both"}

    import matplotlib.pyplot as plt
    from matplotlib.patches import Rectangle

    cm = plt.get_cmap("gist_rainbow")
    ncols = 2 if "prediction_label" in sample else 1
    fig, axs = plt.subplots(ncols=ncols, squeeze=False, figsize=(ncols * 10, 10))

    image = quantile_normalization(sample["image"])
    image = ops.convert_to_numpy(image)
    axs[0, 0].imshow(image)
    axs[0, 0].axis("off")

    if "label" in sample:
        labels = ops.convert_to_numpy(sample["label"])
        boxes = ops.convert_to_numpy(sample["bbox_xyxy"])
        masks = ops.convert_to_numpy(sample["mask"])
        for i in range(len(labels)):
            class_num = int(labels[i])
            color = cm(class_num / len(self.categories))

            if show_feats in {"boxes", "both"}:
                x1, y1, x2, y2 = boxes[i]
                r = Rectangle(
                    (x1, y1),
                    x2 - x1,
                    y2 - y1,
                    linewidth=2,
                    alpha=box_alpha,
                    linestyle="dashed",
                    edgecolor=color,
                    facecolor="none",
                )
                axs[0, 0].add_patch(r)

                label = self.categories[class_num]
                axs[0, 0].text(x1, y1 - 8, label, color="white", size=11, backgroundcolor="none")

            if show_feats in {"masks", "both"}:
                mask = masks[i]
                alpha = mask * mask_alpha
                mask = mask * class_num
                axs[0, 0].imshow(mask, cmap=cm, vmin=0, vmax=10, alpha=alpha)

        if show_titles:
            axs[0, 0].set_title("Ground Truth")

    if "prediction_label" in sample:
        axs[0, 1].imshow(image)
        axs[0, 1].axis("off")

        scores = ops.convert_to_numpy(sample["prediction_score"])
        labels = ops.convert_to_numpy(sample["prediction_label"])
        boxes = ops.convert_to_numpy(sample["prediction_bbox_xyxy"])
        for i in range(len(labels)):
            score = scores[i]
            if score < 0.5:
                continue

            class_num = int(labels[i])
            color = cm(class_num / len(self.categories))

            if show_feats in {"boxes", "both"}:
                x1, y1, x2, y2 = boxes[i]
                r = Rectangle(
                    (x1, y1),
                    x2 - x1,
                    y2 - y1,
                    linewidth=2,
                    alpha=box_alpha,
                    linestyle="dashed",
                    edgecolor=color,
                    facecolor="none",
                )
                axs[0, 1].add_patch(r)

                label = self.categories[class_num]
                caption = f"{label} {score:.3f}"
                axs[0, 1].text(x1, y1 - 8, caption, color="white", size=11, backgroundcolor="none")

            if "prediction_mask" in sample and show_feats in {"masks", "both"}:
                masks = ops.convert_to_numpy(sample["prediction_mask"])
                mask = masks[i]
                alpha = mask * mask_alpha
                mask = mask * class_num
                axs[0, 1].imshow(mask, cmap=cm, vmin=0, vmax=10, alpha=alpha)

        if show_titles:
            axs[0, 1].set_title("Prediction")

    if suptitle is not None:
        fig.suptitle(suptitle)

    return fig

keras_climate.datasets.western_usa_live_fuel_moisture.WesternUSALiveFuelMoisture

WesternUSALiveFuelMoisture(root='data', input_features=all_variable_names, transforms=None, download=False)

Bases: NonGeoDataset

Western USA Live Fuel Moisture dataset.

This tabular style dataset contains fuel moisture (mass of water in vegetation) and remotely sensed variables in the western United States. It contains 2615 datapoints and 138 variables. For more details see the dataset page <https://source.coop/stanford/sar-moisture-conent>_.

Dataset Format:

• .geojson file for each datapoint

Dataset Features:

• 138 remote sensing derived variables, some with a time dependency
• 2615 datapoints with regression target of predicting fuel moisture

If you use this dataset in your research, please cite:

• https://doi.org/10.1016/j.rse.2020.111797

.. note::

This dataset requires the following additional library to be installed:

• azcopy <https://github.com/Azure/azure-storage-azcopy>_: to download the dataset from Source Cooperative.

Initialize a new Western USA Live Fuel Moisture dataset instance.

Args: root: root directory where dataset can be found input_features: which input features to include transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory

Raises: AssertionError: if input_features contains invalid variable names DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/western_usa_live_fuel_moisture.py

def __init__(self, root="data", input_features=all_variable_names, transforms=None, download=False):
    """Initialize a new Western USA Live Fuel Moisture dataset instance.

    Args:
        root: root directory where dataset can be found
        input_features: which input features to include
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory

    Raises:
        AssertionError: if ``input_features`` contains invalid variable
            names
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert set(input_features) <= set(self.all_variable_names)

    self.root = root
    self.input_features = input_features
    self.transforms = transforms
    self.download = download

    self._verify()

    self.dataframe = self._load_data()

plot

plot(sample, show_titles=True, suptitle=None, variables_to_plot=None)

Plot a time series visualization of the LFMC sample.

Source code in keras_climate/datasets/western_usa_live_fuel_moisture.py

def plot(self, sample, show_titles=True, suptitle=None, variables_to_plot=None):
    """Plot a time series visualization of the LFMC sample."""
    import matplotlib.pyplot as plt

    if not variables_to_plot:
        variables_to_plot = [
            "slope",
            "elevation",
            "canopy_height",
            "forest_cover",
            "silt",
            "sand",
            "clay",
            "vv",
            "vh",
            "red",
            "green",
            "blue",
            "swir",
            "nir",
            "ndvi",
            "ndwi",
            "nirv",
            "vv_red",
            "vv_green",
            "vv_blue",
            "vv_swir",
            "vv_nir",
            "vv_ndvi",
            "vv_ndwi",
            "vv_nirv",
            "vh_red",
            "vh_green",
            "vh_blue",
            "vh_swir",
            "vh_nir",
            "vh_ndvi",
            "vh_ndwi",
            "vh_nirv",
            "vh_vv",
        ]

    input_data = ops.convert_to_numpy(sample["input"])

    time_labels = ["t", "t-1", "t-2", "t-3"]

    fig, axs = plt.subplots(
        len(variables_to_plot), 1, figsize=(6, 1.5 * len(variables_to_plot)), sharex=True
    )

    if len(variables_to_plot) == 1:
        axs = [axs]

    for i, var_base_name in enumerate(variables_to_plot):
        values = []

        for t_label in time_labels:
            full_var_name = f"{var_base_name}({t_label})"
            var_position = self.all_variable_names.index(full_var_name)
            values.append(input_data[var_position])

        axs[i].plot(range(len(time_labels)), values, "o-")
        axs[i].grid(True, alpha=0.3)

        if show_titles:
            axs[i].set_title(f"{var_base_name.upper()}")

    axs[-1].set_xticks(range(len(time_labels)))
    axs[-1].set_xticklabels(time_labels)

    lon = input_data[-2]
    lat = input_data[-1]
    lfmc_value = float(ops.convert_to_numpy(sample["label"]))

    axs[-1].text(
        x=0.5,
        y=-0.7,
        s=f"Live Fuel Moisture Content\nat {lon:.4f}, {lat:.4f}: {lfmc_value:.2f}%",
        ha="center",
        transform=axs[-1].transAxes,
    )

    if suptitle is not None:
        fig.suptitle(t=suptitle, y=1.6, transform=axs[0].transAxes)

    plt.tight_layout()

    return fig

keras_climate.datasets.worldstrat.WorldStrat

WorldStrat(root='data', modalities=all_modalities, split='train', transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

WorldStrat dataset.

WorldStrat <https://worldstrat.github.io/>__ is a multi-modal dataset covering nearly 10,000km2 of matched high and low resolution satellite imagery across the globe. High-resolution SPOT 6/7 imagery comes at a resolution of 1.5m/pixel and is matched with a time-series of Sentinel-2 data.

Dataset features:

• High resolution (1.5m/pixel) Airbus SPOT 6/7 imagery with RGBN channels
• Low resolution (8x lower) Sentinel 2 L1C and L2A
• globally distributed areas of interest around the world

Dataset format:

• pixel dimensions vary across AOI tiles
• all modalities are 'tif' files except for 'hr_rgbn' which is an 8-bit 'png'
• 'hr_ps' (pansharpened, 4 bands), 'hr_pan' (panchromatic, 1 band) and 'hr_rgbn' are the 1.5m/pixel SPOT products and live in 'hr_dataset//'
• 'lr_rgbn' is SPOT's native 6m/pixel multispectral acquisition before pansharpening, so roughly 4x lower resolution than 'hr_rgbn'. It is also stored in 'hr_dataset//_rgbn.tiff'
• 'l1c' and 'l2a' are Sentinel-2 time-series with 13 and 12 bands respectively and roughly 8x lower resolution than 'hr_rgbn', downloaded as separate archives

If you use this dataset in your research, please cite:

• https://zenodo.org/records/15382551
• https://arxiv.org/abs/2207.06418

Initialize the WorldStrat dataset.

Args: root: root directory where the dataset can be found modalities: sequence of input modalities to load, choose from 'hr_ps', 'hr_pan', 'hr_rgbn', 'lr_rgbn', 'l1c', 'l2a' split: the dataset split to load, choose from 'train', 'val', 'test' transforms: a function/transform that takes in a dictionary of tensors and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: AssertionError: If split or modalities arguments are invalid. DatasetNotFoundError: If dataset is not found and download is False.

Source code in keras_climate/datasets/worldstrat.py

def __init__(
    self,
    root="data",
    modalities=all_modalities,
    split="train",
    transforms=None,
    download=False,
    checksum=True,
):
    """Initialize the WorldStrat dataset.

    Args:
        root: root directory where the dataset can be found
        modalities: sequence of input modalities to load, choose from
            'hr_ps', 'hr_pan', 'hr_rgbn', 'lr_rgbn', 'l1c', 'l2a'
        split: the dataset split to load, choose from 'train', 'val',
            'test'
        transforms: a function/transform that takes in a dictionary of
            tensors and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: If *split* or *modalities* arguments are invalid.
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
    """
    assert all(modality in self.all_modalities for modality in modalities), (
        f"Invalid modality: {modalities}, please choose from {self.all_modalities}"
    )
    assert split in self.valid_splits, (
        f"Invalid split: {split}, please choose from {self.valid_splits}"
    )

    self.root = root
    self.modalities = modalities
    self.split = split
    self.transforms = transforms
    self.download = download
    self.checksum = checksum

    self._verify()

    self.file_path_df = pd.read_csv(
        os.path.join(self.root, self.file_info_dict["train_val_test_split"]["filename"])
    )

    self.file_path_df = self.file_path_df[self.file_path_df["split"] == self.split].reset_index(
        drop=True
    )
    self.metadata_df = pd.read_csv(os.path.join(self.root, self.file_info_dict["metadata"]["filename"]))

plot

plot(sample, show_titles=True, suptitle=None)

Plot a sample from the dataset.

Source code in keras_climate/datasets/worldstrat.py

def plot(self, sample, show_titles=True, suptitle=None):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    n_panels = len([k for k in sample if k.startswith("image_")])
    n_panels += "prediction" in sample

    fig, axs = plt.subplots(1, n_panels, figsize=(5 * n_panels, 5), squeeze=False)

    for panel, modality in enumerate(self.modalities):
        key = f"image_{modality}"
        if key in sample:
            img = ops.convert_to_numpy(sample[key])

            if modality in ("hr_ps", "hr_pan"):
                img = img[..., 0]
            elif modality == "hr_rgbn":
                img = img[..., 0:3]
            elif modality in ("l1c", "l2a"):
                img = img[0][..., [3, 2, 1]]

            img = ops.convert_to_numpy(quantile_normalization(img))

            axs[0, panel].imshow(img)
            axs[0, panel].axis("off")
            if show_titles:
                axs[0, panel].set_title(self.modality_titles[modality])

    if "prediction" in sample:
        pred = ops.convert_to_numpy(sample["prediction"])
        if pred.shape[-1] == 4:
            pred = pred[..., :3]
        pred = ops.convert_to_numpy(quantile_normalization(pred))
        axs[0, -1].imshow(pred)
        axs[0, -1].axis("off")
        if show_titles:
            axs[0, -1].set_title("Prediction")

    if suptitle:
        fig.suptitle(suptitle)

    return fig

keras_climate.datasets.xbd.XView2

XView2(root='data', split='train', transforms=None, checksum=True)

Bases: xBD

Deprecated alias for the xBD dataset.

.. deprecated:: Use :class:xBD instead.

Source code in keras_climate/datasets/xbd.py

def __init__(self, root="data", split="train", transforms=None, checksum=True):
    """Initialize a new xBD dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: If *split* is invalid.
        DatasetNotFoundError: If dataset is not found.
    """
    assert split in self.metadata
    self.root = root
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    self._verify()

    self.class2idx = {c: i for i, c in enumerate(self.classes)}
    self.files = self._load_files(root, split)

keras_climate.datasets.xbd.xBD

xBD(root='data', split='train', transforms=None, checksum=True)

Bases: NonGeoDataset

xBD dataset.

The xBD <https://xview2.org/dataset>__ dataset is a dataset for building disaster change detection. This dataset object uses the "Challenge training set (~7.8 GB)" and "Challenge test set (~2.6 GB)" data from the xView2 website as the train and test splits. Note, the xView2 website contains other data under the xView2 umbrella that are not included here. E.g. the "Tier3 training data", the "Challenge holdout set", and the "full data".

Dataset format:

• images are three-channel pngs
• masks are single-channel pngs where the pixel values represent the class

Dataset classes:

1. background
2. no damage
3. minor damage
4. major damage
5. destroyed

If you use this dataset in your research, please cite:

• https://arxiv.org/abs/1911.09296

Initialize a new xBD dataset instance.

Args: root: root directory where dataset can be found split: one of "train" or "test" transforms: a function/transform that takes input sample and its target as entry and returns a transformed version checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: AssertionError: If split is invalid. DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/xbd.py

def __init__(self, root="data", split="train", transforms=None, checksum=True):
    """Initialize a new xBD dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: If *split* is invalid.
        DatasetNotFoundError: If dataset is not found.
    """
    assert split in self.metadata
    self.root = root
    self.split = split
    self.transforms = transforms
    self.checksum = checksum

    self._verify()

    self.class2idx = {c: i for i, c in enumerate(self.classes)}
    self.files = self._load_files(root, split)

__getitem__

__getitem__(index)

Return an index within the dataset.

Returns a single T x H x W x C image and a change detection mask.

Source code in keras_climate/datasets/xbd.py

def __getitem__(self, index):
    """Return an index within the dataset.

    Returns a single T x H x W x C image and a change detection mask.
    """
    sample = self._load_sample(self.files[index])

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None, alpha=0.5)

Plot a sample from the dataset.

Source code in keras_climate/datasets/xbd.py

def plot(self, sample, show_titles=True, suptitle=None, alpha=0.5):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    ncols = 2
    colors = _colors_to_rgb(self.colormap)
    images = ops.convert_to_numpy(sample["image"])
    image1 = draw_semantic_segmentation_masks(images[0], sample["mask"], alpha=alpha, colors=colors)
    image2 = draw_semantic_segmentation_masks(images[1], sample["mask"], alpha=alpha, colors=colors)
    if "prediction" in sample:
        ncols += 1
        image3 = draw_semantic_segmentation_masks(
            images[1], sample["prediction"], alpha=alpha, colors=colors
        )

    fig, axs = plt.subplots(ncols=ncols, figsize=(ncols * 10, 10))
    axs[0].imshow(image1)
    axs[0].axis("off")
    axs[1].imshow(image2)
    axs[1].axis("off")
    if ncols > 2:
        axs[2].imshow(image3)
        axs[2].axis("off")

    if show_titles:
        axs[0].set_title("Pre disaster")
        axs[1].set_title("Post disaster")
        if ncols > 2:
            axs[2].set_title("Predictions")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.xbd.xBDDistShift

xBDDistShift(root='data', split='train', id_disaster='hurricane-matthew', id_pre_post='post', ood_disaster='mexico-earthquake', ood_pre_post='post', transforms=None, checksum=True)

Bases: xBD

xBD dataset with a custom, disaster-based train/test split.

Uses disasters as the shift axis and converts damage masks to binary building masks.

If you use this dataset in your research, please cite:

• https://arxiv.org/abs/2412.13394

Initialize a new xBDDistShift dataset instance.

Args: root: root directory where dataset can be found split: one of "train" or "test" id_disaster: disaster used as the in-distribution training set id_pre_post: imagery to use for the in-distribution disaster ood_disaster: disaster used as the out-of-distribution test set ood_pre_post: imagery to use for the out-of-distribution disaster transforms: a function/transform that takes input sample and its target as entry and returns a transformed version checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: AssertionError: If split or the disaster shift configuration is invalid. DatasetNotFoundError: If dataset is not found.

Source code in keras_climate/datasets/xbd.py

def __init__(
    self,
    root="data",
    split="train",
    id_disaster="hurricane-matthew",
    id_pre_post="post",
    ood_disaster="mexico-earthquake",
    ood_pre_post="post",
    transforms=None,
    checksum=True,
):
    """Initialize a new xBDDistShift dataset instance.

    Args:
        root: root directory where dataset can be found
        split: one of "train" or "test"
        id_disaster: disaster used as the in-distribution training set
        id_pre_post: imagery to use for the in-distribution disaster
        ood_disaster: disaster used as the out-of-distribution test set
        ood_pre_post: imagery to use for the out-of-distribution disaster
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        AssertionError: If *split* or the disaster shift configuration is
            invalid.
        DatasetNotFoundError: If dataset is not found.
    """
    assert {id_disaster, ood_disaster} <= set(self.valid_disasters)
    assert id_disaster != ood_disaster
    assert {id_pre_post, ood_pre_post} <= {"pre", "post", "both"}
    self.id_disaster = id_disaster
    self.id_pre_post = id_pre_post
    self.ood_disaster = ood_disaster
    self.ood_pre_post = ood_pre_post
    super().__init__(root, split, transforms, checksum)

plot

plot(sample, show_titles=True, suptitle=None, alpha=0.5)

Plot a sample from the dataset.

Source code in keras_climate/datasets/xbd.py

def plot(self, sample, show_titles=True, suptitle=None, alpha=0.5):
    """Plot a sample from the dataset."""
    import matplotlib.pyplot as plt

    ncols = 1
    colors = _colors_to_rgb(self.colormap)
    image = draw_semantic_segmentation_masks(sample["image"], sample["mask"], alpha=alpha, colors=colors)
    if "prediction" in sample:
        ncols += 1
        prediction = draw_semantic_segmentation_masks(
            sample["image"], sample["prediction"], alpha=alpha, colors=colors
        )

    fig, axs = plt.subplots(ncols=ncols, figsize=(ncols * 10, 10), squeeze=False)
    axs[0, 0].imshow(image)
    axs[0, 0].axis("off")
    if ncols > 1:
        axs[0, 1].imshow(prediction)
        axs[0, 1].axis("off")

    if show_titles:
        axs[0, 0].set_title("Image")
        if ncols > 1:
            axs[0, 1].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig

keras_climate.datasets.zuericrop.ZueriCrop

ZueriCrop(root='data', bands=band_names, transforms=None, download=False, checksum=True)

Bases: NonGeoDataset

ZueriCrop dataset.

The ZueriCrop <https://github.com/0zgur0/multi-stage-convSTAR-network>__ dataset is a dataset for time-series instance segmentation of crops.

Dataset features:

• Sentinel-2 multispectral imagery
• instance masks of 48 crop categories
• nine multispectral bands
• 116k images with 10 m per pixel resolution (24x24 px)
• ~28k time-series containing 142 images each

Dataset format:

• single hdf5 dataset containing images, semantic masks, and instance masks
• data is parsed into images and instance masks, boxes, and labels
• one mask per time-series

Dataset classes:

• 48 fine-grained hierarchical crop categories <https://github.com/0zgur0/multi-stage-convSTAR-network/blob/fa92b5b3cb77f5171c5c3be740cd6e6395cc29b6/labels.csv>_

If you use this dataset in your research, please cite:

• https://doi.org/10.1016/j.rse.2021.112603

.. note::

This dataset requires the following additional library to be installed:

• h5py <https://pypi.org/project/h5py/>_ to load the dataset

Initialize a new ZueriCrop dataset instance.

Args: root: root directory where dataset can be found bands: the subset of bands to load transforms: a function/transform that takes input sample and its target as entry and returns a transformed version download: if True, download dataset and store it in the root directory checksum: if True, verify the checksum of the downloaded files (may be slow)

Raises: DatasetNotFoundError: If dataset is not found and download is False. DependencyNotFoundError: If h5py is not installed.

Source code in keras_climate/datasets/zuericrop.py

def __init__(self, root="data", bands=band_names, transforms=None, download=False, checksum=True):
    """Initialize a new ZueriCrop dataset instance.

    Args:
        root: root directory where dataset can be found
        bands: the subset of bands to load
        transforms: a function/transform that takes input sample and its
            target as entry and returns a transformed version
        download: if True, download dataset and store it in the root
            directory
        checksum: if True, verify the checksum of the downloaded files
            (may be slow)

    Raises:
        DatasetNotFoundError: If dataset is not found and *download* is
            False.
        DependencyNotFoundError: If h5py is not installed.
    """
    lazy_import("h5py")

    self._validate_bands(bands)
    self.band_indices = np.array([self.band_names.index(b) for b in bands], dtype="int64")

    self.root = root
    self.bands = bands
    self.transforms = transforms
    self.download = download
    self.checksum = checksum
    self.filepath = os.path.join(root, "ZueriCrop.hdf5")

    self._verify()

__getitem__

__getitem__(index)

Return a sample containing image, mask, bounding boxes, and target label.

Source code in keras_climate/datasets/zuericrop.py

def __getitem__(self, index):
    """Return a sample containing image, mask, bounding boxes, and target label."""
    image = self._load_image(index)
    mask, boxes, label = self._load_target(index)

    sample = {"image": image, "mask": mask, "bbox_xyxy": boxes, "label": label}

    if self.transforms is not None:
        sample = self.transforms(sample)

    return sample

plot

plot(sample, show_titles=True, suptitle=None, time_step=0)

Plot a sample from the dataset.

Raises: RGBBandsMissingError: If bands does not include all RGB bands.

Source code in keras_climate/datasets/zuericrop.py

def plot(self, sample, show_titles=True, suptitle=None, time_step=0):
    """Plot a sample from the dataset.

    Raises:
        RGBBandsMissingError: If *bands* does not include all RGB bands.
    """
    import matplotlib.pyplot as plt

    rgb_indices = []
    for band in self.rgb_bands:
        if band in self.bands:
            rgb_indices.append(self.bands.index(band))
        else:
            raise RGBBandsMissingError()

    ncols = 2
    image = ops.convert_to_numpy(sample["image"])[time_step]
    image = np.take(image, rgb_indices, axis=-1)
    image = ops.convert_to_numpy(quantile_normalization(image))
    image = np.clip(image * 255, 0, 255).astype("uint8")

    mask = ops.convert_to_numpy(ops.argmax(sample["mask"], axis=0))

    if "prediction" in sample:
        ncols += 1
        preds = ops.convert_to_numpy(ops.argmax(sample["prediction"], axis=0))

    fig, axs = plt.subplots(ncols=ncols, figsize=(10 * ncols, 10))

    axs[0].imshow(image)
    axs[0].axis("off")
    axs[1].imshow(mask)
    axs[1].axis("off")

    if show_titles:
        axs[0].set_title("Image")
        axs[1].set_title("Mask")

    if "prediction" in sample:
        axs[2].imshow(preds)
        axs[2].axis("off")
        if show_titles:
            axs[2].set_title("Prediction")

    if suptitle is not None:
        plt.suptitle(suptitle)

    return fig