Add RasterReader and RasterStackReader implementations (#188)

* add `DatasetReader/Writer` protocols

* add working binary stack reader

* add rasterio as dep

* start implementing the 3d reader

* fix docstyle, allow 1 or 3 slices for stack

* start getting binary file test working

* start hdf5 stack, add binary slices test

* add similar hdf5 tests

* add general gdal raster reader

* start test

* add `BaseStackReader` for common features

* add `num_threads` to other readers

* fix bad typing

* remove `get_max_block_shape`. switch to `StackReader` protocol for `EagerLoader`

* reorg tests into classes

* reorder locations of background loaders

* use `StackReader` in `ps.create_ps`, drop `.iter_blocks`

* fix loading for new `StackReader`, fix `...` as band index

* fix tests for EagerLoader refactor

* bad math

* update changelog for new version. Close #174 [skip ci]

.github/workflows/test-build-push.yml

@@ -33,6 +33,7 @@ jobs:
               pydantic=2.1
               pymp-pypi=0.4.5
               pyproj=3.3
+              rasterio=1.3
               rich=12.0
               ruamel_yaml=0.15
               scipy=1.5

CHANGELOG.md

@@ -1,4 +1,16 @@
-# [Unreleased](https://github.com/isce-framework/dolphin/compare/v0.8.0...main)
+# [Unreleased](https://github.com/isce-framework/dolphin/compare/v0.9.0...main)
+
+# [v0.9.0](https://github.com/isce-framework/dolphin/compare/v0.8.0...v0.9.0)
+
+**Added**
+- `DatasetReader` and `StackReader` protocols for reading in data from different sources
+  - `DatasetReader` is for reading in a single dataset, like one raster image.
+  - `StackReader` is for reading in a stack of datasets, like a stack of SLCs.
+  - Implementations of these have been done for flat binary files (`BinaryReader`), HDF5 files (`HDF5Reader`), and GDAL rasters (`RasterReader`).
+
+**Changed**
+- The `VRTStack` no longer has an `.iter_blocks` method
+  - This has been replaced with creating an `EagerLoader` directly and passing it to the `reader` argument
 
 # [v0.8.0](https://github.com/isce-framework/dolphin/compare/v0.7.0...v0.8.0)
 

conda-env.yml

@@ -15,6 +15,7 @@ dependencies:
   - pydantic>=2.1
   - pymp-pypi>=0.4.5
   - pyproj>=3.3
+  - rasterio>=1.3
   - rich>=12.0
   - ruamel.yaml>=0.15
   - scipy>=1.5 # "scipy 0.16+ is required for linear algebra", numba. 1.5 is the oldest version that supports Python 3.7

mkdocs.yml

@@ -13,6 +13,7 @@ theme:
 plugins:
 - search
 # plugin suggestions from here: https://mkdocstrings.github.io/recipes/
+- autorefs
 - gen-files:
     scripts:
     - docs/gen_ref_pages.py

pyproject.toml

@@ -80,7 +80,8 @@ plugins = ["pydantic.mypy"]
 
 [tool.pytest.ini_options]
 doctest_optionflags = "NORMALIZE_WHITESPACE NUMBER"
-addopts = "  --cov=dolphin  -n auto --maxprocesses=8 --doctest-modules --randomly-seed=1234 --ignore=scripts --ignore=docs --ignore=data --ignore=pkgs"
+# addopts = "  --cov=dolphin  -n auto --maxprocesses=8 --doctest-modules --randomly-seed=1234 --ignore=scripts --ignore=docs --ignore=data --ignore=pkgs"
+addopts = " --doctest-modules --randomly-seed=1234 --ignore=scripts --ignore=docs --ignore=data --ignore=pkgs"
 filterwarnings = [
   "error",
   # DeprecationWarning thrown in pkg_resources for older numba verions and llvmlite

requirements.txt

@@ -6,6 +6,7 @@ opera-utils>=0.1.5
 pydantic>=2.1
 pymp-pypi>=0.4.5
 pyproj>=3.3
+rasterio>=1.3
 rich>=12.0
 ruamel_yaml>=0.15
 scipy>=1.5

src/dolphin/_background.py

@@ -14,6 +14,13 @@
 
 _DEFAULT_TIMEOUT = 0.5
 
+__all__ = [
+    "BackgroundWorker",
+    "BackgroundReader",
+    "BackgroundWriter",
+    "DummyProcessPoolExecutor",
+]
+
 
 class BackgroundWorker(abc.ABC):
     """Base class for doing work in a background thread.