-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
tensorstore #1174
base: master
Are you sure you want to change the base?
tensorstore #1174
Conversation
…bs into tensorstore3
…bs into tensorstore3
…bs into tensorstore3
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks good already. I just have a few questions and suggested some small changes.
if all(a == "" for a in axes): | ||
dimension_names = ("c", "x", "y", "z") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this the default axis order and axis naming for Zarr arrays? Maybe we should take a look on the length of the shape metadata and set the default dimension names for each length accordingly. E.g.:
len(array.shape) == 2
: ("x", "y")
len(array.shape) == 3
: ("x", "y", "z")
...
Otherwise, we might get an IndexError when using the x_index
, y_index
and z_index
for accessing chunk_shape and shard_shape.
|
||
|
||
def _value_error(args: Any) -> str: | ||
return f"VecInt can be instantiated with int values `VecInt(1,2,3,4) or with `VecIntLike` object `VecInt([1,2,3,4]). Got {args}." |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I noticed that the ValueError suggestions for instance creation are wrong, the axes are required.
return f"VecInt can be instantiated with int values `VecInt(1,2,3,4) or with `VecIntLike` object `VecInt([1,2,3,4]). Got {args}." | |
return f"VecInt can be instantiated with int values `VecInt(1,2,3,4, axes=("x","y","z","t"))`, with axes as argument names `VecInt(x=10, y=42, z=3, t=100)` or with two Iterables one with the axes values and the other with axes names `VecInt((1, 2, 3, 4), ("x", "y", "z", "t"))`. Got {args}." |
array = self._array | ||
axes = array.domain.labels | ||
if all(a == "" for a in axes): | ||
dimension_names = ("c", "x", "y", "z") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Same as in the comment above, maybe we should consider the array shape when using these dimension names as default.
) | ||
return kvstore_spec | ||
else: | ||
return str(path) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is a string enough for a KVStoreSpec? I would have assumed that something like {"driver": "file", "path": str(path)}
is required here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think using a string is a shorthand, but we can also make it explicit.
# assert remote_ds.display_name is None | ||
# remote_ds.display_name = "Test Remote Dataset" | ||
# assert remote_ds.display_name == "Test Remote Dataset" | ||
# del remote_ds.display_name | ||
# assert remote_ds.display_name is None |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
# assert remote_ds.display_name is None | |
# remote_ds.display_name = "Test Remote Dataset" | |
# assert remote_ds.display_name == "Test Remote Dataset" | |
# del remote_ds.display_name | |
# assert remote_ds.display_name is None | |
assert remote_ds.display_name is None | |
remote_ds.display_name = "Test Remote Dataset" | |
assert remote_ds.display_name == "Test Remote Dataset" | |
del remote_ds.display_name | |
assert remote_ds.display_name is None |
@@ -222,6 +222,7 @@ def check_properties(annotation: wk.Annotation) -> None: | |||
check_properties(annotation_deserialized) | |||
|
|||
|
|||
@pytest.mark.skip("issue with current wk version") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pytest.mark.skip("issue with current wk version") |
Co-authored-by: Mark Bader <[email protected]>
…bs into tensorstore3
Co-authored-by: Mark Bader <[email protected]>
@@ -189,7 +189,7 @@ jobs: | |||
timeout-minutes: 30 | |||
env: | |||
WK_TOKEN: ${{ secrets.WK_TOKEN }} | |||
run: ./test.sh -vv -p no:faulthandler --splits 3 --group ${{ matrix.group }} --splitting-algorithm least_duration | |||
run: ./test.sh -vv -s -p no:faulthandler --splits 3 --group ${{ matrix.group }} --splitting-algorithm least_duration |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
run: ./test.sh -vv -s -p no:faulthandler --splits 3 --group ${{ matrix.group }} --splitting-algorithm least_duration | |
run: ./test.sh -vv -p no:faulthandler --splits 3 --group ${{ matrix.group }} --splitting-algorithm least_duration |
Description:
Todos: