Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clarification for non-portable behavior for in-place Python operations #828

Open
oleksandr-pavlyk opened this issue Jul 30, 2024 · 5 comments
Labels
Maintenance Bug fix, typo fix, or general maintenance. Narrative Content Narrative documentation content. topic: Type Promotion Type promotion.
Milestone

Comments

@oleksandr-pavlyk
Copy link
Contributor

The wording for in-place operators may be more explicit to warn users that in-place operations, e.g., x1 += x2, where Type Promotion Rules require x1 + x2 to have data type different from data type of x1 are implementation defined.

Present wording hints at it:

For example, after in-place addition x1 += x2, the modified array x1 must always equal the result of the equivalent binary arithmetic operation x1 = x1 + x2.

but states that result of the in-place operation must "equal" the result of the out-of-place operation and the equality may hold true for arrays of different data types.

@asmeurer
Copy link
Member

Did we agree that it should always be implementation defined if the types are different, or should x1 += x2 be OK if the type promoted type is the same as x1.dtype (i.e., x2 upcasts to x1)?

@kgryte kgryte added this to the v2024 milestone Sep 19, 2024
@kgryte kgryte added Maintenance Bug fix, typo fix, or general maintenance. Narrative Content Narrative documentation content. topic: Type Promotion Type promotion. labels Sep 19, 2024
@kgryte
Copy link
Contributor

kgryte commented Sep 19, 2024

should x1 += x2 be OK if the type promoted type is the same as x1.dtype (i.e., x2 upcasts to x1)?

@asmeurer I believe that is what @oleksandr-pavlyk was getting at in the OP. IMO, it should be okay to allow type promotion to x1.dtype and we can be more explicit in stating that the relationship x1 = x1 + x2 must hold provided the constraints of "limited" type promotion are satisfied.

@asmeurer
Copy link
Member

So there are two cases:

Case 1: x1 + x2 promotes to x1.dtype

x1 = asarray([0], dtype=int64)
x2 = asarray([0], dtype=int32)
x1 += x2

Case 2: x1 + x2 promotes to x2.dtype

x1 = asarray([0], dtype=int32)
x2 = asarray([0], dtype=int64)
x1 += x2

I think @oleksandr-pavlyk was asking about case 2:

... where Type Promotion Rules require x1 + x2 to have data type different from data type of x1 ...

In my opinion, this is actually spelled out already https://data-apis.org/array-api/latest/API_specification/array_object.html#in-place-operators:

An in-place operation must not change the data type or shape of the in-place array as a result of Type Promotion Rules or Broadcasting.

In other words, case 2 is currently required to error (which is stronger than implementation defined).

Case 1 is perhaps more ambiguous whether it is required or not. My reading of the current text is that it is. If we want to make it implementation defined, we should explicitly state that. I'm not aware of any reasons why it would be a problem, though.

@ndgrigorian
Copy link

ndgrigorian commented Sep 19, 2024

In other words, case 2 is currently required to error (which is stronger than implementation defined).

Does this make current NumPy behavior non-compliant, then?

Because in NumPy

In [26]: x_np = np.arange(10, dtype="i4")

In [27]: x_np += np.ones(10, dtype="i8")

In [28]: x_np
Out[28]: array([ 1,  2,  3,  4,  5,  6,  7,  8,  9, 10], dtype=int32)

it simply casts the second array into the type of the first array per same_kind casting.

And per the spec

An in-place operation must have the same behavior (including special cases) as its respective binary (i.e., two operand, non-assignment) operation. For example, after in-place addition x1 += x2, the modified array x1 must always equal the result of the equivalent binary arithmetic operation x1 = x1 + x2.

This seems like a very strong condition implying that x1 += x2 of this case should be disallowed, and may be an even stronger condition than what you've quoted, because the data type of x1 (the in-place array) does not change.

@asmeurer
Copy link
Member

asmeurer commented Sep 19, 2024

I think the x1 = x1 + x2 sentence is not really intended to be saying anything about the data type of x1. It's only saying that whatever behavior is specified in __add__ (and thus add()) also applies to the += operator, i.e., all the special cases, nan behaviors, and so on. The wording should probably be improved since this isn't clear.

The sentence before that, which I quoted, "An in-place operation must not change the data type or shape of the in-place array..." does indeed imply that NumPy is currently noncompliant, because it uses must and not should. We could potentially loosen this to be implementation defined. I guess one question is whether the NumPy team agrees that this is nonideal behavior and should be deprecated. I certainly find the behavior surprising (it really is an in-place change of dtype: even views of x1 are updated to the promoted dtype).

Also we should check if other libraries allow this. This is the behavior in PyTorch:

>>> x1 = torch.asarray([0], dtype=torch.int32)
>>> x2 = torch.asarray([1], dtype=torch.int64)
>>> x1 += x2
>>> x1
tensor([1], dtype=torch.int32)

That actually could arguably be within what the spec says, because it didn't change the dtype of x1. So we should be clear whether this is actually OK, or whether it should be an error (or implementation defined).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Maintenance Bug fix, typo fix, or general maintenance. Narrative Content Narrative documentation content. topic: Type Promotion Type promotion.
Projects
None yet
Development

No branches or pull requests

4 participants