Clarify definitions of "default device" and "current device"

[asmeurer]

The inspection API mentions "default device" in several places, for instance

• https://data-apis.org/array-api/latest/API_specification/generated/array_api.info.default_device.html#array_api.info.default_device
• https://data-apis.org/array-api/latest/API_specification/generated/array_api.info.default_dtypes.html#array_api.info.default_dtypes
• https://data-apis.org/array-api/latest/API_specification/generated/array_api.info.dtypes.html#array_api.info.dtypes

However, as far as I can tell, this term is never actually defined anywhere. I thought it might be defined at https://data-apis.org/array-api/latest/design_topics/device_support.html#device-support (or at https://data-apis.org/array-api/latest/purpose_and_scope.html#terms-and-definitions), but it doesn't seem to be. Are there APIs where the default device should be used? I thought this would be the case for creation functions, but the phrase "default device" never appears (e.g., at https://data-apis.org/array-api/latest/API_specification/generated/array_api.empty.html#array_api.empty). Presumably this is what is meant by default device, but are there other instances where "default device" should be used.

Another related concept that appears in some places but is never really defined is "current device" (for example, dtypes() should use the "current device" when device=None https://data-apis.org/array-api/latest/API_specification/generated/array_api.info.dtypes.html#array_api.info.dtypes). It would be helpful to clarify the difference between "current device" and "default device" and when one should be used and when the other should be used.

Actually the creation functions never really state clearly what should happen when device=None, except for asarray and the *_like functions, which say the device should be inferred from x.

My understanding is that for libraries with a context manager, the "current device" is the device set in the current context, whereas the "default device" is the device used when no context is set (the "current device" would be the same as the "default device" in this case). Is this correct? By this reasoning creation functions should actually use the "current device" when device=None, not the "default device". But is it also true that default_device() should always return the "default device" regardless of the current context? It seems like this would be less useful than "current device". Do we need a separate current_device() inspection API?

Or is it actually the case that these terms were both meant to mean the same thing (i.e., both mean the current device used by default in the current context)? If that's the case, then the note at https://data-apis.org/array-api/latest/API_specification/generated/array_api.info.default_dtypes.html#array_api.info.default_dtypes doesn't make much sense to me. And if so, we should unify this terminology to avoid confusion (probably the term "default device" should be preferred since that is the name of the inspection function).

[asmeurer]

Actually maybe current_device() isn't really necessary, since None means "current device" in all APIs that accept a device keyword.

[rgommers]

Are there APIs where the default device should be used? I thought this would be the case for creation functions, but the phrase "default device" never appears

Yes, creation functions indeed. The empty docs you linked are a good example, the description for the device keyword now says: "device on which to place the created array. Default: None." The "what does None mean" part is implicit here. I agree it'd be better to spell out what should happen.

My understanding is that for libraries with a context manager, the "current device" is the device set in the current context, whereas the "default device" is the device used when no context is set (the "current device" would be the same as the "default device" in this case). Is this correct?

Correct indeed. May also be global state rather than a context manager (e.g., PyTorch has both - the global one is torch.set_default_device).

By this reasoning creation functions should actually use the "current device" when device=None, not the "default device". But is it also true that default_device() should always return the "default device" regardless of the current context? It seems like this would be less useful than "current device". Do we need a separate current_device() inspection API?

Yes agreed, creation functions will use the current device. Both may be useful to inspect indeed. I believe we didn't consider this in detail because the standard has no way to set the current device, however I think that shouldn't stop us from adding an inspection function.

Actually maybe current_device() isn't really necessary, since None means "current device" in all APIs that accept a device keyword.

It's not necessary indeed, it'd just be nicer syntax for empty((1,)).device.

[asmeurer]

It's also worth noting that the "default real floating-point dtype" (another notion from the same APIs) can change at runtime in PyTorch using the set_default_dtype function. At data-apis/array-api-compat#166 I've made default_dtypes()['real floating'] return torch.get_default_dtype(), i.e., get the value dynamically. That seems the most obviously useful, but it also disagrees with the notion of "default" for devices.