Lightrun
question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

RFC: Adding complex number support to the specification

See original GitHub issue

Complex Number Support

Plan for complex number support in the array API specification.

What follows is a plan for adding complex number support to the 2022 array API specification. This RFC is comprised of the following sections:

  • Prior discussions
  • High-level summary concerns and questions
  • Individual API changes

Prior Discussions

General Concerns

New APIs

real
conj (complex conjugate)
imag
arg/angle/phase (phase angle)
linalg.eig
linalg.eigvals

Decisions

  • initial data types: complex64 and complex128?

    • Update: only complex64 and complex128 to be added, thus mirroring float32 and float64.

  • rounding complex numbers (component-wise)

    • NumPy supports round, but not ceil, floor, and trunc.
    • Update: ceiling, flooring, and truncating complex numbers is not well-defined. The only rounding operation which is commonly supported is round.

  • ordering (lexicographic?)

  • branch cut policy?

    • branch cut reference: https://en.wikipedia.org/wiki/Branch_point#Branch_cuts
    • should we strictly follow NumPy/C99, or should we allow for implementation flexibility?
    • if we strictly follow something other than NumPy/C99, obvious backward compat concern
    • Update: branch cuts will be included in the 2022 revision under a provisional status. Provided no issues arise during adoption and implementation, the provisional status may be lifted in a future revision of the standard.

  • casting from complex to real

    • discard imaginary component (as in C, NumPy)
    • other choice would to require user to explicitly invoke real before casting
    • Update: decision was to require that array API consumers be explicit in specifying which component they want to cast when casting from complex to real.

  • complex numbers with components which are infinity and/or NaN

    • in Python, a complex number can be both infinite and NaN (according to cmath.isinf, cmath.isnan)

      In [1]: z = complex(float('inf'), float('nan'))

In [2]: z
Out[2]: (inf+nanj)

In [3]: cmath.isinf(z)
Out[3]: True

In [4]: cmath.isnan(z)
Out[4]: True

    • in NumPy (v1.23.5), behavior is consistent with Python

      In [1]: z = np.complex128(complex(float('inf'), float('nan')))

In [2]: z
Out[2]: (inf+nanj)

In [3]: np.isnan([z])
Out[3]: array([True])

In [4]: np.isinf([z])
Out[4]: array([True])

    • in C99 (see “complex floating types” section), one infinity model (e.g., inf + nan*j == inf)

      • “In order to support the one-infinity model of complex number arithmetic, C regards any complex value with at least one infinite part as an infinity even if its other part is a NaN…”
      • See C99 rationale 7.3.9.4 and SO

    • update: decision was made to allow implementation-dependent behavior for modeling complex NaNs and infinities. In particular, for complex multiplication and division, handling of complex infinities and NaNs may not be consistent across implementations.

Creation Functions

arange

asarray

empty

  • No changes. Output array data type has no restrictions.

empty_like

  • No changes. Output array data type has no restrictions.

eye

from_dlpack

  • No changes.

full

full_like

linspace

meshgrid

ones

ones_like

tril

  • No changes necessary. No restrictions on input array data types.

triu

  • No changes necessary. No restrictions on input array data types.

zeros

  • No changes necessary. No restrictions on output array data types.

zeros_like

  • No changes necessary. No restrictions on output array data types.

Data Type Functions

astype

  • Add note stating that, when casting from a complex data type to a real data type stating, the imaginary component is discarded and casting to integral data types is unspecified and thus implementation-dependent.

    • Discarding the imaginary component follows C and is done in NumPy.
    • Could also disallow/omit/not explicitly support casting from complex to real and require users to explicitly call real if they want to discard the imaginary component.
    • Update: casting from a complex data type to a real-valued data type was disallowed in the specification in favor of API consumers being explicit wrt which component should be cast.

  • status: implemented

broadcast_arrays

  • No changes necessary.

broadcast_to

  • No changes necessary.

can_cast

  • No changes necessary. Follows promotion rules for complex numbers.

finfo

  • No changes necessary.

iinfo

  • No changes necessary.

result_type

  • No changes necessary. Follows promotion rules for complex numbers.

Data Types

  • Add complex64 and complex128 following precedent where the numeric suffix specifies the number of bits.

    • The real and imaginary components should be IEEE 754 floating-point numbers.

  • status: implemented

Default Data Types

  • The default complex number data type is dependent on the default floating-point data type. If the latter is float32, the default complex data type must be complex64.

    • PyTorch already does this.

  • status: implemented

Data Type Categories

  • Add complex data types to list of numeric data types.

  • Add “Real Data Types” category which should be equal to the current list of numeric data types.

  • Add “Complex Data Types” category which only includes the complex number data types.

  • status: implemented

Element-wise Functions

abs

acos

acosh

add

asin

asinh

atan

atan2

  • No complex number support.
  • No changes necessary as already limited to real number data types.

atanh

bitwise_and

  • No changes. Complex dtypes are not allowed.

bitwise_left_shift

  • No changes. Complex dtypes are not allowed.

bitwise_invert

  • No changes. Complex dtypes are not allowed.

bitwise_or

  • No changes. Complex dtypes are not allowed.

bitwise_right_shift

  • No changes. Complex dtypes are not allowed.

bitwise_xor

  • No changes. Complex dtypes are not allowed.

ceil

  • Add complex number support. Independently round components (e.g., as in MATLAB).

    • NumPy does not support in np.ceil.
    • No complex number support due to implementation ambiguity.

cos

cosh

divide

equal

exp

expm1

floor

  • Add complex number support. Independently round components (e.g., as in MATLAB).

    • NumPy does not support in np.floor.
    • No complex number support due to implementation ambiguity.

floor_divide

  • No complex number support.
  • No changes necessary as already limited to real number data types.

greater

greater_equal

isfinite

isinf

isnan

  • Add support for complex numbers.

    • Care should be taken for infinities and NaNs.
      • how to classify a complex number as NaN?
      • Update: if any component is NaN, the complex number is considered “NaN”

  • status: implemented

less

less_equal

log

log1p

log2

log10

logaddexp

  • No complex number support. This function is mainly useful for stats.
  • No changes necessary as already limited to real-valued data types.

logical_and

  • No changes necessary. Input arrays expected to have boolean data type.

logical_not

  • No changes necessary. Input arrays expected to have boolean data type.

logical_or

  • No changes necessary. Input arrays expected to have boolean data type.

logical_xor

  • No changes necessary. Input arrays expected to have boolean data type.

multiply

negative

not_equal

positive

pow

remainder

  • No complex number support.
  • No changes necessary as already limited to real-valued data types.

round

sign

sin

sinh

square

sqrt

subtract

tan

tanh

trunc

  • Add complex number support. Independently truncate components (e.g., as in MATLAB).

    • NumPy does not support in np.trunc.
    • No complex number support due to implementation ambiguity.

Linear Algebra Functions

matmul

  • Add complex number support.

    • complex number addition is well-defined.
    • complex number multiplication is less well-defined and depends on how an implementation chooses to model complex infinity.

  • status: implemented

matrix_transpose

  • No changes necessary.

tensordot

  • Add complex number support.

    • complex number addition is well-defined.
    • complex number multiplication is less well-defined and depends on how an implementation chooses to model complex infinity.

  • status: implemented

vecdot

Manipulation Functions

concat

  • No changes necessary.

expand_dims

  • No changes necessary.

flip

  • No changes necessary.

permute_dims

  • No changes necessary.

reshape

  • No changes necessary.

roll

  • No changes necessary.

squeeze

  • No changes necessary.

stack

  • No changes necessary.

Searching Functions

argmax

  • Require that complex numbers be ordered in lexicographic order?

    • While not compatible with multiplication, lexicographic order is a common total order. E.g., NumPy uses lexicographic order.
    • Another alternative is by magnitude and then phase angle. E.g., MATLAB supports this order.
    • The specification could default to lexicographic, and revisit the ordering relation in the future with possible support for optionally specifying an alternative comparison method.
    • Some care needs to be given to sorting when one or more components is NaN.
    • Update: no support for complex numbers due to a lack of a natural order.

  • status: implemented

argmin

nonzero

where

  • No changes necessary.

Set Functions

unique_all

  • Specify complex number value equality.

    • In C, “In order to support the one-infinity model of complex number arithmetic, C regards any complex value with at least one infinite part as an infinity even if its other part is a NaN…”
    • Need to resolve isinf and isnan duality
    • Update: base equality comparison on equal.

  • status: implemented

unique_counts

unique_inverse

unique_values

Sorting Functions

argsort

sort

Statistical Functions

max

mean

  • No complex number support.
  • Change to only real number data types.

min

prod

std

  • No complex number support.
  • Change to only real number data types.

sum

var

  • No complex number support.
  • Change to only real number data types.

Type Promotion

Rules

Mixing arrays with Python Scalars

Utility Functions

all

any

Linear Algebra Extension

cholesky

cross

det (determinant)

diagonal

  • No changes necessary.

eigh

  • Add complex number support.

    • Require that each matrix must be Hermitian and the returned Q unitary.
    • May want to make the dtype of the eigevalues unconditionally real.

  • status: implemented

eigvalsh

inv

matmul

  • Support added in main namespace.

matrix_norm

matrix_power

matrix_rank

matrix_transpose

  • No changes necessary.

outer

pinv

qr

slogdet

solve

svd

svdvals

trace

vecdot

  • Changes made in main namespace.

vector_norm

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Reactions:3
  • Comments:5 (5 by maintainers)

github_iconTop GitHub Comments

2reactions
rgommerscommented, Dec 14, 2022

All PRs have been merged - adding complex number support to the API is complete. Thanks a lot to everyone who contributed - and in particular @kgryte for all the spec writing.

1reaction
leofangcommented, Apr 18, 2022
  • branch cut policy?

I don’t know why I missed it… CuPy uses Thrust (and it’s likely being used by many other projects that depends on CUDA; IIUC the implementation was ported to libcudacxx) so it’s worth mentioning.

Thrust’s complex math funcs were based on FreeBSD’s. In each FreeBSD man page the branch cut is clearly documented so it’s a good starting point. However, as Thrust didn’t document it clearly (I asked internally and was told to inspect the source directly, which is challenging given my lack of bandwidth) and it’s unclear to me if the Thrust impl was a one-to-one translation of FreeBSD’s since I don’t have access to the FreeBSD source code, I’d proceed with caution.

Read more comments on GitHub >

github_iconTop Results From Across the Web

RFC 3261 SIP: Session Initiation Protocol - IETF
Any implementation that supports TLS MUST support the SIPS URI scheme. The To header field allows for a display name. A UAC may...
Read more >
RFC 7950: The YANG 1.1 Data Modeling Language
YANG version 1.1 is a maintenance release of the YANG language, addressing ambiguities and defects in the original specification. There are a small...
Read more >
RFC822: Standard for ARPA Internet Text Messages
Since any number of levels is possible within the domain hierarchy, specification of a fully qualified address can become inconvenient. This standard permits ......
Read more >
BQN Specification: Complex numbers
Complex numbers are an optional extension to BQN's numeric system. If they are supported, the following functionality must also be supported. This extension...
Read more >
OpenAPI Specification - Version 3.0.3 - Swagger
The OpenAPI Specification defines a standard interface to RESTful APIs ... as a type is also supported and is defined as a JSON...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found
Previous page

Use case study: Intheon

Next page

Definition of `vecdot` is ambiguous in the complex case.