Incorrect results with WCS.world_to_pixel for some coordinates (v5.0.4)

Workaround

Use development version until v5.0.5 is released if you are stuck with v5.0.x series.

Description

Note: This issue is fixed in the development version of astropy. Unfortunately, the version of astropy I have to use is fixed at 5.0 and is unlikely to ever be updated, so I would like to gain an insight into why this error is occurring, so I can try to prevent it from happening.

I have discovered that for some WCSs (possibly because they are using TPV distortions?) world_to_pixel gives incorrect results for some input coordinates. Worse still, if processing many coordinates simultaneously, incorrect results can be returned for all points.

My main use case is to check which of a large number (hundreds of thousands) of points are within the FOV of a detector or not. In order to do this I use SkyCoord.contained_by (I have also tried WCS.footprint_contains), however if a “bad” point is in the list of coordinates, the output from these methods is incorrect.

I suspect the issue is that I am calling this function on points very far away from the CCD detector (see example code below) where the distortion is probably horribly inaccurate. For a very small number of these points, the inversion fails, and somehow causes the inversion for all subsequent points to fail. If that is the case, I can implement a workaround that checks how far every point is from the centre of the image, and discard all points definitely not in the image, before passing the remaining candidate points into SkyCoord.contained_by.

Expected behaviour

• WCS.world_to_pixel returns the correct pixel coordinates of a point
• SkyCoord.contained_by returns the correct results for a set of input points

Actual behaviour

• For some points, WCS.world_to_pixel gives wildly incorrect results
• If a bad point is present in the coordinates list, all points after this in the list give incorrect results from SkyCoord.contained_by

Steps to Reproduce

Here’s a snippet with a WCS and a “bad” coordinate, which demonstrates the incorrect behaviour.

import numpy as np
import matplotlib.pyplot as plt

from astropy.wcs import WCS
from astropy.coordinates import SkyCoord
from astropy.units import degree


WCS_HDR = {
    "WCSAXES": 2,
    "CRPIX1": 2048.5,
    "CRPIX2": 2068.5,
    "PC1_1": 2.5775031003848e-05,
    "PC1_2": -1.0229055179621e-05,
    "PC2_1": -1.002710164877e-05,
    "PC2_2": -2.5976175077716e-05,
    "CDELT1": 1.0,
    "CDELT2": 1.0,
    "CUNIT1": "deg",
    "CUNIT2": "deg",
    "CTYPE1": "RA---TPV",
    "CTYPE2": "DEC--TPV",
    "CRVAL1": 231.44766230612,
    "CRVAL2": 30.279811520079,
    "LONPOLE": 180.0,
    "LATPOLE": 30.279811520079,
    "CRDER1": 2.0186312779783e-06,
    "CRDER2": 1.3182628539451e-06,
    "CSYER1": 2.472294743461e-07,
    "CSYER2": -1.886101405461e-07,
    "TIMESYS": "UTC",
    "MJDREF": 0.0,
    "DATE-OBS": "2022-04-25T13:23:30.458686",
    "MJD-OBS": 59694.55799142,
    "RADESYS": "ICRS",
    "PV1_1": 1.0,
    "PV1_4": -0.014168057351631,
    "PV1_5": 0.0064243333222503,
    "PV1_6": -0.0047023798235038,
    "PV1_7": -0.0068384489046016,
    "PV1_8": 0.00075844325976519,
    "PV1_9": -0.0053436189574974,
    "PV1_10": -0.00037824218754685,
    "PV2_1": 1.0,
    "PV2_2": 5.5511151231258e-17,
    "PV2_4": 0.009608591877984,
    "PV2_5": -0.0093441295487718,
    "PV2_6": 0.0031295540585114,
    "PV2_7": -0.0076164769072577,
    "PV2_8": -0.0011483403573003,
    "PV2_9": -0.0085066413113332,
    "PV2_10": 0.00054421248101201,
}


BAD_RA = 231.63289040154802
BAD_DEC = 31.3230668542431

wcs = WCS(WCS_HDR)

# The pixel positions according to world_to_pixel
# returns 0.00155357, -12.33174808
print(
    "Bad point pixel coords according to world_to_pix",
    wcs.world_to_pixel(SkyCoord(BAD_RA, BAD_DEC, unit=degree)),
)

# The pixel positions for a point offsetted by 0.1" - should be rougly 1 pixel different
# returns -6255.40493233, -34915.72059282 - definitely more than 1 pixel different!
print(
    "A point roughly 1 pixel away from the bad point",
    wcs.world_to_pixel(
        SkyCoord(BAD_RA + 1 / 3600.0, BAD_DEC + 1 / 3600.0, unit=degree)
    ),
)


# Now let's generate a number of randomly placed points around the image as described by the CCD:

# determine the sky coordinates of the centre of the detector (say the detector is 100x100 pixels)
nx = ny = 100
wcs.array_shape = (ny, nx)
centre_coord = wcs.pixel_to_world(nx / 2, ny / 2)
ra_c, dec_c = centre_coord.ra.degree, centre_coord.dec.degree

# get 10k random points 10" around detector centre
n_points = 10000
r_points = 10 / 3600
ras = ra_c + np.random.uniform(-r_points, r_points, n_points)
decs = dec_c + np.random.uniform(-r_points, r_points, n_points)

# Use SkyCoords.contained_by to determine which of these points are within the FOV of the detector
sk = SkyCoord(ras, decs, unit=degree)
inds = sk.contained_by(wcs)
print("Number of points in the detector (reference value) = ", np.sum(inds))
# Returns the "correct" answer

# Now append the bad point to the end of the coordinates array and repeat
ras_a = np.append(ras, BAD_RA)
decs_a = np.append(decs, BAD_DEC)
sk_a = SkyCoord(ras_a, decs_a, unit=degree)
inds_a = sk_a.contained_by(wcs)
print("Number of points in the detector (bad point appended) = ", np.sum(inds_a))
# Also the correct answer

# Now prepend the bad point to the coordinates array and repeat
ras_p = np.append(BAD_RA, ras)
decs_p = np.append(BAD_DEC, decs)
sk_p = SkyCoord(ras_p, decs_p, unit=degree)
inds_p = sk_p.contained_by(wcs)
print("Number of points in the detector (bad point prepended) = ", np.sum(inds_p))
# Returns 0... This bad point seems to "break" every point after it

# Now put the bad point in the middle
ras_m = np.append(ras[0 : n_points // 2], np.append(BAD_RA, ras[n_points // 2 :]))
decs_m = np.append(decs[0 : n_points // 2], np.append(BAD_DEC, decs[n_points // 2 :]))
sk_m = SkyCoord(ras_m, decs_m, unit=degree)
inds_m = sk_m.contained_by(wcs)
print("Number of points in the detector (bad point in middle) = ", np.sum(inds_m))
# Returns something in between the true result and zero... wrong results for points after the bad point


# Now examine the behaviour of wcs.world_to_pixel around the bad point

# Consider a cluster of points 0.001" around the bad point
x_off = np.linspace(-0.001, 0.001, 20)
y_off = x_off
xs = []
ys = []
ras = []
decs = []
for dx in x_off:
    for dy in y_off:
        ra = BAD_RA + dx / 3600
        dec = BAD_DEC + dy / 3600
        ras.append(ra)
        decs.append(dec)

        x, y = wcs.world_to_pixel(SkyCoord(ra, dec, unit=degree))

        xs.append(x)
        ys.append(y)

# Plotting these shows a very large scatter in the positions...
# if you zoom in on where the points should be (-6260, -34903)-ish
# you can see that all the points along a line are missing - I assume
# world_to_pixel does not converge for these points.
print(xs[0], ys[0])
plt.scatter(xs, ys)
plt.show()

System Details

Tested on:

• Linux 5.15
• Python 3.9.9
• Numpy 1.20.3
• pyerfa 2.0.0.1
• astropy 5.0
• Scipy 1.7.1
• Matplotlib 3.5.1

[gpsgibb]

Installing v5.0.x with pip via: pip install git+https://github.com/astropy/astropy.git@v5.0.x (Astropy’s reported version is 5.0.5.dev243+g48d7caa96)

I can confirm that the problem is fixed in this branch

[pllim]

Great! This means once 5.0.5 is released and you upgrade, this problem will be fixed. And given that it is already fixed in development branch, I will update the original post to indicate as such and close this issue because there is nothing to fix, just have to wait for the release. Thanks for your patience!