Unhelpful error when alignging (large) nan-padded arrays

[meindertsma]

I have 4656x3520 arrays from fits files that I have padded with numpy.nan to be 13968x10560 (a bit excessive, but I wanted to be sure there would be enough space). Minimum example that gives the error for aligning the images:

import numpy as np
import astroalign as aa
from astropy.io import fits

target = fits.open('A13_G*.fits')[0].data
toalign = fits.open('A12_G*.fits')[0].data
target = np.pad(target, ((target.shape[0], target.shape[0]), (target.shape[1], target.shape[1])), constant_values=np.nan)
toalign = np.pad(toalign, ((toalign.shape[0], toalign.shape[0]), (toalign.shape[1], toalign.shape[1])), constant_values=np.nan)
aa.register(target,  toalign)

I want to combine the images into a mosaic, so my idea was to pad them with NaNs, and then align and stack the images, and then trimming excess NaNs.

And this is what I get from running the code above:

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
/Software/users/modules/7/software/anaconda3/2020.07/lib/python3.8/site-packages/astroalign.py in find_transform(source, target, max_control_points, detection_sigma, min_area)
    263             # Assume it's a 2D image
--> 264             source_controlp = _find_sources(
    265                 _bw(_data(source)),

/Software/users/modules/7/software/anaconda3/2020.07/lib/python3.8/site-packages/astroalign.py in _find_sources(img, detection_sigma, min_area)
    485     thresh = detection_sigma * bkg.globalrms
--> 486     sources = sep.extract(image - bkg.back(), thresh, minarea=min_area)
    487     sources.sort(order="flux")

sep.pyx in sep.extract()

sep.pyx in sep._assert_ok()

Exception: internal pixel buffer full: The limit of 300000 active object pixels over the detection threshold was reached. Check that the image is background subtracted and the detection threshold is not too low. If you need to increase the limit, use set_extract_pixstack.

During handling of the above exception, another exception occurred:

TypeError                                 Traceback (most recent call last)
<ipython-input-10-c88bbfad1678> in <module>
      7 target = np.pad(target, ((target.shape[0], target.shape[0]), (target.shape[1], target.shape[1])), constant_values=np.nan)
      8 toalign = np.pad(toalign, ((toalign.shape[0], toalign.shape[0]), (toalign.shape[1], toalign.shape[1])), constant_values=np.nan)
----> 9 aa.register(target,  toalign)

/Software/users/modules/7/software/anaconda3/2020.07/lib/python3.8/site-packages/astroalign.py in register(source, target, fill_value, propagate_mask, max_control_points, detection_sigma, min_area)
    460 
    461     """
--> 462     t, __ = find_transform(
    463         source=source,
    464         target=target,

/Software/users/modules/7/software/anaconda3/2020.07/lib/python3.8/site-packages/astroalign.py in find_transform(source, target, max_control_points, detection_sigma, min_area)
    268             )[:max_control_points]
    269     except Exception:
--> 270         raise TypeError("Input type for source not supported.")
    271 
    272     try:

TypeError: Input type for source not supported.

Any help with solving my problem would be appreciated.
I can provide the fits files if necessary.
Edit: The type error is probably because of the NaNs, so I think that that is not the main problem.
Edit2: I have tried increasing the detection sigma upto 10, and still got the same error, and then my other images didn't align.
I have multiple images that I want to align, and they should all overlap the target I'm using. Some work, some give MaxIterError (possibly due to them not overlapping), but a lot give this error.

[martinberoiz]

I see, astroalign can't deal with NaN's unfortunately. Try padding with zeros or the median value of the background or compatible noise. Also before sending to register check that the array dimensions are correct and that the type makes sense.

[meindertsma]

I can't only be due to the NaNs though, as some of the padded frames did align, and some gave the (normal) MaxIterError. I don't think I can use 0 or any other number as the fill value, as this will affect the outcome when combing the arrays using a median. I can use nan, because np.nanmedian ignores them, but any other value might already be in the array.
The dimensions make sense as far as I can tell, and I think the TypeError is due first error message not being able to handle NaNs, which is a sep issue.
Is there a way to increase the internal pixel buffer without importing sep? Or is this unlikely to solve the issue?

[martinberoiz]

I think you have several issues going on and it's hard for me to pinpoint what's not working for you. I'll try to advice you as much as I can, but I suggest you try to reduce the problem to the smallest that reproduce the error.

• If you get MaxIterError it means no transformation was found. Not much you can do about it I'm afraid.
• You may use np.nanmedian afterwards, but astroalign just doesn't use it and it may spill the NaN's (or not, just try it)
• I don't specifically reserve any specific internal pixel buffer. I don't see how this can fix any problem since you don't mention any memory issues?

If you want, you can send me a (1) problem image pair and the code you used and I'll try to figure out what's wrong.

[meindertsma]

My code is:

import numpy as np
import astroalign as aa
from astropy.io import fits

target = fits.open('Combined_band_A13_Lum.fits')[0].data.byteswap().newbyteorder("<")
toalign = fits.open('Combined_band_A12_R*.fits')[0].data.byteswap().newbyteorder("<")
target = np.pad(target, ((target.shape[0], target.shape[0]), (target.shape[1], target.shape[1])), constant_values=np.nan)
toalign = np.pad(toalign, ((toalign.shape[0], toalign.shape[0]), (toalign.shape[1], toalign.shape[1])), constant_values=np.nan)
aa.register(target,  toalign, max_control_points=200)

The target is Combined_band_A13_Lum.fits for each of the below:
with max_control_points=50:
Works: Combined_band_A18_G*.fits
MaxIterError: Combined_band_A12_Lum.fits
Exception + TypeError: Combined_band_A12_R*.fits

with max_control_points=200:
Works: Combined_band_A12_Lum.fits
MaxIterError: Combined_band_A12_G*.fits
Exception + TypeError: Combined_band_A12_R*.fits

I have uploaded the files in zip folder here: https://drive.google.com/file/d/1O_mr10SEndYwsTKag0Y7GugyzjFKyUUz/view?usp=sharing
The images do all overlap, I checked using Astrometry.net, however the overlap between A18 and A13 is a lot more than between A13 and A12. I think this 'solved' issue with SEP is also related: sep-developers/sep#15