Skip to content

wip: fix: resize with interpolationType nearest #453

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 11 commits into from
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
28 changes: 24 additions & 4 deletions src/geometry/__tests__/resize.test.ts
Original file line number Diff line number Diff line change
@@ -1,26 +1,46 @@
import { encodePng } from '../../save';
import * as path from 'node:path';

test.skip('compare result of resize with opencv (nearest)', () => {
import { encodePng, write } from '../../save';

test('compare result of resize with opencv (nearest)', async () => {
const img = testUtils.load('opencv/test.png');
const expectedImg = testUtils.load('opencv/testResizeNearest.png');

const resized = img.resize({
xFactor: 10,
yFactor: 10,
interpolationType: 'nearest',
});

const substraction = expectedImg.clone().subtract(resized);
await write(
path.join(__dirname, 'resize_nearest_substraction.png'),
substraction,
);
await write(path.join(__dirname, 'resize_nearest.png'), resized);

expect(resized).toMatchImage('opencv/testResizeNearest.png');
});

test.skip('compare result of resize with opencv (bilinear)', () => {
test('compare result of resize with opencv (bilinear)', async () => {
const img = testUtils.load('opencv/test.png');
const expectedImg = testUtils.load('opencv/testResizeBilinear.png');

const resized = img.resize({
xFactor: 10,
yFactor: 10,
interpolationType: 'bilinear',
});

expect(resized).toMatchImage('opencv/testResizeBilinear.png');
const substraction = expectedImg.clone().subtract(resized);
await write(
path.join(__dirname, 'resize_bilinear_substraction.png'),
substraction,
);
await write(path.join(__dirname, 'resize_bilinear.png'), resized);

// OpenCV bilinear interpolation is less precise for speed.
expect(resized).toMatchImage('opencv/testResizeBilinear.png', { error: 25 });
});
Comment on lines +42 to +43
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

error: 25 is too much (that's 10% of the range of values 0-255)


test('result should have correct dimensions', () => {
Expand Down
52 changes: 19 additions & 33 deletions src/geometry/resize.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,6 @@
import { Image } from '../Image';
import { getClamp } from '../utils/clamp';
import { getBorderInterpolation, BorderType } from '../utils/interpolateBorder';
import {
getInterpolationFunction,
InterpolationType,
} from '../utils/interpolatePixel';
import { BorderType } from '../utils/interpolateBorder';
import { InterpolationType } from '../utils/interpolatePixel';
import { assert } from '../utils/validators/assert';

export interface ResizeOptions {
Expand Down Expand Up @@ -53,36 +49,26 @@ export interface ResizeOptions {
* @returns The new image.
*/
export function resize(image: Image, options: ResizeOptions): Image {
const { interpolationType = 'bilinear' } = options;
const {
interpolationType = 'bilinear',
borderType = 'constant',
borderType = interpolationType === 'bilinear' ? 'replicate' : 'constant',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why? Is it OpenCV behavior?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know but bilinear was better with replicate by default.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is probably not the right way to fix it.

borderValue = 0,
} = options;
const { width, height } = checkOptions(image, options);
const newImage = Image.createFrom(image, { width, height });
const interpolate = getInterpolationFunction(interpolationType);
const interpolateBorder = getBorderInterpolation(borderType, borderValue);
const clamp = getClamp(newImage);
const intervalX = (image.width - 1) / (width - 1);
const intervalY = (image.height - 1) / (height - 1);
for (let row = 0; row < newImage.height; row++) {
for (let column = 0; column < newImage.width; column++) {
const nx = column * intervalX;
const ny = row * intervalY;
for (let channel = 0; channel < newImage.channels; channel++) {
const newValue = interpolate(
image,
nx,
ny,
channel,
interpolateBorder,
clamp,
);
newImage.setValue(column, row, channel, newValue);
}
}
}
return newImage;
const { width, height, xFactor, yFactor } = checkOptions(image, options);

return image.transform(
[
[xFactor, 0, 0],
[0, yFactor, 0],
],
{
width,
height,
borderType,
borderValue,
interpolationType,
},
);
}

/**
Expand Down
5 changes: 3 additions & 2 deletions src/geometry/transform.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ import { Image } from '../Image';
import { getClamp } from '../utils/clamp';
import { BorderType, getBorderInterpolation } from '../utils/interpolateBorder';
import {
getInterpolationFunction,
getInterpolationNeighbourFunction,
InterpolationType,
} from '../utils/interpolatePixel';

Expand Down Expand Up @@ -130,11 +130,12 @@ export function transform(
const interpolateBorder = getBorderInterpolation(borderType, borderValue);
const clamp = getClamp(newImage);

const interpolate = getInterpolationFunction(interpolationType);
const interpolate = getInterpolationNeighbourFunction(interpolationType);
for (let row = 0; row < newImage.height; row++) {
for (let column = 0; column < newImage.width; column++) {
const nx = transformPoint(transformMatrix[0], column, row);
const ny = transformPoint(transformMatrix[1], column, row);

for (let channel = 0; channel < newImage.channels; channel++) {
const newValue = interpolate(
image,
Expand Down
128 changes: 71 additions & 57 deletions src/utils/interpolatePixel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,32 +12,50 @@ export const InterpolationType = {
export type InterpolationType =
(typeof InterpolationType)[keyof typeof InterpolationType];

type InterpolationFunction = (
type InterpolationNeighbourFunction = (
/**
* The image to interpolate.
*/
image: Image,
column: number,
row: number,
/**
* interpolation of original column (float) from target column.
*/
nx: number,
/**
* interpolation of original row (float) from target row.
*/
ny: number,
/**
* channel index.
*/
channel: number,
intepolateBorder: BorderInterpolationFunction,
/**
* Border interpolation function.
*/
interpolateBorder: BorderInterpolationFunction,
/**
* Clamp function.
*/
clamp: ClampFunction,
) => number;

/**
* Get the interpolation function based on its name.
* Get the interpolation neighbour function based on its name.
* @param interpolationType - Specified interpolation type.
* @returns The interpolation function.
*/
export function getInterpolationFunction(
export function getInterpolationNeighbourFunction(
interpolationType: InterpolationType,
): InterpolationFunction {
): InterpolationNeighbourFunction {
switch (interpolationType) {
case 'nearest': {
return interpolateNearest;
return interpolateNeighbourNearest;
}
case 'bilinear': {
return interpolateBilinear;
return interpolateNeighbourBilinear;
}
case 'bicubic': {
return interpolateBicubic;
return interpolateNeighbourBicubic;
}
default: {
throw new RangeError(`invalid interpolationType: ${interpolationType}`);
Expand All @@ -46,97 +64,93 @@ export function getInterpolationFunction(
}

/**
* Interpolate using nearest neighbor.
* Interpolate using the nearest neighbour.
* @param image - The image to interpolate.
* @param column - Column index.
* @param row - Row index.
* @param channel - Channel index.
* @param nx - column (float) of the point to interpolate
* @param ny - row (float) of the point to interpolate
* @param channel - channel index.
* @param interpolateBorder - Border interpolation function.
* @returns The interpolated value.
*/
function interpolateNearest(
function interpolateNeighbourNearest(
image: Image,
column: number,
row: number,
nx: number,
ny: number,
channel: number,
interpolateBorder: BorderInterpolationFunction,
): number {
column = Math.round(column);
row = Math.round(row);
const column = Math.floor(nx);
const row = Math.floor(ny);

return interpolateBorder(column, row, channel, image);
}

/**
* Interpolate using bilinear interpolation.
* @param image - The image to interpolate.
* @param column - Column index.
* @param row - Row index.
* @param channel - Channel index.
* @param nx - column (float) of the point to interpolate
* @param ny - row (float) of the point to interpolate
* @param channel - channel index.
* @param interpolateBorder - Border interpolation function.
* @returns The interpolated value.
*/
function interpolateBilinear(
function interpolateNeighbourBilinear(
image: Image,
column: number,
row: number,
nx: number,
ny: number,
channel: number,
interpolateBorder: BorderInterpolationFunction,
): number {
const px0 = Math.floor(column);
const py0 = Math.floor(row);
nx = Math.max(0, Math.min(nx - 0.5, image.width));
ny = Math.max(0, Math.min(ny - 0.5, image.height));

const px0 = Math.floor(nx);
const py0 = Math.floor(ny);
const px1 = px0 + 1;
const py1 = py0 + 1;

if (px1 < image.width && py1 < image.height && px0 >= 0 && py0 >= 0) {
const vx0y0 = image.getValue(px0, py0, channel);
const vx1y0 = image.getValue(px1, py0, channel);
const vx0y1 = image.getValue(px0, py1, channel);
const vx1y1 = image.getValue(px1, py1, channel);

const r1 = (px1 - column) * vx0y0 + (column - px0) * vx1y0;
const r2 = (px1 - column) * vx0y1 + (column - px0) * vx1y1;
return round((py1 - row) * r1 + (row - py0) * r2);
} else {
const vx0y0 = interpolateBorder(px0, py0, channel, image);
const vx1y0 = interpolateBorder(px1, py0, channel, image);
const vx0y1 = interpolateBorder(px0, py1, channel, image);
const vx1y1 = interpolateBorder(px1, py1, channel, image);

const r1 = (px1 - column) * vx0y0 + (column - px0) * vx1y0;
const r2 = (px1 - column) * vx0y1 + (column - px0) * vx1y1;
return round((py1 - row) * r1 + (row - py0) * r2);
}
const vx0y0 = interpolateBorder(px0, py0, channel, image);
const vx1y0 = interpolateBorder(px1, py0, channel, image);
const vx0y1 = interpolateBorder(px0, py1, channel, image);
const vx1y1 = interpolateBorder(px1, py1, channel, image);

const px1nx = px1 - nx;
const nxpx0 = nx - px0;
const py1ny = py1 - ny;
const nypy0 = ny - py0;

const r1 = px1nx * vx0y0 + nxpx0 * vx1y0;
const r2 = px1nx * vx0y1 + nxpx0 * vx1y1;
return py1ny * r1 + nypy0 * r2;
}

/**
* Interpolate using bicubic interpolation.
* @param image - The image to interpolate.
* @param column - Column index.
* @param row - Row index.
* @param channel - Channel index.
* @param nx - interpolation of original column (float) from target column.
* @param ny - interpolation of original row (float) from target row.
* @param channel - channel index.
* @param interpolateBorder - Border interpolation function.
* @param clamp - Clamp function.
* @returns The interpolated value.
*/
function interpolateBicubic(
function interpolateNeighbourBicubic(
image: Image,
column: number,
row: number,
nx: number,
ny: number,
channel: number,
interpolateBorder: BorderInterpolationFunction,
clamp: ClampFunction,
): number {
const px1 = Math.floor(column);
const py1 = Math.floor(row);
const px1 = Math.floor(nx);
const py1 = Math.floor(ny);

if (px1 === column && py1 === row) {
if (px1 === nx && py1 === ny) {
return interpolateBorder(px1, py1, channel, image);
}

const xNorm = column - px1;
const yNorm = row - py1;
const xNorm = nx - px1;
const yNorm = ny - py1;

const vx0y0 = interpolateBorder(px1 - 1, py1 - 1, channel, image);
const vx1y0 = interpolateBorder(px1, py1 - 1, channel, image);
Expand Down
Loading