Skip to content

Add documentation about ordering no longer being preserved #47

@charlescapps

Description

@charlescapps

Hi, I was looking into upgrading from 1.x to 2.x of tiny-async-pool, and I'm glad that I perused the source code, because it is evident that the order of the results no longer matches the order of the input data!

In the previous implementation, the order is clearly preserved because the array of ret is in the order of the input iterable.

This is rather a trap that someone could easily fall into, since the incorrect result order might not crop up in unit tests or under most circumstances!

Can we update the documentation, and/or add an example of how to preserve order?

Here's some possible code to wrap this in a way that preserves order and has 1.x semantics -

import asyncPool from "tiny-async-pool";

// Wrap tiny-async-pool with semantics of 1.x, also ensuring the order of the results
export const awaitPromisesWithConcurrency = async <IN, OUT>(
  concurrency: number,
  data: readonly IN[],
  iteratorFn: (value: IN) => Promise<OUT>
): Promise<OUT[]> => {
   // Tag the data & result with its index, to ensure the results are in the correct order
  const dataWithIndexes: [IN, number][] = data.map((value, index) => [value, index]);
  const iteratorFnWithIndex: (value: [IN, number]) => Promise<[OUT, number]> =
    ([value, index]: [IN, number]) => iteratorFn(value).then(result => [result, index]);
  const asyncIterable = asyncPool(concurrency, dataWithIndexes, promiseCreatorWithIndex);
  // Insert data in the results array in the correct index
  const results: OUT[] = [];
  for await (const [result, index] of asyncIterable) {
    results[index] = result;
  }
  return results;
};

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions