fixes for parallel reconstruction #7758
Conversation
agnxsh
force-pushed
the
recover-parallel
branch
from
7bf20ac to
2b0a983
Compare
|
please re-run macos job I had to kill it to clean up |
| var spawned = 0 | ||
|
|
||
| # Choose a sane limit for concurrent tasks to reduce peak memory/alloc pressure. | ||
| let maxInFlight = if blobCount < 9: blobCount else: 9 |
|
|
||
| # ---- Spawn phase with time limit ---- | ||
| proc freePendingPtrPair(idxPtr: ptr CellIndex, cellsPtr: ptr Cell) = | ||
| if idxPtr != nil: |
There was a problem hiding this comment.
free(NULL)/free(0) is fine and doesn't have to be guarded against
There was a problem hiding this comment.
https://www.open-std.org/JTC1/SC22/wg14/www/docs/n1124.pdf
7.20.3.2 The free function
...
The free function causes the space pointed to by ptr to be deallocated, that is, made available for further allocation. If ptr is a null pointer, no action occurs
| # Allocate unmanaged C buffers and copy data into them | ||
| let | ||
| idxBytes = csize_t(columnCount) * csize_t(sizeof(CellIndex)) | ||
| cellsBytes = csize_t(columnCount) * csize_t(sizeof(Cell)) |
There was a problem hiding this comment.
This is what calloc does. It also accounts for potential alignment issues (i.e. due to alignment requirements, the size isn't just n*sizeof(y))
| idxPtr = cast[ptr CellIndex](c_malloc(idxBytes)) | ||
| if idxPtr == nil: | ||
| drainPending(0) | ||
| return err("Out of memory") |
There was a problem hiding this comment.
Would be good to have this Out of memory message be slightly more specific about its context and be easily distinguishable should it ever appear.
| if cellsPtr == nil: | ||
| c_free(idxPtr) | ||
| drainPending(0) | ||
| return err("Out of memory") |
There was a problem hiding this comment.
Would be good to have this Out of memory message be slightly more specific about its context and be easily distinguishable should it ever appear.
| localIndices[j] = idxArr[j] | ||
| localCells[j] = cellsArr[j] | ||
| # use the task wrapper which maps string errors to void | ||
| return recoverCellsAndKzgProofsTask(localIndices, localCells) |
4 participants
No description provided.