Skip to content

Commit 052c2a6

Browse files
treeowlkonsumlamm
authored andcommitted
Chunk better
`parListChunk` previously split a list up into chunks, applied the given strategy to each chunk, and then put them all together again. This led to two extra copies of the list. We get very little benefit from actually splitting the list, because the parallel computations need to traverse their part anyway; we can instead just hand off the whole list and let them count out their chunk. We count each chunk twice, but that shouldn't cost enough to matter. Now that `Eval` has a `MonadFix` instance, we can avoid actually having to put together lists at the end; instead, we pass each parallel computation the (as-yet-uncomputed) result of calculating the rest of the list.
1 parent 350e513 commit 052c2a6

File tree

1 file changed

+30
-6
lines changed

1 file changed

+30
-6
lines changed

Control/Parallel/Strategies.hs

Lines changed: 30 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
{-# LANGUAGE BangPatterns, CPP, MagicHash, UnboxedTuples #-}
22
{-# LANGUAGE GeneralizedNewtypeDeriving #-}
3+
{-# LANGUAGE RecursiveDo #-}
34
-----------------------------------------------------------------------------
45
-- |
56
-- Module : Control.Parallel.Strategies
@@ -599,13 +600,36 @@ parListNth n strat = evalListNth n (rparWith strat)
599600
-- 'parList'
600601
--
601602
parListChunk :: Int -> Strategy a -> Strategy [a]
602-
parListChunk n strat xs
603-
| n <= 1 = parList strat xs
604-
| otherwise = concat `fmap` parList (evalList strat) (chunk n xs)
603+
parListChunk = parListChunk'
605604

606-
chunk :: Int -> [a] -> [[a]]
607-
chunk _ [] = []
608-
chunk n xs = as : chunk n bs where (as,bs) = splitAt n xs
605+
parListChunk' :: Int -> (a -> Eval b) -> [a] -> Eval [b]
606+
parListChunk' n strat
607+
| n <= 1 = traverse strat
608+
609+
-- parListChunk n strat xs =
610+
-- concat `fmap` 'parList' ('evalList' strat) (chunk n xs)
611+
-- but we avoid building intermediate lists.
612+
parListChunk' n0 strat = go n0
613+
where
614+
go !_n [] = pure []
615+
go n as = mdo
616+
-- Calculate the first chunk in parallel, passing it the result
617+
-- of calculating the rest
618+
bs <- rpar $ runEval $ evalChunk strat more n as
619+
620+
-- Calculate the rest
621+
more <- go n (drop n as)
622+
return bs
623+
624+
-- | @evalChunk strat end n as@ uses @strat@ to evaluate the first @n@
625+
-- elements of @as@ (ignoring the rest) and appends @end@ to the result.
626+
evalChunk :: (a -> Eval b) -> [b] -> Int -> [a] -> Eval [b]
627+
evalChunk strat = \end ->
628+
let
629+
go !_n [] = pure end
630+
go 0 _ = pure end
631+
go n (a:as) = (:) <$> strat a <*> go (n - 1) as
632+
in go
609633

610634
-- --------------------------------------------------------------------------
611635
-- Convenience

0 commit comments

Comments
 (0)