Skip to content

Commit 2232fb9

Browse files
siliataiderhageboeck
authored andcommitted
[DF][Python] Deserialize Dask results in the main thread
1 parent aa67da8 commit 2232fb9

1 file changed

Lines changed: 5 additions & 4 deletions

File tree

  • bindings/distrdf/python/DistRDF/Backends/Dask

bindings/distrdf/python/DistRDF/Backends/Dask/Backend.py

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -323,10 +323,11 @@ def _process_partial_results(self,
323323
cumulative_plots: Dict[int, Any] = {}
324324

325325
# Collect all futures in batches that had arrived since the last iteration
326-
for batch in as_completed(future_tasks, with_results=True).batches():
327-
for future, result in batch:
328-
merged_results = reducer(merged_results, result) if merged_results else result
329-
326+
for batch in as_completed(future_tasks, with_results=False).batches():
327+
for future in batch:
328+
result = future.result()
329+
merged_results = reducer(merged_results, result) if merged_results else result
330+
330331
mergeables = merged_results.mergeables
331332

332333
for pad_num, (drawable_id, (callbacks_list, index, operation_name)) in enumerate(drawables_info_dict.items(), start=1):

0 commit comments

Comments
 (0)