Repository navigation
Refactor code to remove Pathos #149
Description
The Multiprocess package, used by Pathos, should work:
https://pypi.python.org/pypi/multiprocess
Mike tried, but had trouble limiting the number of processes created.
Even better, look into using standard library multiprocessing directly with the Dill serializer (also used by Pathos):
https://pypi.python.org/pypi/dill
Mike tried, but could not obtain the same performance because the serialization/deserialization code was called unnecessarily often.
- Searchlight
- FCMA
Activity
[-]Refactor searchlight multiprocessing code to remove Pathos[/-][+]Refactor code to remove Pathos[/+]mihaic commentedon Feb 8, 2017
@mjanderson09, can you please also try the pool context manager in the searchlight code, like PR #177?
[searchlight] Switch from pathos to multiprocess
mihaic commentedon Jun 14, 2017
Fixed by #225.
Merge pull request #149 from amplab/getput
Give run_function_on_all_workers to take a worker_info dictionary inc…