Skip to content

replace pathos.multiprocessing with multiprocess - #177

Merged
mihaic merged 2 commits into
brainiak:masterfrom
yidawang:fcma_multiprocess
Feb 8, 2017
Merged

mihaic merged 2 commits into
brainiak:masterfrom
yidawang:fcma_multiprocess

Conversation

@yidawang

@yidawang yidawang commented Feb 8, 2017

Copy link
Copy Markdown
Member

This is to get rid of the dependency of pathos. It reported "Too many open files error" before but is fixed by wrapping the usage of Pool using with.

Comment thread brainiak/fcma/voxelselector.py Outdated
results = list(pool.map(
lambda x: _cross_validation_for_one_voxel
(x[0], x[1], x[2], x[3]),
inlist))

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about:

results = [pool.starmap(_cross_validation_for_one_voxel, inlist)]

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@mihaic
mihaic merged commit 4c8486e into brainiak:master Feb 8, 2017
@mihaic mihaic mentioned this pull request Feb 8, 2017
2 tasks done
@yidawang
yidawang deleted the fcma_multiprocess branch February 8, 2017 19:32
danielsuo pushed a commit that referenced this pull request Nov 16, 2017
Implement launching cluster with shell
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants