Skip to content

Adding statistical tests to ISC and ISFC - #310

Merged
mihaic merged 7 commits into
brainiak:masterfrom
cbaldassano:isfc-stats
Dec 13, 2017
Merged

mihaic merged 7 commits into
brainiak:masterfrom
cbaldassano:isfc-stats

Conversation

@cbaldassano

Copy link
Copy Markdown
Collaborator

Adding statistical testing through phase randomization, as requested by https://groups.google.com/forum/#!topic/brainiak/vZnOgGoV7q4 and as described in Simony et al. (2015, DOI: 10.1038/ncomms12141)

@cbaldassano
cbaldassano requested a review from mihaic December 11, 2017 21:38
Comment thread brainiak/isfc.py Outdated
pos_freq = np.arange(1, (D.shape[1] - 1) // 2 + 1)
neg_freq = np.arange(D.shape[1] - 1, (D.shape[1] - 1) // 2, -1)

shift = np.random.rand(D.shape[0], len(pos_freq), D.shape[2]) * 2 * math.pi

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We try to avoid np.random.rand in favor of np.random.RandomState.rand, according to the Scikit-learn guidelines:
https://github.com/brainiak/brainiak/blob/master/CONTRIBUTING.rst#standards
http://scikit-learn.org/stable/developers/contributing.html#random-numbers

Comment thread tests/isfc/test_isfc.py
D[:, :, 2] = \
[[-0.26593192, -0.56914734, 0.28397317, 0.30276589, -0.42637472],
[-0.67598379, -0.51549055, -0.64196342, 1.60686666, 0.04127293]]
[0.04127293, -0.67598379, -0.51549055, -0.64196342, 1.60686666]]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is the reordering necessary?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment to clarify, I wanted to create a null ISC to make sure it was not significant.

Comment thread tests/isfc/test_isfc.py Outdated
assert D.shape == (4, 5, 2), "Loaded data has incorrect shape"

ISFC = brainiak.isfc.isfc(D)
np.random.seed(0)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't forget to remove this seed setting when you switch to using RandomState.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you actually forget to remove the line? :)

Comment thread brainiak/isfc.py Outdated
return ISFC, p


def phase_randomize(D):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function is not really ISFC specific. I think it fits better in utils/utils.py.

Comment thread brainiak/isfc.py Outdated
back into the time domain. This yields timecourses with the same power
spectrum (and thus the same autocorrelation) as the original timecourses,
but will remove any meaningful temporal relationships between the
timecourses.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have a paper reference by any chance?

Comment thread brainiak/isfc.py Outdated
return np.real(ifft(F, axis=1))


def ecdf(x):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utils/utils.py?

Comment thread brainiak/isfc.py Outdated
return ISFC[..., 0]


def p_from_null(X, two_sided=False):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this function also belongs in utils/utils.py.

Comment thread tests/isfc/test_isfc.py Outdated
assert D.shape == (4, 5, 2), "Loaded data has incorrect shape"

ISFC = brainiak.isfc.isfc(D)
np.random.seed(0)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you actually forget to remove the line? :)

Comment thread brainiak/utils/utils.py
return result


def phase_randomize(D, random_state=0):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add tests for the new functions. They can be as simple as you want.

@mihaic mihaic left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the prompt changes, @cbaldassano.

@mihaic
mihaic merged commit b0a34e7 into brainiak:master Dec 13, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants