hey stats geeks...

[chhotii]

Let's say you have a situation in which you're tempted to use a t-test. However, not all the values came from different subjects. One subject was read 4 times, one subject was read 2 times, and the other 3 reads are each from a different subject. (Don't ask me why this was done, not my experiment...) This so totally violates the assumption of samples being independent, right? What would you do? Take an average for each subject, and do a t-test on the averages for each subject?

Comments

[quietann.livejournal.com]

There are t-tests for paired samples -- two samples per person. But if you have more than 2 per person, you're moving out of t-test land unless there is some reason to drop certain scores.

[chhotii.livejournal.com]

Paired t-test makes sense when the reason that there are 2 samples per person is that each person gives 1 sample before treatment + 1 sample after treatment and what you're really interested in is the difference between before and after. Actually in this case I was doing a paired t-test because for each run of the experiment, there is a "before" number and an "after" number, and the null hypothesis is that before and after are the same, i.e. the mean difference is zero. But then I noticed that it wasn't a different subject every time... A couple of subjects performed the experiment more than once. Now, if they had just had exactly one subject do the experiment over and over, of course we could do a paired t-test, testing the hypothesis that before and after are different for that one subject. But nooo, they had to have 3 subjects do the experiment just once each (thus yielding 2 numbers each) while another one did it 4 times (thus yielding 8 numbers) etc. for some unfathomable reason. So we are moving out of t-test land into... where??? The land of some test I've never heard of, I guess.

[frobzwiththingz.livejournal.com]

But nooo, they had to have 3 subjects do the experiment just once each (thus yielding 2 numbers each) while another one did it 4 times

Question 1: Are there are really only 5 subjects in the trial to begin with?

Let's label the subject that did 4 trials S4 and the one that did 2 trials S2. So basically S2 gave you TMI and S4 gave you way too much I.

Question 2: Suppose you didn't know that was the case. The lab techs secretary
decided to randomly select only one trial from each subject and hand
those results to you. Would you consider the t-test appropriate to
use here? If so, run your test for each possible input set the lab tech
secretary could have handed you. If you get a decent p value in all or
most cases, consider the matter worth looking into and dweeb at whoever
should be dweebed at to design a better followup and run it. Otherwise
reject your hypothesis and move on.

[quietann.livejournal.com]

argh, I didn't notice the only 5 people thing, which kind of rules out a t-test anyway.

If you had more people, and the 4 runs were "ordered" in some way, and knew, when a person had less than 4 runs which runs they did or did not have, *and* that the missing runs were "missing completely at random" or possibly "missing at random", and the data met a bunch of other assumptions, you might be able to move into the territory of "mixed effects modeling."

But barring that, f's approach would be good for poking around and exploring...

However.... I'm thinking from a hypothesis-testing point of view (because that's the stats world I live in the most) and I don't really see a hypothesis anywhere.... so I'd wonder just what hypothesis is being tested anyway.

(Oh, and if you had just one person doing the exp't over and over, you'd need to worry about carryover when setting up the pairs.... e.g. is the result of the 2nd pair influenced by the first, the third pair influenced by the first and second, etc.)