Effect Size

After reading and answering a couple of comments with similar questions I decided to post an explanation that I left out in the video of how to find the amount of "spots" the student moves. I hope this helps anyone with the same question. If you are still confused or have follow ups please comment!

Thanks to those that posted the questions in the first place! tl;dr, it comes down to using ES as a z-score in a standard normal distribution which tells us how many standard deviations the student would have moved when compared with the control. This then gives us a percentile, which allows us to calculate the rank.

This might take a bit to explain, and I'll be using some of the information form the paper I cited for this information. I hope this will answer your question, if not please follow up.
Basically, an effect size is simple way to evaluate the effectiveness of an intervention by comparing two groups, one control and one experimental. "Effect size is exactly equivalent to a 'Z-score' of a standard distribution" (Coe).

When we look at our list of students ranked by performance in two classrooms, like in the video, we see that they are about the same and John is the average student in one of the classes. After giving the intervention to John's class and leaving the other class alone we can compare scores. John was the average student, he was in the 50th percentile before the experiment.

After the experiment we compare the two groups and get an effect size of 0 lets say, which means the intervention had no effect. John is still in the 50th percentile because there was no movement. If we looked at the two normal curves of the two classes they would overlap. If there was an effect of .5 that would be a movement of .5 standard deviations in a positive direction (to the right) which would mean John, originally in the 50th percentile compared to the other group, would move to the 69th percentile when compared to the control group. In a class of 25 that would put him at number 7. If it was an effect of 1 that would be 1 standard deviation to the right and would be in the 84th percentile, or in a class of 25 number 4. If the group size was larger, lets say 100, the ranks would be different as they would correlate to the percentile (84th percentile means they scored higher than 84% of the class). Page 4 of the paper cited has a great table that shows this in action.

Also, this works in the negative direction, so if the average student in the 50th percentile was actually hurt by an intervention and had worse outcomes (which does happen!) the rank would drop. An ES of -.5 would mean the student would be .5 standard deviations to the left (negative direction) and would now be in the 31st percentile, instead of scoring higher than 50% of the class he now only scores better than 31% of the class.

This is how we can assign position based on effect size, we get the effect size, which correlates to Z-score, which tells us how many standard deviations the experimental group moved, which leads us to a conclusion of percentile. From there we take our class size and figure out what it means to be in the nth percentile. I really hope this long-winded explanation helps!

this helped me understand thank you!

Thank you!

I loved you video, thanks.

love john!he makes understanding research papers a lot easier😁thanks heaps!

Thank you! Super clear and concise. I truly appreciate it. 🙂

Thank you, John, You certainly are helping many people to the top of their class!

Useful, thanks!

Does this values you are saying are fixed?

This was very helpful and straight to the point.

I just watched the video, read your explanations in your answers to a few comments, then watched the video again and it makes sense now - you're moving John into bigger control classes as a way to show bigger standard deviations (0 (intervention didn't do anything) -> .3 -> .8 -> 1.3 -> 2 -> 3 (intervention did a LOT when you hold him up to class that didn't get a super cool intervention)).

I have a couple questions - what happens to the 24 other students in John's class?

Also, can you talk about mean vs. mode here?

Is effect size only calculated in pre-post test designs?

Awesome, understood it well, thank you

internal validity could be questioned for this, given that the two classes would have different teachers moving forward and experiences.

dude!!! this was so easy! thanks!!

Thank you

Amazing tutorial. Nice graphics.

Question: is effect size stable for all students in a group? John is average scoring in pre-test. Would students that scored lower or higher have the same advantage or disadvantage of the intervention in terms of effect size?

This explanation is much much much easier! Thankssss

Thank you for providing a simple way to understanding effect size.

i am so thankful to u!
i wish u all the happiness and good luck in ur life bc u made this easy for me to understand ;)

man, i hope im john