On the interpretation of the posterior for effect sizes

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

On the interpretation of the posterior for effect sizes

Alexander
Hello Dr. Kruschke,

I am running an ANOVA model in WinBUGS and want to calculate effect sizes between two experimental groups. So, I have included a variable in the model that estimates the difference between the group means while running the sampler. I basically understand the posterior as expressing uncertainty for the estimation of effect size, given the data and the (uninformative) prior.

Is it then correct to say that the area under the posterior > 0 (or a ROPE value near 0) represents the probability of an effect? If yes, wouldn't it be better to give people these probability values instead of a binary acceptance or rejection of the hypothesis that there is no effect ?

Regards,

Alexander  
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: On the interpretation of the posterior for effect sizes

John K. Kruschke
Administrator

Thanks for your message.

I recommend using JAGS instead of (Win-)BUGS. JAGS is more robust and has a lot of features that make it more flexible. If BUGS is working for you, terrific, but consider switching to JAGS when you can.

"Effect size" has different specific technical definitions in different modeling contexts. I'm not sure if you are using "effect size" with a specific definition in mind. For a description of effect size in the context of two-group comparison, see the article linked here: www.indiana.edu/~kruschke/BEST/

For interpreting the posterior distribution, it depends on your goal: Are you trying to provide a summary description of the posterior, or are you trying to make a discrete decision based on the posterior? If providing a summary description, then all those numbers can be useful to the audience, but especially the central tendency (mean or mode), the limits of the 95% HDI, and, optionally, the %age in/out/above/below a landmark value. If making a discrete decision, then I recommend using the HDI and ROPE, not the %age on the side of a landmark value, because the HDI is referring to genuinely high-credibility (high probability density) values, while the %age above/below a landmark does not give a sense of the shape of the distribution.


On Sat, Jul 20, 2013 at 4:03 AM, Alexander [via Doing Bayesian Data Analysis] <[hidden email]> wrote:
Hello Dr. Kruschke,

I am running an ANOVA model in WinBUGS and want to calculate effect sizes between two experimental groups. So, I have included a variable in the model that estimates the difference between the group means while running the sampler. I basically understand the posterior as expressing uncertainty for the estimation of effect size, given the data and the (uninformative) prior.

Is it then correct to say that the area under the posterior > 0 (or a ROPE value near 0) represents the probability of an effect? If yes, wouldn't it be better to give people these probability values instead of a binary acceptance or rejection of the hypothesis that there is no effect ?

Regards,

Alexander  


To start a new topic under Doing Bayesian Data Analysis, email [hidden email]
To unsubscribe from Doing Bayesian Data Analysis, click here.
NAML

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: On the interpretation of the posterior for effect sizes

John K. Kruschke
Administrator
In reply to this post by Alexander

You might find the examples in this blog post interesting:
http://doingbayesiandataanalysis.blogspot.com/2013/07/decisions-from-posterior-distributions.html


On Sat, Jul 20, 2013 at 11:02 AM, John K. Kruschke <[hidden email]> wrote:

Thanks for your message.

I recommend using JAGS instead of (Win-)BUGS. JAGS is more robust and has a lot of features that make it more flexible. If BUGS is working for you, terrific, but consider switching to JAGS when you can.

"Effect size" has different specific technical definitions in different modeling contexts. I'm not sure if you are using "effect size" with a specific definition in mind. For a description of effect size in the context of two-group comparison, see the article linked here: www.indiana.edu/~kruschke/BEST/

For interpreting the posterior distribution, it depends on your goal: Are you trying to provide a summary description of the posterior, or are you trying to make a discrete decision based on the posterior? If providing a summary description, then all those numbers can be useful to the audience, but especially the central tendency (mean or mode), the limits of the 95% HDI, and, optionally, the %age in/out/above/below a landmark value. If making a discrete decision, then I recommend using the HDI and ROPE, not the %age on the side of a landmark value, because the HDI is referring to genuinely high-credibility (high probability density) values, while the %age above/below a landmark does not give a sense of the shape of the distribution.


On Sat, Jul 20, 2013 at 4:03 AM, Alexander [via Doing Bayesian Data Analysis] <[hidden email]> wrote:
Hello Dr. Kruschke,

I am running an ANOVA model in WinBUGS and want to calculate effect sizes between two experimental groups. So, I have included a variable in the model that estimates the difference between the group means while running the sampler. I basically understand the posterior as expressing uncertainty for the estimation of effect size, given the data and the (uninformative) prior.

Is it then correct to say that the area under the posterior > 0 (or a ROPE value near 0) represents the probability of an effect? If yes, wouldn't it be better to give people these probability values instead of a binary acceptance or rejection of the hypothesis that there is no effect ?

Regards,

Alexander  


To start a new topic under Doing Bayesian Data Analysis, email [hidden email]
To unsubscribe from Doing Bayesian Data Analysis, click here.
NAML


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: On the interpretation of the posterior for effect sizes

Alexander
Thank you very much for the explanation and the great post. Indeed, a decision based on HDI seems to be more convincing.

Btw, I also use Gelman's arm package for R which accepts lmer models. I can then compute the HDI based on the mcmc output using the code in your book. This seems to work fine as well.

Regards,

Alexander

   
Loading...