question-mark
Stuck on an issue?

Lightrun Answers was designed to reduce the constant googling that comes with debugging 3rd party libraries. It collects links to all the places you might be looking at while hunting down a tough bug.

And, if you’re still stuck at the end, we’re happy to hop on a call to see how we can help out.

Allow to make predictions without considering group-specific effects.

See original GitHub issue

Hello all,

First off I just want to say thank you for all the hard work on this amazing package! I love it and it gets better all the time.

I have a question about predicting new data with a hierarchical model, specifically around how to handle the random/group effects. In my particular use case I have a model with two random intercept terms and fixed effects that include the interaction between three variables; two of which are continuous and the other categorical. My approach to exploring the interaction is to build a small ‘test’ dataset that fixes the continuous variables to a given value (e.g. ±2 SD) along with the categorical variable, and then pass them to the .predict method of a fitted model to recover the posterior predictive distribution.

When doing this I notice errors if I omit either random effect term, and specifically that test values of the random effects must have a value in the actual dataset. My understanding is this allows me to predict the distribution of new observed data for my fixed effects set at my test levels for whatever combination of random effect A and B I choose (for example pupil A in school 1).

Previously I’ve done most of my hierarchical models from a frequentist perspective using R’s lmer package; and making predictions there is typically (and perhaps seemingly) random effect free - I guess from some kind of newX @ fixed_beta calculation. My question is whether a) this is a valid way to think about predictions from mixed models in a Bayesian context, and b) if so would this be possible in bambi?

My approach at the moment has been to make a ‘test’ version of each combination of random effects - say for pupil A in school 1 predict the values for the varying combinations of fixed effects; have the model predict each one of these, and subsequently average over the fixed effects and chains/draws to arrive at a final distribution of each of the combinations of the fixed effects. This seems intuitive (‘integrate over nuisance parameters’) but I’m not confident this is correct, so any guidance is appreciated.

Thanks for reading and the awesome package, and happy to share code/data if this is too vague!

Issue Analytics

  • State:closed
  • Created 2 years ago
  • Comments:6 (3 by maintainers)

github_iconTop GitHub Comments

1reaction
alexjonesphdcommented, Mar 14, 2022

I tried a local change and it worked with no problems, it really was that simple.

I will try a PR for this. I don’t know how common marginalization is over certain group-specific effects, but my guess is setting them to zero is more popular and useful functionality to have right now. Thanks so much @tomicapretto!

0reactions
tomicaprettocommented, Apr 10, 2022

Closing since #470 has been merged

Read more comments on GitHub >

github_iconTop Results From Across the Web

Can predictive models make predictions for individuals or only ...
1) Predictive Models can NOT make predictions on individuals - only on groups: My professor from my "Research Methods" class introduced us to ......
Read more >
Making Predictions with Regression Analysis - Statistics By Jim
Learn how to use regression analysis to make predictions and determine whether they are both unbiased and precise.
Read more >
Fixed vs Random vs Mixed Effects Models - Examples
A random-effects model, by contrast, allows predicting something about the population from which the sample is drawn. There can be categories/ ...
Read more >
4. Regression and Prediction - Practical Statistics for Data ...
An implicit assumption when only main effects are used in a model is that the relationship between a predictor variable and the response...
Read more >
Six Rules for Effective Forecasting - Harvard Business Review
We don't, of course: Prediction is possible only in a world in which events are preordained and no amount of action in the...
Read more >

github_iconTop Related Medium Post

No results found

github_iconTop Related StackOverflow Question

No results found

github_iconTroubleshoot Live Code

Lightrun enables developers to add logs, metrics and snapshots to live code - no restarts or redeploys required.
Start Free

github_iconTop Related Reddit Thread

No results found

github_iconTop Related Hackernoon Post

No results found

github_iconTop Related Tweet

No results found

github_iconTop Related Dev.to Post

No results found

github_iconTop Related Hashnode Post

No results found