Multivariate probit model

(Redirected from Multivariate probit)

In statistics and econometrics, the multivariate probit model is a generalization of the probit model used to estimate several correlated binary outcomes jointly. For example, if it is believed that the decisions of sending at least one child to public school and that of voting in favor of a school budget are correlated (both decisions are binary), then the multivariate probit model would be appropriate for jointly predicting these two choices on an individual-specific basis. J.R. Ashford and R.R. Sowden initially proposed an approach for multivariate probit analysis.[1] Siddhartha Chib and Edward Greenberg extended this idea and also proposed simulation-based inference methods for the multivariate probit model which simplified and generalized parameter estimation.[2]

Example: bivariate probit

edit

In the ordinary probit model, there is only one binary dependent variable   and so only one latent variable   is used. In contrast, in the bivariate probit model there are two binary dependent variables   and  , so there are two latent variables:   and  . It is assumed that each observed variable takes on the value 1 if and only if its underlying continuous latent variable takes on a positive value:

 
 

with

 

and

 

Fitting the bivariate probit model involves estimating the values of   and  . To do so, the likelihood of the model has to be maximized. This likelihood is

 

Substituting the latent variables   and   in the probability functions and taking logs gives

 

After some rewriting, the log-likelihood function becomes:

 

Note that   is the cumulative distribution function of the bivariate normal distribution.   and   in the log-likelihood function are observed variables being equal to one or zero.

Multivariate Probit

edit

For the general case,   where we can take   as choices and   as individuals or observations, the probability of observing choice   is

 

Where   and,

 

The log-likelihood function in this case would be  

Except for   typically there is no closed form solution to the integrals in the log-likelihood equation. Instead simulation methods can be used to simulated the choice probabilities. Methods using importance sampling include the GHK algorithm,[3] AR (accept-reject), Stern's method. There are also MCMC approaches to this problem including CRB (Chib's method with Rao–Blackwellization), CRT (Chib, Ritter, Tanner), ARK (accept-reject kernel), and ASK (adaptive sampling kernel).[4] A variational approach scaling to large datasets is proposed in Probit-LMM.[5]

References

edit
  1. ^ Ashford, J.R.; Sowden, R.R. (September 1970). "Multivariate Probit Analysis". Biometrics. 26 (3): 535–546. doi:10.2307/2529107. JSTOR 2529107. PMID 5480663.
  2. ^ Chib, Siddhartha; Greenberg, Edward (June 1998). "Analysis of multivariate probit models". Biometrika. 85 (2): 347–361. CiteSeerX 10.1.1.198.8541. doi:10.1093/biomet/85.2.347 – via Oxford Academic.
  3. ^ Hajivassiliou, Vassilis (1994). "Chapter 40 Classical estimation methods for LDV models using simulation". Handbook of Econometrics. 4: 2383–2441. doi:10.1016/S1573-4412(05)80009-1. ISBN 9780444887665. S2CID 13232902.
  4. ^ Jeliazkov, Ivan (2010). "MCMC perspectives on simulated likelihood estimation". Advances in Econometrics. 26: 3–39. doi:10.1108/S0731-9053(2010)0000026005. ISBN 978-0-85724-149-8.
  5. ^ Mandt, Stephan; Wenzel, Florian; Nakajima, Shinichi; John, Cunningham; Lippert, Christoph; Kloft, Marius (2017). "Sparse probit linear mixed model" (PDF). Machine Learning. 106 (9–10): 1–22. arXiv:1507.04777. doi:10.1007/s10994-017-5652-6. S2CID 11588006.

Further reading

edit
  • Greene, William H. (2012). "Bivariate and Multivariate Probit Models". Econometric Analysis (Seventh ed.). Prentice-Hall. pp. 778–799. ISBN 978-0-13-139538-1.