Empirical Bayes for the Ridge Penalty in Probit Models

Empirical Bayes is a popular procedure to fix hyperparameters in Bayesian models, with estimation often performed via maximization of the marginal likelihood. In the case of Bayesian probit models with independent homoscedastic Gaussian prior distribution, one might be interested in the estimation of the prior variance of the parameters. We develop an expectation-maximization algorithm to obtain maximum marginal likelihood estimates of this quantity, where the expectation step leverages recent implementations of the expectation propagation algorithm for Bayesian probit models. Importantly, the penalty in the Ridge probit models is a 1-to-1 function of such a variance. The performance is validated over synthetic data generated with different values of the hyperparameter of interest.