Multiparameter models.

Classical approach: maximize a joint likelihood, proceed in steps. Bayesian approach: base inference on marginal posterior distributions of interest, eg `p(theta_1 | y) = int p(theta_1, theta_2) dtheta_2 = int p(theta_1 | theta_2, y) p(theta_2 |y)`

Integral not usually calculated directly: take a 2 step simulation process

Normal model

Sampling distribution: `y_i ~ N(mu, sigma^2)`

Non-informative prior `p(mu, sigma^2) prop sigma^(-2)`

• `p(mu, sigma^2 | y) prop sigma^(n-2) exp(-1/(2sigma^2) sum(y_i - mu)^2)`.
• `p(bar y | y) = int int p(bar y | sigma^2, y) p(mu, sigma^2 | y) p(sigma^2 | y) prop t_(n-1)(bar y, (1+ 1/n)^1/2 s)`

Conjugate prior: `mu | sigma^2 ~ N(bar y, sigma^2/kappa_0)`, `sigma^2 ~ Inv chi^2(eta_0, sigma^2_0)`:

• `mu` and `sigma^2` not independent, a priori
• `p(mu, sigma^2 | y) prop N-Inv-chi^2(mu_n, sigma^2_n/kappa_n; nu_n,, sigma^2_n)`
  • `mu_n = k+0/(kappa_0 + n) mu_0 + n / (kappa_0 + n) bary`
  • `kappa_n = kappa_0 + n`
  • `nu_n = nu_0 + n`
  • `nu_n sigma^2_n = nu_0 sigma^2_0 + (n-1)s^2 + (kappa_0 n)/(kappa_0 +n)(bar y -mu_0)^2`

Both have same posteriors:

• `p(mu | sigma^2, y) = N(bar y, sigma^2/n)`
• `p(sigma^2 | y) prop (sigma^2)^(-(n+1)/2) exp(-(n-1)s^2/(2sigma^2)) prop InvChi^2(n-1, s^2)`
• `p(mu | y ) prop [1 + (n(mu-bar y)^2)/((n-1)s^2)]^(-n/2) prop t_(n-1)(bar y, s^2/n)`

Semi-conjugate prior: `mu ~ N(mu_0, tau^2_0)` `sigma^2 ~ inv chi^2(nu_0, sigma^2_0)`. Doesn't end up nice.

Multinomial model

Generalisation of binomial model.

Sampling distribution: `p(y | theta) prop Pi^k_(j=1) theta_j^(y_j)` Conjugate prior: Dirichlet `p(theta | alpha) prop pi^k_(j=1) theta_j^(alpha_j-1)` Posterior prior: Dirichlet: `alpha_j' = alpha_j + y_j`

Can also be represented as the product of `k` independent Poisson rvs `y_j ~ Poi(lambda_j)` with restriction `sum_j y_j = n`, then `theta_j = lambda_j / sum(lambda_i)`

Multivariate normal

Useful distributions and equivalences

Dirichlet-Gamma: Draw `x_1, ..., x_n` from `"Gamma"(delta, alpha_j)` for any common `delta`. Set `theta_j = x_j / sum(x_i)` `Inv-chi^2`-`chi^2`: `sigma^2 / chi^2(nu)`.