Why can we consider expectation in Gibbs Sampling

less than 1 minute read

Suppose we are doing Gaussian Mixture (1D). The histogram of posterior distribution is (we choose a new from this histogram),

Each integral shows posterior predictive distribution of and , respectively. We can consider the expectation of for the first term instead of calculating everything. can take various values, but it follows Normal distribution. Enough amount of data makes the posterior distribution of sharp. The expectation can be a good approximation.

Code can be found here.