# Posts by Year

## Yet another expression of Bernoulli distribution

The probability mass function of Bernoulli distribution is

## Debug, Debug, Debug

We would like to minimize bugs when we write code. If we encounter errors, we know something is going wrong, but some serious bugs don’t stop the code.

## Faster Log Gamma Calculation

We sometimes encounter the calculation of

## Bayesian Multiple Regression

We can replace normal multiple regression with Bayesian Multiple Regression.

## Perplexity

Perplexity is commonly used to evaluate language models.

## Log Likelihood of LDA in CGS

Log likelihood of Latent Dirichlet in Collapsed Gibbs Sampling.

## Another view of Sigmoid function

Sigmoid function (standard logistic function) is defined as

## Softmax without Overflow

Overflow problems are common in neural network-like structures.

## Use tanh instead of exp in sigmoid function

If the data is large, we encounter overflow in sigmoid function.

## Why can we consider expectation in Gibbs Sampling

Suppose we are doing Gaussian Mixture (1D). The histogram of posterior distribution is (we choose a new $$z_i$$ from this histogram),

## Metropolis-Hastings Sampling Tips

You may wonder why $$\log f(x) - {\rm exprand}(1)$$ is the same as $$u \sim {\rm Uniform}(0, f(x))$$.

## Variational Bayes Derivation of Latent Dirichlet Allocation

This is not a smoothed LDA. The model is Beli, Ng, and Jordan (2003) Figure 1. You can find another useful derivation note.