Metropolis-Hastings Sampling Tips

You may wonder why \(\log f(x) - {\rm exprand}(1)\) is the same as \(u \sim {\rm Uniform}(0, f(x))\).

\(\begin{align} &\qquad \min \left( 1, \frac{p(x|\theta^{'})}{p(x|\theta)} \right) > r\\ & \Leftrightarrow \log \min \left( 1, \frac{p(x|\theta^{'})}{p(x|\theta)} \right) > \log r\\ & \Leftrightarrow \log \min \left( 1, \frac{p(x|\theta^{'})}{p(x|\theta)} \right) - \log r > 0\\ & \Leftrightarrow \log \min \left( 1, \frac{p(x|\theta^{'})}{p(x|\theta)} \right) + {\rm exprand}(1) > 0, \end{align}\)

because

\(\begin{align} r &\sim {\rm Uniform}(0,1)\\ \exp(1) &= - \log r \end{align}\)

Added on 13/May/2017:
\(u \sim {\rm Uniform}(0, f(x))\) is the same as

\(\begin{align} u &= r f(x)\\ r &\sim {\rm Uniform}(0,1) \end{align}\)

Hence,

\(\begin{align} u &= r f(x)\\ \Longleftrightarrow &\quad \\ \log u &= \log r + \log f(x)\\ &= \log ({\rm randunif}(0,1)) + \log f(x)\\ &= - {\rm exprand}(\ ) + \log f(x) \end{align}\)

You may find “Non-Uniform Random Variate Generation” useful for sampling.