rwm.Rmd

# <a name='rwm'>Random-Walk Metropolis</a>

(<a name='rwm'>RWM</a>)

<br>
```{r, echo=FALSE}
pander::pander(
  data.frame(
    Aspect = c('Acceptance Rate', 'Applications', 'Difficulty', 'Final Algorithm?', 'Proposal'),
    Description = c('The optimal acceptance rate is based on the multivariate normality of the marginal posterior distributions, and ranges from 44% for one parameter to 23.4% for five or more parameters. The observed acceptance rate may be suitable in the interval [15%,50%].', 'This is a widely applicable, general-purpose algorithm that is best suited to models with a small to medium number of parameters. The proposal covariance matrix must be solved, and this matrix grows with the number of parameters.','This algorithm is moderately difficult for a beginner because the proposal covariance matrix must be tuned.','Yes.', 'Blockwise or Multivariate.')),
  justify='left', style='multiline', split.cells=c(17,83), split.tables=Inf)
```

<br>

Random-Walk Metropolis (RWM) is a multivariate extension of Metropolis-within-Gibbs (MWG). Multiple parameters usually exist, and therefore correlations may occur between the parameters. Many MCMC algorithms attempt to estimate multivariate proposals, usually taking correlations into account through a covariance matrix. Each iteration, a multivariate proposal is made by taking a draw from a multivariate normal distribution, given a proposal covariance matrix.

RWM does not have any required algorithm specifications, though blockwise sampling may be specified with `B`, which accepts a list of proposal covariance matrices equal in length to the number of blocks. By default, blockwise sampling is not performed, so all parameters are updated with one multivariate proposal.

Given the number of dimensions (`K`) or parameters, the optimal scale of the proposal covariance, also called the jumping kernel, has been reported as $2.4^2/K$ based on the asymptotic limit of infinite-dimensional Gaussian target distributions that are independent and identically-distributed (Gelman, Roberts, and Gilks 1996). In applied settings, each problem is different, so the amount of correlation varies between variables, target distributions may be non-Gaussian, the target distributions may be non-IID, and the scale should be optimized. There are algorithms in statistical literature that attempt to optimize this scale, such as the Robust Adaptive Metropolis (RAM) algorithm.

There have been numerous methods introduced for tuning the proposal covariance matrix. Many adaptive MCMC algorithms such as Adaptive Metropolis (AM), Adaptive-Mixture Metropolis (AMM), and RAM will tune the proposal covariance matrix. Alternatively, initially specify an identity matrix with small-scale diagonal values such as `1e-3` as the proposal covariance matrix, update with RWM, and then update again but this time with the observed covariance of the historical samples. Done repeatedly, this may arrive at an acceptable proposal covariance matrix, suitable for longer and more serious updates.

Since RWM is not adaptive, it is suitable as a final algorithm.

## References

- Gelman A, Roberts G, Gilks W (1996). "Efficient Metropolis Jumping Rules." Bayesian Statistics, 5, 599-608.
- Haario H, Saksman E, Tamminen J (2001). "An Adaptive Metropolis Algorithm." Bernoulli, 7(2), 223-242.
- Roberts G, Rosenthal J (2009). "Examples of Adaptive MCMC." Computational Statistics and Data Analysis, 18, 349-367.
- Vihola M (2011). "Robust Adaptive Metropolis Algorithm with Coerced Acceptance Rate." In Forthcoming (ed.), Statistics and Computing, pp. 1-12. Springer, Netherlands.


## See Also

- [AM](#am)
- [AMM](#amm)
- [MWG](#mwg)
- [RAM](#ram)