Conventional simulations of complex systems, which have many degrees of freedom, are hampered by multiple-minima problem. One way to overcome the multiple-minima problem is to perform a simulation in a generalized ensemble where each state is weighted by an artificial, non-Boltzmann weight factor so that a random walk in potential energy space may be realized. Three of well-known generalized-ensemble algorithms are multicanonical, simulated-tempering, and replica exchange method. In previous works, the methods combined with simulated-tempering and replica-exchange method, the one-dimensional replica-exchange simulated-tempering and simulated-tempering replica-exchange method, were developed. For the former method, the weight factor of the one-dimensional simulated-tempering is determined by a short replica-exchange simulation and multiple histogram reweighting techniques. For the latter method, the production run is a replica-exchange simulation with a few replicas not in the canonical ensembles but in the simulated-tempering ensembles. In this article, the general formulation of the multidimensional replica-exchange simulated-tempering and simulated-tempering replica exchange method is reviewed.