We introduce a Bayesian optimization method for estimating the maximum tolerated dose in this article. A number of parametric model-based methods have been proposed to estimate the maximum tolerated dose; however, parametric model-based methods need an assumption that dose-toxicity relationships follow specific theoretical models. This assumption potentially leads to suboptimal dose selections if the dose-toxicity curve is misspecified. Our proposed method is based on a Bayesian optimization framework for finding a global optimizer of unknown functions that are expensive to evaluate while using very few function evaluations. It models dose-toxicity relationships with a nonparametric model; therefore, a more flexible estimation can be realized compared with existing parametric model-based methods. Also, most existing methods rely on point estimates of dose-toxicity curves in their dose selections. In contrast, our proposed method exploits a probabilistic model for an unknown function to determine the next dose candidate without ignoring the uncertainty of posterior while imposing some dose-escalation limitations. We investigate the operating characteristics of our proposed method by comparing them with those of the Bayesian-based continual reassessment method and two different nonparametric methods. Simulation results suggest that our proposed method works successfully in terms of selections of the maximum tolerated dose correctly and safe dose allocations.