Budgeted optimization algorithms based on Gaussian processes have attracted a lot of attention as a way to deal with computationally expensive cost functions. Recently, parallel versions of these algorithms have further improved their ability to address the computation cost bottleneck. This article focuses on those algorithms that maximize the multi-point expected improvement criterion. Synchronous and asynchronous parallel versions of such algorithms are compared. It is shown that asynchronous algorithms are slower than the synchronous ones iteration-wise. In terms of wall clock time however and contrarily to synchronous algorithms, asynchronous implementations benefit from a near-linear speed-up up to the order of 100 computing nodes. This speed-up is bounded by the maximization of the expected improvement time which, by becoming a critical time limitation, may change the way researchers look at budgeted optimization methods.