One of the most important applications of simulation is the comparison of alternatives. Possible decisions, operating procedures, system designs can be compared through simulation experiments before implementing them in a real life. Usually, the comparison is based on post-experimental analysis of the simulation output data. Stochastic simulation leads to variance of the estimated difference between systems performance measures. The goal of simulation experiments is to obtain a point and interval estimates of the difference in mean performance of the models. Different approaches and methods for simulation-based comparison of alternatives are analysed in the paper. A special attention is given to statistical analysis techniques based on confidence interval estimation. In the case study, performances of two alternative replenishment policies in supply chains are simulated and compared. The comparison is restricted to a single performance measure and performed by using a Paired-t confidence interval method.