Due to an increase in computing power and innovative approaches of an end-to-end reinforcement learning (RL) that feed data from high-dimensional sensory inputs, it is now plausible to combine RL and deep learning to perform smart building energy control (SBEC) systems. Deep reinforcement learning (DRL) revolutionizes the existing Q-learning algorithm to deep Q-learning (DQL) profited by artificial neural networks. Deep neural network (DNN) is well trained to calculate the Q-function. To create a comprehensive SBEC system, it is crucial to choose an appropriate mathematical background and benchmark the best framework of a model-based predictive control to manage the building heating, ventilation, and air conditioning (HVAC) system. The main contribution of this paper is to explore a state-of-the-art DRL methodology to smart building control.