The evolution of bio-based composites in the building industry is strongly linked with the growing demand for sustainable development, which is relevant nowadays. Hemp shives are a large group of organic residues that are obtained in the process of oil extraction as well as straw processing. These residues could be utilized along with a binder as constituents in the manufacture of bio-based building composites. This study is focused on the impact of density and relative humidity on the effective thermal conductivity of hemp shive-based bio-composites with a magnesium binder. For this reason, a series of samples with variable densities was manufactured and subjected to conditioning in a climatic chamber at a constant temperature and different relative humidity settings. As soon as samples were stabilized, the guarded hot plate method was applied to determine their thermal conductivities. Before each measurement, great care was taken during sample preparation to ensure minimum moisture loss during long-lasting measurements. The results showed that an increase in sample density from 200 kg/m3 to 600 kg/m3 corresponded to up to a three-fold higher composite thermal conductivity. In the case of sample conditioning, a change in relative humidity from a very low value to 90% also resulted in almost 60% average higher thermal conductivity.