Addressing the significant interest in 5G enabling technologies, this paper aims to evaluate the performance-affecting factors in the converged passive optical network that is utilized for the transmission of both, residential access and radio-over-fiber data channels. Considering that architectures and data-rates for 5G transmission over fiber are still under research, we provide general results, that can be applicable for different solutions. Respectively, we utilize the advantages of mathematical modeling for the independent evaluation of the different performance affecting factors (bandwidth of the electro-optical elements, the dispersion of the optical fiber, received optical signal power, and noise) in the context of different data rates and modulation levels. Thus, the paper defines the most important physical layer transmission impairments to deal with to ensure effective radio over fiber transmission utilizing existing fiber optical network infrastructure.