In this report we have investigated the performance of mixed 2.5 Gbit/s and 10 Gbit/s optical systems with simultaneous propagation of various modulation formats. For mixed WDM systems with 25 GHz channel spacing, nonlinear crosstalk originated from cross phase modulation and four-wave mixing are the major source of system performance degradation. Dispersion limited distances for conventional NRZ external modulated systems at 10 Gbit/s is about 80 km, for the mixed 10 Gbit/s NRZ-RZ-NRZ system these distances is two times shorter. Traditional 10 Gbit/s NRZ and 10 Gbit/s mixed NRZ-Duo-NRZ WDM systems have similar transmission properties and equally efficient for 80 km, and efficient channel interval should be more than 50 GHz. Compared together all investigated modulation formats, it can be seen that all of them can be used for future mixed traffic transmission in WDM networks.