Due to ever increasing channel density in WDM systems the nonlinear optical effects in fibers become a limiting factor for the high-speed data transmission, which necessitates determination of such important characteristic as the coefficient of optical nonlinearity. To measure it, two methods are proposed: continuous wave self-phase modulation (CW-SPM) and pulse phase self-modulation (P-SPM). The research work has been carried out both in the form of calculations and experimentally, taking as wavelength the central one from the third optical transparency window, λ = 1550 nm. Based on the OptSim 5.2 simulations and experimental results, the sought-for nonlinear coefficients have been calculated. In the paper, comparative analysis and estimation of the results are performed for two different optical fibers.