Exceptional points are the most intriguing features of non-Hermitian photonics, providing clues for enhanced sensitivity, asymmetric light reflection, laser-absorber effect, topological switching, and other applications. The materials used in non-Hermitian optical structures are usually supposed to be isotropic. In this paper, we study the archetypal anisotropic non-Hermitian system consisting of the PT-symmetric bilayer (with balanced loss and gain) supplemented with the anisotropic defect (interlayer). Starting from the analysis of isotropic case, we show that anisotropy results in doubling the number of exceptional points and nontrivial splitting of the PT-symmetry-broken regions. We demonstrate that to shift the exceptional points, it is not enough to change the orientation of anisotropic medium optical axis in the transverse plane, but two anisotropic layers with differently oriented axes are needed. In the latter case, an experimentally feasible way is proposed to tune the system to an exceptional point by simply controlling the relative optical axes orientation. Moreover, we show that the exceptional points in anisotropic structures generally lose their association with unity transmission. Our results will be helpful for better understanding exceptional points and other features of non-Hermitian systems and for plenty of applications inspired by the rich physics behind the scene.