Composite biomaterials are of great interest in tissue engineering and regenerative medicine. Composites partially exhibit the properties of their starting materials but, through rational design, they can be endowed with synergistic effects, overcoming the limitations of the single components. Calcium phosphate/polyvinyl alcohol (CaP/PVA) composites are archetypical examples of composite biomaterials, where a soft elastic biologically inert gel is combined with a bioactive inorganic phase, radically improving the mechanical properties. The combination allows engineering a material which better resembles the tissue architecture, for example, osteochondral defects, where organic-inorganic interfaces and gradient properties are present. The combination of components with opposite properties such as polyvinyl alcohol (PVA) and calcium phosphates (CaP) for creating synergistic properties is a grand challenge for the material engineer, where the know-how on PVA preparation must be adapted to the necessity of incorporating a phase with the appropriate size, aspect ratio, chemical composition, homogeneity or even pre-determined composition gradients. In this review, we report how these design challenges were overcome and how different design parameters involved in CaP/PVA influence each other. The review is organized as follows. First, the general properties of organic/inorganic composites and PVA hydrogels are briefly summarized. Then, CaP/PVA composites are described according to their preparation methods, discussing how parameters such as polymer deacetylation degree, tacticity, molecular weight, freeze-thawing parameters, CaP content or CaP addition or in situ preparation modality influence the composite's properties such as friction, boundary lubrication, porosity, and osteointegration. Finally, open research questions and future and desirable developments are given.