Calcium above a threshold is removed, not all and not enough that it matters. To have the two most respected names in brewing water chemistry, AJ Delange and Martin Brungard, begrudgingly agree with each other that it doesn't matter certainly passes my trust threshold.Calcium is important in the brewing process so I'd prefer not to be removing it. With low mineral water it's probably fine, but not in South East England with high levels of calcium in the water.
Additionally phosphate does have a taste, so if you overdo it you will be able to taste it.
If your tap water is super hard (like mine >380ppm alkalinity) then you really are stuck between a limestone and a hard place when brewing pale ales. The amount of CRS you'd need to add could push you into mineral/astringent taste territory. Similarly lactic acid has a taste threshold that you could quite easily pass which leaves phosphoric as the general best, most flavour neutral option.
Or buy an RO machine or use bottled Ashbeck!