If you're going all-grain, I'd suggest reading How To Brew by John Palmer, he covers
water here.
Brewing All-Grain is like driving a car, you can get by with basic knowledge and the process, but you can also pop the hood and tinker with the engine. Messing around with the water chemistry is tinkering with the engine.
To start off with I'd recommend treating your tap water with campden tablets and maybe a 5.2 pH stabiliser for the mash.
Full disclosure - I haven't moved on to all-grain yet, but this is what I've read so far, so if anyone can add to this feel free!
If you wanted to delve in, get a copy of the water report from your area. I'm in London and got mine from Thames Water, but they don't test for certain things that we look for as a brewer (I called them up to confirm this and they actually asked if it was for beer brewing). It may be different where you live.
I'm lucky enough to live near some amazing craft breweries in SE London and an email to one of them resulted in them very kindly sending me a copy of a water report they commissioned recently.
Once you have this info it shouldn't be too difficult to add chemicals to achieve the water profile you're aiming for.