I got the flavour you describe in some brews that got infected with wild yeasts too. The difference is, when fermenting the wild yeast gave off a smell that reminded me of tobacco, quite strongly.
Given that you keep getting it, and you are been careful to remove chlorine from your water (you're giving the campden time to work yes? Not just bunging it in and going straight to it?), I'd be eyeing up your equipment and suspecting some of it may be harbouring a source of infection. A tap, bit of hose etc...
I don't actually usually use chlorine based cleaners etc even, and those who remember my methods know I prepared my water the night before using RO water and a bit of tap treated with campden pro-rata... So in my case it was definitely infection to blame.
So I'd advise going at this 2 pronged. Rule out chlorine as the problem (easy, buy a load of cheap bottle water and brew with that. Stop using chlorine based cleaning products, switch to percarbonate or similar for cleaning, chemisan (or whatever it's called) for sanitising.). Make sure you have very very thoroughly cleaned and sanitised your equipment. If you have any taps on any of your kit, think about replacing them, they're pigs to clean. If you are using a boiler with a tap, make sure you run boiling wort through it multiple times during the boil (just put it back in to the boiler afterwards) to kill anything in it as well. If using a wort chiller, make sure you put it in 15 minutes before the end of the boil... Stuff like this.
It's a PITA when it happens, and a pig to get rid of in my experience. But wild yeasts are everywhere, so yeah, good chance it's gonna happen sooner or later (if it is this). If it's Chlorine, throw a party, you have the easy fix problem. lol