If you are getting overflow, 1 of 2 things (or both) is probably happening: 1) you're getting a stuck mash, try adding some oat husks/rice hulls (4% usually, rinse well) to improve circulation through the grain bed, 2) you're trying to run with the re-circ valve fully open, try slowing it down a bit as you don't need to. Also, check you're not putting more grain in than the machine is rated for, they do have a maximum that they can cope with.
Oh, and check the sparge plate if you're using it, they're prone to clogging up. Thanks in part to the massive holes in the bottom of the grain basket on Klarsteins.... I've taken to milling my own grain, then given it time to settle after doughing in before I turn on the re-circulation, so it doesn't suck all the small stuff down before the husks have a chance to form a filter bed. When I don't do this, my sparge plate clogs constantly, and if I don't use it and run re-circ directly into the basket, the grain bed never settles.
Finally, you are removing the plug from the overflow pipe after you've doughed in? As the only time it should be able to overflow the sides is if you leave this in, otherwise it should just overflow down the overflow pipe if you get a stuck mash etc. I mean, once you get the hang of not getting a stuck mash, how fast to run recirc you don't need the overflow pipe so much, but before that, ending up with the sides overflowing, and the pump sucking air from under the basket and elements heating a dry bottom isn't so good.
Oh, and the baskets in brewers are designed the way they are for a reason, so that the liquid goes all the way through the grain. If you put an open sided basket in there, it'll take the route of least resistance and go straight out the sides, skipping most if not all of the grain. When you use a BIAB bag in a DIY setup, it presses against the sides of the boiler, so most goes down and out through the bottom as that's the route of least resistance, my last bag actually had a more open bottom than sides too. I'm not convinced that pumped re-circulation would be overly useful with a totally open basket like that, as it would just go in the top and out the sides.
I started out with an ACE boiler by the way, which was identical to a Klarstein Fullhorn kettle, but came with the bag included. I modified it with a recirc pump system. I then moved on to a DIY eBIAB rig that I built myself. I'm now learning to use a Klarstein Brauheld Pro. It's thanks to the Brauheld I know about turning the recirc down with the valve at the top, about the sparge plate been prone to clogging and the big holes in the grain basket letting particles through, especially if you use pre-crushed malt.