When creating a "yeast starter" you are growing yeast cells, not fermenting sugars. So, the sugar is immaterial (within limits), and any significant growing is done before obvious signs of fermenting. The yeast does recognise there is sugar (reacting to mystical biological stimuli well outside my knowledge) and grows accordingly. BUT! The nutrients available to the yeast which it will need to grow are not immaterial. And as you already know, plain white sugar is no good, because it contains no nutrient. As for the others you list .... they all contain impurities from which the yeast might find suitable nutrients. But who knows? You can risk it but don't risk harvesting the yeast for future brews.
I've heard of people going to the trouble of creating mini mashes for starter liquor (boiled for a few minutes, but no need to add hops). The information I have suggests you keep starter gravities between 1.030 and 1.040 (ideally).