Derek, Vic, Julian. When they control someone it's creepy and pervy.
Adrian & crew? They're just doing what's natural: spreading love and joy in the SJI Universe.
Bear in mind that in the SJI-verse using mind control on someone isn't considered a huge deal in and of itself. It seems to be viewed as generally undesirable, but if it happens to you you just shrug it off and get on with your life. If memory serves me correctly Daphne has analogized this to the cavalier attitude towards mass property destruction seen in Marvel comic books.
Given that context, I think the difference between the bad guys and the good guys is that the bad guys treat the women they control as objects, with no care to their feelings, goals or interests. The good guys seem to treat those they control as people and express some desire for them to be happy, to enjoy what's happening to them, and to continue to have lives outside of the mind controlled sex episodes. There's a fundamental de-humanization in the attitude the bad guys express towards their thralls that is mostly or entirely absent in the attitudes of the good guys. (And in cases where nominally good characters veer towards crossing that line, as may be the case with Pearl and Heidi, they start to feel more like villains.)
