After looking at the code (both Marine's and how I backported it: see the getMixedColors() method in mkrlinterface.cpp), the way this feature works is by "adding" the new color to the current draw colors list (*): the colors are "added" together (after a conversion in H/S/L components), then averaged (kind of). While I agree it is unintuitive and may (literally) "look" strange, this is how Marine implemented it (intended or not).
Also, there is no distinction between the objects applying the colors: all objects can issue a color change and all changes are "mixed" together...
I suggest you raise your concern by opening an issue on
Marine's RLV code repository (I suppose a @setcamdrawcolor command could be added, that would overwrite all existing colors for the object that emits it).
(*) You may however remove a color, by using the '=rem' action in the RLV command. E.g., if you add Red with 'llOwnerSay("@camdrawcolor:1;0;0=add")' you can later remove it (possibly before applying a new color) with 'llOwnerSay("@camdrawcolor:1;0;0=rem")'...