The viewer does NOT modify the colour by itself (the tag/dot colours are initialized at LLVOAavatar object creation, and can only be changed by the corresponding Lua functions).
However, avatars beyond the draw distance won't be coloured with your custom colour, because the avatar got derendered (i.e. the LLVOAvatar object got destroyed, and with it the associated colour which is contained in a member variable); the (mini-)map info is separate from the objects list, and works with UUIDs, thus why your friends (which are known by their Ids by the viewer) can be coloured and not a random avatar.
The workaround is to increase your draw distance (anyway, what's the point of spotting your enemies on the mini-map if you can't see them to target them ?...). As long as the avatar stays within the draw distance, you do not need to face it (have it in your camera FOV) to see it coloured on the mini-map.
Firestorm deals differently with avatar colouring, by associating each "tagged" avatar Id with a colour in a std::map. This (searching for the UUID in the std::map at each frame) is however
much slower than using a pointer to a member variable in LLVOAvatar to get the colour... So I don't feel like using the same method in my viewer...
Finally, note that there are better ways to track avatars than the mini-map: the radar offers a marking feature, alerts, and allows to track (with the tracker beacon) any single avatar (including beyond draw distance, since it also uses the maps data).