These questions are answered under the extended rules. https://www.seriousgmod.com/threads/ttt-extended-rules.42318/ You cannot kill / KoS a detective, even if you think they are RDMing other players. However, if they to attack you, you can of course attack them back. For a traitor - yes, you can KoS/attack them if they are attacking you. This is considered self defense. Self defense only applies to direct attacks.
So like you are allowed to reasonably defend yourself from t buddies. If they are attempting to kill (shooting you, crowbarring you) you can kill them. You can only kill a detective if he's actively shooting you (damaging you) or if the detective koses you and t baits you (shoots towards you). It's all in the extended rules. If you didn't see it, I recommend taking 15 to 20 minutes to go over them.
KoSing detective is never RDM as the other side knows they are killing a detective(a proven innocent) so your kos really has no value. If an innocent kills detective with the reasoning "they were kosed" its still RDM on their behalf. For Traitor part: "Like RDMs, KOSes can only be called if there is a valid reason to call them." Self defense is a valid KoS reason so going by the rules if you are in the moment of getting attacked by the T buddy you can KoS them and not get slain if someone else attacks them. If your T buddy stops attacking you though you cannot KoS them later on just for revenge. If you call the KoS during the fight and the T buddy dies way later to the KoS then afaik it still counts as a valid KoS as for KoSes it really only matters if at the time you had a valid reason to call it or not for example you dont get slain for a valid KoS that lead to a kill even if the player you KoSed later on gets proven by a tester and you witness it (you dont have to call the KoS off) Might need a confirmation on the last part by a lead admin though.
Jassa got in on the nose gonna change this to answered. Althought I will tag @wink @Graze and @HelixSpiral for the confirmation on that last part.