Antonyms for the phrase "throws in one lot with" could include words like reject, detach, disassociate, divorce, disunite, distance, sever, isolate, dissociate, and disconnect. All of these words imply a sense of separation or division, rather than the strong sense of unity suggested by "throws in one lot with." By using these antonyms, you could describe someone who is walking away from a group or cause, choosing to distance themselves from the larger whole. Alternatively, you might use these words to describe someone who was never a part of the group to begin with, or who actively opposes their values and beliefs.