When we say "I don't envy you/him etc.," we mean that we don't want to be in the same situation or experience the same thing as them. There are various other synonyms of this phrase that can be used in different situations. One of the most common alternative phrases is "I'm glad it's not me." This phrase conveys that you are happy that you are not in their shoes. Another commonly used phrase is "I wouldn't want to be you/him etc. right now." This phrase is a bit more direct in stating that you would not want to be in their situation. Regardless of the phrase used, it shows empathy towards the person's situation.
