There are many phrases that could be used instead of "you had me worried" that convey a similar sense of concern or anxiety. Some alternatives might include "you had me on edge," "you had me anxious," or "you had me fretting." These phrases all communicate a feeling of worry or discomfort, but each has its own unique flavor of distress. For example, "you had me on edge" might imply a heightened sense of tension or urgency, while "you had me fretting" might suggest a quieter, more subtle form of anxiety. Ultimately, the choice of phrase will depend on the context and the speaker's intended tone and emphasis.