"at" refers to a location, whereas "in" refers to the inside. The former, "at", could refer to either the outside or the inside of a building. Both of these are correct:
I'm at the hospital. <The speaker could be inside or outside the building>
I'm in the hospital.
Speakers might use "at" instead of "in" when location itself is more important than being inside that location and doing something; i.e., waiting for the doctor. For example, a speaker, not all, might use "at" and "in" this way,
I'm at the hospital, waiting for the doctor. <the location is important>
I'm in the hospital, waiting for the doctor. <the action is important>
All the best.