Note my corrections above. Given that the question uses the word "earlier", I wouldn't use "just" in the reply (if you were using it to mean "a short time ago" rather than "only"). I would say "I was at the gym" - that's the most natural response to that simple question.