It depends on the rest of the sentence. If you are talking about the office as a place of work, then I'd say "at the office."
For example:
Q. Where's John?
A1. He's at work.
A2. He's at the office.
Q1. What time do you have to be at work?
Q2. What time do you have to be at the office?
A. I have to be at work/at the office by 9am.
If the office is one of the rooms in my house, for example, and someone asks me, "Where's the copy machine?" I'd reply, "It's in the office."
Debra
Teaching ESL to Adults