It would depend on the context, and what Mandy said prior to the excerpt. You can only "take her literally" if she's said something, but you don't give us that information. She means "I think you are taking what I said before too literally. I wasn't really asking you out, or, I didn't really mean tomorrow."
Interested in Language