It's usually an expression of frustration.
I would recommend NOT using it unless it felt very natural to you. Some people are offended by any use of God's name in expressions like this. An alternative is "For Pete's sake" or "For heaven's sake."
For Pete's sake, Mary, would you hurry up!
For Pete's sake, how did you think I would feel about that?
Calm down! I just got here. Give me a chance to catch my breath, for heaven's sake!