You have to overcome your fear- it's just a psychological problem. When you are speaking in a foreign language, you are doing the hard work, not the listener, so remember that. I am at beginner level in a foreign language at the moment and I only care that the listener understands me.