With the comma included, "which was in a blue envelope" is additional description of the letter. If you omit the comma, it tells us which letter he gave you: the one in the blue envelope, not the one in the red envelope.
If you have a question about the English language and would like to ask one of our many English teachers and language experts, please click the button below to let us know: