Note my corrections and comments above. Starting with "The name of the folder" suggested that we already knew what folder you were talking about. Since it was post #1 of the thread, clearly it was impossible for us to know that.
If you want to keep "My" and/or "&" (or the word "and"), you'll have to keep the comma. However, there are alternatives:
Thoughts - Opinions - Experiences
Thoughts Opinions Experiences
Thoughts 000 Opinions 000 Experiences
There are lots of other options. With regard to your original question, it's worth noting that general advice is to avoid using punctuation in file/folder names.
"
Punctuation, symbols, or special characters (periods, commas, parentheses, ampersands, asterisks, etc.) should be avoided. Some of these symbols are used in operating systems to perform certain tasks, such as to identify folder levels in Microsoft products and Mac operating systems. Periods are used to identify file formats such as .jpg and .doc."
Source:
CLICK