"Having said that" - refers to what was said prior to this sentence. Another way of saying this would be, "Now that I (we, you) have said that"
It would seem that whatever was said in the sentence before this passage made some reference to the speaker's prowess - (see the following definition from dictionary.com - I like definition #2)
Dictionary.com Unabridged (v 1.1) - Cite This Source - Share This prow·ess Audio Help /ˈpraʊɪs/ Pronunciation Key - Show Spelled Pronunciation[prou-is] Pronunciation Key - Show IPA Pronunciation –noun 1.exceptional valor, bravery, or ability, esp. in combat or battle. 2.exceptional or superior ability, skill, or strength: his prowess as a public speaker. 3.a valiant or daring deed.
Then, it seems that the speaker wants the listener to ignore whatever reference there was to his or her prowess so that the listener can focus on the most important part of the story -- which has not yet been revealed.
Please let me know if I can clarify further.