My view differs from Gwens only slightly, and be aware that I am speaking as a native English speaker, not as a teacher.
I see "During our conversation..." as implying that at some point in time between the start and the end of our conversation (or now), an event happened.
I don't see much difference in the two forms. That, however, says nothing about whether they're both correct grammatically.
Would you say there is not much difference between:
1-"During this converstaion, we took a wrong turn."
2-"During this conversation, we have taken a wrong turn." ?