He didn't blame her -- if he'd had a sister like that... but all the same, those people in cloaks...

[diamondcutter]

He dashed back across the road, hurried up to his office, snapped at his secretary not to disturb him, seized his telephone, and had almost finished dialing his home number when he changed his mind. He put the receiver back down and stroked his mustache, thinking... no, he was being stupid. Potter wasn't such an unusual name. He was sure there were lots of people called Potter who had a son called Harry. Come to think of it, he wasn't even sure his nephew was called Harry. He'd never even seen the boy. It might have been Harvey. Or Harold. There was no point in worrying Mrs. Dursley; she always got so upset at any mention of her sister. He didn't blame her -- if he'd had a sister like that... but all the same, those people in cloaks...

Source: Harry Potter and the Sorcerer's Stone, J. K. Rowling

I’d like to know if “but all the same” in the last sentence means “but that’s no difference”.

 

[dunchee]

Cambridge Dictionary

all the same

1. despite what has just been said: 2. despite what has just been said:

dictionary.cambridge.org

despite what has just been said