The first doesn't work- you can 'get out of a pub', but 'get rid of' doesn't have that meaning to me, so I'd change that. I'd also use 'start afresh', but whether it works or not it depends on what you mean by this. If you're just unhappy with the place and want a better pub, then 'let's get out of this pub and go to somewhere new' or similar would work better. If you're doing something like having a meeting or important conversation, then you could 'start afresh'.
The second is a bit unclear. It might be easier to understand if you said '...he thinks it's my shirt that's weird and not what I just said'.