Student or Learner
"Bo, please stay and have dinner with us."
"Tom, sorry, I cannot make it because I get something important to do in the office."
"Alright, Tom, in that case I won't keep you."
Is I won't keep you the correct phrase to express Bo will not force Tom to have dinner with her.
Polite forms: 'I won't keep you' => Go as soon as you like - it's up to you; 'You'd better get moving then' => I don't think you should stay