"The books I borrowed from you are on the table" or "The books I need to return to you are on the table" are fine. Note that the verb "owe" sounds unnatural in this context.
There is no way to tell how many books are on the table in total. All we know from your statement is that, somewhere on the table, the person can find the books that you want to give back to him.
Student or Learner