From what I was able to find out, semi-modals are distinguished from modals on the basis of the principle of formal application. This division into modals and semi-modals has nothing to do with semantics.
To be classified as a semi-modal a modal unit has to meet one of the following requirements:
a) to be followed by a full infinitive (e.g. ought to do)
2) to have finite forms (e.g. needed to do, will need to do)
3) contain more than one word (e.g. had better, would rather)
As you can see this division into modals and semi-modals seems to have English teaching value rather than linguistic value. It's meant to help students learn grammar rules and avoid making mistakes..