Gerunds and Infinitives though derived from verbs they do the role of a noun in a sentence ; they can be modified with adjectives ( a feature of a noun ) they can also be modified by adverbs ( a feature of a verb ) . the question is -Which part of speech are they?
If you are teaching the traditional 8 parts of speech
(noun, verb, pronoun, adverb, adjective, conjunction, preposition,
and interjection), then I guess that you have no choice but to list
infinitives and gerunds under the category of verbs.
The Oxford Dictionary of English Grammar says that:
(1) infinitive = base form of a verb; often proceded by to.
(2) gerund = the -ing form of the verb.