I think you are right. One can use certain adverbs such as however, but or expressions such as all in all, or in coclusion to begin a sentence especiallly when one is writing an essay or a letter. But as you already have noticed carefully or quickly, smarty are not making any sense.
If you were to paraphrase the green sentences, you should use a modal.
Example :I should carefully drive my car to my father.
Hope I have been helpful
Interested in Language