The preference is to use an adverb here (adverbs modify verbs, so you use an adverb to describe the verb "speak").
"Well" and "ill" are adverbs. Actually, "Ill" can be an adjective or an adverb (Thankfully, no one makes us say "illly"!), but both are standard English use, as an adjective or an adverb.
"Bad" is an adjective. Only informally (read: not quite right) is it used as an adverb, preferring instead the "badly" construction.
So, you may speak badly, ill, or well or someone, but not bad or good. You may use many other adverbs. You may speak politely (but not polite), effusively (but not effusive), extensively (but not extensive), etc., about someone. Just make sure it's an adverb you speak!
Learn this, and you will be speaking properly (not proper).