5Why do they use "the" in the first sentence? It is first mentioning and it should be without article (if it is uncountable noun) and with a/an if it is countable.
As the Poisson Distribution appears to be an alternative name for the Poisson law of small numbers, the definite article is fine.
[...]would it be OK to write it without "the" completely, such as
"In probability theory and statistics, Poisson distribution ...is a discrete probability distribution that expresses the probability of a"?
If you mean "distribution in the way noted by Poisson", then probably yes.