Both of them are correct. However, "all my money" seems more natural to me.
Before a noun with a determiner (the, my, ... ) all and all of are both possible.
Before a noun with no determiner, we do not normally use "of" .
- For Teachers