Well, it's not my dual 3.2 Pentium several-gigs-of-everything-custom-made-bigboy-box-on-36MDSL either
Often lag on a server is from time accessing a database. I can't see your stats, but this site often has several hundred users browsing it at once so I imagine that is the cause. I don't know what can be done about it, though.
EDIT>> Oops, you did post your stats. Ah well, I have no idea then.