Outskirts is correct. The term comes from a time when many women wore many outskirts and not just one. So, we see the outskirts as one thing, one entity. That is the reason why the factory (singular) can be on the outskirts (perceived as singular). Then again, why should it be any different from: "The fly is on the bananas"?
One fly, somewhere on the bananas.