Reason 1 is that despite having a 7-month-old goddaughter who has me wrapped around her pinky finger (an often drool-covered finger... which is inexplicably not at all gross to me only because it's hers, wtf self), I am rather frightened of small humans, especially if they are sporting miniature fangs, wearing tiny capes, and/or carrying kid-sized weapons.
Reason 2 is that the 14+ teenagers who don't bother with costumes* and show up in groups of 5 or 6 for free candy piss me the fuck off. Get a part-time job and buy your own goddamned candy, you lazy louts. And get off my lawn. You bastards.
*If they're in costume, it's all good -- it's not their age that annoys me, it's the fact that they don't even bother pretending they're celebrating anything. :|