User blog comment:SynDynastie/The Avengers Movie/@comment-4651179-20121120153123

That's true, but because the other Marvel character such as Fantastic Four, Spider-Man or the X-Men have their movie rights with other studios (Fox, Sony, Fox again, respectively) they couldn't appear.

Other reason might be because the Avengers are supposed to be the only heroes in Earth-199999 yet.

But you can also take to account that New York is one of the biggest cities in the world, so other heroes could be fighting the Chitauri off-screen.