The ZIPs on the list other than 10011 and 10004 are not "real" ZIP codes corresponding to neighborhoods. 10167, for example, is the code for a single office building on Park Av. 10105 is an office building on 6th Avenue. I imagine a very small number of wealthy people listed those buildings (likely where their employers or attorneys are located) as their "address" on their tax returns and generated an artificial "average income" for the ZIP codes in which no people actually live.
With the notable exception of 10128, which was a carveout some years ago on the UES because new high rises made 10028 too populous for the USPS to manage, the 101__ and 102__ in Manhattan are generally reserved for particular large buildings and not for actual neighborhoods.
http://www.newyorkjewelry.com/images/NYCmap2.gif
10004 is a "real" neighborhood but might be skewed by very high-income people who don't actually live there, as opposed to just actual residents.