For myself, it would be the drone of continual traffic, and hustle/bustle of so many people. Realize though, that you adapt quickly to noise and tend not to notice it as much once you get used to it.
The west end has really changed over the years, and for the better in my opinion. They've cut off access to many streets in the residential areas, to cut down on traffic. In place are circles and plots of gardens and such, making little parks and quaint walking/sitting areas here and there.
The residential section, although being very populated, mostly high-rise apt buildings, is actually surprisingly quiet. Thirty years ago, it wasn't uncommon to hear gun-shots in the night. Today it seems very tranquille, with lovely grounds and gardens everywhere you look.
There's a hospital situated more near the downtown part of the West End, so you hear sirens quite often, and English Bay is always abound with people. English Bay is a gorgeous bay with beaches for miles, entrance to Stanley Park, etc.
My mother and I were walking around a few of the streets in the residential area not far behind the hospital recently, admiring all the gardens when we stumbled across a really cool little comunity called Mole Hill (awful name).
It's a community of heritage houses fixed up and converted into apts, suites, etc. The gardens are everywhere and so lush! There are all sorts of neat little areas that have been designated into gardens space all over the west end now.
We ended up in the back alley of all things, but it was a huge open area with unbelievable gardens everywhere .. like walking through a park.
Turns out this whole great section is where the Farmer's Market sets up on the wknd. That would be a nice place to live! .. right in the heart of the city amongst gorgeous gardens and produce galore.
Here's the site if you wish to check it out
Mole Hill Community Housing Society
Other than it being too crowded for my liking, the West End would be a fun place to live if you're into city life. Everything is at your fingertips .. easy access to pretty much anywhere .. airport, north shore mountains, great beaches, half of them being on the west end, English Bay, Stanley Park, very convenient transit system, excellent choices for shopping, restaurants, etc.
Parking is a REAL issue, especially in the residential area of the west end. But as a renter you're usually provided a parking spot, so it's only an issue for your visitors.
Other areas that have been fixed up nicely, but very expensive .. Yaletown on the opposite end of the downtown area, and also along Georgia St. and the harbour at Stanley Park entrance .. very beautiful, great sea wall and walking areas, marinas, gorgeous looking apt buildings, but expensive!