We love Melrose Heights, but any of the neighborhoods mentioned here would be good bets. If you do opt for Melrose Heights, you'll want to look at streets from Woodrow east. The little triangle made up of King, Millwood, and Gervais is pretty questionable...
But anything along Woodrow, Fairview, Maple, Gladden, etc is absolutely lovely.
As for groceries, there is a weekly farmer's market at 701 Whaley in the Olympia neighborhood, and one in Forest Acres (technically a suburb, but it's basically nested in the city of Columbia), though I've never been to that one. Also, Publix is perfectly nice for regular mass-market groceries (probably the nicest mainstream grocery store around); Rosewood Market (right across from the Publix on Rosewood) has nice earthy, organic stuff (that's where we get our local-roasted coffee); and there's a local-ish (Asheville, NC-based) chain called Earthfare that's similar to a Whole Foods right in the heart of Shandon (as well as another similar chain, Fresh Market, in Forest Acres).
It's hillier here than you'd expect, so be prepared to cycle like you mean it.