I don't live in the CP area, but have biked to and visited the park many times. IMO it seems like a nice place to live. Actually the homes in the area immediately surrounding the park are very nice (and expensive

) - especially north of Mulberry. The area of Mountain Ave just north of the park is very well kept and charming with turn of the century housing galore. I think the only area the may be a bit "rowdy" would be south of Mulberry (from the park) as that neighborhood does tend to have a large amount of rentals occupied by college students since the housing is more affordable and it is pretty close to the campus. But that is only by reputation - as far as crime goes I think the worst you will see may be a few college parties. No real crime, drugs that I am aware of. A pretty nice area IMO.
However, since I don't actually live there I'll let someone who does offer their opinion
EDIT - wanted to comment on this: "loitering, drugs, etc., or with noise" - BTW this kind of stuff doesn't really exist at City Park from what I have seen (my 2 dozen or so visits). Pretty family oriented and quite nice.