Bronxville is a charming village to just walk around and shop on a nice sunny day. The path along the river in the Bronxville area is a nice place to exercise/recreate and I recommend doing so when the Bronxville farmer's market is open. Kill two birds and all that.
The best parts of the Bronx River itself are on the grounds of the Botanical Gardens and the NY Zoological Society (Bronx Zoo) both located in the Bronx proper. That is also where the 'large' falls is located.