I LITERALLY just did this myself - 100,000 whole life policy, cash value of 17k, yearly premiums almost covered. I paid off some high interest debt and put the rest in my savings which is paying 3.5% right now (a credit union). I wanted to have a larger EF just in case...
Some things to consider - there are tax consequences. I have to pay taxes on about $6300. I waited until January 1 of this year so that I could push that off to next year.
Also, be sure you have other life insurance totally in place before doing so. I ended up going with a 30-year term of $250,000 for $15/month through ING.