I'll second Grill God's motion.. Riverton is larger and more of a place that a 23 yr old might be able to find friends and places to hang out. Plus you'd be more likely to find housing in Riverton. Roughly 25 miles between the two.
That being said, that area of Wyoming is probably my favourite! I lived in Lander (23 miles from Riverton) when I was in my 20's and loved it. There are a lot of outdoor activities to do within a 50 mile radius, and the views of the mountains can't be beat (imho)
Good luck with your decision, but I would suggest that you take a trip out there first before committing to anything.
Out of curiosity, what is the job in Pavillion? and where are you living now?