There is some bouldering and several gyms in the Triangle (Raleigh, Durham, Chapel Hill). I'll let someone from there chime in with the details. There is great climbing in the western part of NC, but that is 3-7 hour drive to the west. Fortunately, most areas have good camping.
Also, while there is some sport climbing in NC, trad routes are far more prevalent. Don't assume that a bolted route is sport unless you like to really run it out (20-50 feet between bolts is not uncommon).
Peri as you've noticed Greenville, NC is not a climbing hotspot. The Triangle Rock Club has a couple of gyms that are around an hour and a half drive from Greenville. There is one in North Raleigh, another in Morrisville and a third in Fayetteville. The North Raleigh one is probably the closest to you. Quality outdoor climbing is quite a bit further west though. You're looking at a 3.5 hour drive to Moores Wall, Pilot or Sauratown unfortunately.
Oh I almost forgot about this but there is a climbing wall at East Carolina University in Greenville. I met a group of students from ECU climbing at Pilot Mountain once who mentioned it. Not sure if it's open to non-students though.