Not much climbing in SE CT. Central and NW, yes. Sandy Hook is about 1 hr from the Gunks and is pretty much due east of Danbury. We usually meet at the Sandy Hook P&R for our Gunks trips (but never on weekends).
Closest CT climbing to Sandy Hook is probably Orenaug Park in Woodbury or Whitestone Cliff in Plymouth
There's a lot more climbing in the state then the folks above will have you know. There's plenty of climbing pretty much all over CT. It varies but all areas pretty much have bouldering, trad, and sport opportunities.
I have grown up living in SW CT and trust me there's plenty. No need to get clusterfu*ked and let your wallet get raped at the Bunks.
What type of climbing are you looking to get into and at what levels?
Take a look at the MP database for CT (it's setup regionally) and let me know if you have any questions about any areas. The map feature would be helpful for u as well...