At the 2000 census, there were 4,443 people, 1,705 households and 1,220 families residing in the town. The population density was 1,014.7 per square mile (391.7/km²). There were 1,740 housing units at an average density of 397.4 per square mile (153.4 /km²). The racial makeup of the town was 93.45% White, 3.74% African American, 0.27% Native American, 0.92% Asian, 0.77% from other races, and 0.86% from two or more races. Hispanic or Latino of any race were 1.44% of the population.

There were 1,705 households of which 26.7% had children under the age of 18 living with them, 55.5% were married couples living together, 12.0% had a female householder with no husband present, and 28.4% were non-families. 23.5% of all households were made up of individuals and 12.3% had someone living alone who was 65 years of age or older. The average household size was 2.61 and the average family size was 3.13.

Age distribution was 22.5% under the age of 18, 7.0% from 18 to 24, 28.8% from 25 to 44, 24.2% from 45 to 64, and 17.6% who were 65 years of age or older. The median age was 40 years. For every 100 females there were 91.8 males. For every 100 females age 18 and over, there were 88.0 males.

The median household income was $50,305, and the median family income was $60,625. Males had a median income of $41,582 versus $32,837 for females. The per capita income for the town was $24,410. About 4.3% of families and 6.6% of the population were below the poverty line, including 12.5% of those under age 18 and 4.2% of those age 65 or over.