He was married to Lucy Rogan and had ten children, including P. J. Carlesimo, the eldest, who was a longtime coach at Seton Hall University and in the NBA.
His 154 wins as a coach places him fourth on the all-time list at Seton Hall behind Honey Russell, P.J. Carlesimo, and Frank Hill.
Carlesimo was known as a humorous speaker, having appeared at countless sports-related dinners and conferences, and was once a guest on The Tonight Show Starring Johnny Carson.