The building is part of the former Saint Martin Royal Palace, built by Henry IV as one of his most frequent city residences since 1455. In 1518 it was turned into an old people´s hospital, whose chapel is still preserved. Later it housed the School for Noble Arts. In 1998 it was refurbished to accommodate the Contemporary Art Museum. The permanent exhibition shows the artworks by the Segovia artist Esteban Vicente, member of the New York School of American Abstract Expressionism.