The existence of the Castle is sure since 1149 A.D. It was destroyed in the XIII Century during the war between Milan and Como. Luchino Visconti started reconstruction in 1345 A.D. calling it la Cittadella (i.e. the small fortress). The Visconti family ruled the Castle with the Aicardi family (who originally used to work for the Visconti) and built the ditch and drawbridge. In 1493 A.D. , the ambassador of Massimiliano I, Emperor of the Sacred Roman Empire, led Bianca Maria Sforza, who belonged to a very powerful Lombard family, to Germany to marry her. The bride was accompanied by Ludovico il Moro and Beatrice d'Este. Part of the group rested for a few days in the Castle of Carimate.
In 1496 the Emperor Massimiliano I himself, passing by Carimate, liked the site so much that he decided to stay for seven days, resting and hunting deer in the surrounding forests. The Castle belonged to the Visconti-Aicardi family until Ludovico Visconti died without legal heirs in 1795: thus the Castle and all its treasures became Government property and were sold by the local authorities to the Arnaboldi family of Como, who held it until the 20th century. Several unimportant Italian proprietors followed in the latest years during which the building was reduced to miserable conditions. Only a small part of the structure war kept alive and hosted a recording house during the '70 and '80.