Construction of the castle began in 1760, under Louis XV by Antoine Le Bègue, Count de Germiny and cavalry captain. Two pavilions, built in the 17th century, frame the entrance to the court; they are the only remains of an earlier construction, the castle of If, which stood on the site of the present castle. Sassy was acquired in 1850 by Chancellor Pasquier. Still today, the estate comprises more than 800 ha in one piece.
The library was converted by the 1st Duke of Audiffret-Pasquier to house the important Parisian library (30,000 volumes) of Chancellor Pasquier. Inside this library including a level, above the carved wooden fireplace is the portrait of Chancellor Pasquier as President of the House of Peers, painted by Horace Vernet in 1842. This library is considered one of the most beautiful private libraries in France.
According to some ancient documents, there has been in Sassy, for many centuries, an annex chapel of the church of St Christopher. Its original name was the chapel of St Evroult; it has changed locations three times. The most remarkable element is this Flemish altarpiece in carved oak. It is remarkable by its age (15th century) and by the number of characters referring to the Passion of Christ.