It is part of their identity. Few civs in history built better castles than the French.
If we go for the historical argument I always thought the castle cost discount should be for a german themed civ as Germany, or the Holy Roman Empire has/had the highest density of castles in all of europe.