The castle was built in 1597 with its original name being "Nuestra Señora de la Palma" (Our Lady of the Palm) and later reformed in 1869. When it was built, the castle was merely used as a "watch tower" but later it was reformed and became a crucial part of the defensive system of the estuary of Ferrol. Later, the castle was also used as a military prison where the most famous prisoner, Antonio Tejero, was held captive. Tejero was responsible for the attempted coup de etat in 1981. Recently, the Palm Castle has been bought by a hotel chain and is pending restoration.