The castle at Loulé formed what would have been the northwestern corner of the city walls. What remains of the original castle are the gate tower and three further towers along with the walls that connect them. All seem in remarkably good condition, and this is largely due to the extensive restoration work during the 19th century.
Loulé Castle suffered extensive damage during the 1755 great earthquake and was again damaged by a much lesser earthquake in 1969.