Originally, the Royal Burgh of Ayr held the Rozelle lands as part of the Barony of Alloway. In 1754, to reduce the Burgh debt, the lands were sold for £2000 to Robert Hamilton who was from a wealthy merchant family which traded in sugar and tobacco from the West Indies. The first mansion house was finished by 1760 and was in the classical style of Robert Adam. During the early 19th Century the house was remodelled by David Bryce to remove the previous classical structure.

By the 1960s the Hamilton family had fallen on hard times which led to the sale of much of the surrounding land. In 1968 the House, with little of its historic interior remaining, was gifted to the Royal Burgh of Ayr and became the Local Authority Museum and Art Gallery. Between 1975-76 the servant's quarters and stable block were converted into the Maclaurin Art Gallery with funds from the bequest of Mrs Mary Ellen Maclaurin.

Rozelle House is the permanent home to Alexander Goudie's brilliant Tam o' Shanter cycle of paintings, a selection of which are on display all year. Guided tours of the paintings are available by prior appointment.

The House is also home to the Ayrshire Yeomanry Museum.

There is also a small permanent exhibition from South Ayrshire Council’s art collection.