The de Grey Family and its heirs, the Lucas Family, held the manorial rights of Harrold for more than 650 years. The first de Grey to be recorded as Lord of the Manor was John de Grey who had acquired the manor before 1278 from Ralph Morin. This John de Grey was probably the second Baron Grey of Wilton (in Herefordshire).
The family was busy acquiring land in Bedfordshire eventually establishing its seat at Wrest Park in Silsoe in the centre of the county. Other branches of this powerful landowning family had respectively acquired bases in North Wales (Ruthyn), Derbyshire (Codnor), Oxfordshire (Rotherfield) and Herefordshire (Wilton).
The de Grey family held the Harrold Manor for some twelve generations from the Honour of Huntingdon.The Barons of Ruthyn and of Wilton subsequently became Earls of Kent, the first earl being Edmond de Grey (1416 – 1490) who was married to Katherine Percy, and their son, George, married Catherine Herbert, daughter of the Earl of Pembroke, whose family by this time held the overlordship of Harrold, the Honour of Huntingdon. Successive advantageous marriages enabled the de Grey family to move up the social ladder and eventually the sixth Earl of Kent held the titles of Duke of Kent, Marquess Grey, Baron Lucas and the Earl of Harrold. These last two titles were passed onto his son, Anthony Grey, who effectively became the first (and last) to use the title of Earl of Harrold. His premature death at the age of 28 and the death of his infant half brother meant that the title reverted to the Duke of Kent and died with him in 1740.
A long male line of de Greys thus came to an end and the Harrold Manor now passed through the female line, as did the titles of Marquess Grey and Baron Lucas. The Duke’s daughter, Amabel predeceased him and his grand-daughter Lady Jemima Campbell became Marchioness Grey at the age of seventeen. Soon afterwards she married Philip Grey, 2nd Earl of Hardwicke of Wimpole Hall in Cambridgeshire. The young couple set about maintaining two great houses, Wrest Park and Wimpole Hall.
The Manor of Harrold now passed through the Barons of Lucas whose surnames changed and occasionally reverted to Grey (by licence rather than direct inheritance). The manorial holders since the death of Henry de Grey, Duke of Kent in 1740 were:
- Jemima Yorke, 2nd Marchioness Grey, Countess of Hardwicke,4th Baroness Lucas (1722–1797)
- Amabel Hume-Campbell, 1st Countess de Grey, 5th Baroness Lucas (1751–1833)
- Thomas Philip de Grey, 2nd Earl de Grey, 6th Baron Lucas(1781–1859)
- Anne Florence Cowper, Countess Cowper, 7th Baroness Lucas (1806–1880)
- Francis Thomas de Grey Cowper, 7th Earl Cowper, 8th Baron Lucas and 4th Lord Dingwall (1834–1905)
- Auberon Thomas Herbert, 9th Baron Lucas and 5th Lord Dingwall (1876–1916)
- Nan Ino Cooper, 10th Baroness Lucas and 6th Lady Dingwall (1880–1958)
- Anne Rosemary Palmer, 11th Baroness Lucas and 7th Lady Dingwall (1919–1991)
The great gardens at Wrest Park were originally laid out by Henry, Duke of Kent and were altered by Philip, Earl of Hardwicke and Jemima, Marchioness Grey to the designs of Lancelot “Capability” Brown in 1758 to 1760.The house was rebuilt in 1834-39, to designs by its owner Thomas de Grey, 2nd Earl de Grey, an amateur architect and the first president of the Royal Institute of British Architects, who was inspired by buildings he had seen on trips to Paris.
It is interesting to note that, following the Act of Enclosure of 1799, large areas of the enclosure map of the village are still labelled with the name Lucas or Lady Lucas (the Lady of the Manor, Amabel Hume-Campbell, Countess de Grey, etc).
The de Grey family, as Lords of Harrold Manor, made contributions towards life in the village. The Buttermarket, an iconic structure in the village, was built by Henry de Grey, Duke of Kent in the early part of the 18th Century (between 1712 and 1720). The maintenance of this market house was undertaken at various times in the next two centuries and, as late as the 1980s correspondence was taking place between Harrold Parish Council and Lady Lucas with regard to its upkeep. In the 19th Century there were various appeals for this upkeep to be maintained and The Bedfordshire Mercury of 21st July 1888 noted: "The residents think that if Earl Cowper were to come back and look at the ancient market house he would have a load or two of gravel put in it as the water, instead of running away from it, runs in it and must do considerable damage to the foundation". The Earl Cowper being referred to was Francis Thomas de Grey Cowper who was also the 8th Baron Lucas. He was the 3rd great grandson of the Duke who first provided the Buttermarket.
Another architectural feature of the Green is the Lock-up and, in 1824, there was an appeal to the Lady of the Manor (who would have been Amabel Hume-Campbell, Countess de Grey) to allow the village to rent a small area of land (no more than 4 square yards) so that a cage could be erected on the Green.. This was agreed and the building was constructed, but the village had to pay a rent of six pence per half year. By 1892 the need to use this to incarcerate wrongdoers was no longer needed and it was used for the storage of lamps and oil, but a rent was to be paid to Earl Cowper.
These rumblings hardly constituted a peasants’ revolt, but they did, perhaps, indicate that the old manorial system was creaking towards its demise. The Property Acts of the 1920s effectively ended manorial rights and thus closed the practice first introduced nine hundred years before and which started in Harrold with the arrival of Gilbert de Blosseville after the Conquest.