Robert Wedderburn, probably born in 1762, was one of at least 5 illegitimate children of James Wedderburn, a doctor and plantation owner in Jamaica. Robert’s grandfather, Sir John Wedderburn of Blackness, was captured, tried for treason, then hanged, drawn and quartered by the English, after the defeat at Culloden in 1746 of the followers of Charles Edward Stuart, who had tried to drive the Hanoverian, George II, off the British throne. Many of the Wedderburns had fled to Jamaica in the aftermath of Culloden; the family had already been in financial straits even before the Jacobite defeat, and needed to recoup their fortunes. James Wedderburn apparently did this very successfully, owning at his death some 8 properties in Westmoreland - Mint, Paradise, Retreat, Endeavor, Inverness, Spring Garden, Moreland, and Mount Edgcombe - worth together over £300,000.
Robert’s mother was Rosanna, his father’s slave housekeeper, whom he treated very harshly and who had two other sons by James Wedderburn. Wedderburn’s next 'housekeeper', Esther Trotter, who was a free Coloured woman, gave him two children, a boy and a girl, who were taken away from her and brought up in Scotland, under the name of Graham.
At the time of Robert's birth, Rosanna had been sold back to Lady Douglas, who had owned her previously; James Wedderburn's brother, Sir John, had protected Rosanna against his brother's violent behaviour, and Lady Douglas, who was Robert's godmother, always protected him. Part of the agreement of the sale of Rosanna to Lady Douglas was that the child she was carrying should be free from birth, so Robert was born free.
The whole story of the Wedderburn family's relationships in Jamaica at this time seems fairly complex. James Wedderburn appears to have been far more vicious in his relationships with slaves, and Blacks and Coloureds in general, than other members of the family. His later rejection of Robert Wedderburn as his son seems to indicate, however, some particular animus against Robert. Robert himself mentions that his eldest half-brother, John, son of James Wedderburn's Scottish wife, who came to Jamaica, presumably in the early or mid-1790s (and died here in 1799) 'acknowledged his father's tawny children, and, amongst them, my brothers as his brothers. He once invited them all to a dinner, and behaved very free and familiar to them.' This was long after Robert left Jamaica.There seems to be much more to the Wedderburns in 18th century Jamaica than the disgusting behaviour of James Wedderburn.
Robert Wedderburn grew up in Kingston under the care of his grandmother, known as 'Talkee/Talky Amy' who was a slave who traded goods for her master, and for other merchants on commission. She was so much in the merchants' confidence that she traded smuggled goods for them. At the age of 11 he saw his 70 year old grandmother flogged within an inch of her life, accused of witchcraft (or presumably obeah).