Officially named John Crichton-Stuart, but known to friends as Johnny Bute, he took on the surname he became known by to race fans worldwide in order to keep his background as low profile as possible.
A statement from his family said that he was "devoted husband, father, brother, uncle, son and grandfather and loved by all.
"The indomitable spirit and energy which Johnny brought to his life will be greatly missed, and the immense warmth and love with which he embraced his family.
"His heart was firmly rooted on the Island of Bute where he spent much of his time. Johnny chaired the Board of Mount Stuart Trust from its active inauguration in 2005 when Mount Stuart and its gardens opened to the public, and its rural estate was vested into the charitable trust.
"He was a moderniser and an inspirational thinker, transitioning a family home to a progressive, working visitor facility and estate."
Dumfries first made his name in FF1600 driving a Ray, having raised funds working as a painter and decorator, and as a van driver for the Williams team. When he arrived on the scene few people knew that he was actually the Earl Of Dumfries, son of the Marquess of Bute.
He quickly progressed to F3, winning the British title in 1984. After a frustrating year in the first season of F3000 in 1985 – a year when he also tested regularly for Ferrari – he was propelled into the Lotus F1 team alongside Ayrton Senna in 1986.
However, he struggled in the Brazilian's shadow, scoring points only twice, with a fifth in Hungary and sixth in Australia. He was dropped at the end of the year, and it would prove to be his only F1 season, although later he tested for Benetton.
He moved to sportscar racing in 1987, initially with Richard Lloyd's Porsche team, before winning Le Mans with TWR Jaguar the following year. He then switched to the TOM's Toyota outfit. He made his final appearance at Le Mans with the Courage team in 1991.
After his racing career wound down his life changed on the death of his father in 1993. Having become the seventh Marquess of Bute, he ran the family's Mont Stuart estate on the Isle of Bute in Scotland.