In 1983, their single “Wings of a Dove” peaked at number 2 in the UK charts, followed by “The Sun and the Rain” (no. 5, November 1983). Their following album, Keep Moving, peaked at number 6 in the UK Albums Chart, and two singles from that album reached the top 20 in the UK Singles Chart. The album received some good reviews, with Rolling Stone magazine giving the album four out of five stars, applauding the band’s changing sound. This was an improvement as the last album reviewed by the magazine, Absolutely, was heavily criticised.
On 5 October 1983, the band were rehearsing and discussing a possible television series, which was being written for them by Ben Elton and Richard Curtis. Barson then informed the band that he would not be able to take part, as he was tired of the music business and wanted to spend more time with his wife. They had recently relocated to Amsterdam. Barson agreed to finish recording the album Keep Moving; he left after playing for the last time with the band at the Lyceum Ballroom on 21 December 1983. After leaving the band James Mackie took Barson’s place appearing with Madness on the US television show Saturday Night Live on 14 April 1984. After leaving the band, Barson returned to the UK for the filming of two music videos as he had played on the tracks, “Michael Caine” and “One Better Day“. He officially left the band in June 1984, following the release of “One Better Day”, however finished a live performance with the band in 1983, Paul Carrack took Barson’s place whilst the band toured America in early 1984. The six remaining members left Stiff Records and formed their own label, Zarjazz Records, which was a sub-label of Virgin Records.
In 1985, the label released the band’s sixth album, Mad Not Mad. Barson’s keyboard parts were filled by synthesisers and Steve Nieve joined the band to take his place. In later years, frontman Suggs described the album as a “polished turd”. The album reached number 16 in the UK charts, which was the band’s lowest position on the album charts to date. Despite the poor chart showing, the album was listed as number 55 in NME’s “All-Time 100 Albums”. The singles for the album fared even worse, with “Yesterday’s Men” peaking at number 18 in the UK charts. The subsequent singles, “Uncle Sam” and “Sweetest Girl“, failed to make the top 20, which was a first for Madness singles.
The band then attempted to record a new album, and 11 demo tracks were recorded. However, musical differences arose between the band members. The untitled album went unreleased, and in September 1986, the band announced that they were to split. Barson rejoined the band for a farewell single, “(Waiting For) The Ghost Train”, but did not appear in the music video. The band officially split following the release of the single, which reached a high of number 18 in the UK. In 1988, four members of the band – Suggs, Chas Smash, Lee Thompson and Chris Foreman – continued under the name The Madness. After one self-titled album and two singles that failed to make the top 40, the band split.