In the early summer of 2004, when Pete Doherty once again found himself cast out of The Libertines because of his drug use, he brought Babyshambles to the fore with Walden on lead guitar. The band's line-up underwent several changes before stabilizing during the late summer of 2004 with Doherty on vocals, Patrick Walden on guitar, Gemma Clarke on drums and Drew McConnell on bass. Walden co-wrote a number of Babyshambles songs with Pete Doherty. One of them, "The Man Who Came To Stay," was released as the B-side to the Killamangiro single in November 2004. Other Doherty/Walden compositions include Top-10 single "Fuck Forever" and "Loyalty Song," "352 Days," "In Love With a Feeling," "Up the Morning," "Pipe Down," "32nd of December," and "8 Dead Boys." He co-wrote six of the sixteen tracks that made it onto Down In Albion, Babyshambles' debut album.
At Babyshambles' live shows, he usually performed using an Olympic White 1960 Fender Jazzmaster, with a 1985 Jazzmaster and a 90s American Stratocaster as backups. Often citing experimental guitarists like J Mascis, Thurston Moore and Jimi Hendrix as early influences, Walden's unusual playing style set Babyshambles apart from other bands in the East London music scene. Marshall magazine "Marshall Law" lists him as using a JCM900 head with a 1960A cab.
In December 2005, Walden left Babyshambles. The band continued to perform under the same name, but did not replace Walden with a different guitarist immediately. In the 10 January 2006 issue of NME, Walden's departure was officially announced. However, on 23 January 2006, Walden turned up to play guitar for the Babyshambles at a gig in the Junction, Cambridge. He returned once more to the band in February and played several of the gigs on that tour, but has not appeared with them onstage since.
Walden's reason to leave the band was his heavy drug abuse. In April 2006 Babyshambles went on tour without Walden who was accused of assaulting his girlfriend, arrested and spent nine days in Pentonville prison. All charges against Walden were eventually dropped. He left London in order to get clean of drugs and Mick Whitnall became Babyshambles' new guitarist. Walden was supposed to have played with Babyshambles on their November–December 2007 Arena tour. The band released a statement saying that he had dropped out at the last minute, even after travelling with them on the tour bus. Walden later stated that he did not appear on stage because there were drugs about, even though it was supposed to be a drug-free tour. Furthermore, Walden stated that his successor, Mick Whitnall, did not want him to play with the band anymore.
Walden was rumoured to have been playing at The Cheltenham Jazz Festival (27 April–2 May) as a special guest with Seb Rochford's band Fulborn Teversham, but cancelled the appearance.
Walden appeared at the Rock Against Racism 30th Anniversary Show at the Hackney Empire on 19 July 2007. He played Babyshambles' classics with his old band mate Drew McConnell. The set list included "The Man Who came To Stay" and "8 Dead Boys"
On 3 August 2007 played a small gig in Hackney with all of his former Babyshambles bandmates (as well as Mick Whitnall) for friend Peter Wolfe's birthday.
On 6 November 2009 Pat played a few songs with Babyshambles including "Pipedown" and "Black Boy Lane" at a gig at Halo in Battersea.
In late 2007 Walden formed the band Big Dave with drummer Seb Rochford and Ruth Goller on bass. In the first half of 2008 the band played a few gigs in small venues. Walden announced that a debut EP will be recorded in 2008. But so far, nothing much has been heard since then.
In August 2010, Waldens career appeared to be off the rails again after he appeared in court charged with receiving stolen goods. After being spotted acting suspiciously in a Sutton branch of Wilkinsons, Walden was pursued by security who found in his possession stolen goods and Methodone prescribed to another user.
In April 2011 Walden completed a spell in rehab and began work on a new project with ex Rebecas member Robert Mannall. The duo have recorded thirty two demos that have been posted on various sites on the internet.
On 1 May 2011 Walden previewed a selection of new material at the Hawley Arms, as part of Camden Crawl 2011, with Drew McConnell.
In November 2014 Walden did his first interview and photoshoot in a number of years with friend, music enthusiast and blogger, Olivia Collins for the website www.themusicalacidtest.com and photographer, Jack Grange. In the interview he says he is studying a Jazz Composition degree in an effort to reconnect with his first love, jazz music.