He is the youngest son of a civil servant, Harold Cocker, and Madge Cocker. According to differing family stories, Cocker received his nickname of Joe either from playing a childhood game called "Cowboy Joe" or from a local window cleaner named Joe. Cocker's first experience singing in public was at age 12 when his elder brother Victor invited him on stage to sing during a gig of his skiffle group.

In 1960, along with three friends, Cocker formed his first group, the Cavaliers. In 1963, they booked their first significant gig when they supported the Rolling Stones at Sheffield City Hall. In 1964, Cocker signed a recording contract as a solo act with Decca and released his first single, a cover of the Beatles' "I'll Cry Instead".

During his career, Cocker creates different musical groups - "The Grease Band " (1966-1969), "Mad Dogs and Englishmen" (1969-1971). He tours the world , playing to large audiences in Europe, Australia and the USA. In 1988, he performed at London's Royal Albert Hall and appeared on The Tonight Show. He also performed for President George W. Bush on the open concert in February 1989.

In October 1972, when Cocker toured Australia, he and six members of his entourage were arrested in Adelaide by police for possession of marijuana. The next day in Melbourne, assault charges were laid after a brawl at the Commodore Chateau Hotel,and Cocker was given 48 hours to leave the country by the Australian Federal Police. Later Cocker sank into depression and began using heroin. In June 1973 he kicked the habit, but continued to drink heavily.

In 1978, Cocker moved onto a ranch owned by Jane Fonda in Santa Barbara, California, United States. Pam Baker, a local summer camp director and fan of Cocker's music, persuaded the actress to let the house to Cocker. Baker began dating Cocker and they eventually married on 11 October 1987.The couple reside on the Mad Dog Ranch in Crawford, Colorado, United States.