From what I understand it was more of the image of drinking that was his trademark. Although he made out to be a heavy drinker on stage, he mostly used apple juice, but off stage was a Jack Daniels man.
Although his image was of a party goer, Martin did not party all night with the rest of the "Rat Pack" crew - actually calling themselves "The Clan". He usually went to bed early so he could play golf the next morning. He was obsessed with golf, and once stated in an interview that he would have preferred to be a professional golfer than an entertainer.