Gardner Williams was descended from an old Boston family. His father, Jeremiah Williams, owned Jeremiah Williams & Co., one of Americaâs largest wool merchants, and was, for a time, the president of the Boston Wool Trade Association. Gardner Williams eventually joined the family business and travelled extensively throughout Europe and the Middle East, and presumably, this familiarity with Europe led him to participate in the Athens Olympics of 1896, as it is noted that he paid his own expenses to Athens.
Writing in 1932 in The Sportsman, Thomas Curtis, 1896 gold medalist in the high hurdles, described Williamsâs problems in the Athens swimming events. Curtis notes that the only American swimming entrant, which was Williams, jumped into the Bay of Piraeus, where the swimming events were held, to start the 100 freestyle, and found the water quite cold, proclaiming, âJesus Christ! Iâm freezingâ and scrambled out of the water. The story is considered to be almost certainly apocryphal.