Born in 1896 in Laurens, South Carolina, Davis claimed that he became blind when a doctor put drops in his eyes, although he gave different versions of the story at various interviews. Regardless, he was faced with the usual prospects for a young blind black person in a southern USA State at that time, music, begging, or learning some simple skill. By his early teens, Davis was playing harmonica, banjo, and guitar in a string band and he met Will Bond, whom Davis called ‘a master guitar player’ and Willie Walker. In 1914 he went to a school for the deaf and blind where he learned Braille and taught music. Davis left the school after six months and hit the road, travelling all over the Carolinas, ending up in Durham by 1926.

It was there that he met and taught a younger guitarist named Fulton Allen, who would later make a name for himself as Blind Boy Fuller. In the early thirties, Davis converted to Christianity, eventually becoming ordained as a Baptist minister. He would mix blues and gospel, playing at revivals and at lumber camps. The vocals on his blues numbers sounded straight out of church, while his finger picking gave his spirituals a bluesy feel. In 1935, Davis went with Fuller to New York to record for the American Record Company. Though Fuller went on to great fame, becoming one of the most recorded bluesmen during the Depression, Davis did not record again until after the War. He eventually moved to New York City and with Brownie McGhee and Sonny Terry, he began recording for a number of folk labels and became known as a blues-playing preacher who worked on the streets of Harlem.

By the end of the 1950's, Davis was sufficiently well-known that he became a major figure of the growing folk music revival. He played the major festivals being one of the most in-demand bluesmen before the men from Mississippi were rediscovered in the 1960's. In 1964 he toured England for the first time playing gigs at mainly small venues in the south. Davis continued to play until his death in 1972.