Kelley was born in Fredericksburg, Virginia on January 11, 1831. At the age of 10, he was sent to Newport, New Hampshire to attend a preparatory school before entering Dartmouth College. Disliking the rigid discipline enforced at the establishment, he ran away, landing in Lowell, Massachusetts, where he went to work on the Advertiser, an influential journal of the area. Knowing that if discovered by his parents he would be obliged to return to the dreaded school, he did not communicate with them for a period of five years. During this time he remained at the Advertiser, thoroughly mastering the details of the printing business. When he found out that his family had moved to Missouri, he joined them there in 1848, and for four years afterward was employed as a salesman in a mercantile house.
In 1852, he started a Democratic paper at Liberty, Missouri called the Democratic Platform and continued its publication until the passage of the Kansas-Nebraska Act, when he discontinued the newspaper and established a pro-slavery paper in Atchison, Kansas, called the Squatter Sovereign, with a partner, Dr. John H. Stringfellow in 1855. This was a time of great tension in Kansas and the newspaper took ultra Southern grounds. Concerning his experience at the time, Kelley would later say: “There was no such thing as concession at the time. We were all extremists, whether advocating or opposing slavery. During my editorial life, I was in constant strife with political opponents.”
He was elected a member of the State Senate under the Lecompton Constitution for the counties of Atchison and Doniphan. On the defeat of that measure by Congress, he sold out his interest in the Squatter Sovereign and moved to Doniphan County, Kansas, where he married Mary L. Foreman, and where his first child was born. He next engaged in merchandising in Kansas City and was thus employed when the Civil War broke out.
The first company of Federal soldiers that entered the city took possession of his store and carried off all his goods. Soon after, he joined the Missouri State Guards as a private and served under General Sterling Price. After the Battle of Pea Ridge, Arkansas he was promoted to captain. After leaving the Confederate service he emigrated to Montana, arriving in Alder Gulch in 1863. From Alder Gulch he went to Helena, and for some time mined in Grizzly Gulch. In 1866 he went to Bear Gulch, where he remained a short time, after which he came to Deer Lodge, where he lived until his death.
For several years he engaged in mercantile pursuits in partnership before becoming extensively in mining operations. In 1885, he was appointed by President Grover Cleveland as U.S. Marshal for Montana and served in that office with official integrity until the day President Henry Harrison was inaugurated. He then resigned, believing that the party in power should have control of all the Federal patronage and be held responsible for it.
He died of a heart ailment in 1890 in Montana.