Miller was born in the Chester District of South Carolina on November 12, 1828. He was educated at Indiana University, where he graduated in 1851, after which he also graduated from the law school at Poughkeepsie, New York. In August 1854 he came to Kansas to support the Free-State cause. As his father had been waylaid and mobbed because of his anti-slavery views, it was but natural that Josiah should be an ardent opponent of slavery, and on January 5, 1855, he began the publication of the Kansas Free State newspaper at Lawrence.
A pro-slavery jury found an indictment against him for maintaining a nuisance in the publication of this paper, and on May 21, 1856, his printing office was destroyed by the territorial authorities. In that year he made speeches in several states for John C. Fremont, the Republican candidate for president, and in 1857 was elected Probate Judge of Douglas County. In 1861 he was a member of the first state senate but resigned his seat to become postmaster at Lawrence. In 1863 he was appointed a paymaster in the army, with the rank of major, and in 1866 was elected a member of the legislature.
His death occurred at Lawrence on July 7, 1870, after having a leg amputated. The inscription on the monument erected to his memory in Oak Hill Cemetery credits him with being the author of the motto, “Ad astra per aspera,” on the Kansas seal of state.