Halderman was born and raised in Kentucky. He was educated at McKendree College in Lebanon, Illinois, and subsequently studied law at the University of Louisville. In the spring of 1854, at the age of 21, he came to Kansas and began the practice of law at Leavenworth.
He served as private secretary to Andrew H. Reeder, the first territorial governor, and in 1855 was Secretary of the first Territorial Council. He was appointed the first probate judge of Leavenworth County; was a major of the First Kansas Regiment in the Civil War, and a major-general of the State Militia.
He also served two terms as mayor of Leavenworth, was a regent of the University of Kansas, a member of the State House of Representatives and in 1870 was elected a member of the State Senate.
In 1872-73 he traveled abroad. In 1880 he was appointed Consul at Bangkok and was soon promoted to Consul-General by President James Garfield. In 1883 he was the first United States Minister to Siam, where the king honored him with the decoration of Knight Commander of the Order of the White Elephant, and later the French government bestowed on him Commander of the Royal Order of Cambodia. He resigned his position in 1885 and returned to Leavenworth.
For some years he resided in Washington, D.C., and at the outbreak of the Spanish-American War promptly tendered his services to the Secretary of War. He was a member of the Kansas Historical Society and a frequent contributor to its publications.
Halderman died in Washington, D.C., in October 1908, and was buried in the government cemetery at Arlington.