The son of businessman-politician Henry B. Payne and Mary Perry, he graduated from Phillips Academy Andover in 1859, then studied at Yale University. At the outbreak of the U.S. Civil War in 1861 he enlisted in the Union Army, though he could have afforded to pay someone to go in his place. In 1863, he became colonel of the 124th Ohio Infantry. He was Brevetted Brigadier General March 13, 1865. Upon the end of the war, he began his career, investing in iron and then oil refining. His oil interests were the first acquired by Standard Oil, and he became a trustee of that firm and acted as a lobbyist. He was charged with bribing members of the Ohio Legislature to attain a Senate seat for his father (before the U.S. Senate was directly elected), and with bribing the Democratic Party to name his brother-in-law United States Secretary of the Navy, though the charges were dropped.
He was the uncle of Payne Whitney. He was also the uncle of Congresswoman Frances Payne Bolton (1885–1977). He was named for Oliver Hazard Perry, a relative of his mother.
He is currently buried at the Lake View Cemetery, which is located in Cleveland, Ohio.