Born one of 13 children in Missouri, James Blue moved to Mississippi. There he became a plantation and slave owner. When the Civil War began, he enlisted in the Confederate army serving as a second lieutenant in the Mississippi Cavalry. Blue was seriously injured serving with Major General Green at Vicksburg. He was shot through the head, the bullet entering his mouth and existing the back of his head below the ear. Initially left for dead, Blue was rescued by his comrades but his regiment was captured.
After he recovered, he re-enlisted, this time as a private with the 2nd Missouri Cavalry, a group comprised of many of his schoolmates and organized around his old homestead in Genevieve County, Missouri. His brother, John, served as sergeant.
Blue was on the staff of General Albert Sydney Johnson during the 1862 Battle of Shiloh, where Johnson bled to death from a bullet to the knee. In the 1870s, Blue moved west, settling in Riverside and working as a carpenter despite his war injuries.