Jim Brumby's father came to Marietta from Mississippi to manage Glover's Tannery, which supplied the Confederacy with essential supplies and was burned when Sherman's army took Marietta in 1864. In the summer of 1863, he and several cadets joined the Confederate Army and enlisted in the Seventh Georgia Calvary under the command of his cousin, Wallis Brumby, oldest son of Anoldus V. Brumby. Eventually, he was sent to the Army of Virginia under General Robert E. Lee.
At the end of the war, when 19-year-old Jim returned to Marietta to mend his broken fortunes, he found his hometown ravaged by Sherman's march through Georgia to the sea. He realized returning soldiers would have to rebuild the South he loved so well, but his first venture, a tannery, was unsuccessful. Broke, but not discouraged, he and an old slave named Washington began making barrels by hand for nearby flour mills. The barrel business became profitable and thus the Marietta Barrel Factory was founded in 1867.
The barrel business began to decline when flour mills switched from barrels to sacks. Jim bought a hand lathe for $25 at a sheriff's sale on the courthouse steps in 1874 and began experimenting making chairs. The barrel and fledgling chair business was located at the corner of Powder Springs and Goss streets near the Confederate Cemetery but was below the Western and Atlantic Railroad line and frequently caught fire from sparks from passing locomotives. As the barrel business floundered, Jim converted totally to chair manufacturing. Although he formed a partnership with his friend, Major Henry Myers, he sent to Mississippi for his younger brother Thomas M. Brumby in 1875, and they operated as Brumby and Brother until the Brumby Chair Co. was incorporated in 1884.
In 1888, because of back problems and on the advice of doctors, Jim Brumby sold his interest in the chair factory to his younger brother Thomas and retired to Dunedin, Florida, traveling there by covered wagon. After his health improved, he returned to Marietta in 1892 and founded the Marietta Chair and Table Co. along the railroad tracks on Church Street in buildings now known as the warehouses. At Marietta Chair he built chairs, tables and cradles but no rockers.
Jim Brumby married Laura Margaret Smith of Beaufort, South Carolina, in 1866, and they had six children. She died in 1912 at their winter home in Dunedin; four years later, Jim married Evelyn Holmes of Flat Rock, North Carolina.
Jim Brumby was a highly innovative and skilled craftsman who was considered a genius with tools. He especially enjoyed tutoring workers at Brumby Chair and Marietta Chair in furniture making. He served as an alderman in Marietta in 1877 and later became an avid boatsman on Florida's Gulf coast. He was the founder of the St. James Church in Dunedin, now known as the Episcopal Church of the Ascension. In Marietta, he built the old Sunday School Chapel at St. James Episcopal Church.
He died on November 25, 1934, at his Florida home, six weeks before his 89th birthday. At the time of his death, he was survived by three daughters: Mrs. Lewis D. Hoppe of Atlanta, Mrs. Robert Leach and Mrs. F.M. Patton of Swannanoa, North Carolina; three sons, R.M Brumby and James R. Brumby both residents of Florida and Bolen G. Brumby of Marietta. In his last illness he wrote his own funeral notice, inviting friends both "white and colored" to his funeral, asked to be buried at sundown, and requested no "bought flowers". He was buried in the St. James Episcopal Cemetery on Polk and Winn streets, only blocks from the two furniture factories he started many years earlier.