The war was officially over, but not for James Bobbitt. He returned to Monroe County and his family. Monroe and Greenbrier counties had many Union sympathizers but James Bobbitt never passed an opportunity to tell them how wrong they were.
There are conflicting legends about what happened after James returned home from the service. One legend says that one of the sisters of James Bobbitt was assaulted by a freed slave, and that James Bobbitt corrected the situation according to Confederate law. A John Alderson was constable and sheriff at the time and urged James Bobbitt to leave the area before the union officers on duty, took him into custody.
Robert Lee Bobbitt, grandson of James Tolliver Bobbitt wrote this account, "due to the unfortunate activities of the carpetbaggers in that vicinity, and the fact that Captain James Bobbitt sought aid for the men of his company who had returned home with him, and who were being persecuted by the carpetbaggers, forced him to leave Monroe County."
For what ever reasons, James Bobbitt did not have a welcome place in Monroe County or Virginia. He left alone on horseback and went to a farm in Logan County, Kentucky. Apparently his wife and children joined him in Kentucky. In the fall of 1865, Tilman Bobbitt, most of his children, and several of James Bobbitt's first cousins, left Monroe County. They stopped in Logan County, Kentucky where they were joined by James Bobbitt and his family. The group in wagons, with all their possessions, journeyed to the village of Springfield, Illinois.
Sometime in 1866, James Bobbitt and his family moved to a village called Louisiana, Missouri, in Pike County. They owned a farm and lived there until about 1877.
When the family left Missouri, they moved to Limestone County, Texas. According to Robert Lee Bobbitt, "Captain James Bobbitt moved from Hill County, Texas to the Panhandle of Texas in the early 1890's and lived for a time at Lockney, Floydada, and Plainview, having aided in the settlement of those communities, which are now beautiful and thriving little cities;
"In 1896, Captain James Bobbitt and his wife moved back to Hill County, Texas. They lived there with his eldest son, Joseph Alderson Bobbitt, who had lost his first wife, Laura (Duff) Bobbitt."