After the war, a couple of good ol' boys that recently 'walked the dog' (the forced loyalty oath returning confederate soldiers had to make to federal troops) met for drinks as they commiserated about their situation.
They eventually formed a social club of sorts and called it the Klu Klux Klan.
They dressed up like ghosts in bedsheets and went on torch lit parades through town.
All of these men were angry, having fought a useless war and now being governed by the same federal troops they fought against. They took much of their anger out on the recently freed slaves who as they saw it, were better off now while they were in a worse situation.
Their ceremonies became more regular and started to focus on scaring the freedmen and keeping those who they thought were 'uppity' in line.
They also eventually morphed into a militant 'Al Quaeda in Iraq' sort of organization and started carrying out murders of freedmen and federal troops for dramatic effect and to change the political situation in their favor. This anger was also turned on the 'carpet baggers' from the north and started to spiral out of control a year or so after the war ended.
Northerners weren't safe walking the streets and whig politicians would be murdered from time to time. Johnson certainly did nothing to stop this and restrained the federal troops from aggressively going after the terrorists.
Only when Grant sent in the full force of the Army did the tide start to turn. Even after that, no freed man could break the 'black codes' without fear of being visited at night by a gang of murderers.