I'll admit I've forgotten most of my US history. But as the film depicts at least, state's rights and slavery are intimately intertwined (or at least, the war becomes about slavery even if it didn't begin that way). Lincoln tries to convince his party that if the slaves are freed, the Confederate states will have no more reason to continue fighting, since (among other reasons) slavery is the basis of their economy. On the other hand, any hint of a peace agreement in the war threatens the amendment passing because it no longer seems necessary to end the war.
It was shocking to see how few people at that time believe in the equality of all people, black or white, and how radical the idea seems. The only two (white) characters who seem to truly believe this are Lincoln and Stevens, who both are forced to go against what they believe to get the amendment passed. Lincoln, for example, passes the Emancipation Proclamation as a war measure based on the idea that slaves were the "property" of the Confederate rebels that he could confiscate. Stevens ultimately testifies that he does not believe in true racial equality, but only equality "under the law."
For most at the time, the idea that blacks and whites were equal, or deserve equal rights, was unimaginable. What gets many of the congressmen riled up is the thought that this could one day lead to blacks or even women voting. Though the amendment was passed in 1865, it wasn't until 100 years later with the 1965 Voting Rights Act that most blacks in the South were really able to exercise their right to vote. Desegregation and women's voting came a century later.
If the idea that blacks were inferior to whites was so widespread, I wonder about people like Lincoln and Stevens. To go against the ideology of almost their entire race and country to truly understand equality seems incredible. Were they visionaries? People who could see beyond their time and circumstances and imagine a different reality? Lincoln did not pass the 13th amendment as a political necessity, or to appease his constituents for reelection. He did it because he believed it was right and true. When can we say that about politics today?