Slaves were brought to the North by Southerners when they went on vacation. This caused a lot of controversy if they were free or not.