What we know know as Tarrytown was once a poor farm in the late 1800's. It was for paupers, convicts, and farmhands, some looking for work, some looking for food in tough times. It was even used for some criminals to pay off debt.