Is there a downside to being social friends with people who report directly to you?

It can turn into a difficult situation. The more you get to know people in a personal friendship, it can become difficult to handle some situations professionally. 

If your social friendship does not get in the way of either you or the employee doing your job effectively, then it can work out. But it can still be tricky. The perception of others in the workplace can be damaging, even if there is not favoritism for the employee.