“If we think of it in terms of race, we would not hold a debate over whether it was appropriate for blacks to be pastors.”