Is it our job to judge people outside the church who are unsaved?
No. Scripture says that those outside the church, the unsaved world, are judged by God; that judgment is His prerogative and never ours. Believers are not to take God's place in judging unbelievers. Church discipline applies only within the church, after dealing with a person's sin.
Les explains that Paul distinguishes between the unsaved world outside the church and sin within the church. Regarding those without, God judgeth; that is His prerogative and never ours, so believers are not called to condemn outsiders. Inside the church, however, Paul says to deal with a wicked person, first trying to get them to recognize and confess their sin and gain victory over it. This correction should be done in a spirit of meekness, not pride, remembering that we also could be tempted and fall into the same trap. If the person refuses, Paul says to put them aside from among yourselves until they come to their senses. Thus the church's responsibility is internal restoration and discipline, not judging the unsaved.
But them that are without God judgeth. Therefore put away from among yourselves that wicked person.