What was the main complaint against Paul's apostleship, especially from the Jewish community?

Les explains that Paul defended his apostleship by pointing to the believers themselves as his work in the Lord. Paul could say that if he were not an apostle to others, he was certainly an apostle to them, because they had been saved out of abject idolatry only through his ministry. He told them, in effect, “You’re the proof of the pudding”; if he were not an apostle, they would still be in paganism. Les compares this to people asking who he thinks he is—just a layman without education—yet his proof is many people saved out of abject sinful backgrounds and brought out of darkness into the light of the Gospel. Paul’s apostleship, then, was proven by the believers he had led out of dark idolatry. The big complaint, especially from the Jewish community, was that he was not one of the Twelve, since he had not walked with Jesus for three years like Peter. Paul’s answer to those who examined him included asking whether he and Barnabas lacked the right to eat and drink, to lead about a sister or a wife as other apostles and the brethren of the Lord and Peter did, or to forbear working.