Does following Paul mean we should throw away the rest of the Bible?

No, the whole Bible fits together, but for Gentiles under grace Paul is the specific apostle to follow.

Watch the Teaching: What Is "My Gospel" That God Will Judge the World By?