Did God hold Israel's immorality and golden calf worship against them?
No. Though they committed immorality and danced around the golden calf in pagan practices while Moses was on the mount, God did not hold those sins against them; He could forgive them. Israel's real problem was unbelief—they could not believe what He said, so they could not enter His rest.
Les teaches that the vile sin of Israel—committing immorality and dancing around the golden calf while Moses was up in the mount—was not what God held against them. He does not even mention it there. As awful as that sin was, God was able to forgive it. The issue was unbelief: they could not believe what He said. Quoting the passage, Les points out that God swore they should not enter His rest because they believed not, and the last verse says they could not enter in because of unbelief. Nothing has changed. God can forgive to the uttermost any sin, the vilest of sin, except unbelief—refusing to believe that Christ died for them, paid their sin debt, and rose from the dead in power. If a person refuses that gospel, his doom is sealed.