Did Paul continue to grieve over his past persecution of believers?
Yes. Paul never quite got over the realization that the one he persecuted was the Old Testament Jehovah; it beat him down. He sorrowed for the day he had persecuted Jewish believers, and the memory never left his mind. He continued to grieve.
Les teaches that after Saul of Tarsus found out that the one he was persecuting was the one who really was his Old Testament Jehovah, it just beat him down and he never could quite get over it. The transcript notes that he could never get it off his mind, and that there are many more accounts of how he sorrowed for the day he had so persecuted these Jewish believers. Saul had voted against them, punished them in every synagogue, and compelled them to blaspheme. Later he called himself the least of the apostles because he persecuted the church, the assembly of God at Jerusalem. He acknowledges this in 1 Corinthians 15, beginning verse 10 with 'But.'