Your comparison is invalid, because the swindler is an unlawful owner. There is nothing unlawful about a kosher marriage itself, despite how the two people got there. A truer comparison might be a person who became a talmid chochom by stealing seforim and/or copyrighted shiurim. He is a talmid chochom, but he must make amends for the means by which he acquired the knowledge. In doing so, he is not regretting being a scholar.
Let’s agree to disagree on this.