MenhemShmei: then almost everyone in the world (or at least places settled by Europeans or Middle Easterners) is a safek Jews. One Jew going off the derekh (or kidnapped) a few thousand years ago has lots of descendants, none of whom have any idea they are related to us.
Of course, one could argue, that everyone created in the image of HaShem is special, and according to Humash, everyone in the world was a relatively close cousin of Avraham Aveinu (remember that at least one of Noah’s kids was still alive at the time). But that sort of argument is very annoying to anyone who wants to feel that he is “special”, and being in Ha-Shem’s image and related to Avraham Aveinu is not special enough for him (and is very anti-racist and is total heresy to the scientific community).