In practice literacy tests were and are (where they are allowed) targetted against undesirabled. To many Americans, especially the secular Jews who have great influence, we frum Jews are especially undesirable.
Better to have the current rule – everyone gets to vote (except convicted criminals and children).