if I am late and a guest has taken my place I ask them to move over a place or two
If you are late you shouldn’t ask him to move in middle of davening. It isn’t his fault you’re late and he shouldn’t have to interrupt his davening due to your lateness. Especially when it’ll now be hard, if at all possible, for him to find another seat.