Would it be possible for you to say that obviously for religious reasons you can’t have lunch at a non-kosher restaurant, and it is preferable that you are NOT seen in one. But obviously, if it is an important business meeting where you will gain information and exchange ideas, and if s/he feels it is necessary for you to be there you will bring along your own lunch and join them.
In that way, you can explain your religious obligation while still acknowledging your willingness to be part of the team.