The guy who took your coat is a sho’el. If you take his you are a shomer avaida. Alternatively you can say that like most of choshen mishpat it’s talui b’minhag and the minhag is that you use it until the next tefilla where you both show up and switch back.
Of course, you can avoid this situation entirely by not buying a coat that looks exactly like everyone else’s.