How do you deal with unfair criticism from other pilots?

The topic title is actually my question: How do you deal with unfair critics from other pilots?

The reason for this question was the following recent situation: During one of my last approaches into LSPV (Wangen-Lachen), I was slightly north of the published traffic pattern, but very minimum and still overhead lake Zurich. At some point, someone talked on the radios in Swissgerman like: “You’re flying above a village, you can’t do that!” I was confused. I looked down and saw water below me, so at first, I thought this call wasn’t for me. But then again: “The aircraft on downwind: You’re still flying above noise-sensitive area, you need to fly over the water!” I looked down again and saw just water. No village, which was clearly further left of me. I continued my downwind and the man was saying again quite angry: “You’re still above the village!!”. But no, I clearly wasn’t, so I just said “negative” on the radio and continued my approach and landed safely on runway 26. I checked my GPS-track on Skydemon, which was clear above the water and outside the red noise-sensitive areas. I was very irritated by it, as I flew a quite new airplane type and wanted to concentrate myself on the landing, instead of dealing with someone how claims something, which is not correct.

At first, I thought someone was complaining from the C-office, but there was nobody there (as usual in LSPV). Later on, I met the one how was complaining about me on the radios. He was an FI and had a student with him. “We both saw you clearly flying above the village. That’s not good, you should keep the traffic pattern very tight.” Then I showed them my GPS-track and both started laughing at me: “Your GPS-track proves nothing, we saw you clearly flying above land and not the lake.” I was completely confused. I just said sorry, wished them a pleasant day, and went away. Later on, I realized that they might had a false perception of me, because they were flying behind more on the left (southern) side. So for them, it might have looked in a way that I flew over the village. But that isn’t a reason to abuse the radios for so long in Swiss-German.

Honestly, I actually hate when “flying colleagues” blame you for something you didn’t do. And why do they want to play air police?! If someone has some helpful feedback, I’m always open to learn something new and thankful if someone points out a real mistake, but this behavior was totally inappropriate. Some Swiss pilots are real nitpickers when it comes to noise reduction policies. It’s a disease on several Swiss airfields. At least I managed to land several times in LSZF (Birrfeld) without any criticism.

To answer the question myself: Deescalate the situation, say sorry, and go away. Even when they blame you for nothing. How would you react to such situations?