It’s in the word: a licensing authority is an authority that issues licenses. A pilot license is a license (= a document recording a permission extended to you), a medical is not a license but a certificate (= a confirmation that a test was passed).
From what bookworm quotes, it seems that the Polish AME/AeMC was not entitled to remove your limitation. Given that you had a class 2 before, one might argue that the Polish class 1 is unrelated and therefore the “removal” provisions don’t apply.
The law says that limitations cannot be removed by the AME/AeMC without the involvement of the authority that issues the PPL. You had a restriction in your class 2 medical, then you went to a foreign country and suddenly you have a class 1 medical without restriction. That — irrespective of your particular circumstances — raises all red flags possible. I suggest you try to work out something with your authorities in Latvia and agree with them on what is required to please them.
Moving your file to Poland might also work but that can be a lengthy process and also requires cooperation from the Latvian authorities. I am not sure you can do that without having your place of residence in Poland.
then you went to a foreign country and suddenly you have a class 1 medical without restriction. That — irrespective of your particular circumstances — raises all red flags possible
You have to admit it is extremely useful though 
I didn’t do that myself but huge numbers of European airline pilots got their initial CV tests done in Hungary. Now, 10-20 medical renewals later, nobody cares.
