When is it appropriate to wear cultural attire from a culture that's not your own? If done out of context, it can easily be seen as offensive to wear a turban or sombrero. But, when a host family invites you to wear a dishdasha or kimono, it's flattering that you are willing to follow their customs.
For me, this is a question of intent. If I'm respectful as I immerse myself in a new culture, everyone wins. But, if there's a perception that I'm appropriating or caricaturing a culture then I've crossed a line.
But, even if I'm being respectful, there's no way for a stranger to distinguish between the two cases when I'm walking down the street. As a result, I'd only wear foreign cultural attire when given an explicit invitation from the host of a culturally significant event.