Defaulting to assuming people are locals

Because I look racially ambiguous, people often don't know what to make of me, especially when I'm traveling. Typically people default to assuming that I'm a local, speaking to me in their native language until I stop them and explain that I'd prefer to switch to English. This is an attitude that I wish more Americans would adopt. Why not err on the side of including someone in your in-group, rather than default to assuming someone is a foreigner?

Ironically, I'm periodically asked in the United States whether I'm comfortable speaking English. Given our country's multi-cultural history, this seems backwards. While some may argue this tendency comes from a desire to be helpful, on the receiving end it feels more like xenophobia. Default to assuming people are locals and then offer to be helpful only if it's necessary. Why presuppose?