I do exactly the same as OP -- refer to the martial artists who gather together to defend the planet as the Z-Warriors, and the whole gang including non-combatants as the Dragon Team. But when chatting in less DB-centric circles, Z-Warriors or Z-Fighters is simply far more recognisable and distinctive than Dragon Team. Simply saying "the main characters" never goes amiss either, I guess.
I was surprised to see a minority backlash against the term Z-Warriors. I get that it sounds like a #radical Funi dubism, but it has a precedent in the Japanese anime. And I get that Z-Warriors being a non-diegetic name irks some people but throughout pop culture there's plenty of examples of character herds having nicknames they don't use themselves. The Rebellion/Resistance Heroes, Joestar Group, Team TARDIS (until recently), Keystone Cops, etc. It applies to real world history as well. The Founding Fathers didn't refer to themselves as that. But it just helps get the point across quicker.
While it's never used in-universe (neither is Dragon Team as far as I'm aware), Z-Warriors does make some thematic sense to me. They gather together as an absolute last resort to defend Earth and it conveys their status as misfit weirdos, kinda like the phrase "Z-list celebrity". It's funny that Toyotaro's manga has also upgraded the Z-Warriors into the 'Super-Warriors' in some chapter titles. Would the GT cast be called the 'Grand Tourers'?

The Super-Warriors are a bit more consistent than before. You can always expect to see Goku, Vegeta, Kuririn, Piccolo, Roshi, Gohan and Tenshinhan (with Jaco and the Androids usually) on the roster.
Earth's Special Forces sounds way too militarised for my liking.