I don't understand how goku is the only person responsible to keep in touch with his friends. His friends didn't visit him as well during the time between DB and DBZ, and I don't see the fandom complaining that Bulma, kuririn and others are bad friends because of this.
I have the impression there's an exaggeration of the idea that Goku is an absent family man. Since he got married he lived 5 years of his life with his wife and son, he kept training not just because he likes it but because of Picolo too, who wasn't an ally at the time. And if he were this detached person he wouldn't live this long in Mountain Paozu and he'd start to search for a new adventure. My point is that he didn't stay because of a promise but because he liked his new family. I don't see Goku as this being incapable of love, he loved his grandpa Gohan imensily and gave his name to his first son. Maybe during this time he didn't train as much as he should because even Picolo reprimends him for it when they formed a team to fight Raditz. And what happened because he wasn't strong enough? He got killed and his son kidnapped. I think this was a lesson he took for the rest of his life.
Now let's analyze the time he was away and the time he was present:
- 1 year he was dead and training to fight saiyans; he's revived, fights Vegeta and Napa after they kill his friends and nearly dies again.
- Next he travels to Namek and nearly dies, spent more than 1 year in Yardrat to learn a useful technic (if he knew of this technic a year before he could prevent his friends's death) and to dominate the super saiyan form;
- He returns to Earth, discover the androids threat and for the next three years he spent training at home, plus almost one year training with Gohan in the time chamber; the week before Cell games with the exception of one day when he looked for a new Kami and the dragon balls, he spent with his family;
- After Cell games he dies and asks not to be revived because he thought he brings danger to the planet and 7 years passed until he comes back and is revived. He discovers he has a younger son and he's surprised - what that implies? He had sex and it wasn't for reproductive purpose. And that mess with a headcanon people enjoys a lot, that Goku is too inocent to be married.
- The 10 years the manga said was of peace, new material (movies and super) retcons this with Beerus, Freeza and Zamasu. To my understanding the two movies are more canon than super (you don't have to agree with me) and I consider events of super from the Champa tournament onwards.
- Goku's alive again and he pays an old promise to his wife. He's a farmer now.
- Goku trains with King Kai (somehow he learnt to sense ki of a dead god

). He knows teletransport and there's no reason for him not to return home after he trains, but people likes to think he lives with King Kai.
- Goku asks Whis for training after he learnt Vegeta was already training for months. I don't know how much time he spent there but I don't think it was more than a year.
- He spent three years training with Vegeta in the time chamber but to their families it was only three days.
- With Zamasu I don't think they spent much time dealing with him, maybe a month? Correct me if I'm wrong.
- And now the current episodes Goku's with his family again.
- The end of the manga showed us Goku took Uub as his apprentice and went to his village (he counts with Mr. Satan's money to help the boy's village). Goku's happy to find a worthy and eager fighter in Uub but he must understand he's also responsable for the boy's life considering he's only alive because he asked. We don't know the time Goku stayed there. GT don't count, but Toriyama drew in Majin Z Goku and Uub in Mount Paozu next to Chichi, Goten and Pan, so after a time maybe Goku asked Uub to live in Mount Paozu during his training. I don't know, Uub's family is poor how could they manage to feed Goku day by day? It makes sense for Goku to invite Uub to be trained at Mount Paozu. And Uub could go to his home at weekends.
A thread to talk about Vegeta and his relationship with his family, and here I am talking about Goku... People usually compare the two of them but I'll always find Goku more interesting.