Because if you’re not, someone else will and when they do, they will no longer be your team, they will now be on this other person’s team.
Org charts do not define who belongs to what team, they only define who you report to.
We all know whose team everyone belongs to.