Collaborative multi-robot navigation is essential for robots to operate as a team in various environments. During navigation, the robot team often needs to maintain specific formations, such as circular formations to protect human teammates in the center. However, robots frequently encounter complex scenarios, such as narrow corridors, where rigidly maintaining these formations is not feasible. Therefore, the robot team must be capable of splitting into smaller subteams and controlling these subteams to navigate while adaptively maintaining their formations. In this paper, we introduce a novel subteaming and adaptive formation control (SAFC) approach for collaborative multi-robot navigation in complex scenarios. SAFC integrates three levels of robot learning in a unified hierarchical framework, including high-level deep graph cut for team splitting, intermediate-level graph learning for collaborative navigation of subteams, and low-level policy learning for controlling individual robots to navigate to the goal without collision. Experimental results in both simulations and real-world robots demonstrate that SAFC enables effective subteaming and adaptive formation control, and achieves promising results in collaborative navigation through complex scenarios.