As in many other countries of Southern Africa, the conventional approach to water resources management in Tanzania has generally suffered from the syndrome of lack of recognition of multi- sectoral water uses and linkages. It has focused more on a sectoral-based development of water resources (e.g., the construction of irrigation infrastructure) and less on holistic management of water resources. However, the new water policy (2002) in the country has set a scene for major changes in the water sector to ensure better integration of water resources management across sectors, which is in essence a move from the conventional (sectoral) water resource management to Integrated Water Resources Management (IWRM). This intends to address participatory, multisectoral, multidisciplinary river basin management and integrate the linkages between land and water uses. Making IWRM happen on the ground is however, a hard work. It requires bringing together on a decision-making table, different stakeholders with diverging interests, complex power relationships and different perceptions: bringing together a very intricate socio-economic reality, the legacy of the conventional water management approach and its embedded practices and beliefs, and the apparently non-reconcilable conflicting demands. It is against this milieu that this paper presents a discussion of the existing water management challenges in Tanzania and the opportunities to build upon. Notwithstanding the complexity of translating the concept of IWRM into practise, the paper underscores the need to have the initial IWRM process focusing on crucial, urgent issues. For Tanzania, the entry point should be that of addressing the existing water resource conflicts, which are becoming rampant, particularly in the Rufiji and Pangani river basins; and facilitate the establishment of functioning Water Use Associations and Apex bodies. These are more likely to solve most of the existing problems of water resources management in the country.