Universities and other higher learning institutions are recognized to be in the knowledge business because they are involved in knowledge creation, dissemination and learning. However, in recent years knowledge has gained increasing economic importance and therefore the role of knowledge management and communities of practice for knowledge creation, sharing and utilization in higher learning institution is becoming crucial. This paper discusses the application of Communities of Practice (CoP) as a means of capturing and sharing organizational knowledge in higher learning institutions. CoPs are informal networks that enable professionals to develop a shared meaning, engage in knowledge building and help to stimulate knowledge flow among the members. Effectively cultivated CoPs can be an important strategy for tapping and harnessing tacit knowledge and facilitating knowledge sharing. This paper draws on an extensive literature review and on various case studies that deal with the concept of communities of practice. The paper discusses the postulation as to how CoPs can be used for harnessing tacit knowledge in higher learning institutions. Relevant theories with regard to CoPs are also highlighted, and applied in practice at the Sokoine University of Agriculture (SUA) in Tanzania as a case study. Lastly, the paper give recommendations on how best CoPs can be implemented in managing tacit knowledge in higher learning institutions, especially in Africa.