Over the past two decades, there has been a high promotion of microfinance institutions (MFI) in Tanzania. In 1990s there was only 825 MFI which increased to 1,875 in 2005. Currently, the country is estimated to have more than 5000 MFI. The promise of MFI lies in the belief that microfinance could empower poor people to fight against poverty through easy access to credit. But what is the actual impact of MFI on the ground? Empirical evidence in this area is inconclusive. The objective of this study was to investigate the impact of microfinance on agricultural productivity by smallholder farmers in Tanzania with the case study of Iramba District. A total of 98 respondents were selected randomly from credit beneficiaries (CB) and noncredit beneficiaries (NCB). The collected data were analyzed through descriptive statistics and multiple regression analysis. Findings revealed that, CB realized high agricultural productivity compared to the NCB respondents. This is partly because the CB were relatively better in accessing markets for agricultural commodities, use of inputs and adoption of improved farming technologies. The major factors hindering smallholder farmers’ access to credit were reported to be lack of information, inadequate credit supply, high interest rates and defaulting.