Using the Mann–Kendall Test to analyze data from a survey of 400 farmers, this study compared the rate of adoption of conservation agriculture (CA) in two contrasting villages of Mnyakongo and Ugogoni locating in the Kongwa District, a semi-arid zone in central Tanzania. Results exhibited that the level of CA adoption was <10% of the total households. The trend of CA adoption was determined at the coefficient of R2 = 0.95, 0.90, 0.68 and 0.57 for mulching, crop rotation, agroforestry and little tillage, respectively. Despite little tillage and crop rotation having high acreage under CA, the rate of mulching adoption was significantly higher than that of others. Furthermore, there were significant correlations between the CA adoption and crop yields or environmental sustainability (p < 0.05). Maize, sorghum and millet yields were significantly greater under CA (1.7 t ha􀀀1) than no-CA (0.7 t ha􀀀1). Particularly, maize yields were increased from 1.3 to 2.3 t ha􀀀1 from 2000 to 2015 under CA when it was intercropped with legumes. The majority farmers (>70%) asserted that CA had optimized their yields for both food and economic incentives. Thus, this study recommends the adoption of CA in the semi-arid agro-ecological zones.