Poverty was at the heart of the tragedy of the commons discourse from the very beginning. The idea was that commoners would inevitably end up deprived due to their own resource overuse. As Hardin saw it, if the initial problem was freedom of the commons, then limiting that freedom would logically reduce poverty. In this article, we argue that alleviating poverty among resource users calls for a broader concept of freedom than Hardin’s – one that is more in line with that of Amartya Sen’s “freedom as agency.” Based on case-studies of smallscale fisheries and poverty in Bangladesh and Tanzania, we claim that the root of the tragedy of the commons is the restriction of freedom rather than unlimited freedom and that it is arguable whether the people who have no other option than to continue fishing for their livelihood, even in over-exploited ecosystems, could be understood to be free.