A world limited to direct exchange would require a "double coincidence of wants" before a trade could occur.
It would be impractical for someone to specialize in the production of services or goods that were of high value but rarely needed, such as (say) a heart surgeon or an orthodontist.
In order to obtain enough food, clothing, and other goods for a comfortable living, the orthodontist would need to find—every week!— someone with meat, bread, etc., who needed work done on his braces, etc.
In such a world, people would have to always be ready to fall back on their own production for necessities, and couldn't spend time becoming a true expert in a specialty.
Note also that the orthodontist wouldn't be able to accept, say, 100 chickens from a farmer in exchange for a lot of dental work, because if the orthodontist then used some of the chickens to obtain other items, it would no longer be direct exchange.