The fundamental flaw of Libertarian thought is the disregard of the tragedy of the commons. The idea that rational actors could deplete a common resource through nothing but the logical pursuit of their own self-interest is a big hole in their platform, as far as I'm concerned.
That's about one of the best nutshell arguments against the philosophy. I think another is risk, where the bigger the businesses or organizations are, the more potential they have to cause harm (intentionally or no) and to a much greater degree. The bank collapse late last decade is a good example.
The one is contained within the other; you're right in that risk scales but libertarianism is basically a mercantilist approach to feudalism anyway. That was one of Piketty's biggest conclusions: without economic protectionism, you will end up with a feudal economy.