One thing that makes me optimistic is that work like this might lead to a culture that puts more stock in data, and less in our presuppositions. We can't help but be biased by the nature of our perceptions and interactions. IMO many avenues are limited to us simply because we cannot agree upon the meaning of the initial data we gather. Programs like this don't need to build a conceptual framework to move on. They only need logical consistency.
Don't get me wrong. I don't see this as a panacea. I am a biomedical researcher, and in you might be surprised how often scientists focus and search for 'a mechanism' when the idea is often ridiculous in a complex system of interconnected positive and negative feedback loops. My hope is that this kind of analysis leads to a more wholistic scientific understanding. That said, I don't buy that this is computational hypothesis creation. That's a very selective way of looking at what the program does, IMO.
Numbers go into a system, numbers come out of a system, and we make decisions based on that. There's problems with agreeing on the input, like you said. Then there's a problem with determining action based on output from the operations, because inevitably decisions will be outsourced and the people using the machine output won't know how it works, know it's limitations. That's effectively magic, and I get antsy whenever people won't have a fundamental grasp of how a piece of tech works but will be using it, especially using it to make decisions. Any thoughts on how you resolve any of these?