Morning Coffee with Models

By Raju Vegesna, Chief Evangelist, Zoho

Coffee houses played a key role in idea generation throughout history. This is where people from different backgrounds and different industries would come and sit together and stumble upon an interesting conversation from a different industry that they could relate to their own, or an idea they could borrow.

Coffee houses (as they were called back then) were a significant upgrade from conversations over a beer, partly because you were replacing a depressant with a stimulant. They became a melting pot of ideas because of diversity of thought, coming from a diversity of backgrounds, experiences and industries. A merchant sitting next to a poet sitting next to a scientist sitting next to a sailor provided the diversity to exchange ideas on the same table.

The magic was not that smart people were in the room. Smart people had always been in rooms. The magic was that smart people from different rooms were now in the same room. Their knowledge had been built by walking different paths. It really was 'Mixture of Experts'.

Not the 'Mixture of Experts' (MoE) we talk about today in AI. The MoE pitch sounds like a coffee house. Many specialists under one roof. But these 'experts' really are slices of the same model, trained on the same books, same Wikipedia, same Reddit, same code from the same GitHub. These 'experts' went to the same school, had the same teachers, took the same tests. So when you ask them a question, you are pretty much getting the same answer, stitched together by the same 'experts'.

Today's frontier models are all drinking from the same well. Their reinforcement learning (RL) is almost the same. They are measured against the same benchmarks. It is the same talent pool moving between the same labs in the same city (very concentrated in SFO). When you ask all of them the same hard question, we don't get diverse viewpoints. It is the same point of view, said in different words. Just the fact that we can replace an API key from one model to another without seeing material difference tells you the entire story. None of the diversity of thought we used to find in coffee houses.

This 'mixture' feels more like a mixture of chemicals. Same inputs, predictable outputs. When we call this 'intelligence', what I am really sensing is chemistry, dressed up as biology.

What we are seeing is mono-cropping of intelligence.

This is a pattern I have been following across industries. Mono-cropping in farming spread to the construction industry, leading to mono-cropping of vertically organized cuboids called high-rises. This is why all buildings around the world look the same. The same pattern is showing up with clothes, food and many other industries. And I am afraid we might be entering a world of mono-cropping of intelligence, which is the most impactful of all. It could kill the diversity of thought, just like the lack of diversity we are already seeing in farming, construction, clothing, food and several other industries.

Diversity is the immune system of any ecosystem, including the ecosystem of ideas.

The old coffee houses worked because they cultivated diversity of thought and perspectives. What we really need is diversity of intelligence. Models trained differently, with different convictions, from different viewpoints. We need to bring a diverse set of models to the same table. They cannot be the Monsantos of the model world.

Perhaps one way to do this is at the company level. Think about how this works with humans. We are all born with the same biological foundation. Same base software, you could say, when we were born. But we become very different people because we live very different lives, with different experiences and different convictions. Models could work the same way. Start with a shared base model. Then let each company shape it through their own experiences, perspectives and convictions. Over time, these models would each carry the personality of the company that raised them. When they finally come to the same coffee table, with genuinely different views, that is when new ideas and cross-pollination can happen.

To some extent, I see parallels with what has happened to the coffee houses themselves. They evolved into the same coffee shop, replicated across every corner of every city, pretending to be coffee houses of the past. They have been mono-cultured too.

I hope we can diversify intelligence before the same thing happens to thinking itself.