Introduction: 2023 - The Year of AI

2023 is obviously the undisputed year of AI. If you are confused about concepts such as ChatGPT, OpenAI, generative AI, and LLM (Large Language Model), then you may be out.  Even since the concept of artificial intelligence (AI) was born at the Dartmouth conference in 1956, AI has never been so close to the public. So naturally, everything around AI has also attracted a lot of attention, especially the few words about AI from the giants standing on the top of the wave can always trigger a lot of heated discussions.

2023 is obviously the undisputed year of AI. If you are confused about concepts such as ChatGPT, OpenAI, generative AI, and LLM (Large Language Model), then you may be out. Even since the concept of artificial intelligence (AI) was born at the Dartmouth conference in 1956, AI has never been so close to the public
Source Pixabay



Jeffrey Hinton's Regret and AI's Potential Risks

Jeffrey Hinton joined the ranks of "AI critics" after resigning from Google not long ago, talked about the risks that AI may bring in the future, and even said,  "I regret what I have studied all my life".


Just when his 180-degree turn was triggered, and the discussion on the dangers of AI by "people who don't know the truth" has not yet come to an end, an internal letter allegedly written by Luke Sernau, a senior software engineer at Google, stirred up waves again.  In fact, there is only one core theme in this leaked document, that is, neither Google nor OpenAI has a moat, and open source AI will take the final victory of this track.


Google's AI Roadmap and Generative AI

In this "war" of generative AI, Google is clearly just an out-and-out catcher.  Although as the creator of Alpha Dog, Google has been playing the role of "AI evangelist" for many years, in the field of generative AI, ChatGPT is undoubtedly the leader. After Google Bard’s public demonstration was overturned, causing Google’s market value to evaporate hundreds of billions of dollars, it finally launched Workspace which integrates generative AI into work scenarios, and Microsoft 365 Copilot, which integrates GPT-4, quickly put  The limelight was stolen.

In this "war" of generative AI, Google is clearly just an out-and-out catcher. Although as the creator of Alpha Dog, Google has been playing the role of "AI evangelist" for many years, in the field of generative AI, ChatGPT is undoubtedly the leader. After Google Bard’s public demonstration was overturned, causing Google’s market value to evaporate hundreds of billions of dollars, it finally launched Workspace which integrates generative AI into work scenarios, and Microsoft 365 Copilot, which integrates GPT-4, quickly put The limelight was stolen.
Source Pixabay


Therefore, in the eyes of the outside world, the term anxiety may be the most authentic portrayal of Google when it comes to generative AI and LLM.

 

At the same time, CEO Pichai, who is leading Google, has a conservative tendency and has been constrained a lot, so Google's AI roadmap is currently unclear.  Under such circumstances, the "sighted people" inside the company will of course be very anxious.


Then the question arises. As a laggard in the generative AI track, Google has a sense of crisis, and it is normal for pessimistic arguments to spread internally, but why is OpenAI involved, or why is an open source AI the ultimate winner?


Flowers are shining brightly and fire is cooking oil, which is undoubtedly the current situation of OpenAI.  Although this company is now almost the hottest investment target, according to a new round of financing documents recently exposed by relevant overseas technology media, OpenAI’s valuation has reached 29 billion US dollars.  What you need to know is that in the current A-share market, the market value of the two AI concepts Cambrian and Kunlun Wanwei alone is already close to this figure.  In other words, OpenAI, which ignited the fire of generative AI, has not been given an astonishing valuation by investors.


Generative AI and the Lack of a Deep Moat

OpenAI’s current problem is that it lacks a clear business model. Their only sure income, at this stage, is the ChatGPT Plus subscription service for $20 a month, and the ChatGPT API for 1k tokens/$0.002. But those two are clearly not enough to make OpenAI profitable.  There is even a view that OpenAI is now very much like QQ at the turn of the century. Although it is also at the forefront of related fields, it also lacks a clear commercialization prospect. But then QQ and others came to the QQ show, but OpenAI has yet to see an opportunity to solve the commercialization problem.

OpenAI’s current problem is that it lacks a clear business model. Their only sure income, at this stage, is the ChatGPT Plus subscription service for $20 a month, and the ChatGPT API for 1k tokens/$0.002. But those two are clearly not enough to make OpenAI profitable. There is even a view that OpenAI is now very much like QQ at the turn of the century. Although it is also at the forefront of related fields, it also lacks a clear commercialization prospect. But then QQ and others came to the QQ show, but OpenAI has yet to see an opportunity to solve the commercialization problem


In fact, what makes Google insiders pessimistic about Google and OpenAI is that generative AI, or LLM itself, does not have a deep moat.  That's right, although products such as ChatGPT and Wenxin Yiyan are so smart, it is actually not as difficult to build a generative AI as everyone imagines.


The theory of LLM is actually very simple, that is, by analysing a large amount of text data for training, to learn the structure and mode of the language, the architecture used is also a long short-term memory network (Long Short-Term Memory, LSTM) or a gated recurrent unit (  Gated Recurrent Unit, GRU) and other traditional recurrent neural network structures.


LLM is more like the result of "powerful brick flying". However, before ChatGPT became a blockbuster, the industry favoured Google's Transformer model.  What the latter pursues is how to design a smaller, faster, but more accurate neural network. Even at that time, OpenAI's GPT-3 was evaluated by the industry as a negative model.  At that time, some people in the industry said, "GPT-3 has shown excellent ability in small sample learning, but it needs to use thousands of GPUs for several weeks of training, so it is difficult to retrain or improve."


ChatGPT's Success and the Emergence of Baidu

The success of ChatGPT lies in the fact that it proposed a new idea. After all, the large model with increased parameters and higher computing power is also a director, and it really makes the large model emerge with intelligence.  But this model does not have a moat.  Don't you see, even if OpenAI keeps its reinforcement learning (RLHF) technology based on human feedback secret, it can't stop the emergence of Baidu Wenxin Yiyan, Ali Tongyi Qianwen, Google Bard and other similar large models.


Li Yanhong mentioned in Baidu's internal speech before, "Computing power cannot guarantee that we can lead in general artificial intelligence technology, because computing power can be bought, but the ability to innovate cannot be bought.  Built", that is to say, computing power and parameters can be bought, and the technical barriers are not high.  So it's no wonder that the "100-model war" in the domestic market has already started in a short period, so the first-mover advantage is hardly worth mentioning here.


Of course, if there is no "accidental" leak of Meta's LLaMA model on 4chan, LLM's technical barriers are not high and it is only relatively large.  However, the forced open-source of the LLaMA model has also allowed the open-source community to dominate the "replacement" craze for ChatGPT, in the recent period.

For example, Alpaca from Stanford, with the help of Llama's pre-training model, only uses a small-scale tuning dataset (52,000 samples) from the GPT model to build an LLM with a dialogue function. Based on the LLaMA model and LoRA (Low-Rank Adaptation of LLM, that is plugin fine-tuning) training, the open source community has successively released models such as ChatLLaMa, Alpaca, Vicuna, and Koala in less than two months, and "The actual effect of "Alpaca Family" is still catching up with GPT-3.5, not even losing to GPT-4.


With the help of the community's brainstorming and collective efforts, and ultimately achieving high-speed iteration at extremely low cost, this is undoubtedly the advantage of open-source AI. As this Google employee put it, "Who would pay for a Google product with a usage limit if there was a free, high-quality alternative with no usage limit?"


Conclusion: The Future of AI and Generative AI

Therefore, in a sense, Google's approach to Android may be the best example, which is to let the open-source community unconsciously serve its own commercial interests.


So I have to say that the leak of the LLaMA model is more like a stroke of genius, which suddenly made Meta, which was originally outdated, surpass Google and catch up with OpenAI. After all, the "Alpaca Family" was born on the structure of Meta.