💬 Deep Dive
Large language models such as ChatGPT are trained by simulating the neural network algorithms of the human brain. You might think that artificial intelligence trained by algorithms, no matter how closely it mimics the human brain, should be purely rational. However, the truth is quite the opposite—ChatGPT is not a product of pure rationality but rather an outcome that resembles sentiment.
The deeper you explore ChatGPT, the more you discover its similarities to humans; and in understanding ChatGPT, you are in fact rediscovering yourself.
1
ChatGPT is not a simple chatbot but an information processing tool—more specifically, one that presents information through a conversational interface. In the process of "input–processing–output," what truly stands out is the large language model responsible for the processing.
The term "large" in large language models refers to the vast amount of training data, reaching tens of billions to hundreds of billions of tokens. The result of this scale is the emergence of intelligence—demonstrated by ChatGPT's ability to understand vague contexts, generate coherent responses, and even exhibit a form of "creativity."
This emergence of intelligence is not based on rational deductions following formal logic, but rather on a sensory outcome simulated by brain-like neural computations. The workings of these large models are akin to a black box; even Sam Altman cannot fully explain the calculations and inferences that occur within ChatGPT, as the operation of these models follows no fixed rules.
This is the magic behind ChatGPT. Traditional natural language processing relies on explicit rules and logical reasoning, yet large language models like ChatGPT, using neural network algorithms, mine those indescribable patterns from massive amounts of data—much like the human brain perceives the world.
For example, I was recently curious about what "disco" truly is. Wikipedia informed me:
Disco originally referred to dance halls that played recorded dance music, and later came to denote a genre of dance music deriving from African-American folk dance and jazz dance in the United States, which became popular worldwide (especially in Europe) in the 1970s. This music is characterized by a blend of funk, jazz, rock, and Latin American rhythmic elements.
Every word in this definition was clear to me, yet together they left me with a vague understanding of "disco," lacking any direct sensory impression. It wasn’t until I listened to Zhang Qiang’s "Stop Asking Me What Disco Is" and "Hand-Held Tractor Skies" that I instantly grasped the rhythm and atmosphere of disco. However, even then, it remains challenging to precisely describe it in words; I can only recognize disco when I hear similar music.
Perhaps some things are beyond the reach of language and must be understood through perception alone. This is how ChatGPT operates—its understanding of "disco" is derived from extensive data training, capturing those patterns that elude verbal description. This is "intelligence"—transcending traditional rational thought, coming closer to a form of sensory cognition.
2
In models, the meaning of a word is determined by its relationships with other words.
A key technology driving the remarkable progress of GPT-style models is the Transformer architecture, which identifies relationships between words and captures deep semantic structures. It can bridge the gaps between words regardless of their order and discover connections within a text.
ChatGPT constantly uses semantics to deduce the next most appropriate word to say. It is not about letters or words alone, but about meanings—all of which are based on relationships. This is why ChatGPT can communicate in dozens of languages; it does not think in language but in semantics and relationships.
Interestingly, aren’t our identities and habits also shaped by all our interactions and relationships? On an individual level, our relationships with family and peers shape our initial personalities; on a societal level, human society is essentially a collection of relationships.
3
After undergoing "pre-training" and "fine-tuning," large models gain access to vast knowledge and learn how to use that knowledge in conversation with users. However, to truly become a reliable information processing tool, an extremely important step known as "alignment" is necessary.
Alignment means, through human feedback, guiding the model’s outputs to align as closely as possible with mainstream values and human preferences, thereby avoiding unreasonable value directions. For instance, when a user seeks comfort, the model responds with more humanistic care instead of cold, unfeeling facts. This process can be seen as one of ethical shaping, and it is precisely where different large model companies exhibit their individual characteristics.
In a sense, the alignment process of a large model is like the way we personally adapt to society. Even after absorbing a great deal of knowledge in school, when we enter a new environment or new relationships, we must continuously adjust our behavior, cognition, and even values based on feedback from people and experiences in order to meet the demands of our context.
The only constant is change, and thus both the model’s alignment and our own adaptation are never a one-time process.
Fin
Large language models like ChatGPT are, in reality, projections of our own language, culture, and thinking. Engaging in dialogue with ChatGPT is essentially conversing with the accumulated knowledge and experience of humanity.
Understanding ChatGPT is, in fact, a process of re-understanding ourselves and the world in which we live.
Sources
💎 Curated Gems
1、在平庸世俗的生活里莫名地觉得悲伤
Documentary director Chen Dongnan once said in a speech, and I’d like to quote a precise description of reality:
We may never be able to entirely avoid coexisting with tremendous pressure and limitations.
Behind trendy phrases like pursuing freedom and becoming oneself lies the possibility that only by facing overwhelming pressure and constraints can a person shape their identity and find freedom.
Following this, there will undoubtedly be one predicament after another—we must continually adapt our way of life, carrying the new form that these challenges forge, and keep moving forward.
2、On Fleeing the Inferno
After experiencing a major fire in Los Angeles, Lawrence Yeo reexamined our everyday obsessions:
I have been pondering over something we fail to achieve in daily life because we are so attached to it. Life often creates an illusion of permanence—as if everything you have today will continue into tomorrow. This illusion extends beyond material wealth to personal identity. We cling to the roles we construct, our achievements, our ongoing projects, and our sense of self. But one day—whether through a nearby fire or the end of life—these attachments will prove meaningless.
3、迎接「低欲望社会的正确姿势」
Investment expert Huang Hai, after studying the Japanese economy, summarized three opportunities in an era of low-consumption desires:
- The first is to pursue quality and cost-effectiveness, exemplified by companies like Uniqlo and Sally.
- The second is emotional value consumption, such as in pets, sports, and tourism.
- The third is health consumption. As society ages, people increasingly pay attention to their health. Against this backdrop, industries ranging from sugar-free beverages, fitness, and health supplements to healthy fast food will continue to develop.