By JIANG Jingling, LI Jinya
ChatGPT, an artificial intelligence-powered chatbot created by OpenAI, is the biggest thing on the internet. Five days after its November 30 release, over a million people used the bot to write poems and songs, debug computer code, and write TV scripts. High school kids cheer for the death of SAT essays. Journalists and programmers are bracing for the end of their jobs.
Sure, artificial intelligence can win at Go, recommend Tiktok videos, and tell cat photos from dog photos. But the progress in large language models (LLM) has been slower than expected. The term refers to gigabytes of datasets that generate sentences with tone and style.
OpenAI started in 2015 with the backing of some of the biggest names in Silicon Valley – Musk, Altman, Brockman and Google Brain’s Ilya Sutskever. It is a leader in the LLM field, alongside Google’s DeepMind.
GPT is short for Generative Pre-trained Transformer. It is the jewel in the OpenAI crown. The idea is to train the AI on a diverse corpus of unlabeled texts - thousands of books- then fine-tune it according to the task. GPT-2 came out in 2019 and used larger data sets and more parameters, but was quickly superseded by GPT-3, which basically read the internet, and shows uncanny language facilities. Since last year, the internet has been plastered with GPT-3 movie scripts and computer code.
ChatGPT is even better. It trains itself while chatting with humans and learns from context. GPT-3 occasionally produced nonsense that had the right tone and style, whereas ChatGPT doesn’t. It answers open questions, writes decent essays, and debates serious issues.
Another OpenAI product is DALL-E2, which conjures “Salvador-Dalí” pictures from written prompts. Dall-E2 and ChatGPT are the same strain of AI-generated content (AIGC). Two of OpenAI’s recent acquisitions – podcast editor Descript and note-taking app Mem – are in this field.
On the release of DALL-E2, Sam Altman wrote that “a decade ago, the conventional wisdom was that AI would first impact physical labor, and then cognitive labor, and then maybe someday it could do creative work. It now looks like it’s going to go in the opposite order.”
The very fact that ChatGPT is free to test shows OpenAI’s determination to perfect its model – more users mean a more extensive training set – but eventually, someone, either investors or users, has to foot the bill. This is a concern for OpenAI who has already poured a billion into GPT-3.
ChatGPT, unsurprisingly, has its detractors. Statckflow, a Q&A site for programmers, has temporarily banned comments generated by ChatGPT over concern that it will produce too many seemingly correct answers that are actually wrong.
The debate is only heating up, but startups that put safeguards on AI abuse, or at least claim to, are already gaining traction. OpenAI employee Dario Amodei’s Anthropic, which aims to make AI more “reliable, interpretable, and steerable,” has raised more than US$700 million in less than two years.