12 trillion The estimated number of tokens used to train OpenAI’s GPT-4, according to Pablo Villalobos, who studies AI for research institute Epoch. He thinks a newer model like GPT-5 would need up to 100 trillion tokens for training if researchers follow the current growth trajectory. OpenAI doesn’t disclose details of the training material for GPT-4.
Attribution: Conor Grant, WSJ
Like this blog? You'll like my books also! Buy them at any online book retailer!
