Have you ever wondered how ChatGPT can create new words even though it has a fixed vocabulary? Or what the longest word in its vocabulary is? 🤔

Here are the answers. 🤓

Tokens Instead of Words

To understand this, we first need to look at the concept of tokens. ChatGPT doesn’t work with words directly but with tokens—small units of characters that are often shorter than a word.

ChatGPT’s vocabulary consists of 100,277 tokens, including special tokens used for commands (e.g., to mark the end of a text). On average, an English word is split into 1.34 tokens, while a German word is about 1.78 tokens long. For example, the word Captain doesn’t exist as a single token but is broken down into “capt” and “ain”.

How Does ChatGPT Create New Words?

Since the model works with individual tokens, it can generate new words by combining existing tokens. This is how creative neologisms emerge. The smallest tokens are individual characters—so in theory, ChatGPT could also “invent” completely new terms by stringing together random characters.

How Long Can a Word Be?

This depends on the context length. ChatGPT can process a maximum of 4,096 tokens at once—so in theory, a single word could be that long! But practically speaking, that wouldn’t make much sense.

And the Longest Token in the Vocabulary?

The answer is unexpected: It’s a token consisting of 128 spaces. 😄 (Token ID 58040)