Generator
Show Probabilities
What are Markov Chains?
A Markov chain predicts the next state based only on the current state, ignoring history. For text generation, we use n-grams (sequences of n characters or words) to predict what comes next.
This generator builds a probability model from the input text, then generates new text by randomly selecting the next character or word based on learned probabilities.
Chinese text has no spaces between words, so a word n-gram model would otherwise treat the whole input as a single token. This generator detects Chinese text and tokenizes it by character instead, joining the generated output back together without spaces.