Split Text

Splitting text into smaller chunks is a surprisingly common operation across writing, programming, and data processing workflows. The need arises any time a piece of text needs to be broken into manageable, consistently sized, or semantically meaningful segments.

One of the most frequent modern use cases is preparing text for large language models (LLMs). Models like GPT-4 or Claude have token limits, typically 4,000 to 128,000 tokens, depending on the model. When you need to process a long document that exceeds the context window, you must split it first, process each chunk separately, and then recombine the results. Splitting by sentence or by a fixed character count (e.g. every 1,000 characters) are common strategies.

Splitting by line is the go-to mode for processing CSV-like data, log files, and line-separated word lists. By-sentence splitting is useful when you want to create flashcards, extract individual facts, or paginate content with natural reading breaks. By-word splitting is useful for generating word-frequency analyses when you want N-word ngrams. By custom delimiter is the most flexible mode: splitting on a comma gives you CSV columns, splitting on `||||` or `---` gives you custom-formatted document sections.

The numbered output makes it easy to refer to a specific chunk ("chunk 14 had a problem") and the stats bar shows total chunks and average chunk size so you can tune the split parameters to produce chunks of a consistent target size.