Fixed-length chunking requires no external services, yet semantic chunking absolutely needs an Embedding API — why? The core idea of semantic chunking is to split text at semantic boundaries. Determining whether "two pieces of text belong to the same topic" requires converting text into vectors and computing similarity — that's exactly what the Embedding API does. Dimension Fixed-Length / Recur
I shipped gni-compression to npm two days ago. One of the first questions I got (from myself, running benchmarks at midnight): does it work on anything other than chat data? Short answer: not yet. Long answer: I found out exactly why, and it led me somewhere more interesting than I expected. After the npm launch I ran GN against Silesia — the standard general text compression benchmark suite. Dick
Why Does Switching Embedding Models Make Such a Huge Difference? In the first four articles, we built the RAG pipeline, tuned parameters, and mastered chunking strategies. But there's one question we haven't dived into: After your documents are chunked, how do they become vectors? This process is called Embedding. It transforms human-readable text into machine-computable vectors. The choice of E