I'm trying to learn the ins and outs of 4coder and was wondering if someone could explain the use of stream_chunks. Do they just provide more convenient ways to work with buffers? What is their main purpose and when would you use them?
The stream chunk stuff is an optimization helper for reading buffers. You can achieve the same thing as they achieve by reading in the entire buffer into your own memory and scanning through that, or by reading the buffer one character at a time, but each of these options is sub-optimal for certain commands. Each individual read has some overhead so lots of one character reads means the overhead adds up, but reading in an entire 10K of text when you're most likely to find what you need in 100 or 200 characters means you're doing an extra big memory copy. Really 99% of the time you could just do one of the slower methods though and it would be fine. I only tried to do most things in the streaming system so that I had some examples of streams in the custom layer.