A long long time ago, at least a year ago, I decided I would contribute the code parsing system in 4coder as free and open software for our community. I released a little example of what I thought it might be like way back then on Patreon, but since then there's been nothing. I've been waiting because I just wanted to make sure that when I actually "released" something it would be really well thought through.
As of this morning I think a release lexer is ready and I have prepared the package!
The lexer supports all sorts of fancy complex features, like arbitrarily chunked source data, output rate limiting, and user controlled allocation. All with an API that leaves the user in control 100% of the time. As well as a convenience level API for making quick tools that just want to pass in a file and get out the tokens with 0 micromanagement.
Today I will make an early release of the library to all of my backers on Patreon since their support has made this side project possible. Two weeks from today I will release the library to everyone. In the mean time I have posted the 4cpp lexer documentation here* so that everyone can feel a little teased.
*The library has been officially released now, so instead of downloading the documentation, you can get the library source right here.