Schedule for New Builds

Heyo everyone!

I've been a bit quiet on the blog here as my life has been in a state of constant change since January. I did some 4coder Friday streams in February and March in which I established a testing system for 4coder - something it has desperately needed for about a year. I didn't do much besides respond to questions and bug reports in April, but I did have time to think over exactly how I want the next few steps forward to look. So here's what I am thinking will come next for 4coder:

I. The last few features for 4.0.X.
Feature creep is a basically never ending process, so the wisest thing to do might be to say "no more features until the new engine is ready!". But I have gone for quite some time without any new editing features, and after all that time I've built a little list of things I can do without *too* much difficulty that will be helpful enough to myself and/or other users that they are worth getting in now, before new features become mostly impossible for a while. These are the last features of 4.0.X.

1. A project file upgrade
The project system has become one of my favorite parts of 4coder, but it needs a few tweaks.

First it needs to be possible for a project to specify a list of directories where code lives. Right now a project just loads everything in the same directory as the project, and recursively everything below that. However as code bases grow, it becomes handy to split code up for different layers of reusability, different frequencies of rebuild, etc, and this can easily lead to cases where the current project system simply doesn't allow you to have one project for your entire code base.

Second the current project system's most powerful feature is the ability to make a single list of common command line operations and bind each one to a single key, but this system could be even more useful with a little reorganization. For starters as a project starts to feature splits in the code, for reasons like different layers of reusability, and different rebuild/rerun frequencies, there start to be a lot of "common" commands. Sometimes you might go for a month without ever rebuilding a particular utility and then one day you are rebuilding it. In the current system I find I either need lots of little projects, or a single big project that I edit and reload whenever I do need to change which commands I am using. Additionally for cross platform use, the project commands are a great way to make an environment practically identical everywhere, but it takes more manual upkeep because of the way it is organized than it would under the new system I have in mind.

2. An ad-hoc list viewer system
A very common request from users is the ability to make custom lists like the file lists that allow you to select one item and that automatically filter towards the text you already have typed. I have resisted just adding in an API for this for a while because any API based on the 4.0.X engine for solving this will be deprecated very early on, if not right away in the new version. However there are a lot of factors in it's favor. One, the new version is a ways off and so we would still get usage out of it for a reasonable period of time. Two, there are a lot of very useful things we can do with this feature. Three, it is not all that difficult to support a very ad-hoc specialized API for doing just this in the 4.0.X engine. Therefore I think the value to cost ratio actually says I should do it.

Using this API the default bindings would get such features as: Execute any command by name with a narrowing list of options. Jump to procedure/type definition with a narrowing list of options. Execute unbound project commands (related to my comments about the new organization of the project system). Convert jump list (error buffer, find all locations, etc) into a narrowing list of jumps.

3. Undo Redo upgrade
Frankly an Undo Redo upgrade has been on the Todo list for way too long (I don't apologize for trying to make that into a pun). There are basically two major things I need to change in Undo/Redo.

First, right now the code is just a mess from lots of experiments that were eventually cut but never cleaned up, and this is holding the system back from accomplishing some relatively easy tasks. In particular there is an extra history buffer that I tried messing with for a while, but I decided it was too bug prone to keep around... but I never actually took it out, I just got rid of commands that used it. This extra history complicates the system in a lot of ways. Without it it would be trivial to implement an Undo Redo customization API that would allow, not only for more creative commands relating to Undo Redo, but also for features such as a multi-buffer scoped undo, and smarter recovery of lost "branches" (if the user wants to implement branches... I'm not planning to do anything like that myself).

Second, right now the way a user has to group edits together into a single unit for the purpose of Undo Redo is by submitting it as a "batch edit" where all the individual edits are specified in an array. Sometimes this is convenient but sometimes it is not. I am not sure whether I want a wrapper API around batch edits, or a grouping system embedded in the history system, but something needs to be done to make it easy for arbitrary sequences of edits to be grouped in the history.

II. The 4coder keyboard input structure
I am separating this from the feature list, to indicate that if you are not someone who wants to rewrite your binding layer, you won't have to, because I am not making this change until after the previous features are in place. The current keyboard input system is flawed, as any users know who have tried to use 4coder with a non-english keyboard know. Since my testing system relies on a stable data format for input events, and I have realized I need to change my input data format, I want to get this fixed ASAP before I invest too deeply in building lots of tests. Therefore I am fixing the primary flaw of the 4coder input system, which is conflation of key events with text input events. I have done enough research and practice with keyboard input to understand what it *really* takes to support arbitrarily complex text input systems while also having command bindings that are independent of keyboard layout.

When I make this change I am hopeful that I will also be able to make a code updating guide to help users quickly switch to the new method of specifying bindings. This is a change that has to happen sooner or later and at this point the more I put it off, the worse the situation gets.

III. Introductions, Tutorials, How Tos, Documentation
4coder's greatest weakness right now is that new users have to ask for help a lot, especially if they want to get into maintaining their own customization. This not only means more work for me, but also probably acts as a barrier to new users who would otherwise get a lot out of 4coder. The forums here on handmade network have done a great job of reducing the strain on me to support 4coder already, but in order to free me up to work on the new 4.1.X engine, I want to have a full corpus of materials to point people to so that I never have to look up answers again. Upgraded documentation in particular will also be useful because I myself rely on my own documentation, and having a more powerful documentation system in place will be a great asset for me.

This stage will also involve getting the 4coder Wiki up and running with example customizations from the community. I have already talked with a few users who will contribute their customization code to these articles, and I am open to adding more if anyone else wants to be on the 4coder Wiki and has an interesting customization to share.

The idea of a 4coder discord server or channel has also been proposed not long ago, and I haven't fully decided what I will do in this direction yet, but I am taking it seriously. I will make a final decision and setup whatever I decided during this stage.

Time Line
I know people love to hear time lines, and putting a rough estimate of time on something does help me to consider more concretely the value vs time trade offs. I think it is a modest prediction to say that this schedule should be seen to completion by the end of 2018. I originally wanted to get a "beta" ready by the end of 2018, but that plan involved rushing this 4.0.X retirement and I'm getting a little fed up with rushing things out and then having to patch my mistakes a week later. It's time to be a grown up and get this thing polished up right, and that takes real time.


Thanks everyone - see you around!