It looks like the next build of 4coder (4.0.29) is going to be ready sometime in the next few weeks. The new build has been in development for a couple months and is loaded to the brim with new features that have all gone through interesting architectural and algorithmic design that I believe are worth sharing for several reasons. One it will prepare 4coder users who want to start writing customizations for how to think about the new features. Two it will help anyone who likes to think about the processes of architecture and algorithm design with some examples of my own process. Three it might expose me to some criticisms or suggestions that could help me improve the specifics of the new 4coder features before I put them into the wild.
Directory of all parts:
- Memory Management Overview
- Memory Management Variables and Objects
- Memory Management Scopes
- Custom UIs and Various Layers for Lister Wrappers
- Custom Cursors, Markers, and Highlights, and the Render Caller
Memory Management Overview
The topic for part one is the new API for easing lifetime management of data created and used by the custom side. This becomes a problem for implementors of custom layer code for a couple of reasons. The first reason this is an issue is that, by design, it is assumed the core can create and destroy views and buffers without the custom side issuing the create/destroy operation, although right now it turns out this only happens at startup when the core creates *scratch* and *messages*, I want to design the API in such a way that, if new cases arise where the core creates or destroys objects, customizations that were written to the API still function correctly.
1 2 3 4 5 6 7 8 | // We want to avoid making every module ever maintain it's own
// lookup tables and write it's own hooks like this:
END_OF_BUFFER_HOOK(cleanup_my_perbuffer_memory){
Attachment *attachment = lookup_attachment_by_id(buffer_id);
if (attachment != 0){
free_attachment_by_id(buffer_id);
}
}
|
The second reason is that even if the custom layer does issue the operation to create or destroy an object, the code that issues that operation should not be forced to be aware of every sub-system present in the custom layer that is interested in creation/destruction of each object. In the most extreme case, imagine you are composing your customizations and you use several openly shared modules that you were written by different authors, and imagine these modules have never before been used together and the authors never spoke to one another. Suppose one of the modules provides features that need to track information on a per-buffer basis, and so this module wants to release information that it has stored whenever a buffer is destroyed. Then suppose the other module provides features that automates opening and closing sets buffers. If the first module was written with the assumption that every other part of the custom layer would be aware of it's presence and call into it when a buffer is created or destroyed, then in order to compose these two modules you will have to manually edit the second module to do the communication the first module requires.
1 2 3 4 5 6 7 8 9 10 11 | CUSTOM_COMMAND_SIG(close_buffer_set)
CUSTOM_DOC("Closes all buffers in the currently active buffer set")
{
for (int i = 0; i < current_set->count; i += 1){
kill_buffer(app, current_set->buffer_ids[i], 0);
// We don't want users to have to manually insert
// stuff like this all over the place
// (wether or not they own the code for both sides).
notify_buffer_killed(app, current_set->buffer_ids[i]);
}
}
|
I chose this as our first topic because it turns out managing this data is foundational to both of the other big things that are included in this build, and even outside of the other new features, solving this particular problem has proven to be important to almost everything in the custom layer in a foundational way, and it has the most interesting architectural and algorithmic design problems, so if someone was only going to read one part, I would want it to be this one.
API Summary
The solution presented in this series is a memory management API, but it is not like any garbage collection, smart pointer, or reference counting system, which I suspect are the sort of things that spring to mind when I say memory management. The design of this system follows directly from the kind of things I was manually doing in the custom layer for features like sticky jumps, previous word-complete state, and persistent per-view states like smooth scrolling and previous paste index, and it fits some patterns I tried to use in the core for tracking per-view-buffer cursor, mark, and scroll states.
Basically, across all these systems, I frequently find myself wanting to allocate memory and set variables tied to a particular buffer, or to a particular view, so that in the future if I have a handle to the view and I know what variable on that view I want to query I can get back whatever I set before, or so that if I allocate some memory that is only relevant to a particular view or buffer, I don't have to register a hook that hears about every buffer that ever closes and checks if there is memory that needs to be freed. This leads to three major types of things I want to talk about. Some things are "variables", they have a name that can be pulled out of "thin air", that is the person who sets the variable doesn't have to pass anything to the person who gets the variable later. Other things are "objects", they have a variable sized amount of memory, and to access them you do so through a handle that was returned when you created the object. Finally there are "scopes", which are the things to which variables and objects can be tied.
1 2 3 | typedef int32_t Managed_Variable_ID; typedef uint64_t Managed_Object; typedef uint64_t Managed_Scope; |
In the next two parts I will go into each of these types and the sort of problems they help solve, and how they interrelate to each other. Then we will look at other new APIs that benefit from the presence of this memory management system.