The Original Lister API Plan
When I first set out to create a Lister API the idea was to just provide a short term hack that would hold over users who want to customize the file lists in 4coder, and who want to create similar lists for other commands. I assumed it would be temporary because I want 4coder's customization layer to expose more than just lists for user controlled UI. I wanted at least to support tables, and probably a number of other basic UI schemes, but I was convinced that to do that right the best option would be to integrate those features into the buffer system, so that there would just be a set of code paths for editing buffers and rendering them and no separate UI mode. Since that was the plan in my head I was convinced I was just going in to make a quick hack API that would be dumped in 4.1.0. Here is a rough outline of how I thought it might look, just based on writing the usage code first:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | // ... command signature/function signature
{
List_Handle list = start_list();
list_add_item(list, "first string", "little description", (void*)1);
list_add_item(list, "second string", "***", ptr_to_a_thing);
// ...
list_add_item(list, "last string", "don't ever click this option!", "last string");
void *result_ptr = execute_list(app, &view, list);
if (result_ptr == 1){
// ...
}
else if (result_ptr == ptr_to_a_thing){
// etc
}
}
|
Actually a lot of credit should go to Casey who first suggested an API direction along these lines.
This design is meant to achieve a few things. First it is very easy to create a list and the user never has to deal with the actual input processing to get back the result of the list. Inside "execute_list" all the necessary work would be done to consume future input and eventually figure out which option was selected, finally returning the user pointer paired with the item. The user pointer would just be a void* leaving it totally up to the user what that means. It could be an integer casted to (void*), a repetition of the string used for the item, a pointer to some structure managed by the user, etc. This API is made possible because 4coder commands run in a separate coroutine which can yield and wait indefinitely for input from the user, which means managing the lifetime of the user data can be as easy as using any scoped temporary storage.
In theory this could be used even for something like the file exploring list, by ending one list and starting a new one every time the user changes the path. It would look something like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | // ... command signature/function signature { for (;;){ List_Handle list = start_list(); File_List files = get_file_list(app); for (int32_t i = 0; i < files.count; i += 1){ list_add_item(list, files.info[i].name, "", &files.info[i]); } File_Info *info = (File_Info*)execute_list(app, &view, list); if (!info->is_folder){ update_path(info->name); } else{ load_buffer(info->name); break; } } } |
But there was another problem, which is that each mode of the list actually behaves a little differently when processing input and matching the user's input string to the listed items. For instance the file path system actually doesn't match the entire input line against the list items for filtering, just the file name found at the end after the last slash in the string. Furthermore in some modes it is not just about what item is selected, but also about the string the user has typed. Most modes use the same wildcarding rules, but maybe users would want to create more powerful matching rules, or disable wildcards. In some modes pressing enter always ends the list successfully and in other modes it might not end the list at all. I also considered the possibility that some custom lists would want input behaviors that I wasn't using anywhere in 4coder yet. All this made me think that maybe the user would want to see every input event and do something with it themselves.
More Complex Control in the API
Before I ever sat down to code up the Lister API I had already thought up an addition to the above API that would let the user inspect incoming events before the system processed them so that the user could write code to respond to any inputs and simulate inputs. I also wanted to separate the string that was used for narrowing the list and the string that would be shown in the text input field.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | // ... command signature/function signature { for (;;){ List_Handle list = start_list(); File_List files = get_file_list(app); for (int32_t i = 0; i < files.count; i += 1){ list_add_item(list, files.info[i].name, "", &files.info[i]); } List_Step_Result step = execute_list(app, &view, list, hot_directory, string_get_front_name(hot_directory)); if (step.in.type == UserInputKey){ // ... } if (step.user_ptr != 0){ File_Info *info = (File_Info*)step.user_ptr; if (!info->is_folder){ update_path(info->name); } else{ load_buffer(info->name); break; } } } } |
But around this time my "fish sensor" was going off. My "fish sensor" is a double slitted sensory organ on the front of my face that detected the distinct smell of something fishy. In particular I realized that either the execute_list procedure needed to return the updated string after processing keyboard input, or processing keyboard input would need to be the responsibility of the user. I also still had no good answer for how to make the matching rule customizable, and it began to dawn on me that if the user is going to write their own filtering code anyway, AND they get notified after every input in case they want to update the list, why shouldn't they just be left to pre-filter the list they are submitting themselves? Anyone who didn't want to write their own filter will just have to use some helper, and anyone who does want to customize it will be able to have a filter rule for literally anything they have the ability to compute.
User Controlled Filtering and Input
Having already considered those things when I sat down to write this API I came to the following idea:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | // ... command signature/function signature { for (;;){ List_Handle list = start_list(); File_List files = get_file_list(app); list_add_text_field(list, "Open:", hot_directory); for (int32_t i = 0; i < files.count; i += 1){ if (wildcard_match(front_of_directory(hot_directory), files.info[i].name){ list_add_item(list, files.info[i].name, "", &files.info[i]); } } List_Step_Result step = execute_list(app, &view, list); if (step.in.type == UserInputKey){ // ... } if (step.user_ptr != 0){ File_Info *info = (File_Info*)step.user_ptr; if (!info->is_folder){ update_path(info->name); } else{ load_buffer(info->name); break; } } } } |
In this version the user now sets explicitly how the text field will look. They submit all the items they want in their list according to the current state. Then they display that list and wait for input that will control how the state changes. After writing something like this I immediately realized another problem with it. The current listing system supports three methods of getting to the item you want. One method is to type in enough of it's name to get it near the top of the list. The second method is to click the item in the list. The third is to use the arrow keys to navigate a selection indicator to the item and then press enter. How was this going to support that behavior? If execute_list processes the arrow keys and keeps the necessary index then we end up in a nasty situation where the customizer and the API both handle some keyboard input. If the user was going to control it, which I would prefer, how would they indicate to the API which item needed the highlight indicator?
I thought, no problem I'll just specify a highlight for each item in the list:
1 | list_add_item(list, files.info[i].name, "", &files.info[i], (highlighted_index == i)); |
Next I was thinking about how this would interact with the mouse, which also changes the highlight values of the items by hovering and clicking them. I could let one override the other, or perhaps say the highest highlight value always wins, but I thought: What if the user could just query the mouse point and the position of the item they are placing and decide for themselves whether to highlight?
User Controlled Mouse Input and Layout
This was going to be tricky though because I was thinking that I would submit these items and let the internal system compute their layout inside the "execute_list" call, so to make this work I would have to emulate the layout on the user side to get the correct rectangle for each item. The next thought to follow that was: If the user is going to end up wanting to emulate the layout anyway, why don't I just let the user control the layout? At this point a full vision for the API formed in my head and I've been stoked about it every since.
Basically the Lister API has transformed into a UI Rendering API. You as the user simply submit a set of items and those items get rendered for you. You are still responsible for 100% of input processing. You can put a view into UI mode, take it out of UI mode, and you can set the UI data for the view. Once you've placed a view into UI mode it will display it's current UI data instead of it's current buffer. In UI mode the view will not be accessible unless you use the AccessHidden or AccessAll flags, so commands that edit or navigate the buffer will not operate (assuming they were written with the correct AccessOpen and AccessProtected flags).
The layout of the UI is specified by directly setting the rectangle each item should cover. This means the API is no longer specifically tied to listing. It could be used to build tables, hierarchical data, profile data, settings editors, etc, etc.
There are two ways you can get input to your custom UI. The first is to treat the entire UI interaction as a single command and call get_user_input to block the command until there is more input from the user. The advantage to this is that you can keep all the code controlling the UI in one place. The disadvantage is that 4coder only allows one in-flight command at a time, the user will have to cancel the operation before they can switch to another view, or if they try to click on another view the command will terminate automatically. The second option is to set the view's command map to a set of commands that understand how to manipulate the state of the UI. In the next version of 4coder, I plan to have rewritten all the old built in UIs with this API using the second method.
Won't this be a lot more cumbersome for users who just want to query the user on a list of options?
Yes this API puts a lot more work on the user, but that can now be wrapped by a helper layer. My plan is to provide a wrapper for all of this that will look a lot like the original API idea. That is still the most convenient way to write interesting new list commands. Throughout all of this my philosophy was that I don't want to have to update this API in a major way for a while as I am trying to wrap up 4.0.X this year. In order to retire it I want to make sure I only add simple general options to the core API. The final design is simpler to implement, since now the core is only responsible for saving UI data and rendering it, and it is fully general to a very wide range of possible UIs.
Progress
So far I have the new API fully implemented, with most internal core details worked out. Some points of complication were transferring the old scrolling system over to this new UI system, and preserving query bars, which were rendered through the old UI system. I have a mostly working buffer switching list that uses the get_user_input method to test out the API, but I want to rewrite it and the other list based commands using the command maps in order to preserve their behavior as much as possible. Along the way I still need to develop the wrapper layer(s), and then when I have rewritten all the old list-based features I will starting adding new ones. This is probably the biggest change in the 4coder code base since adding support for Mac, and at only one or two days a week, I expect it to be a little while before it is all ready to go.
Bye!
I hope some of you have found this interesting or insightful; if I didn't think of something important along the way, or if you have any questions, I'd love to hear about it. Thanks for reading everyone!