I've been thinking about what macros in 4coder should be like, and I realize I have unanswered questions about what a macro system should do in a lot of cases.
The most basic use case
If I have the code:
And if I want to turn it into this code:
Then I have a fairly basic use case for a macro system. I should be able to record a macro for editing the first line and then apply it to all the lines. So how does that process look for me as I literally type?
This is not the case that confounds me, but it does raise an interesting question, which is "What is meant to happen when a macro replays a command that takes sub-inputs?" It could either be that we only replay command streams, but then this basic use case would require the user to re-type ", enter, ), enter for each application of the macro. Obviously the most useful thing would be to recreate the intention of the command. In this case the intention of the those commands are to search for " and ), no question about that. But now that I know there is some need to recreate the "intention" of each command, I can't help but wonder if I can come up with a case where I wouldn't know how 4coder should interpret the intention.
A case where I don't know how 4coder should interpret the intention
It's a little bit hard to imagine the use case that would lead to an ambiguous user intention, not impossible, but it's easier to come up with the input case that is ambiguous, and maybe we can talk about the use case that gives rise to this later.
This could mean one of two things at the "Paste" command. It could mean insert whatever string is currently at the top of the clipboard stack, which would be the easiest thing to implement. It could also mean insert whatever string was at the top of the clipboard stack when the macro was recorded. I could see the second being an easy expectation and even sometimes a useful thing to have available.
If you're not convinced that this is ambiguous, then imagine sticking the output of paste into a sub-input to an I-Search, and then copy pasting something within the macro recording (forgive me for not writing out the beat for beat inputs for this case).
My question is
What does everyone else think about macros? In the case of the paste command how should things work? I'm not exactly sure what the best way to generalize this concept is, but my theory is that this ambiguity is coming from indirection. It is unclear when the macro system records a paste command whether the the intention was to record inserting the "value" pulled from the clipboard or inserting the "reference" to the clipboard. So in the general case how should a macro system be implemented around these ambiguities?
The most basic use case
If I have the code:
1 2 3 4 5 6 7 8 9 10 | init_a_thingy(thingy_ctx, "Foo", 0, foo_func);
init_a_thingy(thingy_ctx, "Bar", 0, bar_func);
init_a_thingy(thingy_ctx, "Fiz", 1, foo_func);
init_a_thingy(thingy_ctx, "Faz", 2, faz_func);
init_a_thingy(thingy_ctx, "Bat", 1, bar_func);
init_a_thingy(thingy_ctx, "Foo Big", 3, foo_func);
init_a_thingy(thingy_ctx, "Bar Big", 3, bar_func);
init_a_thingy(thingy_ctx, "Fiz Gigantic", 100, foo_func);
init_a_thingy(thingy_ctx, "Faz Tiny", -1, faz_func);
init_a_thingy(thingy_ctx, "Bat Goofy", 10, foo_func);
|
And if I want to turn it into this code:
1 2 3 4 5 6 7 8 9 10 | {"Foo", 0, foo_func,},
{"Bar", 0, bar_func,},
{"Fiz", 1, foo_func,},
{"Faz", 2, faz_func,},
{"Bat", 1, bar_func,},
{"Foo Big", 3, foo_func,},
{"Bar Big", 3, bar_func,},
{"Fiz Gigantic", 100, foo_func,},
{"Faz Tiny", -1, faz_func,},
{"Bat Goofy", 10, foo_func,},
|
Then I have a fairly basic use case for a macro system. I should be able to record a macro for editing the first line and then apply it to all the lines. So how does that process look for me as I literally type?
- Navigate to the beginning of the line; this is important because when I don't want the recorded macro to start with some arbitrary navigation.
- Begin macro recording.
- Set Mark.
- I-Search.
- Type " .
- Press enter.
- Delete range.
- Type { .
- I-Search.
- Type ) .
- Press enter.
- Delete.
- Delete.
- Type } .
- Type , .
- Navigate to the the beginning of the next line.
- Complete macro recording.
This is not the case that confounds me, but it does raise an interesting question, which is "What is meant to happen when a macro replays a command that takes sub-inputs?" It could either be that we only replay command streams, but then this basic use case would require the user to re-type ", enter, ), enter for each application of the macro. Obviously the most useful thing would be to recreate the intention of the command. In this case the intention of the those commands are to search for " and ), no question about that. But now that I know there is some need to recreate the "intention" of each command, I can't help but wonder if I can come up with a case where I wouldn't know how 4coder should interpret the intention.
A case where I don't know how 4coder should interpret the intention
It's a little bit hard to imagine the use case that would lead to an ambiguous user intention, not impossible, but it's easier to come up with the input case that is ambiguous, and maybe we can talk about the use case that gives rise to this later.
- Navigate to a good anchor point (like before).
- Begin macro recording.
- I-Search to ( .
- Navigate one character right.
- Paste.
- Type , .
- Type <space> .
- Navigate to the next good anchor point.
- Finish macro recording.
This could mean one of two things at the "Paste" command. It could mean insert whatever string is currently at the top of the clipboard stack, which would be the easiest thing to implement. It could also mean insert whatever string was at the top of the clipboard stack when the macro was recorded. I could see the second being an easy expectation and even sometimes a useful thing to have available.
If you're not convinced that this is ambiguous, then imagine sticking the output of paste into a sub-input to an I-Search, and then copy pasting something within the macro recording (forgive me for not writing out the beat for beat inputs for this case).
My question is
What does everyone else think about macros? In the case of the paste command how should things work? I'm not exactly sure what the best way to generalize this concept is, but my theory is that this ambiguity is coming from indirection. It is unclear when the macro system records a paste command whether the the intention was to record inserting the "value" pulled from the clipboard or inserting the "reference" to the clipboard. So in the general case how should a macro system be implemented around these ambiguities?
Edited by Allen Webster
on
I think it is useful to name things more explicitly here. In Emacs, the type of macro we are contemplating is explicitly called a "keyboard macro", to underscore the fact that it is just a recording of inputs that then get replayed. I think that's a good first thing to implement, because it is clear what it does and doesn't have ambiguities.
In the future, there could be other types of macros, and those macros could get fancier. But you would still want keyboard macros for the purpose of recording and replaying input.
- Casey
In the future, there could be other types of macros, and those macros could get fancier. But you would still want keyboard macros for the purpose of recording and replaying input.
- Casey
I agree with what has already been said with regards to pasting.
Another thing I would like to bring up is applying macros to multiple lines, and what happens on "invalid" input. Imagine you have a thousand lines of the kind you have below, and you want to apply the macro to all of them. However, one of the lines might have a different format than you expected, meaning that applying the same macro to it would produce some mess. I think it is worth concidering how lines of a different format can be detected, and what the macro system should do about them.
As far as I can tell from using vims macro system, invalid lines are detected by seeing whether searches found a match or not. So, if the first I-search in your macro fails to find a ", the entire macro stops there (Note, that vim has a search-for-this-character-on-this-line function, which of course would fail more readily than a I-search which can find a quote on the next line). When applying a macro to a series of lines, this means that the macro will stop before finishing with all lines, leaving the user to see the failed line, undo, and try recording a different macro. In general, I find this to work fine and to be relatively intuitive, meaning one doesn't have to spend a lot of time on learning macros. However, I imagine one can make a better system which solves these same problems.
Another thing I would like to bring up is applying macros to multiple lines, and what happens on "invalid" input. Imagine you have a thousand lines of the kind you have below, and you want to apply the macro to all of them. However, one of the lines might have a different format than you expected, meaning that applying the same macro to it would produce some mess. I think it is worth concidering how lines of a different format can be detected, and what the macro system should do about them.
As far as I can tell from using vims macro system, invalid lines are detected by seeing whether searches found a match or not. So, if the first I-search in your macro fails to find a ", the entire macro stops there (Note, that vim has a search-for-this-character-on-this-line function, which of course would fail more readily than a I-search which can find a quote on the next line). When applying a macro to a series of lines, this means that the macro will stop before finishing with all lines, leaving the user to see the failed line, undo, and try recording a different macro. In general, I find this to work fine and to be relatively intuitive, meaning one doesn't have to spend a lot of time on learning macros. However, I imagine one can make a better system which solves these same problems.
Would it be out of question to have both value and reference variants? A key you hold while doing the operation while recording macros to tell which one was the intended one.
With the value variant you described - how would you make copy and paste inside macro behave?
You could at least make the "by value" and "by reference" be global to the macro. (Also, could you edit the copy buffer from within the macro? example - macro that goes to a specific point in a line and pushes that part to the clipboard)
Also, do you intend to support macro execution inside macros and if so, how should clipboard work there?
With the value variant you described - how would you make copy and paste inside macro behave?
You could at least make the "by value" and "by reference" be global to the macro. (Also, could you edit the copy buffer from within the macro? example - macro that goes to a specific point in a line and pushes that part to the clipboard)
Also, do you intend to support macro execution inside macros and if so, how should clipboard work there?
For your copy-paste example, I think you should paste whatever is in the buffer when the macro is executed; it should be executing the commands that were input. If someone desires to insert a large block of pre-written text than that should be done with some kind of 'snippet' system, not an adhoc fusion of keyboard macros + clipboard.
wrt 'intention', does 4coder support multiple cursors? They accomplish much the same thing as keyboard macros, but more immediately; for a short, one-off edit such as your first example I think they are better, whereas a keyboard macro is better for if you wish to save it and reuse it over and over. Example:
https://www.youtube.com/watch?v=ZmJ0JOLk-Eg
That video doesn't show it, but if you were to select text, copy, move to a different point on the line, and paste, then each cursor would have its own clipboard and your actions would do the expected thing.
wrt 'intention', does 4coder support multiple cursors? They accomplish much the same thing as keyboard macros, but more immediately; for a short, one-off edit such as your first example I think they are better, whereas a keyboard macro is better for if you wish to save it and reuse it over and over. Example:
https://www.youtube.com/watch?v=ZmJ0JOLk-Eg
That video doesn't show it, but if you were to select text, copy, move to a different point on the line, and paste, then each cursor would have its own clipboard and your actions would do the expected thing.
I'd also want to bring up the possibility of using multiple selections and cursors to solve this problem. The main benefit multiple cursors bring over macros is that it is immediately apparent if the keystrokes you're typing won't work for every line, because you'll see the changes happening to each case with instant feedback.
This is in contrast to what normally happens with macros where you slightly mess up you're keystokes or make an assumption that doesn't hold for every case and have to rerecord your macro.
I think if you want macros though it makes sense that they behave as Casey describes. Its just that for the problems you posed in the post, I think the best solution is multiple selection and cursors. For a text editor that does this well and has thought this way of use through check out https://kakoune.org/
It's not perfect, but I think its a good example of how multiple selection can be a powerful and easy to understand tool.
This is in contrast to what normally happens with macros where you slightly mess up you're keystokes or make an assumption that doesn't hold for every case and have to rerecord your macro.
I think if you want macros though it makes sense that they behave as Casey describes. Its just that for the problems you posed in the post, I think the best solution is multiple selection and cursors. For a text editor that does this well and has thought this way of use through check out https://kakoune.org/
It's not perfect, but I think its a good example of how multiple selection can be a powerful and easy to understand tool.
The problem with multiple cursors is that it is not repeatable. So you want keyboard macros either way. I agree that multiple cursors are a good way to visualize your macro as you create it, but they are not a substitute for keyboard macro recording, because you need to be able to create a macro once, then apply it many times, on arbitrary files, and at different times during your editing process.
Stated alternately, keyboard macros are the feature. Multiple cursors are a good way to visualize that feature, but they are not a superset of the feature.
- Casey
Stated alternately, keyboard macros are the feature. Multiple cursors are a good way to visualize that feature, but they are not a superset of the feature.
- Casey
I'd just want to point out that vim solved copying or pasting during a macro by letting the user copy to separate registers, so in vim you could copy while you're inside a macro to a different register than the "main" one you use, and thus prevent polluting the "main" copypasta buffer. I'm not sure if it's the best way (or if it's a good or a bad thing for interfacing with the OS), but IMO it is a thing to consider.
Emacs also allowed you to use "registers" for this purpose. I am not sure whether or not I think that is a good or bad thing, since I literally never used them.
However, I would point out that, generally speaking, what the keys do is completely up to the author of the custom layer anyway. So if you want to make a key that does something differently inside a macro vs. outside, you just could, so long as 4coder provided a call that was like "am I playing back a macro right now". And obviously, it would be trivial for any custom layer author to implement "registers" right now, no changes necessary.
Whether any of this is a good or bad idea to actually use, well, that's up to the author of the custom layer I suppose :) I can vouche for keyboard macro recording because I used it all the time in Emacs. The rest of the stuff I never used, but that may have just been because ELISP is so bad I never wanted to explore it.
- Casey
However, I would point out that, generally speaking, what the keys do is completely up to the author of the custom layer anyway. So if you want to make a key that does something differently inside a macro vs. outside, you just could, so long as 4coder provided a call that was like "am I playing back a macro right now". And obviously, it would be trivial for any custom layer author to implement "registers" right now, no changes necessary.
Whether any of this is a good or bad idea to actually use, well, that's up to the author of the custom layer I suppose :) I can vouche for keyboard macro recording because I used it all the time in Emacs. The rest of the stuff I never used, but that may have just been because ELISP is so bad I never wanted to explore it.
- Casey
When doing macros it would be nice if 4coder could spit out code that you can use to implement and customize the macro in your custom layer.
Hmm, this is all really interesting and useful so far. Thinking about it, another couple of questions comes to my mind. First do keyboard macros allow you to change between views? It seems like they would allow it and it's just up to the user to end the macro where they started if they want to be able to apply the macro over and over, but I could see some value in supporting a type of macro that only records the events in a single view.
All that thought makes me wonder what API would provide the maximum capacity for custom types of macro recording. So far I am thinking something like:
Although this doesn't have the useful concept of "is_recording()" if the core is always storing events, it will allow for arbitrarily overlapped recordings, pulling recordings out of the event stream without having to make a call, and possibly other interesting stuff. Either way, submitting macros this way allows the custom side to compose macros in all sorts of ways. Thoughts welcome.
All that thought makes me wonder what API would provide the maximum capacity for custom types of macro recording. So far I am thinking something like:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | struct Event_Record{
Event_Data event;
Mouse_State mouse; // Do I want to record mouse state, or have commands that pull from
// the mouse state just get the mouse state from the frame that invoked
// the command? I figure include this because if it does make a difference
// a user can just modify this field before submitting anyway.
// None of these would be used in the replay but might be useful data to a user
// if they want to do something like filter for events in a particular view.
Command_Function *command;
View_ID active_view;
Buffer_ID active_buffer;
};
int32_t macro_get_index(Application_Links *app);
void macro_read_events(Application_Links *app, int32_t first_index, int32_t one_past_last_index,
Event_Record *events_out);
void macro_execute(Application_Links *app, Event_Record *events, int32_t event_count);
bool32 macro_is_replaying(Application_Links *app);
|
Although this doesn't have the useful concept of "is_recording()" if the core is always storing events, it will allow for arbitrarily overlapped recordings, pulling recordings out of the event stream without having to make a call, and possibly other interesting stuff. Either way, submitting macros this way allows the custom side to compose macros in all sorts of ways. Thoughts welcome.
Edited by Allen Webster
on
I've used my own macro system inside of 4coder many times for complex search & replace and/or restructures to a code base, so the fact that the macro is NOT locked into a certain view was extremely handful for me. Of course if you implement your own system it should be easy to disable or change the behavior of certain commands during a macro recording, as Casey said, so you can say that while you record a macro you cannot move to another view. But I think this is up to the custom-layer implementer, not something to be concerned about as you design the macro system (apart for getting some info about the state of the program during the recording).
As for having an event log, sounds neat, and the handling of the context of the recording could again be done within the user scope (saving the starting event index and getting the list on the end of the recording).
cool.
As for having an event log, sounds neat, and the handling of the context of the recording could again be done within the user scope (saving the starting event index and getting the list on the end of the recording).
cool.
In Emacs, keyboard macros allowed you to switch between views, and this was in fact crucial for implementing certain operations without fuss.
For example, suppose you wanted to do a complex "cut and paste" where you took some parts of some lines in one file and reassembled those parts in another file. A common trick I would do is to open the source file on the left side and the destination file on the right side, then I would record a keyboard macro that started on the left side. It would search-mark-cut-etc., then hit the view switch, search-paste-edit-etc., then finish with the view switch to return to the right side. Repeating this macro would continue the edit job for all the lines in sequence. It was really quite powerful considering it is only just input playback!
- Casey
For example, suppose you wanted to do a complex "cut and paste" where you took some parts of some lines in one file and reassembled those parts in another file. A common trick I would do is to open the source file on the left side and the destination file on the right side, then I would record a keyboard macro that started on the left side. It would search-mark-cut-etc., then hit the view switch, search-paste-edit-etc., then finish with the view switch to return to the right side. Repeating this macro would continue the edit job for all the lines in sequence. It was really quite powerful considering it is only just input playback!
- Casey
Mr4thDimention
if the core is always storing events, it will allow for arbitrarily overlapped recordings, pulling recordings out of the event stream without having to make a call, and possibly other interesting stuff.
One feature I've kind of been dreaming about in 4coder was if it _was_ always recording, you could make commands that were like "make a macro out of what I did since the last invocation of SEARCH" or something. This would be pretty baller, because a lot of times I do a SEARCH and then I go to something and make an edit, only to realize that I really just want to keep doing that thing over and over, and now I have to stop and covert it to a SEARCH-AND-REPLACE, or whatever. So an API that let the custom layer look back over the recorded series and look for things would be pretty nice.
Another thing I might suggest, although it may be stupid, but I might recommend that macro recording just be recorded into buffers - meaning that really what you have is just an *input* buffer, and every time you type a key, it goes into that buffer. The notation would be something like:
1 | abcdefgthisismetyping\{left}\{right}\{ctrl s}fubar
|
although that is obviously crappy and has had no thought put into it, but you get the idea. Newlines would be ignored, so you can format it however you want.
Then "playing back a macro" just means that you're going to give it some region and it will interpret it in that "input language".
This way, you get everything else for free: you can edit macros after your record them, you can save them into your custom file, you can look at your prior input history and find edits you did before, whatever.
"Start recording keyboard macro" would really just be a custom binding that recorded the point in the *input* buffer where you currently were, then "Stop recording keyboard macro" would just copy the region from that point to the end, and that's the macro. But it's not a "special mode", because why would it be?
- Casey