One of the features I find useful in C++14 is the number literal syntax, where you can add single quotes that are ignored by the compiler. Like 0xAABB'CCDD'EEFF or 1'234'567. 4coder doesn't support this out-of-the-box so I decided to have a stab at extending the lexer.
After adding the missing ../build folder, fixing the compilation error (conversion warning was treated as error) and fixing the read path in 4coder_cpp_lexer.cpp I managed to run the lexer test. But it crashes while reading line 414 (or later, not sure if print buffer got flushed)
All output looks good until that line.
Should this be working or is the lexer code generated using some other method?
I'm using 4coder version 4.1.6.
Edit:
As a side-note, building the lexer, running it and rebuilding the custom layer; it does actually lex the literals properly. I guess the crash is just a false negative.
After adding the missing ../build folder, fixing the compilation error (conversion warning was treated as error) and fixing the read path in 4coder_cpp_lexer.cpp I managed to run the lexer test. But it crashes while reading line 414 (or later, not sure if print buffer got flushed)
1 2 3 4 | #error ``` Foo bar is bad news bears! ``` # define foo bar # define bar baz #define baz bish |
All output looks good until that line.
1 2 3 4 5 6 7 8 9 10 | [ 2475, 2510) LiteralString / PPErrorMessage : 0x0001 / 0x0000 : ``` Foo bar is bad news bears! ```: [ 2510, 2511) Whitespace / Whitespace : 0x0000 / 0x0000 : : [ 2511, 2521) Preprocessor / PPDefine : 0x0001 / 0x0000 :# define: [ 2521, 2522) Whitespace / Whitespace : 0x0001 / 0x0000 : : [ 2522, 2525) Idexited with code -1073741819 |
Should this be working or is the lexer code generated using some other method?
I'm using 4coder version 4.1.6.
Edit:
As a side-note, building the lexer, running it and rebuilding the custom layer; it does actually lex the literals properly. I guess the crash is just a false negative.
Edited by Jens
on
Reason: Further investigation
If you still need help with this, you might want to contact Allen directly on discord.
Everything seems to be working except that the lexer test crashes, but as it works I'm happy.
It was really simple to be honest, to get ' supported (the 123'4567-styled literals) all I had to do was to add ' to the places "0124.." was in 4coder_cpp_lexer_gen.cpp. Technically that's not 100% correct since 0x'aa'bb is an invalid literal but the compiler will catch that one.
For binary literals I more or less just copy-pasted the hex version and replaced x with b :D
It was really simple to be honest, to get ' supported (the 123'4567-styled literals) all I had to do was to add ' to the places "0124.." was in 4coder_cpp_lexer_gen.cpp. Technically that's not 100% correct since 0x'aa'bb is an invalid literal but the compiler will catch that one.
For binary literals I more or less just copy-pasted the hex version and replaced x with b :D