We are currently in the process of converting the website to the new design. Some pages, like this one, are still broken. We appreciate your patience.
4coder»Forums
Jens
54 posts / 1 project
Co-founder and programmer of GameLoom Studio
Extending Lexer
Edited by Jens on Reason: Further investigation
One of the features I find useful in C++14 is the number literal syntax, where you can add single quotes that are ignored by the compiler. Like 0xAABB'CCDD'EEFF or 1'234'567. 4coder doesn't support this out-of-the-box so I decided to have a stab at extending the lexer.

After adding the missing ../build folder, fixing the compilation error (conversion warning was treated as error) and fixing the read path in 4coder_cpp_lexer.cpp I managed to run the lexer test. But it crashes while reading line 414 (or later, not sure if print buffer got flushed)

1
2
3
4
#error ``` Foo bar is bad news bears! ```
#   define foo bar
#    define bar baz
#define baz bish


All output looks good until that line.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
[ 2475,  2510)        LiteralString /       PPErrorMessage : 0x0001 / 0x0000
	: ``` Foo bar is bad news bears! ```:
[ 2510,  2511)           Whitespace /           Whitespace : 0x0000 / 0x0000
	:
:
[ 2511,  2521)         Preprocessor /             PPDefine : 0x0001 / 0x0000
	:#   define:
[ 2521,  2522)           Whitespace /           Whitespace : 0x0001 / 0x0000
	: :
[ 2522,  2525)           Idexited with code -1073741819


Should this be working or is the lexer code generated using some other method?

I'm using 4coder version 4.1.6.


Edit:

As a side-note, building the lexer, running it and rebuilding the custom layer; it does actually lex the literals properly. I guess the crash is just a false negative.
Simon Anciaux
1363 posts
Extending Lexer
If you still need help with this, you might want to contact Allen directly on discord.
Jens
54 posts / 1 project
Co-founder and programmer of GameLoom Studio
Extending Lexer
Everything seems to be working except that the lexer test crashes, but as it works I'm happy.

It was really simple to be honest, to get ' supported (the 123'4567-styled literals) all I had to do was to add ' to the places "0124.." was in 4coder_cpp_lexer_gen.cpp. Technically that's not 100% correct since 0x'aa'bb is an invalid literal but the compiler will catch that one.

For binary literals I more or less just copy-pasted the hex version and replaced x with b :D