Loading Casey's prj files fail to open project

Greetings,

When I am using 4coder_casey, and I try to load a .prj file, nothing appears to happen. Specifically I get 'Permission Denied' as the error returned from errno. But comparing the permissions of the prj file to other files loaded by 4coder_custom (such as config.4coder, which does load), they are equivalent.

Is there other reasons why fopen would fail specifically in Casey's OpenProject? Are there any debugging techniques that I did not try below that I could investigate?

Thanks.


More information regarding my debugging attempts:

4coder: Version 4.0.12.1 power (the patched version, I named it .1)
OS: Windows 7 ultimate
MSVC: Visual Studio 2012 ultimate

When debugging, I find that issue is ProjectFile is null:

1
1102:    FILE *ProjectFile = fopen(ProjectFileName, "r");


I added code to catch what the error from errno would return, which is simply: Permission Denied.

There is no visible reason for why this would be the case. procexp's handle does not return anything is using it, and icacls returns:

1
2
3
4
5
D:\work\Tests\handmade_hero>icacls handmade_main.prj
handmade_main.prj BUILTIN\Administrators:(I)(F)
                  NT AUTHORITY\SYSTEM:(I)(F)
                  NT AUTHORITY\Authenticated Users:(I)(M)
                  BUILTIN\Users:(I)(RX)

I am starting 4ed as administrator, either through a cmd prompt or Visual Studio, both running as Administrator.

To further debug, I tried simply creating a small app that would open the same file, and this works. I even tested using '\\' or '/' for the slashes and again, in this small app it works.

I thought it maybe an issue with 4coder_custom, but if I switch to 4coder_default_bindings.cpp, process_config_file successfully loads.

config.4coder is loaded as rb, so I tried that with the prj file, but again, still get NULL from fopen.

The final thing I did was move the location of the prj file to the 4coder\code folder and it still failed.

What debugging techniques am I missing? Is there other reasons why fopen would fail to open as read if the user has permission, and the file exist?
Continuing to debug, I tried simply hardcoding the value (instead of using ProjectFileName), and found something...odd.

If the value of ProjectFileName differs from the hard-coded path I use, fopen works and the project is properly loaded.

However, for some odd reason, if ProjectFileName is equivalent to the hardcoded path, it fails? (fopen returns null)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
internal void
OpenProject(Application_Links *app, char *ProjectFileName)
{
    int TotalOpenAttempts = 0;

    char *Path = "D:/work/Tests/handmade_hero/handmade_main.prj";
    // ProjectFileName = "D:/work/local/tests/4coder/code/handmade_main.prj"

    FILE *ProjectFile = fopen(Path, "r");
    FILE *ProjectFile2 = fopen(ProjectFileName, "r");

    if (!ProjectFile)
    {
        // This works!
    }

    if (!ProjectFile2)
    {
        // This fails - ProjectFile2 is NULL.
    }

    // rest of function
}


Process explorer does not show any handles on either file (even after successfully using fopen). Is there a way of seeing what else is locking this file?

Is it possible that 4coder is locking the file? This won't answer why it appears to work fine for Casey, but maybe I'm missing something.

Please note that when I was trying ProjectFileName == Path, I did not care what ProjectFile2 returned. ProjectFile still was NULL.
I'm not sure what you are asking. There is a difference in your Path and ProjectFileName strings. Are you asking why one works and other doesn't? My guess would be there is mistake in folder/file name for one that doesn't work.

For looking what files some software has locked and to unlock them I recommend LockHunter utility. I use it all the time.

Edited by Mārtiņš Možeiko on
Thanks Martins. LockHunter does indeed show 4ed has locked the file before it reaches fopen.

It also shows after I successfully run fopen on the hard-coded path, that it is also locked by 4ed. Odd how procexp does not?

Anyway, other than knowing that it's locking the file, I'm not really sure where else to turn? I asked the forum instead of emailing Allen directly for this is more a custom bug than a 4coder issue; so I was trying to reach out to the community instead. Is this a 4ed bug?

Should I investigate other ways to open the project file? Is this a bug in how OpenProject works? Or more a bug on how we are trying to use it with OPEN_FILE_HOOK_SIG?

For now, I will stop debugging for a bit until I get some suggestions. I will hard-code for now, and later will try another implementation.

Edited by Brian on
The issue you may be hitting is that when you call OpenProject it triggers the system to load all the files in a directory. If the project file is inside that directory, the system then loads the project again and triggers OpenProject again. I'm not sure if that's your issue, but it seemed like a likely candidate problem to me.
Hey Allen,

OpenProject fails no matter where I put the .prj file. When I am debugging OpenProject, it is the first time the breakpoint is hit.

The original path is in my handmade hero folder: "D:/work/Tests/handmade_hero/handmade_main.prj"

But I have moved this file all over the place, such as in C:\Users\Brian\handmade_main.prj. I've been simply using the 4coder\code path for that is the project I'm working on when 4ed loads, and thus when I use cmdid_interactive_open, it's in the same folder so I do not need to navigate.

My observations of when using hard-coded path is if say I have the hard-coded path to C:\Users\Brian\handmade_main.prj, but from within 4coder using cmdid_interactive_open I navigate and open C:\Users\Brian\handmade_main.prj, that's when it fails.
I just started messing with this and I appear to be getting the same effect, so I'll probably be able to figure out what's going on soon. I am surprised Casey hasn't hit this issue... hmm... interesting.

EDIT: Alright, this was in fact a 4coder bug, and it's fixed now. I'm pretty surprised Casey hasn't complained about it yet, I wonder if he stopped using fopen in his system. In theory you can make it work by getting rid of fopen and instead read the data right out of the 4coder buffer. If you don't feel like implementing that yourself, I'll be posting a new build in about a week with all the little fixes I've been doing recently.

Edited by Allen Webster on
Thanks Allen for looking into it, and pointing me in the right direction.

Looking at the latest handmade hero (Day 349), Casey appears to be using 4.0.10. Perhaps where you noticed the change, changed since that build? It could explain then why Casey has not yet seen it.

Anyway, if OPEN_FILE_HOOK_SIG means that the file is already loaded by this time and is in the buffer, I think it's perfectly reasonable then to keep that, and us use the buffer instead of trying to open the file again.

Perhaps a suggestion is it is stated in the documentation for using OPEN_FILE_HOOK_SIG?


Due to me having a goal to learn programming, I tried to implement it myself using the buffer and I managed to get a solution. My solution goes to line 61.

Anyone care to criticize my solution? I need to add some error messages and handle looping over more than two lines in the prj file. The thing that stands out that I probably did incorrectly is the use of app->memory.

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
internal void
OpenProject(Application_Links *app, Buffer_Summary *buffer)
{
    int TotalOpenAttempts = 0;

    if (!buffer->exists)
    {
        // TODO(bk):  Handle when buffer fails and print an error
        return;
    }

    Range range = {0};
    range.min = 0;
    
    // TODO(bk):  This assumes there are no spaces in the path.  Need to support spaces.
    range.max = buffer_seek_whitespace_right(app, buffer, range.min);
    
    String buildDir = {0};
    buildDir.str = (char*)app->memory;
    buildDir.size = range.max - range.min;
    assert(buildDir.size < app->memory_size);
    
    if (!buffer_read_range(app, buffer, range.min, range.max, buildDir.str))
    {
        // TODO(bk):  it failed to read the buffer range, print an appropriate error
        return;
    }

    memcpy(BuildDirectory, buildDir.str, buildDir.size);
    size_t BuildDirSize = strlen(BuildDirectory);
    if ((BuildDirSize) && (BuildDirectory[BuildDirSize - 1] == '\n'))
    {
        --BuildDirSize;
    }

    if ((BuildDirSize) && (BuildDirectory[BuildDirSize - 1] != '/'))
    {
        BuildDirectory[BuildDirSize++] = '/';
        BuildDirectory[BuildDirSize] = 0;
    }

    range.min = buildDir.size + 1;
    range.max = buffer_seek_whitespace_right(app, buffer, range.min);

    String codeDir = {0};
    codeDir.str = (char*)app->memory;
    codeDir.size = range.max - range.min;
    assert(codeDir.size < app->memory_size);

    if (!buffer_read_range(app, buffer, range.min, range.max, codeDir.str))
    {
        // TODO(bk):  it failed to read the buffer range, print an appropriate error
        return;
    }

    // TODO(bk):  Looks like Casey wants to loop through the file and get each path and get all the files within each folder.
    char SourceFileDirectoryName[4096];
    char FileDirectoryName[4096];

    // TODO(bk):  Most likely do not need to do this and can use codeDir
    memcpy(SourceFileDirectoryName, codeDir.str, codeDir.size);
    

    // NOTE(allen|a3.4.4):  Here we get the list of files in this directory.
    // Notice that we free_file_list at the end.
    String dir = make_string(FileDirectoryName, 0, sizeof(FileDirectoryName));
    append(&dir, SourceFileDirectoryName);
    if(dir.size && dir.str[dir.size-1] == '\n')
    {
        --dir.size;
    }

    if(dir.size && dir.str[dir.size-1] != '/')
    {
        dir.str[dir.size++] = '/';
    }

    File_List list = get_file_list(app, dir.str, dir.size);
    int dir_size = dir.size;

    for (int i = 0; i < list.count; ++i)
    {
        File_Info *info = list.infos + i;
        if (!info->folder)
        {
            String filename = make_string(info->filename, info->filename_len);
            String extension = file_extension(filename);
            if (IsCode(extension))
            {
                // NOTE(allen): There's no way in the 4coder API to use relative
                // paths at the moment, so everything should be full paths.  Which is
                // managable.  Here simply set the dir string size back to where it
                // was originally, so that new appends overwrite old ones.
                dir.size = dir_size;
                append(&dir, info->filename);

                open_file(app, 0, dir.str, dir.size, true, true);
                ++TotalOpenAttempts;
            }
        }
    }

    free_file_list(app, list);

}


Alright, I've got some tips for you, first the non-4coder specific stuff:

I find this part a bit confusing:
1
2
memcpy(BuildDirectory, buildDir.str, buildDir.size);
size_t BuildDirSize = strlen(BuildDirectory);

I might be misunderstanding something subtle, but it looks like you already have the size for the buildDir string, but then you copy it into BuildDirectory and strlen it. That's a bit odd, just because if you know the size of the string already, you could just do
1
2
size_t BuildDirSize = buildDir.size;
BuildDirectory[BuildDirSize] = 0;

My concern is not just that this might be slow, but the strlen might find the wrong length if there isn't already a null terminator in the correct location. If I am missing some subtle point here then please set my thinking straight!

Now for a few 4coder specific tips:
1
codeDir.str = (char*)app->memory;

Is totally fine and as long as you only ever need one piece of temporary memory, that is the easiest way to do it. If you want to get fancier, there is also a global_part which is just a memory arena for when you need to get a little fancier, but in this case I think you did it the best possible way.

1
2
// TODO(bk):  This assumes there are no spaces in the path.  Need to support spaces.
range.max = buffer_seek_whitespace_right(app, buffer, range.min);

There is already code in the 4coder default include that does a seek to the first newline character, although it's API might not be set up to match this, which is unfortunate (and I should fix that), but it could serve as a starting point for you to make such a procedure for yourself. Look for "seek_end_of_line" or something like that.

Sorry, I don't generally read this forum but I probably should :) Yes, Casey did hit this problem, but Casey just decided to rewrite that routine to stop using fread and start using the buffer contents directly. The problem was that 4coder now seems to have those files opened in some kind of exclusive mode, so you cannot fopen them when 4coder has them open. My guess is that maybe 4coder always opened them in exclusive mode, but maybe it had simply closed them already at the time, or something like this?

Anyway, I rewrote the PRJ loader to just use the contents of the buffer instead. This is better anyways, since now it is more portable since it doesn't depend on the filesystem.

- Casey