pserve --reload in 1.6 is less reliable than in 1.5

[hyperknot]

Somehow in 1.6 the pserve --reload is not as reliable as it used to be in pre-1.6.

Bug:
Pserve can get into a mode, where it forgets about files, and no matter if the file has been saved and resaved, it just doesn't want to reload it. I cannot reproduce it yet, but it happened after I had a NameError in a try-except block. Pserve didn't give any warning in that case, it just 'forgot' about that particular file's updates. It's also possible that one of the savings of the file was during a reload process or something timing related.

I had to manually stop the process with Ctrl + C and start it again, to get pserve back into normal mode.

[stevepiercy]

@hyperknot not all files cause the server to restart. For details see http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/project.html#reloading-code

Provide a link to your project repository, and identify which files do not cause the server to restart and which you think it should do so.

[hyperknot]

@stevepiercy I understand that, however it is not the case here:

• it's a normal Python file in the project, which normally triggers a reload
• it always triggered a reload in 1.5
• once I restarted pserve manually 1.6, the same file kept triggering again

[digitalresistor]

The new reloader won't reload a file after it has had an error unless the file has changed on disk. If you notice the error, change it and save while it is still checking the rest of the files, you may cause a race condition whereby the reloader won't notice your file has changed because it was still busy checking the other files (this is strictly theoretical based upon what I remember from reading the code, I have not verified that this is the case).

We may need to do additional verification that the file hasn't changed while the reloader is active.

[mmerickel]

Does it pick up the file again if you re-save it (with no syntax errors of course)?

[hyperknot]

One thing which was definitely missing in my case was the warning message, which should be displayed normally (based on the source code). It might be timing related or because my error was inside a try-except block (just an idea).

I saved the file at about 3x times afterwards I fixed it, but it didn't pick it up.

[tseaver]

From the description (it used to reload under pyramid < 1.6), I'm assuming that the NameError was occurring at runtime (i.e., inside a function body), rather than during module import. @hyperknot, can you at least post the snippet which contains that function?

[mmerickel]

When I was implementing this feature you can see in #2049 that I ran across a different (but maybe similar timing-related issue) where the reloader got into a loop, thinking that the file had been modified during every poll interval. I haven't been able to reproduce either that issue or this one with the final version of the code though.

[hyperknot]

@tseaver It was happening at runtime, like:

def upload(request):
    try:
        clean_data = clean(json_body['data'])
    except Exception as e:
        logger.error('error: {}'.format(e))

@mmerickel I've just run into a much simpler case, reload was not detecting any change in __init__.py. There was no syntax error or anything, I was just changing a setting and was really surprised that nothing was updated. After many tries, I realised again that I have to Ctrl + C the process, and start it again. After restart the reload detection works again.

I'm on OS X 10.9.5, Python 2.7.11.

[mmerickel]

related #2250