Source map don't refresh after source change/save & node restart. Attaching/re-attaching to Meteor.js server (meteor debug)

[johnf76]

• VSCode Version: 1.1.1
• OS Version: Linux Mint 14 64bit

Steps to Reproduce:

1. Start any meteor application via "meteor debug"
2. When node pauses, attached debugger
   {
   "name": "Attach Server",
   "type": "node",
   "request": "attach",
   "port": 5858,
   "address": "127.0.0.1",
   "restart": true,
   "sourceMaps": true,
   "outDir": null,
   "localRoot": "${workspaceRoot}",
   "remoteRoot": null
   },
3. The program pauses for debugging perfectly..
4. Add, remove or change any line of source or even change location of debugger statement and save.
5. detach debugger.
6. Break out of meteor instance (ctrl-c).
7. Start meteor via "meteor debug" (Open Source VS Code #1 above)
8. Attach debugger (gulp-symdest does not preserve links on electron #2 above)

ISSUE:
9. The program WILL pause at the new debugger location and correctly run the code changes, however the source map shows the source before modification (obviously making debugging impossible). To correct and get the source map recreated/updated I need to closing VSCode and re-starting, then re-attaching the debugger.

Side Issue: The 'F9' debug breaks don't work at all when attaching to meteor application line above, I have to use 'debugger' statements. It'd be so much more convenient use pause with F9 breaks.. F9 works just fine when launching a straight node application, not sure what the difference is or if I'm doing something wrong?

BTW, VSCode kicks ass, thanks for building this, it's so fast and nice to work with.. I've been using Webstorm for node/meteor for years, I switched, the speed of the application is night & day (debugging is where is really hit home though (except for having to type 'debugger' all over :)).

[weinand]

@johnf76 Since I do not (yet) have any experience with debugging meteor with VS Code, I could only speculate what the problem might be (and I will stay away from that until I know more details).

(BTW, a quick Internet search showed that debugging meteor server side is possible with VS Code: https://forums.meteor.com/t/using-visual-studio-code-for-meteor-development/21058)

First some facts about node debugging in VS Code:

• VS Code itself doesn't do anything with source maps; it is completely ignorant about them and sees them only as files (if they are not filtered anyway). So restarting VS Code cannot not 'recreate' or 'update' source maps. So if you are seeing this behaviour there must be something else going on that I'm not aware of.
• the node.js debugging extension of VS Code ('node-debug') manages source maps (besides communicating to the node runtime). Node-debug is a separate process that is started for every debug session that is for every 'launch' or 'attach' (A VS Code debug session is indicated through the orange coloured status bar). Node-debug tries to locate the source maps and caches them until the session ends. So if you 'attach' to meteor and then stop the session, all cached state is gone. The next 'attach' tries to locate the source maps again. So debug sessions are isolated from each other. However, if the underlying node runtime continues to run across session, it could very well keep some state across session.

What does step 6 ("Break out of meteor instance (ctrl-c)") from above mean? Does it kill the underlying node.js?

[johnf76]

@weinand Thanks for the quick response and information.

Regarding being able to use VSCode for server side debugging, yes, I am successfully debugging meteor applications and it works very well for the most part, I apologize I didn't explain that well.

From what I notice (and reading your bullet points), it seems like maybe the cache is persisting somehow? When I'm debugging (both node in debug mode & vscode attached), I detach the debugger, then kill the node process and restart (meteor rebuilds the application and runs "node /path/to/my/app.js debug"). From that point only re-attaching the VSCode debugger after closing & restarting VSCode will have the code changes reflected.

Regarding #6, yes, running "meteor debug" basically builds the applications then executes "node /path/to/my/app.js debug", for example below is the process, the screen wrapping makes it kind of hard to read. When breaking out (ctrl-c), it's just killing the node process.

/home/john/.meteor/packages/meteor-tool/.1.3.2_4.10vjklo++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/dev_bundle/bin/node /home/john/.meteor/packages/meteor-tool/.1.3.2_4.10vjklo++os.linux.x86_64+web.browser+web.cordova/mt-os.linux.x86_64/tools/index.js debug

Also, one thing to note which realized I didn't mention & don't know if it could be part of the issue, meteor runs an old version of node (v0.10.43). Their next release will be updating to v4+ (don't know the exact version).

Thanks,

John

[weinand]

@johnf76 since I'm the author of node-debug, I'm pretty sure that no cache is persisted between debug sessions ;-)

Yes, node v0.10.43 is indeed very old but I don't think that is the source of your problem.

So let's analyse what you said about the initial problem:
"the source map shows the source before modification"

A regular source map is a mapping between a generated JavaScript file and one or more source files that got transpiled into the generated JavaScript.
If you have edited the source without updating the source maps, the mapping will be wrong (wrong character locations), but the paths in the maps still point to the correct source so you will see the edited source after the modification.

Only a source map with "inlined source" can lead to the issue that you are seeing: "the source map shows the source before modification". In this case the original source files are inlined into the source maps when the source map is generated. In this case the mapping and the source are always in synch but they only reflect what's being executed if the source map has been regenerated after you have edited the source.

Could you please try this:

• if VS Code shows "the source before modification" can you verify that the editor header says something like "content from inlined source". If it does we can be sure that meteor is using source maps with inlined source.
• try to locate the source maps on disk. One way to find them is to look into the generated JavaScript. At the end of the file should be a path that points to the location of the source map.
• Verify that the source map gets updated when you edit the source and the generated JavaScript is produced. If I understand your steps from above correctly, you have to restart meteor for this.
• if you cannot find the source maps on disk it could be that the source maps are inlined into the generated JavaScript files. In this case there is no file path at the end of the JavaScript but a long data URL containing the source maps in base64 encoding.

Please let me know what you find.

[johnf76]

Thanks @weinand, I have verified as you stated. It does show "inlined content from source map" in the header. I was able to locate the source map and verified that the "app.js.map" was specified in the node source on the last line.

john@HomeOffice /media/john/ssd_storage/dev/meteor/collections $ meteor debug
[[[[[ /media/john/ssd_storage/dev/meteor/collections ]]]]]

=> Started proxy.                             
=> Started MongoDB.                           
   Starting your app                         /
Your application is now paused and ready for debugging!

To debug the server process using a graphical debugging interface, 
visit this URL in your web browser:
http://localhost:8080/debug?port=5858

I20160602-23:38:20.154(-7)? Kadira: completed instrumenting the app
Paused at /media/john/ssd_storage/dev/meteor/collections/.meteor/local/build/programs/server/app/app.js:1926

Last line of /media/john/ssd_storage/dev/meteor/collections/.meteor/local/build/programs/server/app/app.js
//# sourceMappingURL=app.js.map

Verifying the source map is recreated:

1. ls of the app & source map while paused:

john@HomeOffice /media/john/ssd_storage/dev/meteor/collections/.meteor/local/build/programs/server/app $ ls -lt *
-r--r--r-- 1 john john 6221439 Jun  2 23:37 app.js
-r--r--r-- 1 john john 3064206 Jun  2 23:37 app.js.map

2. ls after updating source, meteor detected changes and rebuilt the application and restarted itself.

john@HomeOffice /media/john/ssd_storage/dev/meteor/collections/.meteor/local/build/programs/server/app $ ls -lt *
-r--r--r-- 1 john john 6221566 Jun  2 23:41 app.js
-r--r--r-- 1 john john 3064230 Jun  2 23:41 app.js.map
john@HomeOffice /media/john/ssd_storage/dev/meteor/collections/.meteor/local/build/programs/server/app $ 

I stopped the debugger, and re-attached, it paused at the new debugger line, however as reported, it's showing the source prior to the change. I can make several changes and stop/restarts of the meteor application and VSCode debugger will only show the original source from when the debugger was initially attached.

Also, just to mention for completeness, if I completely stop the meteor process (break out of "meteor debug" which runs "node debug") and change source, then restart it via "meteor debug" (vs just changing and saving the source in which meteor (if running) catches the source change and rebuilds automatically as shown above). It actually deletes the "app" directory, then recreates is and generates the new app.js & app.js.map files. The behaviour with VSCode is the same either way, but wanted to mention just in case there might be something to it.

1. ls while paused at a debugger line.

john@HomeOffice /media/john/ssd_storage/dev/meteor/collections/.meteor/local/build/programs/server/app $ ls -lt *
-r--r--r-- 1 john john 6221439 Jun  2 23:31 app.js
-r--r--r-- 1 john john 3064206 Jun  2 23:31 app.js.map

2. I broke out of meteor, added a line to change the source, then restarted via meteor debug. I stayed in path at did ls after it restarted and nothing shows. probably a file system thing since the path I was in is now deleted, but since re-created:

john@HomeOffice /media/john/ssd_storage/dev/meteor/collections/.meteor/local/build/programs/server/app $ ls -lt *
ls: cannot access *: No such file or directory

3. I cd back into the same path and I can list the updated files.

john@HomeOffice /media/john/ssd_storage/dev/meteor/collections/.meteor/local/build/programs/server/app $ cd /media/john/ssd_storage/dev/meteor/collections/.meteor/local/build/programs/server/app
john@HomeOffice /media/john/ssd_storage/dev/meteor/collections/.meteor/local/build/programs/server/app $ ls -lt *
-r--r--r-- 1 john john 6221439 Jun  2 23:37 app.js
-r--r--r-- 1 john john 3064206 Jun  2 23:37 app.js.map

Also, just a quick side question, should the F9 break statement work when attaching and debugging a node app? I've found when attaching they don't work, but debugger statements.

If there's anything else I can help with please let me know, thanks for your help!

[weinand]

@johnf76 thanks for this excellent and detailed investigation. That made things clearer and sparked my interest to install meteor and play with it. I could easily reproduce the problem and it is not a caching problem of node-debug, but seems to be a bug in VSCode. The correct inlined source contents is fetched, but the old content is shown. Fix will appear in Alpha next week.

Breakpoints: in theory it should be possible to use breakpoints instead of debugger statements. But after analysing the generated sourcemaps and meteor's bootstrap code, I realized that this whole setup is highly adapted to the Chrome Dev tools. Making it work in VSCode requires quite some work.

Could you get breakpoints to work in node-inspector? I couldn't either.

[johnf76]

Thank you for your help @weinand, much appreciated! I just tried using breakpoints in the chrome node-inspector, same here, breaks do not work. I don't understand enough about how it all works, is the break line issue more of a problem with the generated source maps? Is something to possibly report as an issue w/ Meteor or is more of an item that can only be enhanced to work on the node-inspector / VSCode end? The debugger statements work, so it's not a big deal, but it is convenient to not have to type debugger; all over the place :)

Not quite the place for asking, but curious since you played with it for a few, what are your initial thoughts of Meteor? The biggest issue for me so far is that I really want to use TypeScript and it's not a trivial task to setup with Meteor & React, dynamic types is JS just don't feel right. :) Just curious, what kind of JS frameworks do you/others on the team like to use?

Anyway, thank you again, I'm looking forward to the update.

[weinand]

@johnf76 thanks for trying the breakpoints with node-inspector. This confirms that the Meteor generated source maps are somewhat unusual. So it would be great if you could report an issue w/ Meteor to get at least their opinion on that matter.

I did a very superficial exploration of Meteor, so I do not yet have deep insights. Meteor seems to be very smart and tries to help with an edit/compile/debug/go workflow. I did not fully understand what they do, but one thing that surprised me was that I saw 5 threads in the VS Code node debugger. Yes, I implemented the support for multiple threads in node-debug but I never saw any node.js based system making use of this.

The problem with all this magic is that it is sometimes a bit tricky to add other technologies (e.g. TypeScript) to this. So Meteor watches for file changes (edits) and builds the application, but how to integrate the TypeScript transpiler into this build process requires some insights that I do not yet have.

[weinand]

@johnf76 VS Code alpha has the fix now.