-
Notifications
You must be signed in to change notification settings - Fork 29.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Workaround for machines that do not have glibc
>= 2.28
#203375
Comments
Hey team, just checking in to add context to Will's request above that this is going to hard-block many thousands of developers currently using VS Code. A number of leaders at Amazon are tracking this issue, we are very concerned about the impact to those customers and hopeful we can work with you on a workaround like Will suggested. |
We have documented the proper workaround at https://code.visualstudio.com/docs/remote/faq#_can-i-run-vs-code-server-on-older-linux-distributions which is also included in the error message from the requirements check.
If the alternate glibc is in the ld cache then the check should pick it up, can you please share simple repro steps I can use to repro the issue. It is possible to fix the check if there is a bug in the detection. |
Thank you, @deepak1556. We might need some help in understanding how this would work in practice. If we have a system that uses an older glibc as the default, how can we put a newer glibc in place such that only VS Code and its extensions use the new glibc and every other binary on the host continues to use the old glibc (for the sake of system stability and fidelity with production platforms running the older glibc)? |
Does this mean 1.85 will continue to receive security fixes or is it "as is" going forward? My concern here is leaving a significant developer population relying on an unsupported VS Code release. That is essentially a time bomb waiting for security to tell them they have to change up how they work because of a new security issue that won't be patched in 1.85. So ideally I can keep folks on the latest supported VS Code, even if it means I have to own the duct tape and baling wire to make it work on my org's older distro. Admittedly there's an element of needing update the distro in use, but that's not something that can change quickly in a large deployment.
AFAIK the ld cache is global, so this would effectively mean the entire system has been migrated to an updated glibc, which is not a viable option for my use case (I would just update if it was).
I built glibc-2.28 from source and install it in a secondary location ( This approach was working, but then the dependency on At this point, I could continue to build additional updated dependencies and repeat the binary patching hacking around, but I really don't want to get onto a treadmill with both the requirements checks in the CLI and in the Does that sound reasonable? Are there any other things I need to be concerned about when taking the patching approach? Like does the CLI do any checksum verification when unpacking the binaries that might rollback my changes? Do you have any other suggestions on how to approach this? I'm happy to make any contributions necessary to support my use case, but would like to get some agreement on the right approach before digging in. |
Hi VS Code PM here 👋 From my understanding you all work for Amazon @wlovo, @rexm @ShawnCarnell @bookest. If there is someone here not from Amazon then can you please share why this issue is important for your use case - thank you 🙏 Amazon folk - can you please send me an email to inikolic@microsoft.com and we are happy to meet with you next week to understand the blockers you hit while trying to do the workaround @deepak1556 provided https://code.visualstudio.com/docs/remote/faq#_can-i-run-vs-code-server-on-older-linux-distributions |
Thanks @isidorn. We've reached out over email. Looking forward to connect next week. In the meantime, do you have a planned release date for v1.86 you can share with us? |
Thank you 🙏 |
My org (not amazon) is on a RHEL7 compatible OS for the time being. It's EOL'd later this year, but not for several months if i understand correctly. I'm sure there are many people out there in a similar situation. This thread and the other bugs make it sound like this is not going to be fixed, so people using EL7 will be stranded on 1.85 until they switch OSs (not an easy or quick thing to do), sounds like. EL7 can't build glibc 2.28 from source (needs an updated gcc as well, and i'm sure many other dependencies will need updating). |
My team is in the same situation. I work in academia and my team and I develop software for operations in an astronomical observatory. All our servers are running on CentOS 7 and while we plan to update some of the external-facing gateways after EOL, I don't expect that we'll update most machines until the project is over in a couple years. My team heavily uses VSCode to develop directly on those servers and there won't be a good way for us to do this after the 1.86 release. I appreciate the need to drop support in certain situations, but I it would be great to have a solution that allows these old system to continue working even if it's with limited support or new features. |
@albireox this is fascinating, I love seeing all the different places VS Code is used, thanks for sharing! To add one workaround to the mix: if Docker can be installed in the remote machines, dev containers via SSH is a fully supported scenario. @theswit, @albireox have you considered this before? |
@deepak1556 reminded me that still in this scenario the remote SSH agent needs to run, making it not really a viable workaround. |
Thanks for developing VSCode! VSCode, especially the SSH extension, has been a real game-changer in terms of our workflow, as we often have to edit code on the fly in multiple servers and for many software products.
I considered it, but as you say I think that won't work. Our current alternative, which is not great and depends on IT being willing to do this for us, is to spin a VM more modern OS in each server and then cross-mount some of the resources that we edit with VSCode. This is not a full solution because for example most of our products are Python packages that run in virtual environments. The virtual environments won't work in the new OS since they the linked libraries are not in the same locations. So we would need to create new virtual environments for each Python product also in the new OS and try to not get too confused about what we are running where :) |
I'd also like to state our use case. We have various OS environments we must develop and being in that environment directly speeds up development significantly. They range from CentOS 5 all the way to Debian Bookworm. Currently we are unable to use VSCode remote-server in CentOS 5 and CentOS 6 environments, and losing CentOS 7 will be big loss as it creates a fragmented development process between the various systems we support (it isn't feasible to upgrade all the OS's as they are part of long life equipment). However, if vscode-server was installed either as a static binary, installed with all of it's required libraries in a virtual environment, or had the ability to determine where to find its required glibc libraries via an environment variable or config, that would not only ensure we are able to continue doing remote development in a CentOS 7 environment, but it would also allow us to open up CentOS 6 and CentOS 5. This allows us to unify our development teams' process around VSCode and the Remote SSH extension. If vscode-server was installed on a remote machine bundled with it's required libraries, that would reduce the dependencies on the remote system's OS and make a glibc change no longer mark an entire remote OS as unsupported. |
Similar as @gakio12 CentOS 7 in synchrotron, and no view to update in near future... |
@isidorn Here goes why.
Finally, Best regards, |
Dear developpers, Thank you so much for your work; I lead a team of cancer researchers and part of a consortium of ~100 people with many proudly using now VSCode for data analysis and software development. Our academic systems (HPC of research centers hosting thousands of people) do not satisfy the new minimum requirements and this new update has become absolutely disruptive for us. I hope that you understand that many HPCs, especially in academic contexts, cannot easily update their systems. We beg to please re-consider this decision or add an alternative workaround, which is different than being left behind. Thank you |
Here's the workaround I came up with as the update today broke my whole development team (we use amazonlinux2 as well). We've been dealing with various projects moving on past the glibc level of this distro and we build our own node v18 as a result. On our wsl instances and lima-vm.io vm's we can do:
this works great with limitations like the terminal doesn't work. Seems like it would be a much more simple workaround for people to just build their own node v18 binary than try to manage an alternate glibc installation which is a nightmare. Being able to npm install the dependencies like node-pty (which has a binary component that can be built on the system) would be really nice too. |
I am not from Amazon, the issue affected lots of developers here who still use RHEL 7.x machines. RHEL 7.9 is still supported by Red Hat, you should consider updating this kind of requirement after major Linux vendors ending the support to the OSs. |
Worth noting that Ubuntu 18 is still supported for 5 more years if you pay for security updates. |
Following up here as well. I work in research at MIT, and it looks like our cluster is on CentOS 7. Most research faculty and staff don't have access to/any say in what the servers run, and similar to @simozacca and some others, a system-wide upgrade doesn't seem to be on the immediate horizon---possibly in the late summer when MGHPCC shuts down. |
Here's a funner workaround I've built for myself and my dev teams that are on amazlinux2. So far everything is working for me, including the terminal, run targets, etc. Maybe it can help you out? It requires a locally build node v18 - make sure you change the path for mynode.
|
I'm a postdoc doing computational materials science, I have been using VS Code with remote-SSH for several years and am very appreciative of the efforts of the VS Code team. However, I am in a similar position as others - my work happens on a university owned cluster that runs CentOS 7. I have been (kindly) teasing the emacs/vim users in my department and encouraging them to just try VS Code. Yet unless this issue is resolved in a not excessively technical way I guess they might have been right to stick to their ancient ways. Really hope a solution can be found and thanks again for all the effort. |
Several of our users at @U-S-NRL-Marine-Meteorology-Division are reporting issues with Version 1.86 that was just released with our RHEL 7 systems. |
FYI This update has blocked all our development teams across the board. Looking for solution. |
We met the same issue. |
Chipping in as a software engineer at an MNC developing enterprise-grade software, this issue seems to affect a significant number of developers here as well. While migration to newer Linux distro versions has been planned, meticulous and extensive planning is needed especially if it involves a large number of developers distributed globally. Proper infrastructure and internal tooling support are also needed to ensure a smooth transition. As such, it might take a while until we upgrade to the latest Linux distro versions. The workaround that we are currently using is downgrading VSCode from version 1.86.0 to 1.85.2 and the "Remote - SSH" extension from version 0.108.0 to 0.107.1 and pinning them there (by disabling automatic updates). This is a little concerning since bug fixes, security patches, and new features from the latest versions of VSCode are still very much desirable. Our team (and I'm sure others around the world as well) would definitely appreciate it if a proper solution with official support is implemented. Thank you folks! |
I also work remotely at a company still using some CentOS 7 machines. VSCode remote stopped working on those unfortunately. However, CentOS 7 is only marked as end-of-life in June 2024, so it caught us off guard to have VSCode drop support before the EOL date! |
This comment was marked as spam.
This comment was marked as spam.
This workaround works for me. huge thx for ur sharing! |
"solution" for Linux users so it doesn't update. ofc the flatpak version wont have your extension, project history ,etc - but atleast you can connect :/ you can still keep your system package for newer servers |
This comment was marked as spam.
This comment was marked as spam.
Locking this issue as well. Please read #203967 (comment). We'll provide an update as soon as we have a plan to go forward from here. |
Thank you very much for your feedback and thank you for sharing your passion for VS Code and sharing how it is being used to enable various scenarios. We have discussed this more in the VS Code team and we have decided to allow VS Code to connect to an OS that is not supported by VS Code (no support for We hope this will provide the needed time for you and your companies to migrate to newer Linux distributions. VS Code will show the appropriate dialog and banner that you are connecting to an OS that is not supported by VS Code. We expect these changes to land in our recovery release that will happen later this week. Thank you |
Closing this issue as the recovery release that allows connections to an OS that is not supported by VS Code will be released on Thursday (8 Feb). Thanks again for your feedback. |
Gentle reminder for everyone on this thread. As we announced here #203375 (comment) VS Code will allow to connect to an OS that is not supported by VS Code (no support for glibc >= 2.28) until Feb 2025. Thus this flow will only be supported for 9 more months. Please reach out to your Company admins to update your Linux distributions until then. Thank you for your understanding 🙏 |
Gentle reminder for everyone on this thread that VS Code will no longer connect to unsupported OS from Feb 2025. More details in this issue #231623 |
Problem
Closely related to #201129, there will be a significant number of machines who will be unable to utilize the latest version of Visual Studio Code (VS Code) once v1.86 is made publicly available.
It will make all Amazon Linux 2 (AL2) EC2 machines unable to use VS Code, while also affecting enterprise users who may use customized or parallel versions of the
glibc
library for their own development needs.While I acknowledge the need to update components to suit the development needs of the VS Code team, it seems a bit heavy-handed to completely block any alternative paths for those who have a customized or advanced need.
There previously was a workaround for those who relied on such needs, but it seems the new script introduced to check requirements is rendering them useless.
Solution
I propose the introduction of a mechanism that allows developers to override the checks for
glibc
andlibstdc++
versions that must be explicitly requested. This could take on the form of a system variable that either disables the check or provides an alternate path to the requested libraries (e.g.VSCODE_GLIBC=/path/to/customized/library
), but may not be the only solution available.The text was updated successfully, but these errors were encountered: