Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Requires >500Mb of memory to install rustc #2145

Closed
teknopaul opened this issue Dec 5, 2019 · 9 comments
Closed

Requires >500Mb of memory to install rustc #2145

teknopaul opened this issue Dec 5, 2019 · 9 comments
Labels
enhancement

Comments

@teknopaul
Copy link

Describe the problem you are trying to solve
I was trying to install rustc on a nano micro VM with only 500Mb of RAM, the installer runs out of memory, if I install on a machine with more RAM and rsync the files using rustc works well .

Describe the solution you'd like
Either working in low mem environments or publish the minimum system requirements. Currently it just crashes without reporting the cause.

I still have this VM so let me know if I can do anything to help testing the issue.
@kinnison
Copy link
Contributor

kinnison commented Dec 5, 2019

This is related to a conversation we had here: #2128 (comment)

We would need a way to detect the available RAM limits in a clean way to automate this; otherwise users of deliberately limited VMs need to set RUSTUP_UNPACK_RAM to something suitable in their profile.

@rbtcollins
Copy link
Contributor

rbtcollins commented Dec 5, 2019 via email

@hanusek
Copy link

hanusek commented Jan 27, 2020

I have the same problem - a embedded device with 256 MB RAM. How to solve it?

@rbtcollins
Copy link
Contributor

rbtcollins commented Jan 27, 2020 via email

@teknopaul
Copy link
Author

I managed to install on a machine with more ram and copy the files across.

@daveorzach
Copy link

daveorzach commented Feb 29, 2020 •
edited
Loading

I successfully installed rustc after adjusting the unpick size
export RUSTUP_UNPACK_RAM=200000000

@kaleidawave
Copy link

I successfully installed rustc after adjusting the unpick size
export RUSTUP_UNPACK_RAM=200000000

Haven't managed to get this working. Fails with thread 'main' panicked at 'RUSTUP_UNPACK_RAM must be larger than 220000000', src/dist/component/package.rs:200:13

And still 220mb is too large for my environment... so it seems there is no way to install rustc...?

@kinnison
Copy link
Contributor

We are working toward making it more possible, but frankly if you're not able to give up 220M to install rust, you're not going to be able to build very complex programs.

rbtcollins added a commit to rbtcollins/rustup.rs that referenced this issue Apr 4, 2021
@rbtcollins
Stream large files during unpacking
056905a 
Fixes rust-lang#2632, rust-lang#2145, rust-lang#2564

Files over 16M are now written incrementally chunks rather than buffered
in memory in one full linear buffer. This chunk size is not
configurable.

For threaded unpacking, the entire memory buffer will be used to buffer
chunks and a single worker thread will dispatch IO operations from the
buffer, so minimal performance impact should be anticipated (file
size/16M round trips at worst, and most network file systems will
latency hide linear writes).

For immediate unpacking, each chunk is dispatched directly to disk,
which may impact performance as less latency hiding is possible - but
for immediate unpacking clarity of behaviour is the priority.
@rbtcollins rbtcollins mentioned this issue Apr 4, 2021
Merged
rbtcollins added a commit to rbtcollins/rustup.rs that referenced this issue Apr 5, 2021
@rbtcollins
Stream large files during unpacking
a6e9176 
Fixes rust-lang#2632, rust-lang#2145, rust-lang#2564

Files over 16M are now written incrementally chunks rather than buffered
in memory in one full linear buffer. This chunk size is not
configurable.

For threaded unpacking, the entire memory buffer will be used to buffer
chunks and a single worker thread will dispatch IO operations from the
buffer, so minimal performance impact should be anticipated (file
size/16M round trips at worst, and most network file systems will
latency hide linear writes).

For immediate unpacking, each chunk is dispatched directly to disk,
which may impact performance as less latency hiding is possible - but
for immediate unpacking clarity of behaviour is the priority.
@rbtcollins
Copy link
Contributor

Fixed by #2707

rbtcollins added a commit to rbtcollins/rustup.rs that referenced this issue Apr 27, 2021
@rbtcollins
Stream large files during unpacking
551cc21 
Fixes rust-lang#2632, rust-lang#2145, rust-lang#2564

Files over 16M are now written incrementally chunks rather than buffered
in memory in one full linear buffer. This chunk size is not
configurable.

For threaded unpacking, the entire memory buffer will be used to buffer
chunks and a single worker thread will dispatch IO operations from the
buffer, so minimal performance impact should be anticipated (file
size/16M round trips at worst, and most network file systems will
latency hide linear writes).

For immediate unpacking, each chunk is dispatched directly to disk,
which may impact performance as less latency hiding is possible - but
for immediate unpacking clarity of behaviour is the priority.
rbtcollins added a commit to rbtcollins/rustup.rs that referenced this issue Jul 11, 2021
@rbtcollins
Stream large files during unpacking
8cf07b8 
Fixes rust-lang#2632, rust-lang#2145, rust-lang#2564

Files over 16M are now written incrementally chunks rather than buffered
in memory in one full linear buffer. This chunk size is not
configurable.

For threaded unpacking, the entire memory buffer will be used to buffer
chunks and a single worker thread will dispatch IO operations from the
buffer, so minimal performance impact should be anticipated (file
size/16M round trips at worst, and most network file systems will
latency hide linear writes).

For immediate unpacking, each chunk is dispatched directly to disk,
which may impact performance as less latency hiding is possible - but
for immediate unpacking clarity of behaviour is the priority.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@rbtcollins @teknopaul @kinnison @daveorzach @hanusek @kaleidawave