Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tmpfs var/log] mount /var/log as tmpfs for some platforms #2780

Merged
merged 1 commit into from
Apr 15, 2019

Conversation

yxieca
Copy link
Contributor

@yxieca yxieca commented Apr 12, 2019

- What I did

SONiC is a heavy writer to /var/log partition, we noticed that this
behavior causes certain flash drive to become read-only over time.
To avoid this issue, we mount /var/log parition on these devices as
tmpfs.

  • Mount /var/log as tmpfs
  • /var/log default size is 128M
  • Adjust size according to existing var-log.ext4 file size.
  • Adjust size to between 5% to 10% of total memory size.

Signed-off-by: Ying Xie ying.xie@microsoft.com

SONiC is a heavy writer to /var/log partition, we noticed that this
behavior causes certain flash drive to become read-only over time.
To avoid this issue, we mount /var/log parition on these devices as
tmpfs.

- Mount /var/log as tmpfs
- /var/log default size is 128M
- Adjust size according to existing var-log.ext4 file size.
- Adjust size to between 5% to 10% of total memory size.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
# NOTE: certain platforms, when reaching initramfs stage, have a small
# limit of mounting tmpfs partition, potentially due to amount
# of RAM available in this stage. e.g. Arista 7050 QX32
[ X"$aboot_platform" = X"x86_64-arista_7050_qx32" ] && return
Copy link
Contributor

@MichelMoriniaux MichelMoriniaux Apr 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we extend this to the 7060-CX32 and CX32s as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Return here would cause small /var/log partition of 128M, are you sure you want 7060 here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

128M is already large IMHO so OK for me

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:-)
I looked at a device. I think we currently mount /var/log for 380M in loop file system. With the 5% to 10% adjustment will mount tmpfs of 512M. That doesn't sound too bad to me.

aboot_platform=""
. ${rootmnt}/host/machine.conf
if [ X"$aboot_platform" = X"x86_64-arista_7050_qx32" ] ||
[ X"$aboot_platform" = X"x86_64-arista_7050_qx32s" ]
Copy link
Contributor

@MichelMoriniaux MichelMoriniaux Apr 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same comment as above: Can we extend this to the 7060-CX32 and CX32S as well?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need more time for this change, because:

  • The change I put up here is tested change.
  • We didn't include 7060 in this list because we didn't observe the same issue.

That doesn't mean that we won't add 7060 to the list. It just mean that we would like to have a discussion, if we decide to proceed, then we need to test it before putting up change for review.

Are you ok with this change go in as infrastructural change first and improve later? :-)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, do the merge I can contribute the change later. both platforms come from the same lineage, I have both in my network and both show the same symptoms

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @MichelMoriniaux !

# NOTE: certain platforms, when reaching initramfs stage, have a small
# limit of mounting tmpfs partition, potentially due to amount
# of RAM available in this stage. e.g. Arista 7050 QX32
[ X"$aboot_platform" = X"x86_64-arista_7050_qx32" ] && return
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

x86_64-arista_7050_qx32" [](start = 27, length = 24)

the best way to do this is to move the configuration part in device/platform/ , but this may requires more work.

Copy link
Collaborator

@lguohan lguohan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@yxieca yxieca merged commit 8bf9247 into sonic-net:master Apr 15, 2019
@yxieca yxieca deleted the tmpfs branch April 15, 2019 05:46
MichelMoriniaux pushed a commit to criteo-forks/sonic-buildimage that referenced this pull request May 28, 2019
…#2780)

SONiC is a heavy writer to /var/log partition, we noticed that this
behavior causes certain flash drive to become read-only over time.
To avoid this issue, we mount /var/log parition on these devices as
tmpfs.

- Mount /var/log as tmpfs
- /var/log default size is 128M
- Adjust size according to existing var-log.ext4 file size.
- Adjust size to between 5% to 10% of total memory size.

Signed-off-by: Ying Xie <ying.xie@microsoft.com>
MichelMoriniaux added a commit to criteo-forks/sonic-buildimage that referenced this pull request May 28, 2019
as per discussion in sonic-net#2780 and sonic-net#2774 added the 7060 platform to the list
of hosts

Signed-off-by: Michel Moriniaux <m.moriniaux@criteo.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants