Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Signal uses too much data on storage / can't clear data with signal #5515

Closed
2 tasks done
peterpanondrugs opened this issue May 30, 2016 · 53 comments
Closed
2 tasks done

Signal uses too much data on storage / can't clear data with signal #5515

peterpanondrugs opened this issue May 30, 2016 · 53 comments
Labels
help wanted

Comments

@peterpanondrugs
Copy link

peterpanondrugs commented May 30, 2016 •
edited
Loading

I have:

Bug description

Signal uses too much data on storage.

Steps to reproduce

  • Send videos pictures whatever with a fresh install.
  • delete all chats. for group chats handle like this: clean completly from inside the group.
  • data wont drop on install size.
    e.g. with 2 group chats (1st: 4user; 2nd: 10 user), Signal consumes 187MB in Total!
  • i checked for empty signal download folders "movies" and "pictures"
  • 3-4 days ago i cleared (clear data) a total empty signal (same 2 groups) with storage use of 1,38GB! these days later: im back on 187MB.

Actual result: maybe the group chats need that much storage??? i dont hope so!

Expected result: When i delete all like my describtion above, signal should drop on install size. (correct me if im wrong, but 150MB for 2 emtpy group chats is too much)

Screenshots

d4bee971-3f6f-4cf0-86ab-3e3803ed2d0f

Device info

Device: S3 LTE (Intl.)
Android version: 5.1.1
Signal version: 3.16.0

Link to debug log

https://gist.github.com/01a0bdc4fa5671bb4e4635e586d04264
@peterpanondrugs peterpanondrugs changed the title Storage: Signal uses too much data on storage Signal uses too much data on storage May 30, 2016
@peterpanondrugs peterpanondrugs changed the title Signal uses too much data on storage Signal uses too much data on storage / can't clear data with signal May 30, 2016
@Esokrates
Copy link

I can confirm this issue under Android 5+. Signal occupied over 1,5 GB app data although the Data (Text of conversation + Images + Videos + Audio) were really around 400MB in total (confirmed by exporting text and all media!!!).
Deleting every Signal conversation dropped the App Data to 1,1GB as expected. So there are 1,1 GB "shadow data" that Signal itself does not free again. Only solution is to delete the AppData from the Application Manager in Android.

I have also an Android 4.4 phone where the exact same data are present and there Signal only takes up around 400MB!.

@peterpanondrugs
Copy link
Author

i can confirm this (around 220MB) also for Android 6.0.1

@Esokrates
Copy link

Can you also confirm that Android 4.4 is not affected?

@floriangosse
Copy link

I have the same problem on my Nexus 5 with Android 6 running and on a Motorola Moto G 2nd Gen. Are there some more informations about the problem?

It would be nice to have a functionality to clear/remove messages in conversations which are old than X. I don't wont to delete all my messages.

@floriangosse floriangosse mentioned this issue Jul 27, 2016
Closed
@patzi
Copy link

patzi commented Jul 27, 2016 •
edited
Loading

@floriangosse

It would be nice to have a functionality to clear/remove messages in conversations which are old than X. I don't wont to delete all my messages.

You can already trim conversations and remove old messages. You can trim conversations in the following way:
Go to settings -> conversations and media -> trim messages: activate and set treshold

I don't know the exact naming because my signal runs in German. There it is:
Einstellungen -> Unterhaltung und Medieninhalte -> Unterhaltung kürzen: aktivieren und Limit einstellen

@floriangosse
Copy link

Thanks @patzi. I didn't see it.

@Esokrates
Copy link

@floriangosse @patzi Unfortunately deleting all messages wont help at all! Thats what this bug is about!
I can delete all conversations and there is still 1 GB of data allocated. Signal allocates storage without needing it, this is a severe bug, because it eats up internal storage and the only way to resolve is to delete all app data, but then you need to reregister signal.

@mopsie
Copy link

mopsie commented Jul 31, 2016

I can also confirm this. Depending on how many pictures, media, etc. get send either in group oder single user conversations the data folder gets clogged up quite fast. After just a couple of weeks of moderate usage with just 3 users Signal has become the single largest app on my phone by far. Its currently about 1.2 GB and growing...

As mentioned, the trimming option in the settings doesnt work. Messages get cut, but the overall size remains unchanged.

This happens on Android 4.4 as well as Android 2.3. In both cases specifically the app_captures folder under data/org.thoughtcrime.securesms/ is full with lots of .blob files segmented at 165.39 MB each. This alone amounts for about 1 GB of the 1,2 GB size.

The same problem/behaviour occurs on the other end at the 3 users as well. Their Signal also grows horrendously big.

Please fix this if possible, cause for long term media conversations it doesnt seem very promising in this state.

@2-4601 2-4601 added the bug label Aug 15, 2016
@ForGorNorPor
Copy link

I have the same issue. Signal is using 310 MB and is still growing. Android 4.4.2.

@Esokrates
Copy link

Esokrates commented Sep 29, 2016 •
edited
Loading

I think this issue may be harder to reproduce now. Until August photos in group conversations have been sent in full resolution, see #5461
This is "fixed" now (in quotes because the picture quality is a real pain now, but thats another issue).
I have not fully tested, but I guess the storage problem may be a little bit less clear to reproduce now, since it's not easily possible now to send a picture with more than 200kB due to pictures getting compressed so heavily!

@haffenloher
Copy link
Contributor

haffenloher commented Jan 22, 2017 •
edited
Loading

Failed to reproduce this with a one-on-one chat in an API 23 emulator. I did the following:

  • send a message with a GIF attached
  • go back to the inbox
  • delete thread

The attachment was correctly deleted and Signal's storage usage went back to where it was before.
Anything else I need to do to trigger this?

@peterpanondrugs
Copy link
Author

peterpanondrugs commented Jan 23, 2017 •
edited
Loading

(Sorry. Missclicked)
I think this is an old issue and can be closed maybe? People should report for the newest version.
In my case its working now fine.

@kaneelschep
Copy link

This is definetely not over. Signal uses a lot mb on the phone. And it wont just go away. I just saved all the pictures in my chats to sd card and then deleted all the chats. At apps it shows 589mb data on device for signal. After reset and app update its still the same allthough there is no simgle chat or picture or whatever in signal. This is on a sony m4 aqua. The only way I got it away on my own phone. a sony z2 is by deleting data in the phones app settings screen. But then the app completely resetted and, I got a new security code and all my friends had to accept that again. This makes the app unusable on phones with less phone memory.

@Trolldemorted
Copy link
Contributor

@kaneelschep is your device rooted, and could you have a look where the space is going?

@kaneelschep
Copy link

kaneelschep commented Jan 29, 2017 via email

@Trolldemorted
Copy link
Contributor

I do not believe there is an easy way without having root privileges, but maybe someone knows a way around.

Does anyone who has this problem have a rooted android, can could give us some hints where the memory goes?

root@kenzo:/ # du -sh /data/data/org.thoughtcrime.securesms/*
4.0K    /data/data/org.thoughtcrime.securesms/app_captures
134M    /data/data/org.thoughtcrime.securesms/app_parts
4.0K    /data/data/org.thoughtcrime.securesms/app_textures
56K     /data/data/org.thoughtcrime.securesms/app_webview
219M    /data/data/org.thoughtcrime.securesms/cache
92K     /data/data/org.thoughtcrime.securesms/code_cache
2.0M    /data/data/org.thoughtcrime.securesms/databases
20M     /data/data/org.thoughtcrime.securesms/files
0       /data/data/org.thoughtcrime.securesms/lib
20K     /data/data/org.thoughtcrime.securesms/shared_prefs

This is from my phone, i am not sure which sizes would be normal though. According to my android's app settings, signal consumes 157MB "data" and 219MB "cache".

@kaneelschep
Copy link

Save all the pictures from the chats, and see how much mb it is. Then if you want. Remove them from the chats and see if the data dissapears from setting/apps/signal....

@kaneelschep
Copy link

I think using 360mb is allready a lot ;-) whatsapp would never have that. And i never ever in years deleted any chats in whatsapp.

@Trolldemorted
Copy link
Contributor

Does that require me manually going through all chats?

360MB isn't that much if you receive many media attachments, i think. If no one with this problem can enlighten us where the space is going, we cannot do much about it.

Maybe there are workarounds like moving signal to the sdcard, and mounting the sdcard on your pc? Or does adb pull work on non-rooted devices for every folder? Or can you use run-as?

@kaneelschep
Copy link

kaneelschep commented Feb 1, 2017 via email

@Trolldemorted Trolldemorted mentioned this issue Feb 22, 2017
Closed
@kaneelschep
Copy link

From last time till now after clearing all fotos from chats. and clearing data in settings. So one month. Signal went from100mb to 1,5 gb. This is totally useless for people with small internal storage phones. And is clearly a serious problem for usability of signal. is is that hard to be possible to move to sdcard?

@kaneelschep
Copy link

kaneelschep commented Jun 19, 2017 via email

@Trolldemorted
Copy link
Contributor

@i7u if you could find out where all the space is going i would be great.

@i7u
Copy link

i7u commented Jun 21, 2017

I'm kinda embarrassed to say I don't know how to do that. I installed a terminal emulator app on my phone (Termux) but I can't ls anything under /data - I get permission denied. With the built-in file manager, I can only see one folder under Device Storage > Android > data > org.thoughtcrime.sms - it's called "files" and allegedly has "0 items".

Could you point me to instructions to list the contents of the right folder?

@ghost
Copy link

ghost commented Jun 22, 2017

@i7u

but I can't ls anything under /data - I get permission denied.

yep, you need root

@Trolldemorted
Copy link
Contributor

Some stackoverflow answers and the adb doc suggest that adb run-as can be used to access an app's storage.

However, the app needs to be debuggable - i don't know whether the official playstore build is, nor if you can switch to the beta version.

@i7u
Copy link

i7u commented Jun 22, 2017

I'm afraid I can't experiment with rooting my phone to collect this information. I only have one phone, and I need it for work. More than happy to collect the info as long as it doesn't jeopardize the functionality of the phone.

@ghost
Copy link

ghost commented Jun 22, 2017

for the sake of completeness

bullhead:/ $ run-as org.thoughtcrime.securesms
run-as: Package 'org.thoughtcrime.securesms' is not debuggable

@kaneelschep
Copy link

kaneelschep commented Jun 22, 2017 via email

@Trolldemorted
Copy link
Contributor

damn.

are the beta builds debuggable?

@ghost
Copy link

ghost commented Jun 22, 2017

@Trolldemorted
nope, tested with a beta v4.7.1

@ForGorNorPor
Copy link

ForGorNorPor commented Jun 22, 2017 •
edited
Loading

I have tracked down the data and it all leads to the app_parts folder. There are hundreds of small files named "partXYZ.mms" with a size varying from a few kB up to 50 MB. Does anyone know what those files contain and if they are really needed?

@cascheberg cascheberg mentioned this issue Jun 28, 2017
Closed
2 tasks
@ghost
Copy link

ghost commented Jun 28, 2017

@ForGorNorPor yep, AFAIK partXYZ.mms files are encrypted attachments (pictures, videos, ...).
For me deleting attachments in Signal works as expected e.g. corresponding partXYZ.mms file under app_parts is deleted as well.

tested with Signal v4.6.1 and v4.7.4

@bme13
Copy link

bme13 commented Jun 30, 2017

My Signal installation consumed more than 1 GB. Deleting attachments etc. seems to work well however I was only able to reduce the size to approx. 1GB. Checking the disk usage revealed:
root@klte:/ # du -h /data/data/org.thoughtcrime.securesms/*
875M /data/data/org.thoughtcrime.securesms/app_captures
141M /data/data/org.thoughtcrime.securesms/app_parts
4.0K /data/data/org.thoughtcrime.securesms/app_textures
4.0K /data/data/org.thoughtcrime.securesms/app_tmp
4.0K /data/data/org.thoughtcrime.securesms/app_webview/paks
64K /data/data/org.thoughtcrime.securesms/app_webview
76K /data/data/org.thoughtcrime.securesms/cache/image_manager_disk_cache
80K /data/data/org.thoughtcrime.securesms/cache
52K /data/data/org.thoughtcrime.securesms/code_cache
1.9M /data/data/org.thoughtcrime.securesms/databases
4.0K /data/data/org.thoughtcrime.securesms/files/sessions
332K /data/data/org.thoughtcrime.securesms/files/prekeys
28K /data/data/org.thoughtcrime.securesms/files/signed_prekeys
72K /data/data/org.thoughtcrime.securesms/files/sessions-v2
500K /data/data/org.thoughtcrime.securesms/files
0B /data/data/org.thoughtcrime.securesms/lib
24K /data/data/org.thoughtcrime.securesms/shared_prefs

So most space seems to be allocated in app_captures. Checking that directory showed:
root@klte:/data/data/org.thoughtcrime.securesms/app_captures # ls -l
-rw------- u0_a210 u0_a210 2946276 2015-11-20 21:47 1448052436315.jpg
-rw------- u0_a210 u0_a210 216667204 2016-02-25 22:32 1456435939816.jpg
-rw------- u0_a210 u0_a210 216667204 2016-02-25 22:32 1456435954745.jpg
-rw------- u0_a210 u0_a210 216667204 2016-02-25 22:32 1456435966189.jpg
-rw------- u0_a210 u0_a210 23791124 2016-08-28 08:39 1472366370604.blob
-rw------- u0_a210 u0_a210 5327268 2016-11-07 13:19 1478521163927.blob
-rw------- u0_a210 u0_a210 5327268 2016-11-07 13:20 1478521204109.blob
-rw------- u0_a210 u0_a210 4804820 2016-11-19 16:04 1479567855890.blob
-rw------- u0_a210 u0_a210 5332 2016-11-29 22:42 1480455759791.blob
-rw------- u0_a210 u0_a210 56302164 2017-01-17 19:49 1484678975591.blob
-rw------- u0_a210 u0_a210 56302164 2017-01-17 19:50 1484679012797.blob
-rw------- u0_a210 u0_a210 56302164 2017-01-17 19:50 1484679042624.blob
-rw------- u0_a210 u0_a210 56302164 2017-01-17 19:52 1484679172310.blob
-rw------- u0_a210 u0_a210 190756 2017-03-24 17:55 1490374547235.blob
-rw------- u0_a210 u0_a210 190756 2017-03-24 17:56 1490374562292.blob

What does the app_captures directory contains? Some files are pretty old. Is it safe to remove (some of) those files?
ps. Running signal version 4.6.1

@Trolldemorted
Copy link
Contributor

@ForGorNorPor can you check whether deleting an attachment decreases the file count in that directory?

@bme13 could you open some of the jpegs and have a look?

@bme13
Copy link

bme13 commented Jun 30, 2017

@Trolldemorted they don't appear to be jpg files. Tried to determine the file type using the file command and that just says data file. Also the .blob files are data files according to the file command.

@bme13
Copy link

bme13 commented Jun 30, 2017

Just had some more time for testing. It seems that the app_captures temporally stores the images / sound recordings (captures :-) ) e.g. when I take a picture I see a new file in the app_captures directory. Then after I send that picture the file is removed from the app_captures directory and shows up in the app_parts directory. So now my assumption is that this occasionally goes wrong and the capture file remains. I will try to reproduce that.

@ForGorNorPor
Copy link

@Trolldemorted sure, but how exactly do I remove media from my chats? :) I know that I can save them in the context menu and I found an option to globally trim all chats, but I would rather only pick some particular media files to delete and keep all text. Is this possible?

@janvlug
Copy link

janvlug commented Jul 1, 2017

@ForGorNorPor By long pressing on the media in the chat, a menu appears that allows you to delete that specific media item.
See #6111 for a pending request to multi-select and delete media.

@ForGorNorPor
Copy link

Ok, that's a bit cumbersome, I hope that this fix will be applied some day.
But good news is, that I indeed managed to get rid of 150 MB by deleting some videos and pictures of my chats. I guess that the encryption bloats those attachments up in size, compared to WhatsApp where I did not encounter this issue ever.
If the handling of attachments would be improved, I'd consider this issue fixed from my point of view. Thanks for your help.

@bme13
Copy link

bme13 commented Jul 1, 2017

It seems that if you share a file this also is placed in the app_captures directory. After trying I found 2 ways to leave (large) files in that directory.

  1. Take a picture, or share a file but don't send it. Leave it as concept. Restart the phone. After the restart the concept is gone however the file remains in app_captures.
  2. Try to share a large file that exceeds the accept limits for attachments. You get a message in the conversation that the message is too large however the file appears in the app_captures directory and remains there.

I think the first option is not very likely to happen however the second one is.
Don't know a proper solution, perhaps check the filesize prior to placing it in the capures directory or maybe that directory can be cleared at startup?

moxie0 added a commit that referenced this issue Jul 6, 2017
@moxie0
Start creating "persistent" blobs in cache directory
526d510 
Helps #5515
// FREEBIE
@whoabuddy
Copy link

I noticed my Signal app was taking up over 1gb of space today, and after searching and finding this issue, I tested the results from my phone. It is a Samsung Galaxy S6 running Android 7, with Signal 4.10.12.

Size before cleanup: 1.16 GB

  • Turned on Message trimming (Delete old messages)
  • Set Conversation length limit to 5 (default: 500)
  • Clicked Trim all conversations now
  • Process started at 0600, ended at 0601
    • Deleting old messages... shows 100 total
    • Progress bar took longer to start then run
  • Turned off Message trimming (Delete old messages)

Size after cleanup: 128 MB

@dalanmiller
Copy link

There are some similar issues open on the iOS repo as well so it may not be specific to a mobile OS. Does Signal have a Patron or anything where we can support to get these things fixed?

@jessekunze
Copy link

On iOS 11 (iPhone 6S) , I'm seeing signal use 1.71 GB with all messages deleted. I also notice that the performance (opening, typing, and sending messages) degrades greatly when the app gets so high into space usage.

@kaneelschep
Copy link

kaneelschep commented Oct 22, 2017 via email

@bme13
Copy link

bme13 commented Oct 24, 2017

Perhaps an other cause for the application size is #6909

@automated-signal
Copy link

GitHub Issue Cleanup:
See #7598 for more information.

@signalapp signalapp locked and limited conversation to collaborators Apr 3, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
help wanted
Milestone
No milestone
Development

No branches or pull requests