High storage cost due to too many IOpS in Azure #1023
Unanswered
GaugeTheory
asked this question in
Q&A
Replies: 1 comment
-
If you are ok with just not having a PVC, you can set storage size to 0: https://www.rabbitmq.com/kubernetes/operator/using-operator.html#persistence |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hello everybody,
I have started use RabbitMQ operator in Azure AKS cluster to deliver my messages. I have also enabled persistent storage to keep my messages in a PVC for data redundancy. In my previous setup I was using bitnami RabbitMQ without operator and without PVC
After a month of usage, I realised 20GB PVC for the RabbitMQ pods cost too much. Disk size costs few Euros/month, but disk operations costs ~100Euros/month. I suspect this is due to IOpS to disk. I see that RabbitMQ PVCs get ~200 IOpS. This is also Message rate I get per second. That means every single messages get written into disk.
My question is more generic one. Is there a way to reduce IOpS to disk? I don't need to write every messages to disk at all, it just needs to write to disk when the messages in memory reaches some size (i.e.1GB) for disaster recovery. I want nothing to be written into disk as long as size of the messages in memory is below a certain threshold.
I have also tried quorum queues, which has 512MB WAL size by default. I have hoped this would reduce IOpS since WAL would never reach 512 MB in normal operations. Still I see similar IOpS as before
https://blog.rabbitmq.com/posts/2020/04/quorum-queues-and-why-disks-matter/
If there is no option to reduce IOpS, i think I will have to get rid of PVCs for RabbitMQs. Is it also possible to disable PVC with RabbitMQ operator?
Or maybe there is a better StorageClass option that fits better with high IOpS cost wise in Azure? I could also pay a little for disk operations but definitely not 100x the disk cost. Or maybe local disk of the VM?
Thanks for any help
Beta Was this translation helpful? Give feedback.
All reactions