download specific dbs

[atlanteh]

Hey, is there a way to only download a specific database and not all them?

[GitSquared]

No, this is not currently supported. Could you tell me a bit more about your usecase for this?

[atlanteh]

Thanks for the super fast reply!
Well I only need the country iso_code to redirect the user to the right place based on that so I don't need the rest of the dbs..
I don't want to give the container any write access to any persistence for security reasons, so I want it to write to in memory "/tmp" but the city db would take a lot of memory without me needing it..

[GitSquared]

Thanks for the explanation. Downloading only the country DB would save you about 37MB tops, which is really negligible on most systems those days, so I'm not entirely convinced it's worth adding a new feature for. Are you working in an IoT context?

[atlanteh]

@GitSquared actually on my computer I'm seeing different values
City is 70MB
ASN is 7MB
Country is 5.5MB

I don't feel this is negligible. I'm not using IOT, but I'm working on an app that should consume a lot of memory, and should be very fast, so I'm trying to save up as much as possible, and saving 77MB of memory on each container just by removing these DBs sounds great to me.

Is there a reason not to implement this? I mean this feels like a basic feature request - download only what you need, no?

[GitSquared]

Sorry, you're right — I was looking at the compressed archives that the lib download, not the size of the unpacked databases.

I'm working on an app that should consume a lot of memory ... so I'm trying to save up as much as possible

Note that if you're writing data to /tmp, that's disk storage on most systems, not RAM - unless you mount your whole container in a ramdisk

Is there a reason not to implement this? I mean this feels like a basic feature request - download only what you need, no?

It is pretty basic indeed, but I haven't touched this lib in a long time and wanted to clean it up a bit, so its a bit more of an investment on my end. But I guess i'll take the opportunity to finally do that ;)

Should have something for you very soon, this week at least.

[atlanteh]

That's amazing. thank you so much!
Regarding the /tmp please have a look here:

https://aws.amazon.com/about-aws/whats-new/2018/03/amazon-ecs-adds-support-for-shm-size-and-tmpfs-parameters/

The tmpfs parameter allows you to mount a temporary volume in memory. This removes the need to store data in your container instance and allows containers to run in a read-only mode, which can improve the security of your service.

[GitSquared]

interesting, thanks for the link!

[GitSquared]

Hey — just published https://github.com/GitSquared/node-geolite2-redist/releases/tag/v3.0.0 with this feat added! You can simply open() a DB or a list of DB like ['GeoLite2-Country', 'GeoLite2-ASN'] and it'll just download what you need. Detailed examples here.

Lmk if you have any questions or trouble!

[atlanteh]

Thanks so much for this awesome work!
I tested this and it works great.
The customStorageDir didn't work, so I created a quick PR:
#21