Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Request] Add qs backend #104

Open
artemklevtsov opened this issue May 28, 2019 · 1 comment
Open

[Request] Add qs backend #104

artemklevtsov opened this issue May 28, 2019 · 1 comment

Comments

@artemklevtsov
Copy link

Hi,

It would be nice to add qs as drop in replacement for the RDS.

@torfason
Copy link

torfason commented Nov 2, 2022

And perhaps even more pertinent to add option to use qs::qserialize() as an option for driver_dbi (and perhaps other). A very quick and dirty replacement got some example data (character vectors) in SQLite down by almost 90%, from 4.4MB down to 550Kb. The time to write it was very similar, qs may be a little bit slower, but no more than 5-10%, and sometimes it comes out faster.

What I changed was just two functions:

serialize_to_raw <- function(x, ascii, xdr) {
  #serialize(x, NULL, ascii = ascii, xdr = xdr, version = 2L)
  qs::qserialize(x)
}

unserialize_safe <- function(x) {
  if (is.character(x)) {
    #unserialize(charToRaw(x))
    qs::qdeserialize(charToRaw(x))
  } else if (is.raw(x)) {
    #unserialize(x)
    qs::qdeserialize(x)
  } else {
    stop("Invalid input")
  }
}

I'm sure more would have to be changed for an actual implementation, and a decision would need to be made on how to offer configuration, as well as whether to save the serialization format as metadata in the underlying store in order to allow it to be read without knowing the serialization method beforehand. But for a storage engine, 90% space savings are pretty significant.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants