Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add bigstring interfaces for deflate/inflate #7

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

Laakeri
Copy link

@Laakeri Laakeri commented Jul 27, 2017

Add {deflate,inflate}_bigstring similar to {delfate,inflate} that
read from / write to bigstrings instead of bytes.

bigstring is defined as:
type bigstring = (char, Bigarray.int8_unsigned_elt, Bigarray.c_layout) Bigarray.Array1.t
See https://github.com/ocaml/ocaml/blob/trunk/bytecomp/lambda.ml#L138

The Z_BUF_ERROR fix (GPR #6) is included in the new C functions camlzip_deflate_bigstring and camlzip_inflate_bigstring.

Add {deflate,inflate}_bigstring similar to {delfate,inflate} that
read from / write to bigstrings instead of bytes.

bigstring is defined as:
  type bigstring = (char, Bigarray.int8_unsigned_elt, Bigarray.c_layout) Bigarray.Array1.t
See https://github.com/ocaml/ocaml/blob/trunk/bytecomp/lambda.ml#L138

The Z_BUF_ERROR fix (GPR xavierleroy#6) is included in the new C functions camlzip_deflate_bigstring and camlzip_inflate_bigstring.
@xavierleroy
Copy link
Owner

I know some projets use "bigstrings" as out-of-heap buffers, and others to work around the length limitation on strings on 32-bit platforms. So, this is a reasonable proposal. On the other hand I note that only the low-level primitives are provided for bigstrings but not the high-level APIs in modules Zip and Gzip. Just to convince myself that this is really what we want, could you tell me about the intended usage scenario for these new primitives?

@Laakeri
Copy link
Author

Laakeri commented Jul 28, 2017

In the intended use case we compress/uncompress an infinite stream of data that uses bigstrings as buffers.

@brendanlong
Copy link
Contributor

I would also like this (in the high level API's) so I can unzip things that aren't files (HTTP responses and other things in memory).

@hcarty
Copy link

hcarty commented Jan 19, 2018

@brendanlong I wrote this somewhat hacked together library on top of cryptokit's zlib support to handle in-memory g(un)zip handling of strings. It may be useful to you until something better is available.

@XVilka
Copy link

XVilka commented Aug 20, 2018

Is there anything I can do to help get this merged?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants