Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wip] decodeFileData and canDecodeFileData #82

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 52 additions & 39 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,46 +218,54 @@ Calling this method after calling `close()` will cause undefined behavior.
`callback` gets `(err, readStream)`, where `readStream` is a `Readable Stream` that provides the file data for this entry.
If this zipfile is already closed (see `close()`), the `callback` will receive an `err`.

`options` may be omitted or `null`, and has the following defaults:
`options` may be omitted or `null`.
If an option value is `null` or `undefined`, it is effectively equivalent to being omitted.
`options` has the following structure and effective default values:

```js
{
decompress: entry.isCompressed() ? true : null,
decrypt: null,
start: 0, // actually the default is null, see below
end: entry.compressedSize, // actually the default is null, see below
decodeFileData: true,
start: 0,
end: entry.compressedSize,
decompress: null, // deprecated
decrypt: null, // deprecated
}
```

If the entry is compressed (with a supported compression method),
and the `decompress` option is `true` (or omitted),
the read stream provides the decompressed data.
Omitting the `decompress` option is what most clients should do.
When `decodeFileData` is `true` or `null` (or omitted), yauzl will attempt to decode the file data.
Currently the only supported non-trivial encoding is Deflate compression,
in which case the file data will piped through a zlib inflate filter.
If `entry.canDecodeFileData()` returns `false`, yauzl will not be able to decode the file data,
and the `callback` will receive an error.
See `canDecodeFileData()` for more information.

The `decompress` option must be `null` (or omitted) when the entry is not compressed (see `isCompressed()`),
and either `true` (or omitted) or `false` when the entry is compressed.
Specifying `decompress: false` for a compressed entry causes the read stream
to provide the raw compressed file data without going through a zlib inflate transform.
When `decodeFileData` is `false`, the `readStream` will provide the raw file data as it is encoded in the zipfile.
In this case, `entry.canDecodeFileData()` is irrelevant.

If the entry is encrypted (see `isEncrypted()`), clients may want to avoid calling `openReadStream()` on the entry entirely.
Alternatively, clients may call `openReadStream()` for encrypted entries and specify `decrypt: false`.
If the entry is also compressed, clients must *also* specify `decompress: false`.
Specifying `decrypt: false` for an encrypted entry causes the read stream to provide the raw, still-encrypted file data.
(This data includes the 12-byte header described in the spec.)
If the file data is encoded trivially (which means among other things that `entry.compressionMethod === 0`,
"stored" as it's called in the spec), then `entry.canDecodeFileData()` returns `true`,
and `decodeFileData` has no effect;
the encoded file data is the same as the decoded file data without needing to be piped through any transform filter.

The `decrypt` option must be `null` (or omitted) for non-encrypted entries, and `false` for encrypted entries.
Omitting the `decrypt` option (or specifying it as `null`) for an encrypted entry
will result in the `callback` receiving an `err`.
This default behavior is so that clients not accounting for encrypted files aren't surprised by bogus file data.
*DEPRECATED*:
Before the `decodeFileData: false` option was introduced to this API,
the `decompress` and `decrypt` options were used to get the file's raw data.
Their usage is rather complicated, and is no longer documented or recommended,
but their behavior is maintained for compatibility.

The `start` (inclusive) and `end` (exclusive) options are byte offsets into this entry's file data,
The `start` (inclusive) and `end` (exclusive) options are byte offsets into this entry's raw file data,
and can be used to obtain part of an entry's file data rather than the whole thing.
If either of these options are specified and non-`null`,
then the above options must be used to obain the file's raw data.
Speficying `{start: 0, end: entry.compressedSize}` will result in the complete file,
If either of these options is specified and non-`null`, then `decodeFileData` must be `false`.
Specifying `{start: 0, end: entry.compressedSize}` will result in the complete file,
which is effectively the default values for these options,
but note that unlike omitting the options, when you specify `start` or `end` as any non-`null` value,
the above requirement is still enforced that you must also pass the appropriate options to get the file's raw data.
you must also provide `decodeFileData` as described above.

*DEPRECATED*:
For compatibility with earlier versions of yauzl, `start` and `end` can be used for entries when
`entry.canDecodeFileData()` returns `true` and `entry.isCompressed()` returns `false`.
Additionally, `start` and `end` can be used when the deprecated `decompress` and `decrypt` options are used appropriately.
The recommended usage is to simply specify `decodeFileData: false` whenever `start` and `end` are used.

It's possible for the `readStream` provided to the `callback` to emit errors for several reasons.
For example, if zlib cannot decompress the data, the zlib error will be emitted from the `readStream`.
Expand Down Expand Up @@ -394,28 +402,33 @@ Effectively implemented as:
return dosDateTimeToDate(this.lastModFileDate, this.lastModFileTime);
```

#### isEncrypted()
#### canDecodeFileData()

Returns is this entry encrypted with "Traditional Encryption".
Effectively implemented as:
If this method returns `false`, then calling `openReadStream()` with `decodeFileData` effectively `true` will result
in the `callback` receiving an error, such as an error for an unsupported compression method.
If this method returns `true`, yauzl is not aware of any reason why attempting to decoding the file data would fail.
This method is currently implemented effectively like this:

```js
return (this.generalPurposeBitFlag & 0x1) !== 0;
return (this.compressionMethod === 0 || this.compressionMethod === 8) &&
(this.generalPurposeBitFlag & 0x1) === 0;
```

See `openReadStream()` for the implications of this value.
If yauzl adds support for more compression methods, or adds early detection for more cases where decoding file data will fail,
this implementation will change to reflect those additions.

Note that "Strong Encryption" is not supported, and will result in an `"error"` event emitted from the `ZipFile`.
#### isEncrypted()

#### isCompressed()
*DEPRECATED*: Use `canDecodeFileData()`, or check this entry's metadata yourself.

Effectively implemented as:
#### isCompressed()

```js
return this.compressionMethod === 8;
```
If `canDecodeFileData()` returns `true`,
then this method indicates that decoding the file data will perform non-trivial decompression.
If `canDecodeFileData()` returns `false`,
then this method is not meaningful.

See `openReadStream()` for the implications of this value.
See `openReadStream()`.

### Class: RandomAccessReader

Expand Down
76 changes: 49 additions & 27 deletions index.js
Original file line number Diff line number Diff line change
Expand Up @@ -432,30 +432,43 @@ ZipFile.prototype.openReadStream = function(entry, options, callback) {
callback = options;
options = {};
} else {
// validate options that the caller has no excuse to get wrong
if (options.decrypt != null) {
if (!entry.isEncrypted()) {
throw new Error("options.decrypt can only be specified for encrypted entries");
if (options.decodeFileData === false) {
// new, simple option
if (options.decrypt != null) {
throw new Error("cannot use options.decrypt when options.decodeFileData === false");
}
if (options.decrypt !== false) throw new Error("invalid options.decrypt value: " + options.decrypt);
if (entry.isCompressed()) {
if (options.decompress !== false) throw new Error("entry is encrypted and compressed, and options.decompress !== false");
if (options.decompress != null) {
throw new Error("cannot use options.decompress when options.decodeFileData === false");
}
}
if (options.decompress != null) {
if (!entry.isCompressed()) {
throw new Error("options.decompress can only be specified for compressed entries");
}
if (!(options.decompress === false || options.decompress === true)) {
throw new Error("invalid options.decompress value: " + options.decompress);
// start and end are allowed
} else {
// old, complicated options
// validate options that the caller has no excuse to get wrong
if (options.decrypt != null) {
if (!entry.isEncrypted()) {
throw new Error("options.decrypt can only be specified for encrypted entries");
}
if (options.decrypt !== false) throw new Error("invalid options.decrypt value: " + options.decrypt);
if (entry.isCompressed()) {
if (options.decompress !== false) throw new Error("entry is encrypted and compressed, and options.decompress !== false");
}
}
}
if (options.start != null || options.end != null) {
if (entry.isCompressed() && options.decompress !== false) {
throw new Error("start/end range not allowed for compressed entry without options.decompress === false");
if (options.decompress != null) {
if (!entry.isCompressed()) {
throw new Error("options.decompress can only be specified for compressed entries");
}
if (!(options.decompress === false || options.decompress === true)) {
throw new Error("invalid options.decompress value: " + options.decompress);
}
decompress = options.decompress;
}
if (entry.isEncrypted() && options.decrypt !== false) {
throw new Error("start/end range not allowed for encrypted entry without options.decrypt === false");
if (options.start != null || options.end != null) {
if (entry.isCompressed() && options.decompress !== false) {
throw new Error("start/end range not allowed for compressed entry without options.decompress === false");
}
if (entry.isEncrypted() && options.decrypt !== false) {
throw new Error("start/end range not allowed for encrypted entry without options.decrypt === false");
}
}
}
if (options.start != null) {
Expand All @@ -470,12 +483,15 @@ ZipFile.prototype.openReadStream = function(entry, options, callback) {
if (relativeEnd < relativeStart) throw new Error("options.end < options.start");
}
}

// any further errors can either be caused by the zipfile,
// or were introduced in a minor version of yauzl,
// so should be passed to the client rather than thrown.
if (!self.isOpen) return callback(new Error("closed"));
if (entry.isEncrypted()) {
if (options.decrypt !== false) return callback(new Error("entry is encrypted, and options.decrypt !== false"));
if (!(options.decrypt === false || options.decodeFileData === false)) {
return callback(new Error("entry is encrypted"));
}
}
// make sure we don't lose the fd before we open the actual read stream
self.reader.ref();
Expand Down Expand Up @@ -505,14 +521,20 @@ ZipFile.prototype.openReadStream = function(entry, options, callback) {
// 30+n - Extra field
var localFileHeaderEnd = entry.relativeOffsetOfLocalHeader + buffer.length + fileNameLength + extraFieldLength;
var decompress;
if (entry.compressionMethod === 0) {
// 0 - The file is stored (no compression)
if (options.decodeFileData === false) {
decompress = false;
} else if (entry.compressionMethod === 8) {
// 8 - The file is Deflated
decompress = options.decompress != null ? options.decompress : true;
} else {
return callback(new Error("unsupported compression method: " + entry.compressionMethod));
switch (entry.compressionMethod) {
case 0: // stored
decompress = false;
break;
case 8: // deflate
decompress = options.decompress != null ? options.decompress : true;
break;
default: // unsupported
// this error needs to happen after the error checks above for compatibility with earlier versions of yauzl.
return callback(new Error("unsupported compression method: " + entry.compressionMethod));
}
}
var fileDataStart = localFileHeaderEnd;
var fileDataEnd = fileDataStart + entry.compressedSize;
Expand Down