will/documentit
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
8ab1dadf54
documentit/node_modules/yauzl/README.md
Queen Bee b3c993d9e3 installed electron
2018-01-05 10:48:13 -08:00

468 lines
19 KiB
Markdown
Raw Blame History

# yauzl
[![Build Status](https://travis-ci.org/thejoshwolfe/yauzl.svg?branch=master)](https://travis-ci.org/thejoshwolfe/yauzl)
[![Coverage Status](https://img.shields.io/coveralls/thejoshwolfe/yauzl.svg)](https://coveralls.io/r/thejoshwolfe/yauzl)
yet another unzip library for node. For zipping, see
[yazl](https://github.com/thejoshwolfe/yazl).
Design principles:
* Follow the spec.
Don't scan for local file headers.
Read the central directory for file metadata.
(see [No Streaming Unzip API](#no-streaming-unzip-api)).
* Don't block the JavaScript thread.
Use and provide async APIs.
* Keep memory usage under control.
Don't attempt to buffer entire files in RAM at once.
* Never crash (if used properly).
Don't let malformed zip files bring down client applications who are trying to catch errors.
* Catch unsafe filenames entries.
A zip file entry throws an error if its file name starts with `"/"` or `/[A-Za-z]:\//`
or if it contains `".."` path segments or `"\\"` (per the spec).
## Usage
```js
var yauzl = require("yauzl");
var fs = require("fs");
var path = require("path");
var mkdirp = require("mkdirp"); // or similar
yauzl.open("path/to/file.zip", {lazyEntries: true}, function(err, zipfile) {
if (err) throw err;
zipfile.readEntry();
zipfile.on("entry", function(entry) {
if (/\/$/.test(entry.fileName)) {
// directory file names end with '/'
mkdirp(entry.fileName, function(err) {
if (err) throw err;
zipfile.readEntry();
});
} else {
// file entry
zipfile.openReadStream(entry, function(err, readStream) {
if (err) throw err;
// ensure parent directory exists
mkdirp(path.dirname(entry.fileName), function(err) {
if (err) throw err;
readStream.pipe(fs.createWriteStream(entry.fileName));
readStream.on("end", function() {
zipfile.readEntry();
});
});
});
}
});
});
```
## API
The default for every optional `callback` parameter is:
```js
function defaultCallback(err) {
if (err) throw err;
}
```
### open(path, [options], [callback])
Calls `fs.open(path, "r")` and gives the `fd`, `options`, and `callback` to `fromFd()` below.
`options` may be omitted or `null`. The defaults are `{autoClose: true, lazyEntries: false}`.
`autoClose` is effectively equivalent to:
```js
zipfile.once("end", function() {
zipfile.close();
});
```
`lazyEntries` indicates that entries should be read only when `readEntry()` is called.
If `lazyEntries` is `false`, `entry` events will be emitted as fast as possible to allow `pipe()`ing
file data from all entries in parallel.
This is not recommended, as it can lead to out of control memory usage for zip files with many entries.
See [issue #22](https://github.com/thejoshwolfe/yauzl/issues/22).
If `lazyEntries` is `true`, an `entry` or `end` event will be emitted in response to each call to `readEntry()`.
This allows processing of one entry at a time, and will keep memory usage under control for zip files with many entries.
### fromFd(fd, [options], [callback])
Reads from the fd, which is presumed to be an open .zip file.
Note that random access is required by the zip file specification,
so the fd cannot be an open socket or any other fd that does not support random access.
The `callback` is given the arguments `(err, zipfile)`.
An `err` is provided if the End of Central Directory Record Signature cannot be found in the file,
which indicates that the fd is not a zip file.
`zipfile` is an instance of `ZipFile`.
`options` may be omitted or `null`. The defaults are `{autoClose: false, lazyEntries: false}`.
See `open()` for the meaning of the options.
### fromBuffer(buffer, [options], [callback])
Like `fromFd()`, but reads from a RAM buffer instead of an open file.
`buffer` is a `Buffer`.
`callback` is effectively passed directly to `fromFd()`.
If a `ZipFile` is acquired from this method,
it will never emit the `close` event,
and calling `close()` is not necessary.
`options` may be omitted or `null`. The defaults are `{lazyEntries: false}`.
See `open()` for the meaning of the options.
The `autoClose` option is ignored for this method.
### fromRandomAccessReader(reader, totalSize, [options], [callback])
This method of creating a zip file allows clients to implement their own back-end file system.
For example, a client might translate read calls into network requests.
The `reader` parameter must be of a type that is a subclass of
[RandomAccessReader](#class-randomaccessreader) that implements the required methods.
The `totalSize` is a Number and indicates the total file size of the zip file.
`options` may be omitted or `null`. The defaults are `{autoClose: true, lazyEntries: false}`.
See `open()` for the meaning of the options.
### dosDateTimeToDate(date, time)
Converts MS-DOS `date` and `time` data into a JavaScript `Date` object.
Each parameter is a `Number` treated as an unsigned 16-bit integer.
Note that this format does not support timezones,
so the returned object will use the local timezone.
### Class: ZipFile
The constructor for the class is not part of the public API.
Use `open()`, `fromFd()`, `fromBuffer()`, or `fromRandomAccessReader()` instead.
#### Event: "entry"
Callback gets `(entry)`, which is an `Entry`.
See `open()` and `readEntry()` for when this event is emitted.
#### Event: "end"
Emitted after the last `entry` event has been emitted.
See `open()` and `readEntry()` for more info on when this event is emitted.
#### Event: "close"
Emitted after the fd is actually closed.
This is after calling `close()` (or after the `end` event when `autoClose` is `true`),
and after all stream pipelines created from `openReadStream()` have finished reading data from the fd.
If this `ZipFile` was acquired from `fromRandomAccessReader()`,
the "fd" in the previous paragraph refers to the `RandomAccessReader` implemented by the client.
If this `ZipFile` was acquired from `fromBuffer()`, this event is never emitted.
#### Event: "error"
Emitted in the case of errors with reading the zip file.
(Note that other errors can be emitted from the streams created from `openReadStream()` as well.)
After this event has been emitted, no further `entry`, `end`, or `error` events will be emitted,
but the `close` event may still be emitted.
#### readEntry()
Causes this `ZipFile` to emit an `entry` or `end` event (or an `error` event).
This method must only be called when this `ZipFile` was created with the `lazyEntries` option set to `true` (see `open()`).
When this `ZipFile` was created with the `lazyEntries` option set to `true`,
`entry` and `end` events are only ever emitted in response to this method call.
The event that is emitted in response to this method will not be emitted until after this method has returned,
so it is safe to call this method before attaching event listeners.
After calling this method, calling this method again before the response event has been emitted will cause undefined behavior.
Calling this method after the `end` event has been emitted will cause undefined behavior.
Calling this method after calling `close()` will cause undefined behavior.
#### openReadStream(entry, callback)
`entry` must be an `Entry` object from this `ZipFile`.
`callback` gets `(err, readStream)`, where `readStream` is a `Readable Stream`.
If the entry is compressed (with a supported compression method),
the read stream provides the decompressed data.
If this zipfile is already closed (see `close()`), the `callback` will receive an `err`.
It's possible for the `readStream` it to emit errors for several reasons.
For example, if zlib cannot decompress the data, the zlib error will be emitted from the `readStream`.
Two more error cases are if the decompressed data has too many or too few actual bytes
compared to the reported byte count from the entry's `uncompressedSize` field.
yauzl notices this false information and emits an error from the `readStream`
after some number of bytes have already been piped through the stream.
Because of this check, clients can always trust the `uncompressedSize` field in `Entry` objects.
Guarding against [zip bomb](http://en.wikipedia.org/wiki/Zip_bomb) attacks can be accomplished by
doing some heuristic checks on the size metadata and then watching out for the above errors.
Such heuristics are outside the scope of this library,
but enforcing the `uncompressedSize` is implemented here as a security feature.
It is possible to destroy the `readStream` before it has piped all of its data.
To do this, call `readStream.destroy()`.
You must `unpipe()` the `readStream` from any destination before calling `readStream.destroy()`.
If this zipfile was created using `fromRandomAccessReader()`, the `RandomAccessReader` implementation
must provide readable streams that implement a `.destroy()` method (see `randomAccessReader._readStreamForRange()`)
in order for calls to `readStream.destroy()` to work in this context.
#### close()
Causes all future calls to `openReadStream()` to fail,
and closes the fd after all streams created by `openReadStream()` have emitted their `end` events.
If the `autoClose` option is set to `true` (see `open()`),
this function will be called automatically effectively in response to this object's `end` event.
If the `lazyEntries` option is set to `false` (see `open()`) and this object's `end` event has not been emitted yet,
this function causes undefined behavior.
If the `lazyEntries` option is set to `true`,
you can call this function instead of calling `readEntry()` to abort reading the entries of a zipfile.
It is safe to call this function multiple times; after the first call, successive calls have no effect.
This includes situations where the `autoClose` option effectively calls this function for you.
#### isOpen
`Boolean`. `true` until `close()` is called; then it's `false`.
#### entryCount
`Number`. Total number of central directory records.
#### comment
`String`. Always decoded with `CP437` per the spec.
### Class: Entry
Objects of this class represent Central Directory Records.
Refer to the zipfile specification for more details about these fields.
These fields are of type `Number`:
* `versionMadeBy`
* `versionNeededToExtract`
* `generalPurposeBitFlag`
* `compressionMethod`
* `lastModFileTime` (MS-DOS format, see `getLastModDateTime`)
* `lastModFileDate` (MS-DOS format, see `getLastModDateTime`)
* `crc32`
* `compressedSize`
* `uncompressedSize`
* `fileNameLength` (bytes)
* `extraFieldLength` (bytes)
* `fileCommentLength` (bytes)
* `internalFileAttributes`
* `externalFileAttributes`
* `relativeOffsetOfLocalHeader`
#### fileName
`String`.
Following the spec, the bytes for the file name are decoded with
`UTF-8` if `generalPurposeBitFlag & 0x800`, otherwise with `CP437`.
If `fileName` would contain unsafe characters, such as an absolute path or
a relative directory, yauzl emits an error instead of an entry.
#### extraFields
`Array` with each entry in the form `{id: id, data: data}`,
where `id` is a `Number` and `data` is a `Buffer`.
This library looks for and reads the ZIP64 Extended Information Extra Field (0x0001)
in order to support ZIP64 format zip files.
None of the other fields are considered significant by this library.
#### comment
`String` decoded with the same charset as used for `fileName`.
#### getLastModDate()
Effectively implemented as:
```js
return dosDateTimeToDate(this.lastModFileDate, this.lastModFileTime);
```
### Class: RandomAccessReader
This class is meant to be subclassed by clients and instantiated for the `fromRandomAccessReader()` function.
An example implementation can be found in `test/test.js`.
#### randomAccessReader._readStreamForRange(start, end)
Subclasses *must* implement this method.
`start` and `end` are Numbers and indicate byte offsets from the start of the file.
`end` is exclusive, so `_readStreamForRange(0x1000, 0x2000)` would indicate to read `0x1000` bytes.
`end - start` will always be at least `1`.
This method should return a readable stream which will be `pipe()`ed into another stream.
It is expected that the readable stream will provide data in several chunks if necessary.
If the readable stream provides too many or too few bytes, an error will be emitted.
Any errors emitted on the readable stream will be handled and re-emitted on the client-visible stream
(returned from `zipfile.openReadStream()`) or provided as the `err` argument to the appropriate callback
(for example, for `fromRandomAccessReader()`).
The returned stream *must* implement a method `.destroy()`
if you call `readStream.destroy()` on streams you get from `openReadStream()`.
If you never call `readStream.destroy()`, then streams returned from this method do not need to implement a method `.destroy()`.
`.destroy()` should abort any streaming that is in progress and clean up any associated resources.
`.destroy()` will only be called after the stream has been `unpipe()`d from its destination.
Note that the stream returned from this method might not be the same object that is provided by `openReadStream()`.
The stream returned from this method might be `pipe()`d through one or more filter streams (for example, a zlib inflate stream).
#### randomAccessReader.read(buffer, offset, length, position, callback)
Subclasses may implement this method.
The default implementation uses `createReadStream()` to fill the `buffer`.
This method should behave like `fs.read()`.
#### randomAccessReader.close(callback)
Subclasses may implement this method.
The default implementation is effectively `setImmediate(callback);`.
`callback` takes parameters `(err)`.
This method is called once the all streams returned from `_readStreamForRange()` have ended,
and no more `_readStreamForRange()` or `read()` requests will be issued to this object.
## How to Avoid Crashing
When a malformed zipfile is encountered, the default behavior is to crash (throw an exception).
If you want to handle errors more gracefully than this,
be sure to do the following:
* Provide `callback` parameters where they are allowed, and check the `err` parameter.
* Attach a listener for the `error` event on any `ZipFile` object you get from `open()`, `fromFd()`, `fromBuffer()`, or `fromRandomAccessReader()`.
* Attach a listener for the `error` event on any stream you get from `openReadStream()`.
## Limitations
### No Streaming Unzip API
Due to the design of the .zip file format, it's impossible to interpret a .zip file from start to finish
(such as from a readable stream) without sacrificing correctness.
The Central Directory, which is the authority on the contents of the .zip file, is at the end of a .zip file, not the beginning.
A streaming API would need to either buffer the entire .zip file to get to the Central Directory before interpreting anything
(defeating the purpose of a streaming interface), or rely on the Local File Headers which are interspersed through the .zip file.
However, the Local File Headers are explicitly denounced in the spec as being unreliable copies of the Central Directory,
so trusting them would be a violation of the spec.
Any library that offers a streaming unzip API must make one of the above two compromises,
which makes the library either dishonest or nonconformant (usually the latter).
This library insists on correctness and adherence to the spec, and so does not offer a streaming API.
### Limitted ZIP64 Support
For ZIP64, only zip files smaller than `8PiB` are supported,
not the full `16EiB` range that a 64-bit integer should be able to index.
This is due to the JavaScript Number type being an IEEE 754 double precision float.
The Node.js `fs` module probably has this same limitation.
### ZIP64 Extensible Data Sector Is Ignored
The spec does not allow zip file creators to put arbitrary data here,
but rather reserves its use for PKWARE and mentions something about Z390.
This doesn't seem useful to expose in this library, so it is ignored.
### No Multi-Disk Archive Support
This library does not support multi-disk zip files.
The multi-disk fields in the zipfile spec were intended for a zip file to span multiple floppy disks,
which probably never happens now.
If the "number of this disk" field in the End of Central Directory Record is not `0`,
the `open()`, `fromFd()`, `fromBuffer()`, or `fromRandomAccessReader()` `callback` will receive an `err`.
By extension the following zip file fields are ignored by this library and not provided to clients:
* Disk where central directory starts
* Number of central directory records on this disk
* Disk number where file starts
### No Encryption Support
Currently, the presence of encryption is not even checked,
and encrypted zip files will cause undefined behavior.
### Local File Headers Are Ignored
Many unzip libraries mistakenly read the Local File Header data in zip files.
This data is officially defined to be redundant with the Central Directory information,
and is not to be trusted.
Aside from checking the signature, yauzl ignores the content of the Local File Header.
### No CRC-32 Checking
This library provides the `crc32` field of `Entry` objects read from the Central Directory.
However, this field is not used for anything in this library.
### versionNeededToExtract Is Ignored
The field `versionNeededToExtract` is ignored,
because this library doesn't support the complete zip file spec at any version,
### No Support For Obscure Compression Methods
Regarding the `compressionMethod` field of `Entry` objects,
only method `0` (stored with no compression)
and method `8` (deflated) are supported.
Any of the other 15 official methods will cause the `openReadStream()` `callback` to receive an `err`.
### Data Descriptors Are Ignored
There may or may not be Data Descriptor sections in a zip file.
This library provides no support for finding or interpreting them.
### Archive Extra Data Record Is Ignored
There may or may not be an Archive Extra Data Record section in a zip file.
This library provides no support for finding or interpreting it.
### No Language Encoding Flag Support
Zip files officially support charset encodings other than CP437 and UTF-8,
but the zip file spec does not specify how it works.
This library makes no attempt to interpret the Language Encoding Flag.
## Change History
* 2.4.1
* Fix error handling.
* 2.4.0
* Add ZIP64 support. [issue #6](https://github.com/thejoshwolfe/yazl/issues/6)
* Add `lazyEntries` option. [issue #22](https://github.com/thejoshwolfe/yazl/issues/22)
* Add `readStream.destroy()` method. [issue #26](https://github.com/thejoshwolfe/yazl/issues/26)
* Add `fromRandomAccessReader()`. [issue #14](https://github.com/thejoshwolfe/yazl/issues/14)
* Add `examples/unzip.js`.
* 2.3.1
* Documentation updates.
* 2.3.0
* Check that `uncompressedSize` is correct, or else emit an error. [issue #13](https://github.com/thejoshwolfe/yazl/issues/13)
* 2.2.1
* Update dependencies.
* 2.2.0
* Update dependencies.
* 2.1.0
* Remove dependency on `iconv`.
* 2.0.3
* Fix crash when trying to read a 0-byte file.
* 2.0.2
* Fix event behavior after errors.
* 2.0.1
* Fix bug with using `iconv`.
* 2.0.0
* Initial release.
Reference in New Issue View Git Blame Copy Permalink