Skip to content

fix: call underlying implementation properly #183

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 38 commits into
base: v2
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
f4a997e
fix: call underlying implementation properly
fernandolguevara Jul 10, 2022
3ad026f
test(response): test response write with cb
fernandolguevara Jul 10, 2022
03361ec
test(response.write): fix test expect
fernandolguevara Jul 10, 2022
d76b3cb
fix(response.write): call underlying implementation properly
fernandolguevara Jul 10, 2022
4653825
test(response.write): should have Content-Encoding header
fernandolguevara Jul 10, 2022
2440598
fix(response.write): handle optional arguments
fernandolguevara Jul 10, 2022
2991c82
fix(response.write): handle end(cb)
fernandolguevara Jul 10, 2022
2b3369c
refactor(response-proxy): match nodejs behavior
fernandolguevara Jul 12, 2022
3493e8c
test(compression): avoid arrow fn
fernandolguevara Jul 12, 2022
50ca417
test(compression): add ERR_STREAM_ALREADY_FINISHED
fernandolguevara Jul 12, 2022
e87ca6b
fix(package): add missing dep
fernandolguevara Jul 13, 2022
0ec6742
Merge branch 'master' into master
fernandolguevara Jul 13, 2022
48c7ad6
fix(package): format
fernandolguevara Jul 13, 2022
1abb056
fix(package): use sinon 4.5.0 target
fernandolguevara Jul 16, 2022
6d20bec
Merge branch 'master' of github.com:expressjs/compression
fernandolguevara Jul 23, 2022
ee0c68c
refactor(lib): remove assert-is-uint8array dependency
fernandolguevara Aug 6, 2022
b96d7ac
fix(package): remove semver
fernandolguevara Aug 6, 2022
0736a7f
fix(package): typo in test script
fernandolguevara Aug 6, 2022
27c391f
use sinon 3.3.0
fernandolguevara Aug 6, 2022
96bccad
remove sinon
fernandolguevara Aug 6, 2022
bd4ff2b
remove deconstruct
fernandolguevara Aug 6, 2022
50aa251
listen error event
fernandolguevara Aug 6, 2022
1ecf4da
fix res.write proxy
fernandolguevara Aug 6, 2022
088b23c
emit stream error on res
fernandolguevara Aug 6, 2022
3c69597
fix node 0.8/0.10/0.12 issues
fernandolguevara Aug 6, 2022
5ed9a90
fix lint error
fernandolguevara Aug 6, 2022
8d9fb6c
remove console.log
fernandolguevara Aug 6, 2022
17e6933
fix(test): add code coverage
fernandolguevara Aug 16, 2022
68ee980
fix: ask for runtime version once
fernandolguevara Apr 11, 2023
696e58f
chore(ci):added new node versions
fernandolguevara Mar 3, 2024
3a98b98
chore(ci): bring back node 8/9
fernandolguevara Mar 3, 2024
b89ebed
Merge pull request #1 from fernandolguevara/add-node-versions
fernandolguevara Mar 3, 2024
2d6fa70
Merge branch 'v2' of github.com:expressjs/compression into fernandolg…
bjohansebas Apr 16, 2025
c327bdd
refactor: remove old runtime checks and simplify isUint8Array function
bjohansebas Apr 16, 2025
a664564
test: remove skips tests
bjohansebas Apr 17, 2025
41d4f20
Merge branch 'v2' of github.com:expressjs/compression into fernandolg…
bjohansebas Apr 17, 2025
860790f
refactor: simplify error handling in res.write tests
bjohansebas Apr 17, 2025
9bceb8a
test: remove version check for Node.js in res.write error handling test
bjohansebas Apr 17, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 46 additions & 10 deletions index.js
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ const isFinished = require('on-finished').isFinished
var onHeaders = require('on-headers')
var vary = require('vary')
var zlib = require('zlib')
var ServerResponse = require('http').ServerResponse

/**
* Module exports.
Expand All @@ -36,6 +37,7 @@ module.exports.filter = shouldCompress
* @private
*/
var cacheControlNoTransformRegExp = /(?:^|,)\s*?no-transform\s*?(?:,|$)/

var SUPPORTED_ENCODING = ['br', 'gzip', 'deflate', 'identity']
var PREFERRED_ENCODING = ['br', 'gzip']

Expand Down Expand Up @@ -87,23 +89,49 @@ function compression (options) {

// proxy

res.write = function write (chunk, encoding) {
if (ended) {
res.write = function write (chunk, encoding, callback) {
if (res.destroyed || res.finished || ended) {
// HACK: node doesn't expose internal errors,
// we need to fake response to throw underlying errors type
var fakeRes = new ServerResponse({})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel like this solution is quite hacky and has a high likelihood of breaking in a future Node.js. we really should try to think on a better solution for this than creating a response object and trying to fake a socket object just to get an error. 🤔

Copy link
Author

@fernandolguevara fernandolguevara Apr 11, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't have access to the node internal errors... newer runtime versions add new ones, that's the main reason for this hack

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I understand, but the Node.js project seems to be agressive is breaking projects who are "using the api wrong" even though citgm tells them it is broken, at lesst in the https components. Probably should check if this module it is citgm...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this module is not included it citgm, so really cannot use a hack like this without some blessing from node.js... is this needed if we drop some node.js versions?

fakeRes.on('error', function (err) {
res.emit('error', err)
})
fakeRes.destroyed = res.destroyed
fakeRes.finished = res.finished || ended
// throw ERR_STREAM_DESTROYED or ERR_STREAM_WRITE_AFTER_END
_write.call(fakeRes, chunk, encoding, callback)
return false
}

if (!res.headersSent) {
this.writeHead(this.statusCode)
}

if (chunk) {
chunk = toBuffer(chunk, encoding)
}

return stream
? stream.write(toBuffer(chunk, encoding))
: _write.call(this, chunk, encoding)
? stream.write(chunk, encoding, callback)
: _write.call(this, chunk, encoding, callback)
}

res.end = function end (chunk, encoding) {
if (ended) {
return false
res.end = function end (chunk, encoding, callback) {
if (!callback) {
if (typeof chunk === 'function') {
callback = chunk
chunk = encoding = undefined
} else if (typeof encoding === 'function') {
callback = encoding
encoding = undefined
}
}

if (this.destroyed || this.finished || ended) {
this.finished = ended
// throw ERR_STREAM_WRITE_AFTER_END or ERR_STREAM_ALREADY_FINISHED
return _end.call(this, chunk, encoding, callback)
}

if (!res.headersSent) {
Expand All @@ -116,16 +144,20 @@ function compression (options) {
}

if (!stream) {
return _end.call(this, chunk, encoding)
return _end.call(this, chunk, encoding, callback)
}

// mark ended
ended = true

if (chunk) {
chunk = toBuffer(chunk, encoding)
}

// write Buffer for Node.js 0.8
return chunk
? stream.end(toBuffer(chunk, encoding))
: stream.end()
? stream.end(chunk, encoding, callback)
: stream.end(chunk, callback)
}

res.on = function on (type, listener) {
Expand Down Expand Up @@ -216,6 +248,10 @@ function compression (options) {
res.removeHeader('Content-Length')

// compression
stream.on('error', function (err) {
res.emit('error', err)
})

stream.on('data', function onStreamData (chunk) {
if (isFinished(res)) {
debug('response finished')
Expand Down
240 changes: 234 additions & 6 deletions test/compression.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,211 @@ var http2 = require('http2')
var compression = require('..')

describe('compression()', function () {
describe('should work end and write with valid types (string, Buffer, Uint8Array)', function () {
it('res.write(string)', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
res.end('hello world')
})

request(server)
.get('/')
.set('Accept-Encoding', 'gzip')
.expect('Content-Encoding', 'gzip')
.expect(200, done)
})

it('res.write(Buffer)', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
res.end(Buffer.from('hello world'))
})

request(server)
.get('/')
.set('Accept-Encoding', 'gzip')
.expect('Content-Encoding', 'gzip')
.expect(200, done)
})

it('res.end(cb)', function (done) {
var callbackCalled = false

var server = createServer({ threshold: 0 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
res.write(Buffer.from('hello world'))
res.end(function () {
callbackCalled = true
})
})

request(server)
.get('/')
.set('Accept-Encoding', 'gzip')
.expect('Content-Encoding', 'gzip')
.expect(200, function () {
assert.ok(callbackCalled)
done()
})
})

it('res.end(string, cb)', function (done) {
var callbackCalled = false

var server = createServer({ threshold: 0 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
res.end(Buffer.from('hello world'), function () {
callbackCalled = true
})
})

request(server)
.get('/')
.set('Accept-Encoding', 'gzip')
.expect('Content-Encoding', 'gzip')
.expect(200, function () {
assert.ok(callbackCalled)
done()
})
})

it('res.write(Uint8Array)', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
res.end(new Uint8Array(1))
})

// TODO: see body response
request(server)
.get('/')
.set('Accept-Encoding', 'gzip')
.expect('Content-Encoding', 'gzip')
.expect(200, done)
})
})

describe('should throw with invalid types', function () {
it('res.write(1) should fire ERR_INVALID_ARG_TYPE', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
try {
res.write(1)
} catch (err) {
assert.ok(err.code === 'ERR_INVALID_ARG_TYPE')
res.flush()
res.end()
}
})

request(server)
.get('/')
.set('Accept-Encoding', 'gzip')
.expect(200, done)
})

it('res.write({}) should fire ERR_INVALID_ARG_TYPE', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
try {
res.write({})
} catch (err) {
assert.ok(err.code === 'ERR_INVALID_ARG_TYPE')
res.flush()
res.end()
}
})

request(server)
.get('/')
.set('Accept-Encoding', 'gzip')
.expect(200, done)
})

it('res.write(null) should fire ERR_STREAM_NULL_VALUES', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
try {
res.write(null)
} catch (err) {
assert.ok(err.code === 'ERR_STREAM_NULL_VALUES')
res.flush()
res.end()
}
})

request(server)
.get('/')
.set('Accept-Encoding', 'gzip')
.expect(200, done)
})
})

it('res.write() should return false or throw ERR_STREAM_ALREADY_FINISHED when stream is already finished', function (done) {
var onError = function (err) {
assert.ok(err.toString().indexOf('write after end') > -1 || err.code === 'ERR_STREAM_WRITE_AFTER_END')
}
var server = createServer({ threshold: 0 }, function (req, res) {
res.on('error', onError)
res.setHeader('Content-Type', 'text/plain')
res.end('hello world')

var canWrite = res.write('hola', function (err) {
assert.ok(err.toString().indexOf('write after end') > -1 || err.code === 'ERR_STREAM_ALREADY_FINISHED')
})

assert.ok(!canWrite)
})

request(server)
.get('/')
.set('Accept-Encoding', 'gzip')
.expect(shouldHaveHeader('Content-Encoding'))
.expect(shouldHaveBodyLength('hello world'.length))
.expect(200, done)
})

it('res.write() should call callback if passsed', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')

res.write('hello, world', function () {
res.end()
})
})

request(server)
.get('/')
.set('Accept-Encoding', 'gzip')
.expect(shouldHaveHeader('Content-Encoding'))
.expect(shouldHaveBodyLength('hello, world'.length))
.expect(200, done)
})

it('res.write() should call callback with error after end', function (done) {
var onErrorCalled = false
var onError = function (err) {
assert.ok(err.code === 'ERR_STREAM_WRITE_AFTER_END')
onErrorCalled = true
}

var server = createServer({ threshold: 0 }, function (req, res) {
res.on('error', onError)
res.setHeader('Content-Type', 'text/plain')
res.end('hello, world')

res.write('hello, world', onError)

process.nextTick(function () {
assert.ok(onErrorCalled)
})
})

request(server)
.get('/')
.set('Accept-Encoding', 'gzip')
.end(done)
})

it('should skip HEAD', function (done) {
var server = createServer({ threshold: 0 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
Expand Down Expand Up @@ -417,9 +622,9 @@ describe('compression()', function () {
.expect('Content-Encoding', 'gzip', done)
})

// TODO: why no set Content-Length?
// res.end(str, encoding) broken in node.js 0.8
var run = /^v0\.8\./.test(process.version) ? it.skip : it
run('should handle writing hex data', function (done) {
it('should handle writing hex data', function (done) {
var server = createServer({ threshold: 6 }, function (req, res) {
res.setHeader('Content-Type', 'text/plain')
res.end('2e2e2e2e', 'hex')
Expand Down Expand Up @@ -473,17 +678,30 @@ describe('compression()', function () {
})

it('should return false writing after end', function (done) {
var onErrorCalled = false
var onError = function (err) {
assert.ok(err.toString().indexOf('write after end') > -1 || err.code === 'ERR_STREAM_WRITE_AFTER_END')
onErrorCalled = true
}

var server = createServer({ threshold: 0 }, function (req, res) {
res.on('error', onError)
res.setHeader('Content-Type', 'text/plain')

res.end('hello, world')
assert.ok(res.write() === false)
assert.ok(res.end() === false)

assert.ok(res.write('', onError) === false)

process.nextTick(function () {
assert.ok(onErrorCalled)
})
})

request(server)
.get('/')
.set('Accept-Encoding', 'gzip')
.expect('Content-Encoding', 'gzip', done)
.expect('Content-Encoding', 'gzip')
.end(done)
})
})

Expand Down Expand Up @@ -1259,9 +1477,12 @@ describe('compression()', function () {
})
})

function createServer (opts, fn) {
function createServer (opts, fn, t) {
var _compression = compression(opts)
return http.createServer(function (req, res) {
if (t) {
res.on('finish', function () { console.log(t.title, 'server closed') })
}
_compression(req, res, function (err) {
if (err) {
res.statusCode = err.status || 500
Expand Down Expand Up @@ -1321,6 +1542,13 @@ function shouldHaveBodyLength (length) {
}
}

function shouldHaveHeader (header) {
return function (res) {
var ok = (header.toLowerCase() in res.headers)
assert.ok(ok, 'should have header ' + header)
}
}

function shouldNotHaveHeader (header) {
return function (res) {
assert.ok(!(header.toLowerCase() in res.headers), 'should not have header ' + header)
Expand Down