How do I get an octet-stream in Node.js using http (or request)? -
i'm writing node.js script parse wikidata database dumps , insert them own database. i'm having troubles downloading files, seems both node.js's http module , request module npm ending response stream early. i'm getting following error:
events.js:85 throw er; // unhandled 'error' event ^ syntaxerror: unexpected end of input @ parse (native) @ emit (/var/app/current/node_modules/split/index.js:27:17) @ next (/var/app/current/node_modules/split/index.js:48:7) @ stream.<anonymous> (/var/app/current/node_modules/split/index.js:53:5) @ stream.stream.write (/var/app/current/node_modules/split/node_modules/through/index.js:26:11) @ gunzip.ondata (_stream_readable.js:540:20) @ gunzip.emit (events.js:107:17) @ readableaddchunk (_stream_readable.js:163:16) @ gunzip.readable.push (_stream_readable.js:126:10) @ gunzip.transform.push (_stream_transform.js:140:32)
i'm using following code download database dump:
var streamer = request.get("http:/dumps.wikimedia.org/other/wikidata/20150330.json.gz").pipe(require('zlib').creategunzip()).pipe(split(json.parse)); streamer.on('data', function(obj){ ... });
the error same whether use request module or built-in http module.
Comments
Post a Comment