Home
Node.js
fs createReadStream Example, Text File
This page was last reviewed on Dec 20, 2023.
Dot Net Perls
CreateReadStream. Sometimes a file may be too big to read into a single string. In Node.js, we can use a stream returned by createReadStream to process the file instead.
fs readFile
With the useful on() method, we can process a file in chunks—each chunk has a maximum number of bytes. And we must deal with a Buffer that contains the data.
Buffer
Example. This program opens a large text file from the disk and counts its newlines in 2 ways. To run it, make sure the path points to a large, local text file.
Version 1 This loop within the on() function acts upon the Buffer. It uses indexOf on the buffer to count newline characters.
for
Version 2 Here we also count newline chars, but we access individual bytes from the Buffer in a for-loop.
String charAt
Result When we run this program, the target file is opened, and each version of the code returns the same newline count for each chunk.
const fs = require("node:fs"); // Create a read stream for this file. const stream = fs.createReadStream("programs/words.txt"); // Call on to read the file until its end in chunks. stream.on("data", (chunk) => { // Version 1: count newlines in stream buffer with indexOf loop. var newlines = 0; while (true) { var index = chunk.indexOf("\n", i); if (index == -1) { break; } newlines += 1; i = index + 1; } // Version 2: count newlines in stream buffer with for-loop. let newlines2 = 0; const newlineInt = "\n".charCodeAt(0); for (var i = 0; i < chunk.length; i++) { if (chunk[i] == newlineInt) { newlines2 += 1; } } // Print results. console.log("Newlines:", newlines, "; newlines2:", newlines2); });
Newlines: 6390 ; newlines2: 6390 Newlines: 6643 ; newlines2: 6643 Newlines: 7051 ; newlines2: 7051 Newlines: 6582 ; newlines2: 6582 Newlines: 5961 ; newlines2: 5961 Newlines: 6408 ; newlines2: 6408 Newlines: 6199 ; newlines2: 6199 Newlines: 6465 ; newlines2: 6465 Newlines: 6640 ; newlines2: 6640 Newlines: 7022 ; newlines2: 7022 Newlines: 6528 ; newlines2: 6528 Newlines: 5478 ; newlines2: 5478 Newlines: 7078 ; newlines2: 7078 Newlines: 6677 ; newlines2: 6677 Newlines: 6420 ; newlines2: 6420 Newlines: 6201 ; newlines2: 6201 Newlines: 6239 ; newlines2: 6239 Newlines: 6207 ; newlines2: 6207 Newlines: 6099 ; newlines2: 6099 Newlines: 6284 ; newlines2: 6284 Newlines: 6884 ; newlines2: 6884 Newlines: 6947 ; newlines2: 6947 Newlines: 6487 ; newlines2: 6487 Newlines: 6449 ; newlines2: 6449 Newlines: 6430 ; newlines2: 6430 Newlines: 6571 ; newlines2: 6571 Newlines: 4480 ; newlines2: 4480
Summary. For efficiency, processing a file in chunks with createReadStream and on() is a clear improvement. The entire file does not need to be read into memory or processed at once.
Dot Net Perls is a collection of tested code examples. Pages are continually updated to stay current, with code correctness a top priority.
Sam Allen is passionate about computer languages. In the past, his work has been recommended by Apple and Microsoft and he has studied computers at a selective university in the United States.
This page was last updated on Dec 20, 2023 (new).
Home
Changes
© 2007-2024 Sam Allen.