Working with files

Node.js provides several core modules, including the fs module. fs stands for File System, and this module provides the APIs to interact with the file system.

In this recipe, and throughout the book, we will make use of the node: prefix when importing core modules.

In this recipe, we’ll learn how to read, write, and edit files using the synchronous functions available in the fs module.

Getting ready

Let’s start by preparing a directory and files for this recipe:

1. Create another directory for this recipe:

   $ mkdir working-with-files
   $ cd working-with-files

2. And now, let’s create a file to read. Run the following in your shell to create a file containing some simple text:

   $ echo Hello World! > hello.txt

3. We’ll also need a file for our program—create a file named readWriteSync.js:

   $ touch readWriteSync.js

Important note

The touch utility is a command-line utility included in Unix-based operating systems that is used to update the access and modification date of a file or directory to the current time. However, when touch is run with no additional arguments on a non-existent file, it will create an empty file with that name. The touch utility is a typical way of creating an empty file.

How to do it…

In this recipe, we’ll synchronously read the file named hello.txt, manipulate the contents of the file, and then update the file using synchronous functions provided by the fs module:

1. We’ll start by requiring the fs and path built-in modules. Add the following lines to readWriteSync.js:

   const fs = require('node:fs');
   const path = require('node:path');

2. Now, let’s create a variable to store the file path of the hello.txt file that we created earlier:

   const filepath = path.join(process.cwd(), 'hello.txt');

3. We can now synchronously read the file contents using the readFileSync() function provided by the fs module. We’ll also print the file contents to stdout using console.log():

   const contents = fs.readFileSync(filepath, 'utf8');
   console.log('File Contents:', contents);

4. Now, we can edit the content of the file – we will convert the lowercase text into uppercase:

   const upperContents = contents.toUpperCase();

5. To update the file, we can use the writeFileSync() function. We’ll also add a log statement afterward indicating that the file has been updated:

   fs.writeFileSync(filepath, upperContents);
   console.log('File updated.');

6. Run your program with the following:

   $ node readWriteSync.js
   File Contents: Hello World!
   File updated.

7. To verify the contents were updated, you can open or use cat in your Terminal to show the contents of hello.txt:

   $ cat hello.txt
   HELLO WORLD!

You now have a program that, when run, will read the contents of hello.txt, convert the text content into uppercase, and update the file.

How it works…

As is commonplace, the first two lines of the file require the necessary core modules for the program.

The const fs = require('node:fs'); line will import the core Node.js File System module. The API documentation for the Node.js File System module is available at https://nodejs.org/api/fs.html. The fs module provides APIs to interact with the file system using Node.js. Similarly, the core path module provides APIs for working with file and directory paths. The path module API documentation is available at https://nodejs.org/api/path.html.

Next, we defined a variable to store the file path of hello.txt using the path.join() and process.cwd() functions. The path.join() function joins the path sections provided as parameters with the separator for the specific platform (for example, / on Unix and \ on Windows environments).

The process.cwd() function is a function on the global process object that returns the current directory of the Node.js process. This program is expecting the hello.txt file to be in the same directory as the program.

Next, we read the file using the fs.readFileSync() function. We pass this function the file path to read and the encoding, UTF-8. The encoding parameter is optional—when the parameter is omitted, the function will default to returning a Buffer object.

To perform manipulation of the file contents, we used the toUpperCase() function available on string objects.

Finally, we updated the file using the fs.writeFileSync() function. We passed the fs.writeFileSync() function two parameters. The first parameter was the path to the file we wished to update, and the second parameter was the updated file contents.

Important note

Both the readFileSync() and writeFileSync() APIs are synchronous, which means that they will block/delay concurrent operations until the file read or write is completed. To avoid blocking, you’ll want to use the asynchronous versions of these functions, covered in the There’s more… section of the current recipe.

There’s more…

Throughout this recipe, we were operating on our files synchronously. However, Node.js was developed with a focus on enabling the non-blocking I/O model; therefore, in many (if not most) cases, you’ll want your operations to be asynchronous.

Today, there are three notable ways to handle asynchronous code in Node.js—callbacks, Promises, and async/await syntax. The earliest versions of Node.js only supported the callback pattern. Promises were added to the JavaScript specification with ECMAScript 2015, known as ES6, and subsequently, support for Promises was added to Node.js. Following the addition of Promise support, async/await syntax support was also added to Node.js.

All currently supported versions of Node.js now support callbacks, Promises, and async/await syntax – you may find any of these used in modern Node.js development. Let’s explore how we can work with files asynchronously using these techniques.

Working with files asynchronously

Asynchronous programming can enable some tasks or processing to continue while other operations are happening.

The program from the Working with files recipe was written using the synchronous functions available in the fs module:

const fs = require('node:fs');
const path = require('node:path');
const filepath = path.join(process.cwd(), 'hello.txt');
const contents = fs.readFileSync(filepath, 'utf8');
console.log('File Contents:', contents);
const upperContents = contents.toUpperCase();
fs.writeFileSync(filepath, upperContents);
console.log('File updated.');

This means that the program was blocked waiting for the readFileSync() and writeFileSync() operations to complete. This program can be rewritten to make use of asynchronous APIs.

The asynchronous version of readFileSync() is readFile(). The general convention is that synchronous APIs will have the term “sync” appended to their name. The asynchronous function requires a callback function to be passed to it. The callback function contains the code that we want to be executed when the asynchronous task completes.

The following steps will implement the same behavior as the program from the Working with files recipe but using asynchronous methods:

1. The readFileSync() function in this recipe could be changed to use the asynchronous function with the following:

   const fs = require('node:fs');
   const path = require('node:path');
   const filepath = path.join(process.cwd(),
     'hello.txt');
   fs.readFile(filepath, 'utf8', (err, contents) => {
     if (err) {
       return console.log(err);
     }
     console.log('File Contents:', contents);
     const upperContents = contents.toUpperCase();
     fs.writeFileSync(filepath, upperContents);
     console.log('File updated.');
   });

   Observe that all the processing that is reliant on the file read needs to take place inside the callback function.

2. The writeFileSync() function can also be replaced with the writeFile() asynchronous function:

   const fs = require('node:fs');
   const path = require('node:path');
   const filepath = path.join(process.cwd(),
     'hello.txt');
   fs.readFile(filepath, 'utf8', (err, contents) => {
     if (err) {
       return console.log(err);
     }
     console.log('File Contents:', contents);
     const upperContents = contents.toUpperCase();
     fs.writeFile(filepath, upperContents, (err) => {
       if (err) throw err;
       console.log('File updated.');
     });
   });

   Note that we now have an asynchronous function that calls another asynchronous function. It’s not recommended to have too many nested callbacks as it can negatively impact the readability of the code. Consider the following to see how having too many nested callbacks impedes the readability of the code, which is sometimes referred to as “callback hell”:

   first(args, () => {
       second(args, () => {
           third(args, () => {});
       });
   });

3. Some approaches can be taken to avoid too many nested callbacks. One approach would be to split callbacks into explicitly named functions. For example, our file could be rewritten so that the writeFile() call is contained within its own named function, updateFile():

   const fs = require('node:fs');
   const path = require('node:path');
   const filepath = path.join(process.cwd(), 'hello.txt');
   fs.readFile(filepath, 'utf8', (err, contents) => {
     if (err) {
       return console.log(err);
     }
     console.log('File Contents:', contents);
     const upperContents = contents.toUpperCase();
     updateFile(filepath, upperContents);
   });
   function updateFile (filepath, contents) {
     fs.writeFile(filepath, contents, function (err) {
       if (err) throw err;
       console.log('File updated.');
     });
   }

   Another approach would be to use Promises, which we’ll cover in the Using the fs Promises API section of this chapter. But as the earliest versions of Node.js did not support Promises, the use of callbacks is still prevalent in many npm modules and existing applications.

4. To demonstrate that this code is asynchronous, we can use the setInterval() function to print a string to the screen while the program is running. The setInterval() function enables you to schedule a function to happen after a specified delay in milliseconds. Add the following line to the end of your program:

   setInterval(() => process.stdout.write('**** \n'), 1).unref();

   Observe that the string continues to be printed every millisecond, even in between when the file is being read and rewritten. This shows that the file reading and writing have been implemented in a non-blocking manner because operations are still completing while the file is being handled.

Important note

Using unref() on setInterval() means this timer will not keep the Node.js event loop active. This means that if it is the only active event in the event loop, Node.js may exit. This is useful for timers for which you want to execute an action in the future but do not want to keep the Node.js process running solely.

5. To demonstrate this further, you could add a delay between the reading and writing of the file. To do this, wrap the updateFile() function in a setTimeout() function. The setTimeout() function allows you to pass it a function and a delay in milliseconds:

   setTimeout(() => updateFile(filepath, upperContents), 10);

6. Now, the output from our program should have more asterisks printed between the file read and write, as this is where we added the 10-millisecond delay:

   $ node readFileAsync.js
   ****
   ****
   File Contents: HELLO WORLD!
   ****
   ****
   ****
   ****
   ****
   ****
   ****
   ****
   ****
   File updated.

We can now see that we have converted the program from the Working with files recipe to handle the file operations asynchronously using the callback syntax.

Using the fs Promises API

The fs Promises API was released in Node.js v10.0.0. The API provides File System functions that return Promise objects rather than callbacks. Not all the original fs module APIs have equivalent Promise-based APIs, as only a subset of the original APIs were converted to provide Promise APIs. Refer to the Node.js API documentation for a full list of fs functions provided via the fs Promises API: https://nodejs.org/docs/latest/api/fs.html#promises-api.

A Promise is an object that is used to represent the completion of an asynchronous function. The naming is based on the general definition of the term “promise”—an agreement to do something or that something will happen. A Promise object is always in one of the three following states:

• Pending
• Fulfilled
• Rejected

A Promise will initially be in the pending state and will remain pending until it becomes either fulfilled—when the task has completed successfully—or rejected—when the task has failed.

The following steps will implement the same behavior as the program from the recipe again but using fs Promises API methods:

1. To use the API, you’ll first need to import it:

   const fs = require('node:fs/promises');

2. It is then possible to read the file using the readFile() function:

   fs.readFile(filepath, 'utf8').then((contents) => {
       console.log('File Contents:', contents);
   });

3. You can also combine the fs Promises API with the use of the async/await syntax:

   const fs = require('node:fs/promises');
   const path = require('node:path');
   const filepath = path.join(process.cwd(),
     'hello.txt');
   async function run () {
     try {
       const contents = await fs.readFile(filepath,
         'utf8');
       console.log('File Contents:', contents);
     } catch (error) {
       console.error(error);
     }
   }
   run();

Two notable aspects of this implementation are the use of the following:

• async function run() {...}: Defines an asynchronous function named run(). Asynchronous functions enable the use of the await keyword for handling promises in a more synchronous-looking manner.
• await fs.readFile(filepath, 'utf8'): Uses the await keyword to asynchronously read the contents of the file specified.

Now, we’ve learned how we can interact with files using the fs Promises API.

Important note

Owing to using CommonJS in this chapter, it was necessary to wrap the async/await example in a function as await must only be called from within an asynchronous function with CommonJS. From Chapter 5 onward, we’ll cover ECMAScript modules, where this wrapper function would be unnecessary due to top-level await being supported with ECMAScript modules.

See also

• The Fetching metadata recipe in this chapter
• The Watching files recipe in this chapter
• Chapter 5