Hosting a Node.js application on Google Compute Engine

We'll implement a Node.js application (http://keystonejs.com/) on Google Compute Engine (GCE). GCE is Google's offering for all IaaS needs. Our simple application is built on expressjs and MongoDB. expressjs is a simple web application framework for Node.js and MongoDB is a document-oriented NoSQL database. KeystoneJS also uses a templating engine along with Node.js and MongoDB.

The architecture of our recipe is depicted as follows:

Single-tiered Node.js application on GCE

We will follow a single-tiered approach to host the application and the database on the same VM. Later in this chapter, we'll host the same Node.js application on Google App Engine and Kubernetes Engine.

You'll be using the following services and others for this recipe: 

• GCE
• Google Cloud logging
• Google Cloud Source Repositories

Getting ready

The following are the initial setup verification steps to be taken before the recipe can be executed:

1. Create or select a GCP project.
2. Enable billing and enable the default APIs (some APIs such as BigQuery, storage, monitoring, and a few others are enabled automatically).
3. Install Google Cloud SDK on your development machine. Please follow the steps from https://cloud.google.com/sdk/docs/.
4. Install Node.js and MongoDB on your development machine.

How to do it...

We'll approach this recipe in two stages. In the first stage, we'll prepare our development machine to run our sample Node.js application. Then we'll push the working application to the Compute Engine.

Running the application on the development machine

Follow these steps to download the source code from GitHub and configure it to work on your development machine:

1. Clone the repository in your development space:

$ git clone https://github.com/legorie/gcpcookbook.git 

You can also download the code from: https://github.com/PacktPublishing/Google-Cloud-Platform-Cookbook.

2. Navigate to the directory where the mysite application is stored:

$ cd gcpcookbook/Chapter01/mysite

3. With your favorite editor, create a filename .env in the mysite folder:

COOKIE_SECRET=d44d5c45e7f8149aabc068244 
MONGO_URI=mongodb://localhost/mysite 

4. Install all the packages required for the application to work:

$ npm install 

5. Start the mongod service in your development machine
6. Run the application:

$ node keystone.js 

7. You'll see the following message logged on the Terminal:

------------------------------------------------
Applying update 0.0.1-admins...

------------------------------------------------ 
mySite: Successfully applied update 0.0.1-admins. 

Successfully created:

* 1 User


------------------------------------------------
Successfully applied 1 update.
------------------------------------------------

------------------------------------------------
KeystoneJS Started:
mySite is ready on port 3000
------------------------------------------------

8. The application is now available on http://localhost:3000, as shown:

9.  You can stop the local server by pressing Ctrl + C.

Deploying the application on GCP

To deploy the application to GCP, we'll first upload the working code from our development machine to Google Source Repositories. Then, instead of setting up the VM manually, we'll modify and use a start up script provided by Google to bootstrap the VM with the necessary packages and a runnable application. Finally, we'll create the VM with the bootstrap script and configure the firewall rules so that the application is accessible from the internet.

Moving the code to Google Source Repositories

Each project on GCP has a Git repository which can be accessed by the GCE instances. Though we can manually move the code to an instance, moving it to Source Repositories gives the ability for the compute instances to pull the code automatically via a start up script:

1. If you have made any changes to the code, you can commit the code to the local repository:

git commit -am "Ready to be committed to GCP"

2. Create a new repository under the project:

3. Follow the steps to upload the code from the local repository to Google Source Repositories. In the following example, the project ID is gcp-cookbook and the repository name is gcpcookbook:

4. After the git push command is successful, you'll see the repository updated in the Source Repositories:

Creating the start up script

The start up script is used to initialize the VM during a boot or a restart with the necessary software (MongoDB, Node.js, supervisor, and others) and loads the application from the source code repository. The following script can be found in the /Chapter01/ folder of the Git repository. The start up script performs the following tasks:

1. Installs the logging agent which is an application based on Fluentd.
2. Installs the MongoDB database to be used by the KeystoneJS application.
3. Installs Node.js, Git, and supervisor. Supervisor is a process control system which is used to run our KeystoneJS application as a process.
4. Clones the application code from the repository to the local folder. Update the code at #Line 60, to reflect your repository's URL:

git clone https://source.developers.google.com/p/<PROJECT ID>/r/<REPOSITORY NAME> /opt/app #Line 60

5. Installs the dependencies and creates the .env file to hold the environment variables:

COOKIE_SECRET=<Long Random String>

6. The application is configured to run under the supervisor:

#! /bin/bash 
# Source url: https://github.com/GoogleCloudPlatform/nodejs-getting-started/blob/master/7-gce/gce/startup-script.sh 
# The startup-script is modified to suit the Chapter 01-Recipe 01 of our book
#  Copyright 2017, Google, Inc. 
# Licensed under the Apache License, Version 2.0 (the "License"); 
# you may not use this file except in compliance with the License. 
# You may obtain a copy of the License at 
# 
#    http://www.apache.org/licenses/LICENSE-2.0 
# 
# Unless required by applicable law or agreed to in writing, software 
# distributed under the License is distributed on an "AS IS" BASIS, 
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 
# See the License for the specific language governing permissions and 
# limitations under the License. 
 
# [START startup] 
set -v  

7. Talks to the metadata server to get the project ID:

PROJECTID=$(curl -s "http://metadata.google.internal/computeMetadata/v1/project/project-id" -H "Metadata-Flavor: Google")
# Install logging monitor. The monitor will automatically pick up logs sent to
# syslog.
# [START logging]
curl -s "https://storage.googleapis.com/signals-agents/logging/google-fluentd-install.sh" | bash
service google-fluentd restart &
# [END logging]

8. Installs MongoDB:

apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv EA312927
echo "deb http://repo.mongodb.org/apt/ubuntu xenial/mongodb-org/3.2 multiverse" | sudo tee /etc/apt/sources.list.d/mongodb-org-3.2.list
apt-get update
apt-get install -y mongodb-org
cat > /etc/systemd/system/mongodb.service << EOF
[Unit]
Description=High-performance, schema-free document-oriented database
After=network.target
[Service]
User=mongodb
ExecStart=/usr/bin/mongod --quiet --config /etc/mongod.conf
[Install]
WantedBy=multi-user.target
EOF
systemctl start mongodb
systemctl enable mongodb 

9. Installs dependencies from apt:

apt-get install -yq ca-certificates git nodejs build-essential supervisor

10. Installs Node.js:

mkdir /opt/nodejs
curl https://nodejs.org/dist/v4.2.2/node-v4.2.2-linux-x64.tar.gz | tar xvzf - -C /opt/nodejs --strip-components=1
ln -s /opt/nodejs/bin/node /usr/bin/node
ln -s /opt/nodejs/bin/npm /usr/bin/npm

11. Gets the application source code from the Google Cloud Source Repositories:

# git requires $HOME and it's not set during the startup script.
export HOME=/root
git config --global credential.helper gcloud.sh
git clone https://source.developers.google.com/p/<Project ID>/r/gcpcookbook  /opt/app

12. Installs the app dependencies:

cd /opt/app/Chapter01/mysite
npm install
cat >./.env << EOF
COOKIE_SECRET=d44d5c45e7f8149aabc06a830dba5716b4bd952a639c82499954
MONGODB_URI=mongodb://localhost:27017
EOF

13. Creates a nodeapp user. The application will run as this user:

useradd -m -d /home/nodeapp nodeapp
chown -R nodeapp:nodeapp /opt/app

14. Configures the supervisor to run the nodeapp:

cat >/etc/supervisor/conf.d/node-app.conf << EOF
[program:nodeapp]
directory=/opt/app/Chapter01/mysite
command=npm start
autostart=true
autorestart=true
user=nodeapp
environment=HOME="/home/nodeapp",USER="nodeapp",NODE_ENV="production"
stdout_logfile=syslog
stderr_logfile=syslog
EOF
supervisorctl reread
supervisorctl update
# Application should now be running under supervisor
# [END startup]

Creating and configuring a GCE instance

After creating the start up script, follow these steps:

1. With the start up script ready, we can create an instance using the gcloud command:

$ gcloud compute instances create mysite-instance \
--image-family=debian-8 \
--image-project=debian-cloud \
--machine-type=g1-small \
--scopes userinfo-email,cloud-platform \
--metadata-from-file startup-script=./startup-script.sh \
--zone us-east1-c \
--tags mysite-server 

2. You can check the progress of the instance creation using the following command:

$ gcloud compute instances get-serial-port-output \
mysite-instance --zone us-east1-c

3. Create a firewall rule to allow access to port 3000 to the instance:

$ gcloud compute firewall-rules create default-allow-http-3000 \
--allow tcp:3000 \
--source-ranges 0.0.0.0/0 \
--target-tags mysite-server \
--description "Allow port 3000 access to mysite-server" 

   The following screenshot shows the details of the firewall rule:

The tags on the firewall rule and the create instance commands should match.

4. Get the public IP of the instance from the Google Cloud Console or by using the following command:

$ gcloud compute instances list

5. Navigate to http://<public IP of the instance>:3000 to see the application running.