It is unclear how much time I spent setting up a Discourse forum platform. In term of days I took a little less than 2 days getting my way through the installation, taking notes, browsing for solutions to my problems, and getting the production version of the Discourse up and running on the server.

For detailed version of the notes I’d taken during the whole course of installation procedure, please refer to activity history 011_1 and activity history 011_2 on my GitHub. This article is going to highlight a few noteworthy reminders and tips.

01. Vagrant

Aside from learning how to deploy a Discourse forum, the goal of this stint was to learn how to use Vagrant for software testing and development. I had this thought to directly deploy a Discourse forum on VPS, but since this was my first attempt, I thought to iron out the deployment procedures so I had it in my muscle memory. This would cost me less hour on DigitalOcean (save money a bit here lol).

Another tiny part of this stint was to try Ubuntu Snappy Core, but to no avail I couldn’t get the Snappy Core to run as expected. Running sudo snappy update returned me this error:

(amd64)ubuntu@ubuntu-core-stable-3:~$ sudo snappy update
sudo: unable to resolve host ubuntu-core-stable-3

After googling, the solutions I found on the internet instructed me to edit the /etc/host file. I tried editing that, but Snappy Core has no vim nor nano. My attempt to install either one also resulted in the same error.

Perhaps not my luck then.

The best thing about Vagrant (as far as my knowledge and experience are concerned), the configuration is much more flexible than of Docker. For example I always can reconfigure the networking (IP and ports) of my Vagrant virtual machine. On the other hand, once a Docker container is deployed, there is no way to edit the ports, which I think a minus when it comes to application development and testing.

I shouldn’t compare Vagrant with Docker though, because both belong in two different worlds. Vagrant is a wrapper for OS virtualization technology, while Docker is an application container technology. Vagrant is more concerned about application development, while Docker is more about production of microservices. The areas in which our Venn diagram overlaps here are reproducibility and averting dependency hell.

02. pre-installing Discourse

I always prefer bridged adapter over NAT. Having my virtual machine looks like a physical computer on the network is less troublesome than having it exists behind the host computer’s NAT.

>>> Vagrantfile
config.vm.network "public_network"

The minimum required RAM to deploy a Discourse forum is 1GB accompanied with at least 1GB SWAP space. I allocated 1500MB (1.5GB) RAM because I was expecting myself to break something.

>>> Vagrantfile
config.vm.provider "virtualbox" do |vb|
    vb.memory = "1500"
end

Allocating a SWAP space with swapfile was simply done by running these commands

# enabling 1G swapfile
sudo fallocate -l 1024M /swapfile
ls -lh /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile

# ensure swap is activated on boot
echo "/swapfile       swap    swap    auto      0       0" | sudo tee -a /etc/fstab

There are two ways to install Docker: through the Ubuntu main repository or via the Docker official install script.

# Ubuntu main repo
sudo apt-get install docker.io

# Docker official install script
wget -qO- https://get.docker.com/ | sh

# give $USER access to run Docker without root
sudo usermod -aG docker $USER

# running docker (choose one), ctrl+C required for the latter
sudo service docker start
sudo docker -d &

# install basic packages
sudo apt-get install git wget curl htop dialog locate zsh nano vim

It is recommended to install Docker by using the install script. For installing Discourse, follow the official installation tutorial.

03. installing Discourse

# from official installation guide
mkdir /var/discourse
git clone https://github.com/discourse/discourse_docker.git /var/discourse
cd /var/discourse
cp samples/standalone.yml containers/app.yml

Pay a very close attention to the email configuration. My first attempt at installing Discourse locally was screwed up big time because I didn’t pay sufficient attention to the mail config. Remember to uncomment each line for Discourse SMTP settings, and do not leave white spaces after each value.

After that, execute ./launcher bootstrap app to, erm, bootstrap the new Docker instance for running Discourse. This should take about 8 minutes to complete, then followed with ./launcher start app to launch the Discourse. If you are not sure about the virtual machine’s IP, check it via ifconfig or ip addr show. Rule of thumb: always check the eth0 adapter.

04. the issue with SMTP

Discourse requires obligatory email verification for new user registration, first user (superadmin) included. To check whether the mailer is working on not, ./launcher can help you with that.

./launcher mailtest app

If I am right, mailtest will execute a script /var/discourse/scripts/mailtest and it will send an email based on the configuration in containers/app.yml. If you edit anything on containers/app.yml file, you need to bootstrap Discourse (again) to reflect the new changes.

The problem with SMTP was very annoying, and I think it is due to the Discourse instance being deployed locally. After the Discourse was started, I registered my admin account. I waited so long for the registration confirmation email, but I didn’t get any. My hypotheses were:

1. SMTP port was blocked. I used 587 and 2525, failed.
2. Invalid keys (checked twice, tried both Mailgun and Mandrill, not working).
3. The ISP blocked SMTP, maybe (to avoid mass spam).

Since I couldn’t finish the superadmin account registration, of course I couldn’t finish the installation because I couldn’t get into the admin dashboard on Discourse. The next available solution was to create admin account via console.

# enter the container
./launcher enter app

# create admin account, follow on-screen instruction
rake admin:create

After I managed to log into the Discourse admin dashboard, I figured out that sidekiq failed to send email.

And that left me hanging.

05. MailCatcher

I came up with two (oversimplified) hypotheses to my email config problem: 1) the container and the VM were problematic, 2) the SMTP server couldn’t process the request.

I was fortunate. When I was scrolling the developer advanced installation tutorial, I found a Rubygem called MailCatcher. By configuring the MailCatcher as (local) SMTP server, MailCatcher won’t send email to the target recipient, instead it catches emails that run through it, and you can read the email on MailCatcher’s web UI.

MailCatcher requires ruby-dev for its installation to complete.

# some required dependencies
sudo apt-get install build-essentials libsqlite3-dev

# install ruby-dev
sudo apt-get install ruby-dev

# install MailCatcher
sudo gem install mailcatcher

# running MailCatcher
mailcatcher --http-ip 0.0.0.0 --smtp-ip 0.0.0.0

The MailCatcher’s web interface is accessible at port 1080 through the virtual machine’s IP, and the SMTP port is accessible at 1025. However, there is a problem with this setup: Discourse is running on a Docker container, while the MailCatcher is running on the virtual machine. There is no way the container can connect to the service on its host (the virtual machine), unless they share the same network.

Unless they share the same network. That’s right.

The host (virtual machine that runs the Docker) and the Docker container communicate through the docker0 network adapter. By figuring out the addresses of both host and container via the docker0 adapter, we can have the container to talk to the MailCatcher on the host.

# finding host's IP on docker0
sudo ip addr show docker0

# finding container's IP (container's eth0 --> host's docker0)
ip addr show eth0

In most setup, the host’s IP on docker0 is 172.17.42.1. So now we’ve found the IP, let’s continue with Discourse’s SMTP configuration.

  DISCOURSE_SMTP_ADDRESS: 172.17.42.1
  DISCOURSE_SMTP_PORT: 1025

Each email sent through MailCatcher’s SMTP is accessible on the MailCatcher’s web UI.

Still remember the sidekiq? Soon after MailCatcher was confirmed to work on my setup, sidekiq was able to send registration email to my account, and I could get my account activated. I’d say the error with SMTP just now was due to the Mandrill and Mailgun were unable to process request from localhost Discourse. To further validate this finding, when I launched the live version of my Discourse instance, Aizan’s Coffee Machine, the Mandrill had no problem sending my superadmin user registration email on the first try.

06. extra notes

The Docker container that runs Discourse is stateless. All Discourse-specific data is not stored inside the Docker container, but inside the /var/discourse/shared folder. Inside the Docker container, this folder is mounted at /shared. The Docker container only hosts the systems needed to run a functional forum, such as database system PostgreSQL, Sidekiq, et cetera. Rebuilding or destroying the Docker container won’t affect the Discourse-specific data, unless you choose to delete the content inside /var/discourse/shared folder.

Regarding the resource usage, during bootstrap it will eat 100% CPU resource. When I was running the Discourse on virtual machine, the RAM usage was hovering around 900MB. About the CPU usage, it is not alarming at all. With no traffic, it is around 3%. The spike, apparently, happens during bootstrapping process.

On the live server (which hosts Aizan’s Coffee Machine), the RAM usage is in the neighborhood of 700MB.

Which processes guzzle the RAM usage? ps aux reveals that… unicorns are eating the RAM!

I would like to see the memory consumption on much more lightweight OS, for example, CoreOS. I had this thought to redo the installation by hosting the Discourse on CoreOS instead of Ubuntu, with the apparent benefit is that I don’t have to put much concern on the distro and package upgrade.