It is unclear how much time I spent setting up a Discourse forum platform. In term of days I took a little less than 2 days getting my way through the installation, taking notes, browsing for solutions to my problems, and getting the production version of the Discourse up and running on the server.
For detailed version of the notes I’d taken during the whole course of installation procedure, please refer to activity history 011_1 and activity history 011_2 on my GitHub. This article is going to highlight a few noteworthy reminders and tips.
Aside from learning how to deploy a Discourse forum, the goal of this stint was to learn how to use Vagrant for software testing and development. I had this thought to directly deploy a Discourse forum on VPS, but since this was my first attempt, I thought to iron out the deployment procedures so I had it in my muscle memory. This would cost me less hour on DigitalOcean (save money a bit here lol).
Another tiny part of this stint was to try Ubuntu Snappy Core, but to no avail I couldn’t get the Snappy Core to run as expected. Running
sudo snappy update returned me this error:
(amd64)[email protected]:~$ sudo snappy update sudo: unable to resolve host ubuntu-core-stable-3
After googling, the solutions I found on the internet instructed me to edit the
/etc/host file. I tried editing that, but Snappy Core has no
nano. My attempt to install either one also resulted in the same error.
Perhaps not my luck then.
The best thing about Vagrant (as far as my knowledge and experience are concerned), the configuration is much more flexible than of Docker. For example I always can reconfigure the networking (IP and ports) of my Vagrant virtual machine. On the other hand, once a Docker container is deployed, there is no way to edit the ports, which I think a minus when it comes to application development and testing.
I shouldn’t compare Vagrant with Docker though, because both belong in two different worlds. Vagrant is a wrapper for OS virtualization technology, while Docker is an application container technology. Vagrant is more concerned about application development, while Docker is more about production of microservices. The areas in which our Venn diagram overlaps here are reproducibility and averting dependency hell.
02. pre-installing Discourse
I always prefer bridged adapter over NAT. Having my virtual machine looks like a physical computer on the network is less troublesome than having it exists behind the host computer’s NAT.
>>> Vagrantfile config.vm.network "public_network"
The minimum required RAM to deploy a Discourse forum is 1GB accompanied with at least 1GB SWAP space. I allocated 1500MB (1.5GB) RAM because I was expecting myself to break something.
>>> Vagrantfile config.vm.provider "virtualbox" do |vb| vb.memory = "1500" end
Allocating a SWAP space with
swapfile was simply done by running these commands
# enabling 1G swapfile sudo fallocate -l 1024M /swapfile ls -lh /swapfile sudo chmod 600 /swapfile sudo mkswap /swapfile sudo swapon /swapfile # ensure swap is activated on boot echo "/swapfile swap swap auto 0 0" | sudo tee -a /etc/fstab
There are two ways to install Docker: through the Ubuntu main repository or via the Docker official install script.
# Ubuntu main repo sudo apt-get install docker.io # Docker official install script wget -qO- https://get.docker.com/ | sh # give $USER access to run Docker without root sudo usermod -aG docker $USER # running docker (choose one), ctrl+C required for the latter sudo service docker start sudo docker -d & # install basic packages sudo apt-get install git wget curl htop dialog locate zsh nano vim
It is recommended to install Docker by using the install script. For installing Discourse, follow the official installation tutorial.
03. installing Discourse
# from official installation guide mkdir /var/discourse git clone https://github.com/discourse/discourse_docker.git /var/discourse cd /var/discourse cp samples/standalone.yml containers/app.yml
Pay a very close attention to the email configuration. My first attempt at installing Discourse locally was screwed up big time because I didn’t pay sufficient attention to the mail config. Remember to uncomment each line for Discourse SMTP settings, and do not leave white spaces after each value.
After that, execute
./launcher bootstrap app to, erm, bootstrap the new Docker instance for running Discourse. This should take about 8 minutes to complete, then followed with
./launcher start app to launch the Discourse. If you are not sure about the virtual machine’s IP, check it via
ip addr show. Rule of thumb: always check the
04. the issue with SMTP
Discourse requires obligatory email verification for new user registration, first user (superadmin) included. To check whether the mailer is working on not,
./launcher can help you with that.
./launcher mailtest app
If I am right,
mailtest will execute a script
/var/discourse/scripts/mailtest and it will send an email based on the configuration in
containers/app.yml. If you edit anything on
containers/app.yml file, you need to bootstrap Discourse (again) to reflect the new changes.
The problem with SMTP was very annoying, and I think it is due to the Discourse instance being deployed locally. After the Discourse was started, I registered my admin account. I waited so long for the registration confirmation email, but I didn’t get any. My hypotheses were:
- SMTP port was blocked. I used
- Invalid keys (checked twice, tried both Mailgun and Mandrill, not working).
- The ISP blocked SMTP, maybe (to avoid mass spam).
Since I couldn’t finish the superadmin account registration, of course I couldn’t finish the installation because I couldn’t get into the admin dashboard on Discourse. The next available solution was to create admin account via console.
# enter the container ./launcher enter app # create admin account, follow on-screen instruction rake admin:create
After I managed to log into the Discourse admin dashboard, I figured out that
sidekiq failed to send email.
And that left me hanging.
I came up with two (oversimplified) hypotheses to my email config problem: 1) the container and the VM were problematic, 2) the SMTP server couldn’t process the request.
I was fortunate. When I was scrolling the developer advanced installation tutorial, I found a Rubygem called MailCatcher. By configuring the MailCatcher as (local) SMTP server, MailCatcher won’t send email to the target recipient, instead it catches emails that run through it, and you can read the email on MailCatcher’s web UI.
ruby-dev for its installation to complete.
# some required dependencies sudo apt-get install build-essentials libsqlite3-dev # install ruby-dev sudo apt-get install ruby-dev # install MailCatcher sudo gem install mailcatcher # running MailCatcher mailcatcher --http-ip 0.0.0.0 --smtp-ip 0.0.0.0
The MailCatcher’s web interface is accessible at port
1080 through the virtual machine’s IP, and the SMTP port is accessible at
1025. However, there is a problem with this setup: Discourse is running on a Docker container, while the MailCatcher is running on the virtual machine. There is no way the container can connect to the service on its host (the virtual machine), unless they share the same network.
Unless they share the same network. That’s right.
The host (virtual machine that runs the Docker) and the Docker container communicate through the
docker0 network adapter. By figuring out the addresses of both host and container via the
docker0 adapter, we can have the container to talk to the MailCatcher on the host.
# finding host's IP on docker0 sudo ip addr show docker0 # finding container's IP (container's eth0 --> host's docker0) ip addr show eth0
In most setup, the host’s IP on
172.17.42.1. So now we’ve found the IP, let’s continue with Discourse’s SMTP configuration.
DISCOURSE_SMTP_ADDRESS: 172.17.42.1 DISCOURSE_SMTP_PORT: 1025
Each email sent through MailCatcher’s SMTP is accessible on the MailCatcher’s web UI.
Still remember the
sidekiq? Soon after MailCatcher was confirmed to work on my setup,
sidekiq was able to send registration email to my account, and I could get my account activated. I’d say the error with SMTP just now was due to the Mandrill and Mailgun were unable to process request from localhost Discourse. To further validate this finding, when I launched the live version of my Discourse instance, Aizan’s Coffee Machine, the Mandrill had no problem sending my superadmin user registration email on the first try.
06. extra notes
The Docker container that runs Discourse is stateless. All Discourse-specific data is not stored inside the Docker container, but inside the
/var/discourse/shared folder. Inside the Docker container, this folder is mounted at
/shared. The Docker container only hosts the systems needed to run a functional forum, such as database system PostgreSQL, Sidekiq, et cetera. Rebuilding or destroying the Docker container won’t affect the Discourse-specific data, unless you choose to delete the content inside
Regarding the resource usage, during bootstrap it will eat 100% CPU resource. When I was running the Discourse on virtual machine, the RAM usage was hovering around 900MB. About the CPU usage, it is not alarming at all. With no traffic, it is around 3%. The spike, apparently, happens during bootstrapping process.
On the live server (which hosts Aizan’s Coffee Machine), the RAM usage is in the neighborhood of 700MB.
Which processes guzzle the RAM usage?
ps aux reveals that… unicorns are eating the RAM!
I would like to see the memory consumption on much more lightweight OS, for example, CoreOS. I had this thought to redo the installation by hosting the Discourse on CoreOS instead of Ubuntu, with the apparent benefit is that I don’t have to put much concern on the distro and package upgrade.