Putting the title “server maintenance” kinda makes me look like an expert sysadmin, while I am actually not. I have 4 servers running right now (bittorent server, blog server, Wikipedia server, forum server). Yesterday I did an operation to shut down my NodeBB forum because of two reasons: I was not actively using my NodeBB forum, and I wanted to do a test-drive on Discourse forum system for my upcoming project soon.
My NodeBB forum was hosted on a 256MB RamNode OpenVZ VPS. I had a VPS running a MediaWiki installation on DigitalOcean 512MB KVM VPS. The MediaWiki installation was using about 190MB memory. I came up with the plan to clean install the RamNode VPS, then set up the software stack (PHP, Nginx, MariaDB) for the MediaWiki, and then spin down the DigitalOcean VPS.
A full rundown of the process is available here on aixnr/linuxconf activity history. This piece is going to highlight a few interesting notes for future reference.
01. SSHFS, NAT, and Bridged Adapter
I didn’t want to use sftp to retrieve and upload files from local to the server. Sometimes I could easily find myself lost in the middle of blinking cursor on the terminal. To save myself from risking my sanity, I would prefer to have GUI for transferring files back and forth. I opted for sshfs, but not on my OSX.
To use sshfs on OSX it requires me to install OSXFuse. I am not a fan installing additional driver on my OSX (I don’t even have NTFS driver on my OSX), not to mention the the installation package is hosted on SourceForge. I have a VirtualBox instance running on Xubuntu, so let’s just use that.
# start the virtual machine
VBoxManage startvm <machine> --type GUI
# installing sshfs
sudo apt-get install sshfs
# configuring sshfs so $USER can use it
sudo modprobe fuse
sudo adduser $USER fuse
sudo chown root:fuse /dev/fuse
But before connecting the server via sshfs or even to ssh into the server from the virtual machine, if the virtual machine is configured to connect to the internet behind NAT, you might not be able to reach the server. From the VirtualBox’s setting, change the connection method to Bridged Adapter.
The difference between NAT and Bridged Adapter is that when NAT is used, the host computer is acting like a router/switch to the virtual machine, whereas when bridged adapter is used the virtual machine is registered to the network (registered to the router/switch) directly without having the host as the intermediary, which means the virtual machine and the host machine exist on the same level on the network.
I don’t understand why sshfs and ssh fail to connect to the server from behind the NAT, but let’s assume that due to extra layer of networking introduced, and somehow the sshfs and ssh don’t understand the network topology, both don’t know how to reach the server. To say that NAT can cause sshfs and ssh to fail is an overgeneralization, because the router itself that the computer connects to also functions as a NAT.
02. mirroring the whole website
I even suprised myself when I came up with the idea to use full website mirror technique as a means of backing up the website. When I was about to take down talk.aixnr NodeBB forum, I said to myself “we have gems in here, so it is quite a waste to keep it to myself only in form of database dump”.
To create a mirror of talk.aixnr, I first used httrack (installed via brew on OSX). The process is quite easy. If you just run httrack on the terminal you will be greeted with interactive wizard to mirror a website. If you already got the hang of it, simply run:
httrack http://talk.aixnr.me/ -W -O "./talk.aixnr" -%v -r4
-r4 here means it will go 4 links deep (I am not sure what does that actually mean). My first mirror copy of the talk.aixnr created by using the httrack worked well on my local computer, but (plot twist), it broke on my server. I did my homework to find a viable alternative… guess what… wget.
# full command line
wget --mirror --convert-links --adjust-extension --page-requisites --no-parent http://example.org
# ridiculously shortened version
wget -mkEpnp http://example.org
And it worked very fantastic, both local and on my server. The problem I had with this setup was that each thread had .html extension when reached through the browser instead of trailing slash (original configuration of the NodeBB uses trailing slash for each thread). To solve this, let’s tell the nginx to rewrite every trailling slash to .html.
>>> /etc/nginx/site-enabled/site.conf
rewrite ^/(.*)/$ /$1.html permanent;
Why bother rewriting the URL? Because I have a few articles on my blogs linking my afterthoughts and discussions to talk.aixnr. The original URLs are all ended with trailing slash, so it will hit error 404. Instead of editing the articles to reflect the URL changes (only noob does it), tell the nginx this: “when people come here with trailing slash, silently behind the screen, modify the trailing slash to .html so that he or she won’t get error 404”.
03. Markdown footnote: redcarpet vs kramdown
Kramdown can’t parse fenced code block, redcarpet can. On the other hand, redcarpet can’t parse markdown footnote. Also, redcarpet can’t parse markdown strikethrough, ~~strike~~ to produce this strike.
Kramdown version installed: 1.6.0
Redcarpet version installed: 3.3.1
I favor fenced code block compared to 4-space indent for <pre> block. That way I can tell the syntax highlighter the language I am babbling.
04. swapfile isn’t possible inside OpenVZ and Virtuozzo
MariaDB installation process hit a wall, hard. This time I was installing MariaDB on 256MB instance, the installation process was interrupted by errors that I didn’t recognize. Some people on the internet said it might be due to low memory (which could be a factor). To remedy this, one would go with adding swap space with swapfile, because adding RAM is not possible in most cases.
sudo fallocate -l 512M /swapfile
sudo chmod 600 /swapfile
sudo mkswap /swapfile
sudo swapon /swapfile
I prefer fallocate, some prefer dd. I think fallocate is a bit more elegant. When I issued sudo swapon, the terminal returned me this:
swapon: /swapfile: Operation not permitted
Why?
OpenVZ and Virtuozzo don’t support user to run swapon command. The reason is that OpenVZ and Virtuozzo are not true virtualization technology. They are just chroot on steroid. OpenVZ virtual machine container doesn’t emulate hardware like virtual machine technology (VirtualBox) does. My explanation might not be accurate, but this is how I think: swapon command exists as a part of the tool to manipulate hardware resource, but since OpenVZ and Virtuozzo serves chrooted environment on the userspace level, it is not possible to control hardware resource.
I might be gravely inaccurate with my definition, but that’s how I think.
Plot twist: it was not because of the memory the MariaDB installation failed.
05. dialog, locate, and updatedb
MariaDB post-installation configuration runs atop ncurses, which requires dialog to run. I didn’t have dialog installed, so the MariaDB configuration process looked a bit disorderly awkward.
sudo apt-get install dialog
When I was configuring PHPmyadmin, I had to edit php.ini and nginx.conf to overcome the 2MB upload limitation (the SQL dump file I had was 8MB in size). I was lazy to google for the location of php.ini, so I relied on locate to find the configuration file.
# install locate
sudo apt-get install locate
# update the database
sudo updatedb
# find php.ini
locate php.ini
Done.