System update

posted on 09:00 AM on Saturday 04 May 2019

One of the hard drives on the server is dead so it was replaced with a new 12TB drive. The restarts showed that there were issues with the system drive which is a Crucial M500 960GB SSD. After a couple of fsck repairing the filesystem, there was a missing file which I had to get from one of the LXD containers as it was causing the networking to not start. That fixed things but things are definitely not right with the SSD which by now has ran for 40649 hours (about 4 years and 8 months). This has gotten me worried so I got a new Samsung 860 EVO 500GB SSD as a replacement. Probably a good time to swap to Ubuntu 18.04. There are also bad sectors on one of the Toshiba 6TB drives so we might get another drive failure soon. The resilvering is done so we are at least proof against that.

I still have the old Fractal Design Node 304 which is compete except for a disk. I will take out the one from the Silverstone RL06 which is not working. I will probably keep the casing for now since it could be useful for another build. The mainboard is gone so other than the casing, power supply and fans, there is really nothing of value.

So the procedure would be as follows:

  1. Get the SSD from the Silverstone RL06 and install it into the Node 304.

  2. Download Ubuntu 18.04 LTS and install it into the Node 304.

  3. Migrate the LXD containers from the server to the Node 304. This would probably a good time to update the container environment to make sure that they are not reliant on the home directory which they are now.

  4. Install the new SSD on the server.

  5. Install Ubuntu 16.04 LTS on the server and get the system up and running.

  6. Migrate the LXD containers back to the server.

This is provisional on the Node 304 working. If it is not working, then plan B would be to get a new mainboard, CPU and memory and install that into the Silverstone RL06 and get that running Ubuntu.

So going forward, we will have 2 Ubuntu servers. One main and one backup, this would reduce the downtime for the website.

For the server, I have ordered 5 new Noctua NF-A14 PWM 140mm fans. Will be replacing the existing two 140mm fans with these new ones. The interior temperatures appears to be rather high so the hard drives are near the limits of their operating range. I cleaned the front filter and that seems to have helped a lot. But would still be good to get more air flowing. One of the additional fans will go to the front of the case to help draw air over the hard drives. The other 2 fans will go to the top of the case to help draw air out. I will have to remove the solid panels at the top of the case and this also means that I would have to move the Windows machine onto a shelf since the server will need some air gap at the top.