SWGEmu – Using a RAM Drive to Reduce SSD Wear When Compiling

If we could go back nearly twenty years to the internet of yore, we could read a post I made where I dreamed about having enough RAM to do something useful with a RAM drive in Windows 2000. Since then I think the most use I have gotten out RAM drives has been their use in Linux distributions such as Damn Small Linux and Knoppix. That was until I recently upgraded to 24GB of RAM from 12GB, because the price on DDR3 finally dropped to sane levels and I really wanted to give this a whirl.

What Are We Doing?
We’re using our extra system RAM as a “scratch pad” file system so the GCC compiler can put its junk there while it does its thing.

Why Are Doing This?
The main reason is because SSDs (solid state drives) wear out a little bit every time they are written to and the process of compiling the SWGEmu server (Core3) writes a hell of a lot files to the drive, thereby reducing the drive’s operational life time. This is likely not significant for enterprise MLC based SSDs, but it absolutely is important for consumer grade TLC and 3D NAND drives, as they are designed to be more for reading data than constant writing of data – Compiling large software is an edge-case, abnormal usage for this kind of hardware. The other reason is that doing this is significantly faster, because it eliminates the write and seek speed bottle necks that exist even when using a fast SSD (well, probably not significantly over an NVME M.2 SSD, but I don’t have one to test that assumption).

Requirements
– A VirtualBox based SWGEmu development environment running Debian/Devuan Linux.
– 20GB RAM will just barely squeeze by with a Windows 10 host OS, so more than that would be ideal.
– A solid state drive and a rotational hard drive.

The Setup
Being completely honest here, this setup is a little complicated, but it makes sense when you look at it on the whole, because the configurations tries to play to strengths of all the hardware that is available. We use the SSD to store the development environment, because that allows it to boot quickly and dramatically improves the performance of Grep searches of the code base and other read-heavy activities. We use a standard old rotational hard drive to store the RAM disk image, log files, and Berkeley database files, because they are frequently written to, but in such a way that the slowness of the drive isn’t really a problem for a testing/development environment. And of course, we use the RAM Drive as the place where GCC spits out its object code and does its executable linking, because that’s an epic ass load of disk writes for what are essentially a bunch of temporary files.

The first thing you will need to do is create the empty virtual disk where you will be storing the RAM drive images. On the settings panel for your VM in VirtualBox go to Storage > Sata Controller > Plus Sign and follow the steps to create a new 20GB VDI image somewhere handy on your rotational hard drive (I put mine in S:\Rob\VirtualBoxVMs\ for instance). In the VirtualBox world, this is the same thing as plugging an empty hard drive into the machine, so the next time you boot up your VM you will need to manually configure the new drive. The steps for that are,

1. Use “sudo cfdisk /dev/sdb” in a terminal to create a partition table that has a single 20GB linux partition. It’s visual and pretty straight forward.

2. Format the partition using “sudo mkfs.ext4 /dev/sdb1”.

3. Create a mount point for the drive.
mkdir /home/swgemu/hd

4. Add the partition to the /etc/fstab file so it will automatically be mounted at startup.
sudo nano /etc/fstab
Add the following line to the file and save it:
/dev/sdb1 /home/swgemu/hd ext3 errors=remount-ro 0 1

Next we need to setup VirtualBox so it can handle the RAM drive. This is where your system specs are important. Compiling Core3 requires approx 850MB of RAM per core used, the Xfce based dev environment needs about 600MB, Windows 10 needs roughly 2GB, and the RAM drive will need a whopping 10GB to house Core3’s crazy volume of object files. On my system I compile using 6 of the 8 cores available on my system and I like to round up to 1GB/core to be on the safe side. So that means that on the System page of settings for my virtual machine I allocate 16,384MB of RAM to the VM, ensuring that the system has enough RAM to avoid ever using the swap file. If you use more or fewer cores, adjust your RAM value accordingly.

Side Note: In the past, Core3 used only 650MB per core and the compiled objects only took up about 4GB on the drive. How times have changed!

Having allocated the RAM, we now need to create a mount point for the RAM drive and add it to fstab so that it will also be ready on boot up.
1. Create a mount point for the drive.
mkdir /home/swgemu/ramdrive

2. Use fstab to create the RAM drive.
sudo nano /etc/fstab
Add the following line to the file and save it:
tmpfs /home/[yourUserName]/ramdrive tmpfs nodev,nosuid,noexec,nodiratime,size=10240M 0 0

Before you reboot, go ahead and open /etc/fstab so you can add the following “mount binds” which will seamlessly allow Core3 to read/write the log and Berkeley database files to different physical hardware than what is normally mounted at these locations. Such is the power of *NIX file systems!

/home/swgemu/hd/logs /home/swgemu/workspace/Core3/MMOCoreORB/bin/log none defaults,bind 0 0
/home/swgemu/hd/databases/ /home/swgemu/workspace/Core3/MMOCoreORB/bin/databases/ none defaults,bind 0 0

Finally, before you reboot you will need to copy the following files from the /home/swgemu/workspace/Core3/MMOCoreORB/build/unix directory to your documents folder, as they are required by the build process and will end up being hidden by a mount bind later on.

/home/swgemu/workspace/Core3/MMOCoreORB/build/unix/conf3400.sh
/home/swgemu/workspace/Core3/MMOCoreORB/build/unix/stamp-h1
/home/swgemu/workspace/Core3/MMOCoreORB/build/unix/config/config.h.in

Now is the time to reboot!

After the system has loaded, you will need to a “first build and save” to prime the system, after which you will be able to simply use the ramdrive.sh bash script to load and save your RAM drive image.

1. Setup the RAM drive environment by creating some mound points and bind in the RAM drive. It’s important to keep in mind that everything in ~/ramdrive gets deleted when the system reboots.
cd ~/ramdrive
mkdir tmp
mkdir build
cd build
mkdir unix
sudo mount –bind /tmp /home/swgemu/ramdrive/tmp
sudo mount –bind /home/swgemu/ramdrive/build/unix /home/swgemu/workspace/Core3/MMOCoreORB/build/unix

2. Copy the above saved files into the ramdrive directory
/home/swgemu/ramdrive/build/unix/conf3400.sh
/home/swgemu/ramdrive/build/unix/stamp-h1
/home/swgemu/ramdrive/build/unix/config/config.h.in

3. Build the server as you would normally. I use a simple build script that tells me how long the compilation took, because that’s fun for me.

4. Run the “./ramdrive.sh save” command to copy the contents of /home/swgemu/ramdrive/build to a Tarball image that can be restored each time you reboot the system. The image gets saved to /home/swgemu/hd/ with a file name like so, ram-image-2019_05_23-.tar.gz.

Here is a copy of ramdrive.sh

Usage:
./ramdrive.sh load
./ramdrive.sh save

Both of those uses should self explanatory. When you start the system, first thing you should do is open an terminal and run ./ramdrive.sh load to configure the RAM drive and populate it with the previously compiled object code. Any time you have made significant C++ changes, you will want to save a new RAM drive image so that the compiler doesn’t have to catch up compiling changes from way back when, as well as the ones you’re currently working on. As complex as it this whole thing is, it’s really that simple to use it once it’s all setup!

Unfortunately the WordPress theme I am using butchers code, which is why I put link to a copy of the file above, but here’s the code anyway:


#!/bin/bash
# Moves SWGEmu build files to and from hard drive back / ram drive

# Bail if not passed argument
if [ -z "$1" ]
then
echo "Usage: ramdrive "
exit 1
fi

ARG=$1
DIR="/home/swgemu/ramdrive/build"
HD="/home/swgemu/hd"
STAMP=$(date +"%Y_%m_%d")

if [[ $ARG == *"load"* ]]
then
echo "Mounting /tmp to RAM drive..."
cd ~/ramdrive
mkdir tmp
mkdir build
cd build
mkdir unix
sudo mount --bind /tmp /home/swgemu/ramdrive/tmp
echo "Loading most recent RAM disk image...."
cd $HD
tar -zxvf ram-image-*.tar.gz -C $DIR
echo "Mounting Core3 build/unix to RAM drive..."
sudo mount --bind /home/swgemu/ramdrive/build/unix /home/swgemu/workspace/Core3/MMOCoreORB/build/unix
echo "RAM Drive ready for use!"
exit 1
elif [[ $ARG == *"save"* ]]
then
echo "Saving...."

cd $HD

if [ -f "ram-"* ]
then
echo "Archiving previous file..."
FILENAME=$(ls ram-*)
mv $FILENAME old_$FILENAME
fi

cd $DIR
tar -zcvf $HD/ram-image-$STAMP-.tar.gz unix
echo "Contents of RAM Drive saved to hard drive!"
exit 1
else
echo "Usage: ramdrive "
exit 1
fi

With all that said, I’m not sure how practical this is for the general SWGEmu modding community to use, but here it is anyway! Mostly, I just wanted to share this, because the younger me inside thinks it’s “so cool!” to be able to do these kinds of things now lol… However, using this setup cut my compilation times down by about 300 seconds while I was working on mods for Tarkin’s Revenge and it definitely saves my 250GB SK Hynix SSD from a whole lot write cycles, which should increase its life span as my Windows 10 system disk and the place where I keep my favorite games and projects.

Advertisements

Benchmarking My FX-8320 with Core3 and TrinityCore in VirtualBox

One of the biggest disappoints I have had when it comes to computers was buying an early version of the Intel Core2 Quad Q8200, because Intel disabled their hardware virtualization support (VT-d) on it, as part of their arbitrary and consumer-unfriendly pricing scheme. That was back in 2008 and at $185, it was the best new CPU I could afford at the time. Certainly, it was better than the Pentium DualCore I was using!

When it came time to upgrade, I spent a long time researching all of my options for new hardware, so that I could not only get the best performance for my dollar, but so I could have access to all the features and functionality of a desktop. I really wanted to get into using virtual machines for something truly useful… something like modding a SWGEmu server (Core3)! I also wanted to get better performance in games, such as Planetside 2 and Guild Wars 2, but that was secondary.

This was fall 2013 and at that time, hands down the best deal was the AMD FX-8320 if you could catch on sale for $135 CAD or so (with the FX-6300 being the next best for the same price). Absolutely, a Core i7 3770 (non-K, because VT-d is was disabled on the K version…) would have been way better, but it was also $340 – $370 CAD, which was basically my whole upgrade budget. Obviously I couldn’t buy a CPU without a motherboard and RAM, so I waited until the FX-8320 went on sale and bought it. I’ve been nothing but pleased with it since – seriously, it’s a super computer!

I reused my Silverstone Heatsink/Fan tower, which is enough to keep the cpu around 45C while compiling with all 8 threads natively using gcc in Linux. It’s stock speed is 3.5GHz and it turbos up to 4.0GHz. I’ve played around with over clocking on it and it is most happy when sitting at 4.0GHz with turbo and power management options disabled. In Windows, it sits at 4GHz all the time and Linux it down clocks to 1.8GHz while idle. Letting it down clock in Windows causes noticeable performance issues while playing games and while compiling in a virtual machine, but Linux seems fine either way.

Usually I have a single VM open using VirtualBox, where I work using the Xfce desktop environment in my Debian 8 Linux guest, inside my Windows 10 host. This gives me the best of both worlds – all the GNU software I love, functioning pretty much the same as running it on the hardware directly, and all the Windows software I use (mostly DirectX based games) can take full advantage of my AMD R9 270 video card. As much as I appreciate the WINE project, honestly, Windows games work way better in Windows. A lot of GNU software on the other hand seems to work just fine in a virtual machine, which is awesome.

I like using VMs, because they are their own self contained systems that can share files with the host system and with each other, without messing each other up. For instance, while I could build TrinityCore directly in Windows and get a decent performance boost while compiling, it would also mean I would need to have a MySQL database running in the background too and… I don’t want that running all the time. Yes, I could put just the MySQL DB in a VM, but… you know what, I prefer working in Linux anyway, so it’s just better to have the whole thing as one self contained “work environment”. So that’s what I have, a VM for Legend of Hondo, a VM for helping with the Tarkin 2.0 server, and a VM for Solozeroth (TrinityCore). And some other ones, such as old timey Slackware, just because!

Anyhow, with all that background out of the way, here is what compiling Core3 and TrinityCore looks like on my machine!

Host System
AMD FX-8320 (Locked at 4.0GHz with Turbo disabled)
8GB DDR3 2133 RAM (2x 4GB)
SK Hynix SL300 250GB SATA3 SSD
Windows 10 64Bit Build 15063.138
VirtualBox 5.1.20

Core3 Environment
Debian 8.5
Linux kernel 3.16.0-4-amd64
GCC 4.9.2
Core3 (SWGEmu) 2016.10.06

TrinityCore Environment
Debian 8.7
Linux kernel 3.16.0-4-amd64
GCC 4.9.2
TrinityCore 3.3.5 2017.04.22
A chart goes here...
As you can see there, TrinityCore takes a hell of a lot longer to compile from scratch that Core3! It also uses more RAM on average and has a much higher peak RAM usage as well. Apart from that, both projects appear to scale similarly when they have access to more threads.

I should note that the 8 cores on my FX-8320, as far as gcc compiling goes, are indeed a 8 physical pieces of hardware handling one job each, unlike an Intel i7, which would be 4 physical pieces of hardware doing two jobs each. For floating point math operations, my FX-8320 only has 4 physical lumps of hardware that can only handle 4 jobs, unlike an Intel i7, which could handle 8 floating point math jobs. Thankfully the gcc compiler uses the “integer units”, of which I have 8! So with that said, if you have an FX processor and you’re working with gcc, you can safely ignore the warning in VirtualBox about assigning more CPU cores than you really have – crank it to the max and make sure you have enough RAM!

My problem is, 8GB of RAM isn’t really enough for compiling with 8 cores AND running the game in the host system. So, I tend to leave the VMs at 6 cores with 3.5GB RAM, which leaves plenty of RAM for working in both the host and the guest (running the server while playing the game, for instance – which works great btw!). Yes, that does mean that the computer takes long to compile, but nice part is that much of the time I don’t need recompile the entire projects. So in reality, most of the time the difference is more like shaving off 10 seconds from a 40 second compilation, which isn’t worth worry about.

Knocking 10 minutes off that 30 minute compile of TrinityCore might be worth the 30 seconds it takes to shutdown, move the RAM slider, and boot up though. Unless it’s lunch time or “AFK for hours on end, because distractions!” time…

On a related note, I have been thinking lately that it would be interesting to see how this compares to compiling on the same setup using a new AMD Ryzen processor or a recent Intel i5 or i7 processor. I’ve read several benchmarks/reviews, including this Linux gcc compiling related test on XDA, and it’s safe to say that yup, when you spend more money, you get a better processor!

Unfortunately, for the $135 CAD that I spent for my FX-8320 3.5 years ago, it’s still the best option for my work load in its price range. I was hoping the new 4 core, 8 thread Ryzen R5 1400 would be priced around $165 CAD, but it’s $225. The 8 core FX-8300 (a slightly lower clocked, but still fully unlocked, FX-8320) at $145 is only $15 more than the 6 core FX-6300 and honestly it’s a steal for Linux programming and VM work (which is basically the best case scenario for the Bulldozer/Piledriver based CPUs, as their 8 real hardware ALUs are great, but their 4 real hardware FPUs, slow cache, and crowded input pipeline are not so hot for stuff like playing games, music encoding, and some photo editing tools).

It’s kind of a bummer that today I can’t spend less to effectively double my performance, as I did when I made the jump to the $135 CAD FX-8320 from the $185 Core2 Q8200. I was overjoyed back then when my compile times in Rescue Girlies (based on Supertux 0.3.3, an SDL based project) were literally cut in half. That’s my kinda upgrade! Yeah, so anyway, I won’t be upgrading any time soon, because it doesn’t make sense to shell out $295 for the 6 core, 12 thread Ryzen R5 1600 (plus motherboard and RAM) that will almost double my performance. That kind of money would be better spent elsewhere, for all the difference it would actually make in my life! 🙂

When it comes time to upgrade, I am hoping that AMD will have a nice 4 core, 8 thread APU with 512 shaders for around $165. I don’t play any new games and an APU like that would give me a 15% to 25% boost in performance, while dramatically reducing the power usage of my desktop. Yes, it would have half the shaders of my R9 270, so I would probably have to dial back the graphics settings a bit but meh, my old eyes are getting blurry anyway! So we’ll see what 2018 or 2019 brings. Hopefully we’ll get some micro-ATX motherboards with 4GB GDDR6 Video RAM for the APUs, because that would be cool!