Category Archives: SWGEmu Mods

A topic to hold information about my mods for the Star Wars Galaxies Emulator.

SWGEmu – Using a RAM Drive to Reduce SSD Wear When Compiling

If we could go back nearly twenty years to the internet of yore, we could read a post I made where I dreamed about having enough RAM to do something useful with a RAM drive in Windows 2000. Since then I think the most use I have gotten out RAM drives has been their use in Linux distributions such as Damn Small Linux and Knoppix. That was until I recently upgraded to 24GB of RAM from 12GB, because the price on DDR3 finally dropped to sane levels and I really wanted to give this a whirl.

What Are We Doing?
We’re using our extra system RAM as a “scratch pad” file system so the GCC compiler can put its junk there while it does its thing.

Why Are Doing This?
The main reason is because SSDs (solid state drives) wear out a little bit every time they are written to and the process of compiling the SWGEmu server (Core3) writes a hell of a lot files to the drive, thereby reducing the drive’s operational life time. This is likely not significant for enterprise MLC based SSDs, but it absolutely is important for consumer grade TLC and 3D NAND drives, as they are designed to be more for reading data than constant writing of data – Compiling large software is an edge-case, abnormal usage for this kind of hardware. The other reason is that doing this is significantly faster, because it eliminates the write and seek speed bottle necks that exist even when using a fast SSD (well, probably not significantly over an NVME M.2 SSD, but I don’t have one to test that assumption).

– A VirtualBox based SWGEmu development environment running Debian/Devuan Linux.
– 20GB RAM will just barely squeeze by with a Windows 10 host OS, so more than that would be ideal.
– A solid state drive and a rotational hard drive.

The Setup
Being completely honest here, this setup is a little complicated, but it makes sense when you look at it on the whole, because the configurations tries to play to strengths of all the hardware that is available. We use the SSD to store the development environment, because that allows it to boot quickly and dramatically improves the performance of Grep searches of the code base and other read-heavy activities. We use a standard old rotational hard drive to store the RAM disk image, log files, and Berkeley database files, because they are frequently written to, but in such a way that the slowness of the drive isn’t really a problem for a testing/development environment. And of course, we use the RAM Drive as the place where GCC spits out its object code and does its executable linking, because that’s an epic ass load of disk writes for what are essentially a bunch of temporary files.

The first thing you will need to do is create the empty virtual disk where you will be storing the RAM drive images. On the settings panel for your VM in VirtualBox go to Storage > Sata Controller > Plus Sign and follow the steps to create a new 20GB VDI image somewhere handy on your rotational hard drive (I put mine in S:\Rob\VirtualBoxVMs\ for instance). In the VirtualBox world, this is the same thing as plugging an empty hard drive into the machine, so the next time you boot up your VM you will need to manually configure the new drive. The steps for that are,

1. Use “sudo cfdisk /dev/sdb” in a terminal to create a partition table that has a single 20GB linux partition. It’s visual and pretty straight forward.

2. Format the partition using “sudo mkfs.ext4 /dev/sdb1”.

3. Create a mount point for the drive.
mkdir /home/swgemu/hd

4. Add the partition to the /etc/fstab file so it will automatically be mounted at startup.
sudo nano /etc/fstab
Add the following line to the file and save it:
/dev/sdb1 /home/swgemu/hd ext3 errors=remount-ro 0 1

Next we need to setup VirtualBox so it can handle the RAM drive. This is where your system specs are important. Compiling Core3 requires approx 850MB of RAM per core used, the Xfce based dev environment needs about 600MB, Windows 10 needs roughly 2GB, and the RAM drive will need a whopping 10GB to house Core3’s crazy volume of object files. On my system I compile using 6 of the 8 cores available on my system and I like to round up to 1GB/core to be on the safe side. So that means that on the System page of settings for my virtual machine I allocate 16,384MB of RAM to the VM, ensuring that the system has enough RAM to avoid ever using the swap file. If you use more or fewer cores, adjust your RAM value accordingly.

Side Note: In the past, Core3 used only 650MB per core and the compiled objects only took up about 4GB on the drive. How times have changed!

Having allocated the RAM, we now need to create a mount point for the RAM drive and add it to fstab so that it will also be ready on boot up.
1. Create a mount point for the drive.
mkdir /home/swgemu/ramdrive

2. Use fstab to create the RAM drive.
sudo nano /etc/fstab
Add the following line to the file and save it:
tmpfs /home/[yourUserName]/ramdrive tmpfs nodev,nosuid,noexec,nodiratime,size=10240M 0 0

Before you reboot, go ahead and open /etc/fstab so you can add the following “mount binds” which will seamlessly allow Core3 to read/write the log and Berkeley database files to different physical hardware than what is normally mounted at these locations. Such is the power of *NIX file systems!

/home/swgemu/hd/logs /home/swgemu/workspace/Core3/MMOCoreORB/bin/log none defaults,bind 0 0
/home/swgemu/hd/databases/ /home/swgemu/workspace/Core3/MMOCoreORB/bin/databases/ none defaults,bind 0 0

Finally, before you reboot you will need to copy the following files from the /home/swgemu/workspace/Core3/MMOCoreORB/build/unix directory to your documents folder, as they are required by the build process and will end up being hidden by a mount bind later on.


Now is the time to reboot!

After the system has loaded, you will need to a “first build and save” to prime the system, after which you will be able to simply use the bash script to load and save your RAM drive image.

1. Setup the RAM drive environment by creating some mound points and bind in the RAM drive. It’s important to keep in mind that everything in ~/ramdrive gets deleted when the system reboots.
cd ~/ramdrive
mkdir tmp
mkdir build
cd build
mkdir unix
sudo mount –bind /tmp /home/swgemu/ramdrive/tmp
sudo mount –bind /home/swgemu/ramdrive/build/unix /home/swgemu/workspace/Core3/MMOCoreORB/build/unix

2. Copy the above saved files into the ramdrive directory

3. Build the server as you would normally. I use a simple build script that tells me how long the compilation took, because that’s fun for me.

4. Run the “./ save” command to copy the contents of /home/swgemu/ramdrive/build to a Tarball image that can be restored each time you reboot the system. The image gets saved to /home/swgemu/hd/ with a file name like so, ram-image-2019_05_23-.tar.gz.

Here is a copy of

./ load
./ save

Both of those uses should self explanatory. When you start the system, first thing you should do is open an terminal and run ./ load to configure the RAM drive and populate it with the previously compiled object code. Any time you have made significant C++ changes, you will want to save a new RAM drive image so that the compiler doesn’t have to catch up compiling changes from way back when, as well as the ones you’re currently working on. As complex as it this whole thing is, it’s really that simple to use it once it’s all setup!

Unfortunately the WordPress theme I am using butchers code, which is why I put link to a copy of the file above, but here’s the code anyway:

# Moves SWGEmu build files to and from hard drive back / ram drive

# Bail if not passed argument
if [ -z "$1" ]
echo "Usage: ramdrive "
exit 1

STAMP=$(date +"%Y_%m_%d")

if [[ $ARG == *"load"* ]]
echo "Mounting /tmp to RAM drive..."
cd ~/ramdrive
mkdir tmp
mkdir build
cd build
mkdir unix
sudo mount --bind /tmp /home/swgemu/ramdrive/tmp
echo "Loading most recent RAM disk image...."
cd $HD
tar -zxvf ram-image-*.tar.gz -C $DIR
echo "Mounting Core3 build/unix to RAM drive..."
sudo mount --bind /home/swgemu/ramdrive/build/unix /home/swgemu/workspace/Core3/MMOCoreORB/build/unix
echo "RAM Drive ready for use!"
exit 1
elif [[ $ARG == *"save"* ]]
echo "Saving...."

cd $HD

if [ -f "ram-"* ]
echo "Archiving previous file..."
FILENAME=$(ls ram-*)

cd $DIR
tar -zcvf $HD/ram-image-$STAMP-.tar.gz unix
echo "Contents of RAM Drive saved to hard drive!"
exit 1
echo "Usage: ramdrive "
exit 1

With all that said, I’m not sure how practical this is for the general SWGEmu modding community to use, but here it is anyway! Mostly, I just wanted to share this, because the younger me inside thinks it’s “so cool!” to be able to do these kinds of things now lol… However, using this setup cut my compilation times down by about 300 seconds while I was working on mods for Tarkin’s Revenge and it definitely saves my 250GB SK Hynix SSD from a whole lot write cycles, which should increase its life span as my Windows 10 system disk and the place where I keep my favorite games and projects.

SWGEmu – 2019 Verbose Server Setup Guide (for Devuan Linux)

Far and away, the most popular post on this blog has been the original “SWGEmu Verbose Server Setup Guide“, so as a sort of parting gift to the handful of readers of this blog, I figured I would post an up to date version of the guide that works with the current iteration of the SWGEmu source code. I’ve decided to keep the blog live after I stop paying for “no-ads”, but I will be posting new content over on GitHub Pages rather than here.

SWGEmu Simple Server Setup using Devuan 2.0 ASCII

This guide is for people who want to setup a custom SWGEmu server, but aren’t interested in using SWGEmu project’s ZonamaDev tool. If you plan on contributing to the SWGEmu project, it’s better to use ZonamaDev, because it’s the standardized work environment for their project and they will help you solve problems with it. Using this guide will give you a working system that will build and run the server code, but you’re on your own for tech support.

Devuan 2.0 is Debian 9 without SystemD. It uses Xfce as its default desktop environment, which is convenient, because it’s both efficient and easy to use. While one could setup a server and build environment without a GUI, Xfce is so small and handy that it may as well be used, given how powerful and cheap PCs are these days.

Keep in mind that it’s not a good idea to use the generic passwords that I have provided in this guide if you’re going to be using the resulting system to host a public server!

1. Download and install VirtualBox
– I used version 5.2.16 in Window 10 Home 64Bit

2. Doanload the “netinst” network installer image for Devuan
Main Page:
Direct File:

3. Open VirtualBox and create a new virtual machine.
– Click the New button
– Name: My SWGEmu Server
– Type: Linux
– Version: Debian 64bit
– Memory: 2048MB minimum, more is better though, so use most or all of the green area if you won’t be doing much with the host system as well.
– Create a new virtual hard disk now
– VDI (VirtualBox Disk Image)
– Dynamically allocated
– 20GB

The final size of the VirtualBox image file will only be about 6GB on the disk, but it can expand up to 20GB without you needing do anything, which is nice – probably unnecessary, but a nice potential time saver down the road.

4. Configure the setting of the new virtual machine.
– Click the settings button

General Window > Advanced Tab
– Shared Clipboard: Bidirectional

System Window > Processor Tab
– CPU Slider: Set it to the max green area.

If you have 6-8 CPUs available, but only 3 or 4 are shoring in the green area, in my experience with AMD FX processors, it’s safe to use up to 4 of 6 or 6 of 8. More CPUs exposed to the virtual machine will reduce the compile time of the SWGEmu server considerably.

– Extended Features: Enable PAE/NX

System Window > Acceleration
– Paravirtualization Interface: Default
– Hardware Virtualziation: Enable VT-x/AMD-V and Enable Nexted Paging.

Display Window > Screen Tab
– Video Memory: 64MB or more

Storage Window
– Background Information: This is where you can add “loop back file systems”, in the form of a new VDI files, which can be mounted in Linux as extra storage. The nice thing about Unix-like operating systems is that other devices, such as remote computers, other hard drives, or one of these loop back files, are seamlessly grafted onto the file system, so the OS treats it like any other part of the file system. You can use this for such things as using “mount bind in your fstab file” to put the Berkeley DB files for the server into their own file or just having a handy Linux formated backup file that is located on a different hard drive than the VDI file for your OS.
I personally have a small solid state drive (SSD), so I have my OS VDI file on the SSD with only a 512MB swap partition inside it to save space on the SSD. To augment the limited amount of swap space, I have another 8GB VDI file on my much larger normal hard drive, which acts as a Linux swap file. Linux allows for setting the priority of swap space, so it will first write to the fast 512MB on the SSD, and if that gets full it will use the slower, larger 8GB on the hard drive. Generally speaking, the swap space rarely gets used at all, even when compiling. There’s lots of powerful customizations you can come up with here, if you’re willing to research on how to use it!

– Storage Tree: Click your FILENAME.vdi and if it is installed on an SSD, put a check in the Solid-state Drive box that pops up on the right side of the window.

Network Window > Adapter 1 Tab
– Attached to: Bridged Adapter

This will allow the virtual machine to get an IP address directly from your router, but we are going to setup a static ip address in Linix so that the game client can always find the server.

Shared Folders Window
– To save some sanity and storage space, you can have one folder to store the TRE files for both the game client and the server. It’s really easy to do by first sharing the game client folder in Windows (right click the folder > Properties > Sharing Tab > Share button > choose “Everyone” in the drop down and click the Share button), then accessing that shared folder from Linux in the virtual machine. . The Windows share will need to be given a name, I suggest calling it SWGEmu.

– Click the [+] button the right
– Folder Path: Other > Navigate to where you have SWGEmu installed > OK > OK

5. Install Devuan Linux
– Make sure you are connected to the internet.
– Select your VM and click the Start button
– In the Select Startup Disk window, navigate to where you saved the iso file for Devuan, select it, press Open, press Start
– Use the keyboard arrow keys to select Graphical Install and hit the Enter key.
– Language: Select yours!
– Country: Select yours!
– Keyboard Conig: American English is the default
– *wait a while*
– Host Name: swgemu (or any other single word you’d like)
– Domain name: leave it blank
– Root Password: 123456 or anything else you would like. 🙂
– Username: swgemu
– Username for account: swgemu
– User password: 123456 or anything else you would like. 🙂
– Timezone: Select yours!
– Partitioning: Guided – Use entire disk

If you know lots about Linux or you want a custom partition setup, this is the step where you can set that up. There are lots of guides on the internet to help you with this step. However, the default “Guided – Use entire disk” will suffice!

– Had disk selection: It’s the VDI you created earlier, so just click continue.
– Partitioning: All files in one partition (recommended for new users)
– Partitioning: Finish partitioning and write changes to disk
– Warning Message: Select Yes, press Continue again. 🙂
– *wait a while*
– Configure Package Manager: Select your country
– FTP Mirror: default Devuan mirror
– HTTP Proxy: Generally speaking, leave this blank. If you need to setup such a thing, look up how to do so on the internet.
– *wait a while*
– Popularity Contest: No
– Software Selection: Devuan desktop environment, Xfce, standard system utilities
– *wait a while*
– Grub boot loader: Yes
– Device for boot loader installation: /dev/sda (the VDI hard drive)
– Installation Complete: Continue and the system will reboot

6. Configure the desktop
– First system boot: login with swgemu / your password
– Panel (toolbar) config popup: Select single empty panel
– Grab the panel drag it to the top or bottom of the screen.

– Right click Panel > Panel Preferences
– Set the length to 100%, Row Size to 26 or so, and click Lock Panel
– Click the Items tab in the and add the following stuff to the toolbar: Applications Menu, Window Buttons, Separator, CPU Graph, Notification Area, Workspace Switcher. Feel free to add anything else you’d like too, of course.

– Right click the Separator on the Panel (it’s a vertical line) > Properties > click the Expand box and change the style to Transparent if so desired. This makes the Panel function the same as the Windows Taskbar.

7. Configure the Sudo
– Open a terminal and add the swgemu user account to the sudoers file
*enter password for the root user account*
*arrow down to the line below root ALL=(ALL:ALL) ALL and type*
swgemu ALL=(ALL:ALL) ALL
*ctl+x, yes save the file, press enter*
sudo apt-get update
*enter password for the swgemu user account*

The system is already up to date, due to the network installer downloading the latest versions of everything! 🙂

8. Configure VirtualBox
– Install the system software we will need to use the VirtualBox Guest Additions
– Open a terminal
sudo apt-get install build-essential dkms linux-headers-amd64 linux-headers-4.9.0-6-amd64

– In the VirtualBox window click Devices > Insert Guest Additions CDROM image…
– Applications Menu > File Manager
– In the File Manager click the CD icon that is labeled VBox_GAs… to mount the CDROM image.
– File > Open Terminal Here
– In the terminal type
sudo sh
– reboot (Applications Menu > Logout > Restart)
– Login again and rejoice in your ability to copy and paste between the host and the guest!
– Open a terminal and add the user to the VirtualBox share group so it can access the Windows shared directory.
sudo adduser $USER vboxsf

9. Install the other software that is required and is available from Devuan/Debian
– Open a terminal
sudo apt-get install git autoconf libdb5.3-dev gdb libtool libreadline-dev libdb-dev libssl-dev libboost-dev openjdk-8-jre openjdk-8-jre-headless mariadb-server-10.1 mariadb-server-core-10.1 libboost-dev libdb5.3 libmariadbclient-dev-compat cmake
– This will install a bunch of other packages automatically, which is normal/good.

10. Install Lua 5.3.2 from source
– Open a terminal
cd ~/Downloads
*wait for it to download*
tar xvzf lua-5.3.2.tar.gz
cd lua-5.3.2
sudo make linux install

– If it gives the errors:
install: cannot stat ‘lua’: No such file or directory
install: cannot stat ‘luac’: No such file or directory
– just press the up arrow and run “sudo make linux install” and it will properly copy the newly compiled binary files to the system directories…

11. Clone the SWGEmu server software
– Open a terminal
mkdir workspace
cd workspace
git clone
*wait a while*
git clone
*wait a while*

You now have two new directories with the SWGEmu software:

12. Configure the SWGEmu environment
– Build the idl compiler and put it in the system path
– Open terminal
cd ~/workspace/PublicEngine/MMOEngine
chmod +x bin/idlc
sudo cp bin/idlc /usr/local/bin/idlc
– Add the idl compiler to the class path so Java can find it
echo ‘export CLASSPATH=”/home/swgemu/workspace/PublicEngine/MMOEngine/bin/idlc.jar”‘ >> /home/swgemu/.profile

Update 2019.05.27: Benjamin Castelli kindly wrote to me (thanks!) to indicate that he had to do the following to get the class path to work for the IDL compiler:
– Add CLASSPATH=”/home/swgemu/workspace/PublicEngine/MMOEngine/bin/idlc.jar” to /etc/environment

– Add a symbolic link to MMOEngine so Core3 can find it
cd ~/workspace/Core3
ln -s ../PublicEngine/MMOEngine MMOEngine
– reboot

13. Configure and build SWGEmu
– Open a terminal
cd ~/workspace/Core3/MMOCoreORB
git checkout unstable
git pull
– Note: You should have already been on the unstable branch and up to date.
make config
make cleanidl
make -j4 build-cmake
– Note: use make -j# where # is the number of cores you allocated to the virtual machine. When in doubt, count the bars in the CPU graph on the Panel.
*wait a while*

14. Give the server access to the game client files
– The server references a significant amount of data directly from the files that get installed with the game itself (“TRE” files). As such, you will need to install SWG from your original CDs, download the official SWGEmu launcher, and run the launcher so it can update your SWG installation to be compatible with SWGEmu.
– As suggested in step 4, you can simply point your virtual machine to the Windows folder where you installed SWGEmu, thus granting the server access to the TRE files that it needs. Alternately you can open that shared folder from the VM and copy the TRE files into the VM itself. I’ll describe how to do both; you can choose which suits your needs.

Pointing to the TRE files on the Windows host system:
– The windows share is automatically mounted at /media/sf_SWG
– Open /home/swgemu/workspace/Core3/MMOCoreORB/bin/conf/config.lua (with the default text editor, Mousepad, by double clicking the file).
– Scroll down to
TrePath = “/home/swgemu/Desktop/SWGEmu”
– and change it to
TrePath = “/media/sf_SWG”
– Save the file and close it.

Copying the TRE files into the virtual machine:
– Open a terminal
mkdir ~/Desktop/SWGEmu
cp -v /media/sf_SWG/*.tre ~/Desktop/SWGEmu

15. Set a static IP Address
– In Windows open a command prompt
Start > Search Box > Type cmd > Hit the Enter Key

– Get your computer’s IP address

In the output, look for the heading Ethernet adapter Local Area Connection. Under it you will find your IPv4 address, which could look like or or on most home networks. The fist three numbers in those chains are the network address and the last number is your computer on the network. Your router (also referred to as a ‘gateway’) is most often the first device on the network, so its number would look like or etc. For your Linux VM, you want to use a unique number that won’t be used by another device as well. 87 is a safe bet!

– In the VM, create the static IP address, being sure to edit the correct ip addresses for your network.
– Open a terminal
sudo nano /etc/network/interfaces
– Arrow down and comment out the following line by putting a # before it
iface eth0 inet dhcp
– Add the following lines
iface eth0 inet static
– ctl+x to exit and save the file.
– reload the network interface to use the static ip address (or reboot)
sudo /etc/init.d/networking reload

16. Configure the MySQL database server
SWGEmu uses two database technologies, MySQL and Berkeley. The MySQL server is used to store account related information, such as character names, passwords, permission levels, and other various other bits of data. The majority of the “game data” that makes up the world, such as quest states, event timers, inventories, etc. is stored in a series of Berkeley database files. You don’t need to configure anything for the Berkeley DB, but you do need to setup the MySQL DB.

– Open up a terminal and configure the root password (to 123456). Initial password is blank. Answer Y to all questions.
sudo mysql_secure_installation

– Create the database
echo ‘CREATE DATABASE swgemu;’ | sudo mysql -uroot -p123456
echo ‘GRANT ALL ON *.* TO `swgemu`@`localhost` IDENTIFIED BY “123456”;’ | sudo mysql -uroot -p123456

– Install the SWGEmu database templates
mysql -p123456 -e source -e ~/workspace/Core3/MMOCoreORB/sql/swgemu.sql;
mysql -p123456 swgemu -e source -e ~/workspace/Core3/MMOCoreORB/sql/datatables.sql;
mysql -p123456 swgemu -e source -e ~/workspace/Core3/MMOCoreORB/sql/mantis.sql;

– Update the server IP address in the SQL database
mysql -p123456 swgemu -v -e “update galaxy set address=’′”

17. Boot the server for the first time
– Open a terminal
cd ~/workspace/Core3/MMOCoreORB/bin

– You will notice some error messages like the following, because only the planet Tatooine is enabled by default.
ERROR: Unable to load screenplay SidorasBey, zone rori is not enabled.
– If all is well, you will see the following message:
(47 s) [Core] initialized
– After that, the first time the server boots it will calculate the navigation meshes that are used by the AI, which can take a long time (and must be done for every planet).

18. Configure the game client
– Go to the directory on the host system where SWGEmu is installed…

– Edit swgemu.cfg so that it reads:
.include “swgemu_login.cfg”
.include “swgemu_live.cfg”
.include “swgemu_preload.cfg”
.include “options.cfg”
.include “user.cfg”

– Edit swgemu_login.cfg so that it points to your server:

– Edit user.cfg for Godmode, multiple instances, frame rate, and extra information when examining items:

0fd345d9 = true



19. Log into the server with the game client on the admin account
– Username: swgemu
– Password: 123456
– Log out and close the game.
– On the server, open a terminal and give the swgemu game account admin privilages
mysql -p123456 swgemu -v -e “UPDATE swgemu.accounts SET admin_level=’15’ WHERE username=’swgemu'”
– Log back into the game on the swgemu account a create a character that will now have “God Mode” privilages.

That’s all that is really needed to run a server. You could even forgo using the Xfce desktop and only install the basic command line of Devuan, but it only saves about 3GB of hard drive space by doing so, which isn’t much in today’s world. Also it should be noted that you can skip all the VirtualBox stuff and simply install Devuan directly onto the hard drive of a real computer and then copy the TRE files from your desktop to your server using a USB stick. Nothing says you must use a virtual machine, it’s just much more convenient in today’s world of multi core CPUs with 8GB+ of RAM and huge, fast storage drives.

Also of note is that I basically have fumbled my way through the SQL portion of this guide – it works, but it’s not my area of expertise. If you’re going use this guide to build a server that you plan on hosting to the public, it’s important to understand that, as with any public facing networked computer, you need to pay close attention to the configuration and security of your server. While this guide gets you a functioning SWGEmu server, it does that and ONLY that. The rest is up to you! 🙂

Update: Bad News for fellow AMD FX CPU Owners

So after playing with this setup, I discovered that compiling Core3 took literally twice as long as it used to! With the Legend of Hondo VM, based on the 2016 version of Core3 and running in a Debian 8 VM running under Windows 7 64 bit, the compile time was 492 seconds using 6 cores (as seen in this post). Today the compilation too a whopping 1140 seconds in my Devuan 2.0 guest, on a Windows 10 64bit host! WTF, eh?

Attempting to mitigate the issue, I installed the backported version of kernel 4.19, upgraded VirtualBox to 5.2.26, disabled the Spectre mitigation in Windows 10 (Meltdown is N/A for this CPU, according to Inspectre.exe, which I believe means the patch isn’t running in Windows), and added “nopti noibrs” to the kernel parameters in grub on the VM, but nothing helped.

What a bummer – compiling software with GCC was the one thing this o’l FX-8320 was great at!

For comparison, I recompiled Legend of Hondo in its Debian 8 VM on this Windows 10 host and it only took 474 seconds. That’s 2.4 times faster!

For further comparison, I compiled Core3 in a current version of the ZonamaDev environment (Debian 9 with kernel 4.9) and it also took way too long, coming in at 1061 seconds.

Well, it’s clearly not a problem with the Windows 10 host or VirtualBox. The problem is either the new version of Core3 or the new version Devuan/Debian/Linux.

Bonus: How to use Linux Kernel 4.19 in Devuan 2.0

This is something I had to do on my computer when I installed Devuan 2.0 directly, as version 4 kernels before 4.12 don’t work very well with my AMD R9 270 video card. It’s not a hard process, but it can be a bit confusing if you’ve never done it before.

1. Add the backports repo to the sources list
– Backports are software that is newer than versions that normally are available with a version of Debian/Devuan. In this case we’re using a newer version of the linux kernel itself.
– Open a terminal and type
sudo echo “deb ascii-backports main” >> /etc/apt/sources.list
sudo echo “deb-src ascii-backports main” >> /etc/apt/sources.list

2. Update the system
– Open a terminal and type
sudo apt-get update
sudo apt-get -t ascii-backports upgrade

3. Search for available kernels
– Open a terminal and type
apt-cache search linux-image

– This will return a list of kernel images that are available. They have specific names. At this time, we’re looking for linux-image-4.19.0-0.bpo.2-amd64, but some of the numbers might be higher by the time you read this, so change them as you need.

4. Install the kernel and kernel headers
– Open a terminal and type
sudo apt-get install linux-image-4.19.0-0.bpo.2-amd64 linux-headers-4.19.0-0.bpo.2-amd64

5. Reboot to start using the new kernel
– The system will automatically boot using the newest kernel.

Legend of Hondo is Playable Solo Again

I took a bit of break from working on RocketTux to bring to Legend of Hondo back to a point where it was playable as a solo experience again. When I started LoH, before the project even had a name, that was really the whole point of the excerise, to make a SWG based … thing I could do when I felt like it. Later I sort of branched out into making “systems” that might be useful to others as well as myself, which prompted me to try and organize the Legend of Hondo repo in such a way that my mods could be more easily visualized or consumed. That ideal was what prompted me to restart Legend of Hondo again in 2016, putting it into a state where it really wasn’t playable solo (without making up for the lack of other players by using an admin account to do various fundamental things).

Given that I just don’t have the time nor do I have the inclination to constantly revisit content that I have already completed for my project, because the upstream folks felt the need to change fundamental aspects of their project, I locked the version of SWGEmu that Legend of Hondo is based on last fall. Then I took a break from working on it from December 2016 to July 2017, in part because I wasn’t able to sit down and have any fun with it when I felt like it. And so I ended my break by rolling through a raft of mods that brought Legend of Hondo to a state where one can sit down and successfully do everything other than use the Pirate System that I have yet to fully design and implement.

The point form feature notes can be read on GitHub here and the commit history (which I am no longer “squishing”) details each mod as I made them (and each patch/fix as I made them as well).

Some of the highlights are…

– Previously, I rebuilt the Skill Tree, removed/modified some professions, and added the Pirate progression trees. This patch ensures that one can use all of it up to Pirate 4xxx (the Bounty Hunter line) without needing to use an admin account to unblock anything (such as, needing Pistol XP, but not having access to a pistol to earn that kind of XP, etc). The only big thing that is missing are the crafted decorations that were made by the Architect profession, but they will return later as things one can buy from merchants (or loot while pirating).
– My version of the Mos Espa city layout, using the Hondo Housing System, is complete. I haven’t gotten moving some NPCs around and populating the new areas though.
– Resource collection got configured for the single player, “no need to run the server 24/7” concept, by making gathering more active and using BazaarBot to fill in the gaps.
– The Hondo Merchant System has returned, with a few merchants who sell important items and a couple who require standing with Jabba the Hutt. Eventually, there will many more (as they are the primary “credit sink” in the game).

Down the road I will hopefully find the time to make the quests, systems, and content that make up the Pirate progression system, but at this point I am not in any hurry to do so. At least as it stands right now, that’s basically “all” that’s left for me to do – the rest of the game is functionally complete (unforeseen bugs/consequences aside). As such, this will probably be the last update for Legend of Hondo in 2017, while I focus my efforts on completing RocketTux.

See the main Legend of Hondo page on this site or the README file on GitHub for more information on how one can play the game (it’s not easy to setup and, no, I won’t help you with it or do it for you, sorry).

Benchmarking My FX-8320 with Core3 and TrinityCore in VirtualBox

One of the biggest disappoints I have had when it comes to computers was buying an early version of the Intel Core2 Quad Q8200, because Intel disabled their hardware virtualization support (VT-d) on it, as part of their arbitrary and consumer-unfriendly pricing scheme. That was back in 2008 and at $185, it was the best new CPU I could afford at the time. Certainly, it was better than the Pentium DualCore I was using!

When it came time to upgrade, I spent a long time researching all of my options for new hardware, so that I could not only get the best performance for my dollar, but so I could have access to all the features and functionality of a desktop. I really wanted to get into using virtual machines for something truly useful… something like modding a SWGEmu server (Core3)! I also wanted to get better performance in games, such as Planetside 2 and Guild Wars 2, but that was secondary.

This was fall 2013 and at that time, hands down the best deal was the AMD FX-8320 if you could catch on sale for $135 CAD or so (with the FX-6300 being the next best for the same price). Absolutely, a Core i7 3770 (non-K, because VT-d is was disabled on the K version…) would have been way better, but it was also $340 – $370 CAD, which was basically my whole upgrade budget. Obviously I couldn’t buy a CPU without a motherboard and RAM, so I waited until the FX-8320 went on sale and bought it. I’ve been nothing but pleased with it since – seriously, it’s a super computer!

I reused my Silverstone Heatsink/Fan tower, which is enough to keep the cpu around 45C while compiling with all 8 threads natively using gcc in Linux. It’s stock speed is 3.5GHz and it turbos up to 4.0GHz. I’ve played around with over clocking on it and it is most happy when sitting at 4.0GHz with turbo and power management options disabled. In Windows, it sits at 4GHz all the time and Linux it down clocks to 1.8GHz while idle. Letting it down clock in Windows causes noticeable performance issues while playing games and while compiling in a virtual machine, but Linux seems fine either way.

Usually I have a single VM open using VirtualBox, where I work using the Xfce desktop environment in my Debian 8 Linux guest, inside my Windows 10 host. This gives me the best of both worlds – all the GNU software I love, functioning pretty much the same as running it on the hardware directly, and all the Windows software I use (mostly DirectX based games) can take full advantage of my AMD R9 270 video card. As much as I appreciate the WINE project, honestly, Windows games work way better in Windows. A lot of GNU software on the other hand seems to work just fine in a virtual machine, which is awesome.

I like using VMs, because they are their own self contained systems that can share files with the host system and with each other, without messing each other up. For instance, while I could build TrinityCore directly in Windows and get a decent performance boost while compiling, it would also mean I would need to have a MySQL database running in the background too and… I don’t want that running all the time. Yes, I could put just the MySQL DB in a VM, but… you know what, I prefer working in Linux anyway, so it’s just better to have the whole thing as one self contained “work environment”. So that’s what I have, a VM for Legend of Hondo, a VM for helping with the Tarkin 2.0 server, and a VM for Solozeroth (TrinityCore). And some other ones, such as old timey Slackware, just because!

Anyhow, with all that background out of the way, here is what compiling Core3 and TrinityCore looks like on my machine!

Host System
AMD FX-8320 (Locked at 4.0GHz with Turbo disabled)
8GB DDR3 2133 RAM (2x 4GB)
SK Hynix SL300 250GB SATA3 SSD
Windows 10 64Bit Build 15063.138
VirtualBox 5.1.20

Core3 Environment
Debian 8.5
Linux kernel 3.16.0-4-amd64
GCC 4.9.2
Core3 (SWGEmu) 2016.10.06

TrinityCore Environment
Debian 8.7
Linux kernel 3.16.0-4-amd64
GCC 4.9.2
TrinityCore 3.3.5 2017.04.22
A chart goes here...
As you can see there, TrinityCore takes a hell of a lot longer to compile from scratch that Core3! It also uses more RAM on average and has a much higher peak RAM usage as well. Apart from that, both projects appear to scale similarly when they have access to more threads.

I should note that the 8 cores on my FX-8320, as far as gcc compiling goes, are indeed a 8 physical pieces of hardware handling one job each, unlike an Intel i7, which would be 4 physical pieces of hardware doing two jobs each. For floating point math operations, my FX-8320 only has 4 physical lumps of hardware that can only handle 4 jobs, unlike an Intel i7, which could handle 8 floating point math jobs. Thankfully the gcc compiler uses the “integer units”, of which I have 8! So with that said, if you have an FX processor and you’re working with gcc, you can safely ignore the warning in VirtualBox about assigning more CPU cores than you really have – crank it to the max and make sure you have enough RAM!

My problem is, 8GB of RAM isn’t really enough for compiling with 8 cores AND running the game in the host system. So, I tend to leave the VMs at 6 cores with 3.5GB RAM, which leaves plenty of RAM for working in both the host and the guest (running the server while playing the game, for instance – which works great btw!). Yes, that does mean that the computer takes long to compile, but nice part is that much of the time I don’t need recompile the entire projects. So in reality, most of the time the difference is more like shaving off 10 seconds from a 40 second compilation, which isn’t worth worry about.

Knocking 10 minutes off that 30 minute compile of TrinityCore might be worth the 30 seconds it takes to shutdown, move the RAM slider, and boot up though. Unless it’s lunch time or “AFK for hours on end, because distractions!” time…

On a related note, I have been thinking lately that it would be interesting to see how this compares to compiling on the same setup using a new AMD Ryzen processor or a recent Intel i5 or i7 processor. I’ve read several benchmarks/reviews, including this Linux gcc compiling related test on XDA, and it’s safe to say that yup, when you spend more money, you get a better processor!

Unfortunately, for the $135 CAD that I spent for my FX-8320 3.5 years ago, it’s still the best option for my work load in its price range. I was hoping the new 4 core, 8 thread Ryzen R5 1400 would be priced around $165 CAD, but it’s $225. The 8 core FX-8300 (a slightly lower clocked, but still fully unlocked, FX-8320) at $145 is only $15 more than the 6 core FX-6300 and honestly it’s a steal for Linux programming and VM work (which is basically the best case scenario for the Bulldozer/Piledriver based CPUs, as their 8 real hardware ALUs are great, but their 4 real hardware FPUs, slow cache, and crowded input pipeline are not so hot for stuff like playing games, music encoding, and some photo editing tools).

It’s kind of a bummer that today I can’t spend less to effectively double my performance, as I did when I made the jump to the $135 CAD FX-8320 from the $185 Core2 Q8200. I was overjoyed back then when my compile times in Rescue Girlies (based on Supertux 0.3.3, an SDL based project) were literally cut in half. That’s my kinda upgrade! Yeah, so anyway, I won’t be upgrading any time soon, because it doesn’t make sense to shell out $295 for the 6 core, 12 thread Ryzen R5 1600 (plus motherboard and RAM) that will almost double my performance. That kind of money would be better spent elsewhere, for all the difference it would actually make in my life! 🙂

When it comes time to upgrade, I am hoping that AMD will have a nice 4 core, 8 thread APU with 512 shaders for around $165. I don’t play any new games and an APU like that would give me a 15% to 25% boost in performance, while dramatically reducing the power usage of my desktop. Yes, it would have half the shaders of my R9 270, so I would probably have to dial back the graphics settings a bit but meh, my old eyes are getting blurry anyway! So we’ll see what 2018 or 2019 brings. Hopefully we’ll get some micro-ATX motherboards with 4GB GDDR6 Video RAM for the APUs, because that would be cool!

Core3 vs. TrinityCore – Which is better to work with?

KISS, that fun phrase every computer science teacher loves to share with his students, is the at the heart of this whole post; TrinityCore (a World of Warcraft server emulator) is a simple, straight forward C++/MySQL project, while Core3 (a Star Wars Galaxies server emulator) is about as “Mad Scientist” as one can get with programming. The difference is… well, it’s truly dumbfounding – so much so that I thought I would take some time to write about it.

If we go way back to my early 20s, I had a good laugh at my friend for playing Everquest, until he bought me a copy and it absorbed me into its oozing mass of unkempt humans and never ending flows of pretty text too. Man that game had a text message for everything!

You fart
You giggle
Kazantoopia runs away in disgust!
You take two steps
You take two more steps
You stop and look around

OK, maybe it wasn’t that bad, but it sure was game that had a funny way of drawing you into it and it basically made me give up single player games for more than a decade. Sure, I moved on from EQ to SWG and WoW (primarily), but I was always playing some kind of MMO. Of course, being a creative person who really wanted to be a part of the worlds I was visiting, I always wished that I could hop in there and make the tweaks and changes to the games that would make them perfect (for me!).

Luckily for me, after I had my fill of online interactions with an increasingly shitty community of people who play online games, I discovered that a bunch of folks far smarter than myself had built systems to emulate the servers of some of the MMOs I liked to play. I did play Star Wars Galaxies, on and off, until it shutdown in December of 2012, but after playing the Mist of Pandaria beta for World of Warcraft, I threw in my towel and walked away from WoW – Blizzard had simply taken a game that I liked and made it into a game that I no longer liked. Unfortunately, for the longest time I had a computer that wasn’t capable of running the 64Bit virtual machines used by these mysterious server emulators, so it took me until early 2014 to start puttering around with them. Since that time, I have learned way more about programming, GNU/Linux, client modding, and project management than I ever knew before and as a result, I have some worldly opinions about what is and isn’t fun work with.

I think the following summary really speaks for itself.

TrinityCore Requires:
– Windows or Linux
– A moden C++ compiler
– Some C++ standard libraries
– A MySQL database
– A text editor (or programming a IDE)

Core3 Requires:
– Linux 64Bit
– Specific versions of the gcc C++ compiler
– Some C++ standard libraries
– Knowledge of the included non-standard C++ libraries
– A whole other custom java program called Public Engine
– IDL compiler (automated C++ creator with its own syntax)
– Specific versions of Lua
– Specific versions of Java
– An understanding of custom C++ hooks for Lua that SWGEmu created
– A MySQL database
– An array of obfuscated Berkeley databases
– A text editor (or programming a IDE)
– A prayer that you don’t need to look something up, because the documentation usually doesn’t exist

I am by no means being harsh, nor am I being soft, that’s just the reality of the design philosophies of the two projects. As you can see from the requirements alone, TrinityCore is obviously easier to work with than Core3, but it gets better. Wait until you get a load of this!

Things that TrinityCore does/has that Core3 can’t do / doesn’t have:

– Excellent documentation.
– A massive list of commands built into the server’s command line.
– A help system that describes the function of all of those commands.
– The ability to use all those server commands on an admin account inside the game.
– Even more commands that can be used inside the game for damned near everything, from database lookups to moving characters from one account to another.
– A sensible relational database for all object and account data that can be manipulated with standardized tools.
– The ability to add new interactive content by simply adding new data to the database.
– C++ that’s “just plain old .cpp and .h files” lol…
– A preconfigured system designed to easily add custom C++ and database content.

And finally…

Things that really fucking suck about Core3 that aren’t an issue with TrinityCore:

– The documentation sucks. There’s not enough of it and what is there is disorganized and often out-dated.
– Client tools are required to look up some important information.
– Said client tools only work in Windows.
– The need to create/edit an arse load of Lua files to make even the most simple content.
– The lack of discipline and planning shown by the way previously completed content is frequently broken by new content and/or arbitrary changes to dependencies.
And my personal favourite:
– The development team has a history of being hostile in general, but their especially disinterested in discussing anything related to the creation of content beyond the scope of their desire to emulate SWG patch 14.1 (bugs and all…).

Now, I have nothing but respect for the good people who have spent thousands of hours building SWGEmu’s Core3 architecture and indeed, it’s better than anything I could put together on my own, but the plain truth of the matter is that in many ways, it’s really not fun to work with at all. When you add the convoluted nightmare of the client, modding SWGEmu quickly becomes a tedious quagmire of, well, stuff that feels more like boring work than fun hobby programming. On the hand, while I can’t say what it’s like to modify the WoW client, as I haven’t had the desire to do so, I can say that modifying the TrinityCore server is pretty much a dream come true.

I really enjoyed my time playing World of Warcraft during the Burning Crusade and Wrath of the Lich King. Now, thanks to TrinityCore, I have a fun little WotLK themed programming hobby, which I call “Solozeroth“, to play around with and learn from. And I must proclaim with utmost of joy and exuberance, puttering with TrinityCore is a genuinely fun way to practice and learn industry standard MySQL and C++!

PSA Soap Box Happy Hour:
Make sure you buy a copy of World of Warcraft or Star Wars Galaxies if you’re going to be working with either, eh. Having spent well over $1,000 on each over the years myself, funnelling my hard earned dollars into Blizzard and Sony Online Entertainment by purchasing game boxes, subscriptions, expansions, and services, I don’t feel bad at all for continuing to use my CD/DVDs of the games (especially given that it’s otherwise impossible to enjoy those versions of the games). Don’t be thieving arsehole, buy the games. 🙂

SWGEmu – Motivations for Modding and other Ramblings

I’m sick and tired, literally I am afraid to say, I… I have a “man-cold” and while my body may be exhausted for really no good reason at all, my not so very sleepy mind desires more than to stare into the darkness. So… I may as use this down time to post about why I spent most of November working on an “Armor Upgrade” for Legend of Hondo.

Feature Documentation:
I suppose the primary motivation was to solve the problem of composite armor being the only really useful design. In part, because it’s ugly and let’s face it, even we men like our digital dollies to look cool (yeah, I went there). But I also looked it at from the perspective of, “gee, wouldn’t it be nice if these other dozen or so armor suits and looted armor parts were actually useful?”. And yes, I did change the white palette so I could run around in a hot pink suit of bone amor when no one is around.

Having the good fortune of playing Star Wars Galaxies in all its incarnations (and not being religiously obsessed with any particular version), as well being exposed to many different computer games and gameplay systems over the years, I went about creating an armor crafting system that is kinda sorta like what was in the final NGE version, but not. I knew that I wanted to nerf the cheese that was Armor Slicing (that thing where any idiot playing a Smuggler can use a piece of garbage to get a 37% bonus to armor, when the Armorsmiths of the galaxy have fight tooth and nail for every single percent), and that I wanted the looted pieces to play a prominent roll in the process. 

Why loot? Well, given that loot comes from enemy’s you need to fight, having a use for the loot encourages the player to go out and get it; In an open-ended single player game like LoH, it’s important to provide opportunities for the player to create generalized goals, such as, “I wonder if can make a piece of armor with low Encumbrance and good Stun resistance? Hmm… What parts would I need to make that happen and where would I get them?”. I like that sort of content, especially the exploration aspects, so I looked at armor crafting as an opportunity to provide that sort of “emergent gameplay”.

At one point I was going to create draft schematics for all the armor types that are available, but upon completing one item I realized that the boring tedium of all that paper pushing (OK, “data entry”) would be a health hazard. I mean, I am quite capable of focusing on the job, but…. JOB people – creating Lua templates and editing IFF files felt like a day at the office, rather than an enjoyable hobby. So, I did something different and made a system that allows the player to upgrade looted and NPC purchased/rewarded armor instead. Honestly, it probably took about the same amount of time, but at least I had fun making it and I added some more variety to the game (and crafting in general, as to upgrade requires a crafted armor segment). When I add the Merchant System, amongst other things the player will be able to purchase all the armor suits in the game, depending on their reputation with the various world factions. Then if they like, they can level some Armorsmith to make segments to upgrade that armor or they can take advantage of the standardized crafting system to make their own from scratch.

Thrax emailed me and asked if I had planned on doing anything with the Armor Rating system, which I had not. I never really got into the high end combat in SWG, so I wasn’t really aware of the details of the system. All I can is, yup, I totally understand why SOE had to do the CU and NGE. The pre-CU combat system and profession system are as unmaintainable and spaghetti coded as systems come. I’m all for complexity, but people, there’s a difference between being complex and being convoluted. The latter is bad and SWG has no shortage of it! After reading the guides in good o’l Biophilia’s Notebook, I concluded that it would be in my best interest to remove both Armor Rating and Armor Piercing entirely. Dumbing down the game? Yes, if you can call dealing with more combat variables than most other games even without those systems, “dumb”. 

I spent a couple days taking it apart and testing combat against various high level MOBs and when all was said and done I found that when no one had Armor Piercing and everyone had Light Armor, the player took far too much damage, far too frequently. It threw off all the calculations I had made in my previous “Life and Death Changes” patch and it became obvious that the only way to truly balance it would be to … Edit every MOB template in the game. You know, just get them all lined up reasonable-like with sensible damage values and… What do I look like, some kind of insanity-man! 

I might, but that’s beside the point. 

So, I scrapped that testing and went about testing it in the opposite direction, by embracing the Dark (convoluted…) Side and allowing the player to upgrade their Armor Rating. Turned out it wasn’t half bad after all! And that’s good, because as much as I mod the crap out of Star Wars Galaxies, I do it because I love the game; I don’t hate it and wish it was something else, I just want it to be something I understand and enjoy – completely (because, hey, I can actually DO that!). Anyway, I wanted the upgrade process to require time and effort, but also be another potentially diverse goal, rather than just the typical slap some resources together BAM we have a Landspeeder? …! With that in mind, I repurposed the looted armor segments that I had previously removed and brought back a reason to hunt them thar Peko Peko Albatrosses, etc. And I suppose now would be a good time to mention that with the two Armor Upgrade procedures, there is finally a real reason to craft an awesome Clothing and Armor Crafting Station! 

Ah, the cough medicine induced ramblings of a middle aged game modder who has nothing better to do with his smartphone as he lay helplessly man-colded in bed… Now to figure out how to make more weapons useful without jacking up SWG and bolting a whole new game underneath. Or sleep… Yeah, maybe that…

BazaarBot – A System to Automatically Sell Items on the Bazaar in SWGEmu

I’ve been taking a little break from working on Legend of Hondo and other projects to help Kinshi and the good folks from the Tarkin community set up Tarkin 2.0 for their small role playing community. That’s nice of me and all, but truth be told, a lot of the stuff I am working on is either stuff I have used in Hondo or stuff I will use all or part of for Hondo. Anyhow, today I am posting about the BazaarBot!

This is an intentionally “dumb auction bot” that periodically lists items for sale on the Bazaar Terminals in the city of Mos Entha. It can be extended to list items in other regions by providing the ObjectIDs of snapshot-file-loaded Bazaar Terminals located in different regions, but for the purposes of Tarkin that wasn’t desired (And I won’t be using it in LoH either, the functionality is there none the less). When I say that the system is dumb, I mean that it doesn’t query the market and decide what it should post. Rather, the system simply posts what the administrator wants it to post, based on their configuration. Being simple in this way gives the admin excellent control of the system, while simultaneously denying players the opportunity coerce the system into listing items more frequently or in greater quantity than the admin desires.

Here is what it can do:

  1. Create and sell stacks of any currently spawning resource.
  2. Create and sell any piece of loot.
  3. Create and sell crafted items.
  4. Log its listing and sales in a custom log file
  5. Be managed using a normal character, as it’s based around using a real character called BazaarBot.

You can checkout the source code for this feature here and read more about it in the file.

Thrax of the Project Pheonix server informed me that this mod won’t compile using the latest version of SWGEmu. I have to say, I really don’t care. The utter lack of discipline shown by the SWGEmu developers is sad, truly sad. It’s no wonder to me that it has taken them more than a decade to produce what they have, because they are constantly throwing wrenches in their machinery by making unnecessary changes to library versions and other ridiculous stuff that doesn’t NEED to be changed. It’s poor project management, plain and simple, and I can’t be arsed to chase their tail with them. So, if a SWGEmu mod that I have made doesn’t work with a version of SWGEmu newer than September 30th, 2016, then you’re on your own to fix whatever minor annoyance they have changed, because I don’t care, it’s a waste of my time.