update readme

add subcommands for drop meseta, trade proposal, place trap, trigger trap

identified command ID for process trade on gamecube (0xD0)

CMakeLists.txt

@@ -11,12 +11,14 @@ set(CMAKE_CXX_STANDARD_REQUIRED True)
 if (MSVC)
     add_compile_options(/W4 /WX)
 else()
-    add_compile_options(-Wall -Wextra -Werror)
+    add_compile_options(-Wall -Wextra -Werror -Wno-address-of-packed-member)
 endif()
 
 include_directories("/usr/local/include")
 link_directories("/usr/local/lib")
 
+set(CMAKE_BUILD_TYPE Debug)
+
 
 
 # Executable definitions
@@ -24,45 +26,44 @@ link_directories("/usr/local/lib")
 find_path     (LIBEVENT_INCLUDE_DIR NAMES event.h)
 find_library  (LIBEVENT_LIBRARY     NAMES event)
 find_library  (LIBEVENT_CORE        NAMES event_core)
-find_library  (LIBEVENT_THREAD      NAMES event_pthreads)
 set (LIBEVENT_INCLUDE_DIRS ${LIBEVENT_INCLUDE_DIR})
 set (LIBEVENT_LIBRARIES
         ${LIBEVENT_LIBRARY}
-        ${LIBEVENT_CORE}
-        ${LIBEVENT_THREAD})
+        ${LIBEVENT_CORE})
 
 add_executable(newserv
-  src/FileContentsCache.cc
-  src/Menu.cc
-  src/PSOProtocol.cc
-  src/Client.cc
-  src/Lobby.cc
-  src/ServerState.cc
-  src/Server.cc
-  src/License.cc
-  src/PSOEncryption.cc
-  src/Player.cc
-  src/SendCommands.cc
   src/ChatCommands.cc
-  src/ReceiveSubcommands.cc
-  src/ReceiveCommands.cc
-  src/Version.cc
-  src/Items.cc
-  src/LevelTable.cc
+  src/Client.cc
   src/Compression.cc
-  src/Quest.cc
-  src/RareItemSet.cc
-  src/Map.cc
-  src/NetworkAddresses.cc
-  src/Text.cc
   src/DNSServer.cc
-  src/ProxyServer.cc
-  src/Shell.cc
-  src/ServerShell.cc
-  src/ProxyShell.cc
+  src/FileContentsCache.cc
   src/IPFrameInfo.cc
   src/IPStackSimulator.cc
+  src/Items.cc
+  src/LevelTable.cc
+  src/License.cc
+  src/Lobby.cc
   src/Main.cc
+  src/Map.cc
+  src/Menu.cc
+  src/NetworkAddresses.cc
+  src/Player.cc
+  src/ProxyCommands.cc
+  src/ProxyServer.cc
+  src/PSOEncryption.cc
+  src/PSOProtocol.cc
+  src/Quest.cc
+  src/RareItemSet.cc
+  src/ReceiveCommands.cc
+  src/ReceiveSubcommands.cc
+  src/SendCommands.cc
+  src/Server.cc
+  src/ServerShell.cc
+  src/ServerState.cc
+  src/Shell.cc
+  src/StaticGameData.cc
+  src/Text.cc
+  src/Version.cc
 )
 target_include_directories(newserv PUBLIC ${LIBEVENT_INCLUDE_DIR})
 target_link_libraries(newserv phosg ${LIBEVENT_LIBRARIES})

README.md

@@ -1,56 +1,121 @@
 # newserv
 
-newserv is a game server for Phantasy Star Online (PSO).
+newserv is a game server and proxy for Phantasy Star Online (PSO).
 
 This project includes code that was reverse-engineered by the community in ages long past, and has been included in many projects since then. It also includes some game data from Phantasy Star Online itself; this data was originally created by Sega.
 
-This project is a rewrite of a rewrite of a game server that I wrote many years ago. So far, it works rather well with PSO GC Episodes 1 & 2, and lobbies (but not games) are implemented on Episode 3. newserv probably doesn't work at all for other versions of PSO (DC/PC/BB), since I haven't tested them yet.
+This project is a rewrite of a rewrite of a game server that I wrote many years ago. So far, it works well with PSO GC Episodes 1 & 2, and lobbies (but not games) are implemented on Episode 3. Some basic functionality works on PSO PC and PSO BB, but there are probably still some cases that lead to errors (which will disconnect the client).
 
 Feel free to submit GitHub issues if you find bugs or have feature requests. I'd like to make the server as stable and complete as possible, but I can't promise that I'll respond to issues in a timely manner.
 
-## History
-
-In ages long past (probably 2004? I honestly can't remember), I wrote a proxy for PSO, which I named khyps. This haphazardly-glued-together mess of Windows GUI code and socket programming provided an interface to insert commands into the connection between PSO and its server, enabling some fun new features. Importantly, it also automatically blocked malformed commands which would have crashed the client, providing a safe way to navigate the wasteland that the official Sega servers had turned into after the Action Replay enable code for the game was released.
-
-khyps soon reached "maturity" and became uninteresting, so in 2005 I began writing a PSO server. This project became known as khyller, evolving into a full-featured environment supporting all versions of the game that I had access to - PC, GC, and BB. But as this evolution occurred, the code became increasingly ugly and hard to work with, littered with debugging filth that I never cleaned up and odd coding patterns that I had picked up over the years.
-
-Sometime in 2006 or 2007, I abandoned khyller and rebuilt the entire thing from scratch, resulting in newserv. But this newserv was not the project you're looking at now; 2007's newserv was substantially cleaner in code than khyller but was still quite ugly, and it lacked a few of the more esoteric features I had originally written (for example, the ability to convert any quest into a download quest). I felt better about working with this code, but it still had some stability problems. It turns out that 2007's newserv's concurrency implementation was simply incorrect - I had derived the concept of a mutex myself (before taking any real computer engineering classes) but implemented it incorrectly. No wonder newserv would randomly crash after running seemingly fine for a few days.
-
-A little-known fact is that no version of khyller or newserv was ever tested with the DreamCast versions of PSO. Both projects claimed to support them, but the DC server implementations were based only on chat conversations (likely now lost to time) with other people in the community who had done research on the DC version.
-
-Sometime in October 2018, I had some random cause to reminisce. I looked back in my old code archives and came across newserv. Somehow inspired, I spent a weekend and a couple more evenings rewriting the entire project again, cleaning up ancient patterns I had used eleven years ago, replacing entire modules with simple STL containers, and eliminating even more support files in favor of configuration autodetection. The code is now suitably modern and it no longer has insidious concurrency bugs because it's no longer concurrent - the server is now entirely event-driven.
-
 ## Future
 
 This project is primarily for my own nostalgia; I offer no guarantees on how or when this project will advance.
 
 Current known issues / missing features:
-- Download quests are mostly implemented, but the client doesn't always accept them. It's probably a format issue in file generation.
 - Test all the communication features (info board, simple mail, card search, etc.)
-- PSO PC and PSOBB are essentially entirely untested. Only GC is fairly well-tested.
-- Add all the chat commands that khyller used to have. (Most, but not all, currently exist in newserv.)
+- The trade window isn't implemented yet.
+- PSO PC and PSOBB are not well-tested and likely will disconnect when clients try to use unimplemented features. Only GC is known to be stable and mostly complete.
 
 ## Usage
 
-Currently this code should build on macOS and Ubuntu. It will likely work on other Linux flavors too, but probably will not work on Windows.
+Currently this code should build on macOS and Ubuntu. It will likely work on other Linux flavors too. It should work on Windows as well, but I haven't tested it - the build process could be very manual.
 
-So, you've read all of the above and you want to try it out? Here's what you do:
-- Make sure you have CMake and libevent installed.
+There is a probably-not-too-old macOS release on the newserv GitHub repository (look in the right sidebar).
+
+If you're running Linux or want to build newserv yourself, here's what you do:
+- Make sure you have CMake and libevent installed. (`brew install libevent` on macOS, `sudo apt-get install cmake libevent-dev` on most Linuxes)
 - Build and install phosg (https://github.com/fuzziqersoftware/phosg).
-- Run `cmake . && make`.
-- Edit system/config.json to your liking.
-- Run `./newserv` in the newserv directory. This will start the game server and run the interactive shell. (You can disable the interactive shell later by editing config.json.) You may need `sudo` if newserv's built-in DNS server is enabled.
+- Run `cmake . && make` on the newserv directory.
+- In the system/ directory, make a copy of config.example.json named config.json, and edit it appropriately.
+- Run `./newserv` in the newserv directory. This will start the game server and run the interactive shell. You may need `sudo` if newserv's built-in DNS server is enabled.
 - Use the interactive shell to add a license. Run `help` in the shell to see how to do this.
 
+### Installing quests
+
+newserv automatically finds quests in the system/quests directory. To install your own quests, or to use quests you've saved using the proxy's set-save-files option, just put them in that directory and name them appropriately.
+
+There are multiple PSO quest formats out there; newserv supports most of them. Specifically, newserv can use quests in any of the following formats:
+- bin/dat format: These quests consist of two files with the same base name, a .bin file and a .dat file.
+- Unencrypted GCI format: These quests also consist of a .bin and .dat file, but an encoding is applied on top of them. The filenames should end in .bin.gci and .dat.gci. (Note that there also exists an encrypted GCI format, which newserv does not support.)
+- Encrypted DLQ format: These quests also consist of a .bin and .dat file, but downlaod quest encryption is applied on top of them. The filenames should end in .bin.dlq and .dat.dlq.
+- QST format: These quests consist of only a .qst file, which contains both the .bin and .dat files within it.
+
+Standard quest file names should be like `q###-CATEGORY-VERSION.EXT`; battle quests should be named like `b###-VERSION.EXT`, and challenge quests should be named like `c###-VERSION.EXT`. The fields in each filename are:
+- `###`: quest number (this doesn't really matter; it should just be unique for the version)
+- `CATEGORY`: ret = Retrieval, ext = Extermination, evt = Events, shp = Shops, vr = VR, twr = Tower, gov = Government (BB only), dl = Download (these don't appear during online play), 1p = Solo (BB only)
+- `VERSION`: d1 = Dreamcast v1, dc = Dreamcast v2, pc = PC, gc = GameCube Episodes 1 & 2, gc3 = Episode 3, bb = Blue Burst
+- `EXT`: file extension (bin, dat, bin.gci, dat.gci, bin.dlq, dat.dlq, or qst)
+
+When newserv indexes the quests during startup, it will warn (but not fail) if any quests are corrupt or in unrecognized formats.
+
+If you've changed the contents of the quests directory, you can re-index the quests without restarting the server by running `reload quests` in the interactive shell.
+
+All quests, including those originally in GCI or DLQ format, are treated as online quests unless their filenames specify the dl category. newserv allows players to download all quests, even those in non-download categories.
+
+### Chat commands
+
+The server's shell supports a variety of administration commands. If the interactive shell is enabled, you can enter these commands at any time, even if the prompt isn't visible. Run `help` in the server's shell to see all of the commands and how to use them.
+
+newserv also supports a variety of commands players can use via the chat interface. These commands work on the game server (that is, in lobbies and games hosted by newserv); they do not work on the proxy server. The chat commands are:
+
+* Information commands
+    * `$li`: Shows basic information about the lobby or game you're in.
+    * `$what`: Shows the type, name, and stats of the nearest item on the ground.
+
+* Personal state commands
+    * `$arrow <color-id>`: Changes your lobby arrow color.
+    * `$secid <section-id>`: Sets your override section ID. After running this command, any games you create will use your override section ID for rare drops instead of your character's actual section ID. To revert to your actual section id, run `$secid` with no name after it.
+
+* Blue Burst player commands
+    * `$bbchar <username> <password> <1-4>`: Use this command when playing on a non-BB version of PSO. If the username and password are correct, this command converts your current character to BB format and saves it on the server in the given slot.
+    * `$edit <stat> <value>`: Modifies your character data.
+    * `$item <data>`: Sets the next item to be dropped from an enemy or box.
+
+* Game state commands
+    * `$maxlevel <level>`: Sets the maximum level for players to join the current game.
+    * `$minlevel <level>`: Sets the minimum level for players to join the current game.
+    * `$password <password>`: Sets the game's join password. To unlock the game, run `$password` with nothing after it.
+
+* Cheat mode commands
+    * `$cheat`: Enables or disables cheat mode for the current game. All other cheat mode commands do nothing if cheat mode is disabled.
+    * `$infhp` / `$inftp`: Enables or disables infinite HP or TP mode. Applies to only you. In infinite HP mode, one-hit KO attacks will still kill you.
+    * `$warp <area-id>`: Warps yourself to the given area.
+    * `$next`: Warps yourself to the next area.
+    * `$swa`: Enables or disables switch assist. When enabled, the server will attempt to automatically unlock two-player doors in solo games if you step on both switches sequentially.
+
+* Configuration commands
+    * `$event <event>` / `$allevent <event>`: Sets the current holiday event in the current lobby, or in all lobbies. Holiday events are documented in the "Using $event" item in the information menu.
+    * `$song <song-id>`: Plays a specific song in the current lobby (Episode 3 only).
+
+* Administration commands
+    * `$ann <message>`: Sends an announcement message. The message text is sent to all players in all games and lobbies.
+    * `$ax <message>`: Sends a message to the server's terminal. This cannot be used to run server shell commands; it only prints text to stderr.
+    * `$silence <identifier>`: Silences a player (remove their ability to chat) or unsilences a player. The identifier may be the player's name or guild card number.
+    * `$kick <identifier>`: Disconnects a player. The identifier may be the player's name or guild card number.
+    * `$ban <identifier>`: Bans a player. The identifier may be the player's name or guild card number.
+
+### Using newserv as a proxy
+
+If you want to play online on remote servers rather than running your own server, newserv also includes a PSO proxy. Currently this works with PSO GC and may work with PC; it also works with some BB clients in specific situations.
+
+To use the proxy, add an entry to the ProxyDestinations dictionary in config.json, then run newserv and connect to it as normal (see below). You'll see a "Proxy server" option in the main menu, and you can pick which remote server to connect to.
+
+A few things to be aware of when using the proxy server:
+- On PC and GC, using the Change Ship or Change Block actions from the lobby counter will bring you back to newserv's main menu, not the remote server's ship select. You can go back to the server you were just on by choosing it from newserv's proxy server menu again.
+- The remote server will probably try to assign you a guild card number that doesn't match the one you have on newserv. The proxy server rewrites the commands on the fly to make it look like the remote server assigned you the same guild card number as you have on newserv, but if the remote server has some external integrations (e.g. forum or Discord bots), they will use the guild card number that the remote server believes it has assigned to you. The number assigned by the remote server is shown to you when you connect to the remote server.
+- The proxy server blocks chat commands that look like newserv commands by default, but you can change this with the `set-chat-safety off` shell command if needed.
+- There are shell commands that affect clients on the proxy (run 'help' in the shell to see what they are). All proxy commands in the shell only work when there's exactly one client connected through the proxy, since there isn't (yet) a way to say via the shell which session you want to affect.
+
 ### Connecting local clients
 
-If you're running PSO on a real GameCube, you can make it connect to newserv by setting its default gateway and DNS server addresses to newserv's address. Note that newserv's DNS server is disabled by default; you'll have to enable it in config.json.
+If you're running PSO on a real GameCube, you can make it connect to newserv by setting its default gateway and DNS server addresses to newserv's address. newserv's DNS server must be running on port 53 and accessible.
 
-If you have PSO Plus or Episode III, it won't want to connect to a server on the same local network as the GameCube itself, as determined by the GC's IP address and subnet mask. In the old days, one way to get around this was to create a fake network adapter on the server with an IP address on a different subnet (or use an existing real one), tell the GameCube that the server is the default gateway, and have the server reply to the DNS request with its non-local IP address. To do this with newserv, just set LocalAddress in the config file to a different interface. For example, if the GameCube is on the 192.168.0.x network and your other adapter has address 10.0.1.6, set LocalAddress to 10.0.1.6. This may not work on modern systems or on non-Windows machines - I haven't tested it in many years.
+If you have PSO Plus or Episode III, it won't want to connect to a server on the same local network as the GameCube itself, as determined by the GameCube's IP address and subnet mask. In the old days, one way to get around this was to create a fake network adapter on the server (or use an existing real one) that has an IP address on a different subnet, tell the GameCube that the server is the default gateway, and have the server reply to the DNS request with its non-local IP address. To do this with newserv, just set LocalAddress in the config file to a different interface. For example, if the GameCube is on the 192.168.0.x network and your other adapter has address 10.0.1.6, set newserv's LocalAddress to 10.0.1.6 and set PSO's DNS server and default gateway addresses to the server's 192.168.0.x address. This may not work on modern systems or on non-Windows machines - I haven't tested it in many years.
 
-If you're emulating PSO using Dolphin on macOS, you can make it connect to a newserv instance running on the same machine via the tapserver interface. This works for all PSO versions, including Plus and Episode III, without the trickery described above. To do this:
-- Use a build of Dolphin that has tapserver support, and set the BBA type to tapserver (Config -> GameCube -> SP1).
-- Enable the IP stack simulator according to the comments in config.json, and start newserv. You do not need to install or run tapserver.
+If you're emulating PSO using a version of Dolphin with tapserver support (currently only the macOS version), you can make it connect to a newserv instance running on the same machine via the tapserver interface. This works for all PSO versions, including Plus and Episode III, without the trickery described above. To do this:
+- Set Dolphin's BBA type to tapserver (Config -> GameCube -> SP1).
+- Enable newserv's IP stack simulator according to the comments in config.json, and start newserv. You do not need to install or run tapserver.
 - In PSO, you have to configure the network settings manually (DHCP doesn't work), but the actual values don't matter as long as they're valid IP addresses. Example values:
   - IP address: `10.0.1.5`
   - Subnet mask: `255.255.255.0`
@@ -61,23 +126,6 @@ If you're emulating PSO using Dolphin on macOS, you can make it connect to a new
 
 ### Connecting external clients
 
-If you want to accept connections from outside your local network, you'll need to set ExternalAddress to your public IP address in the configuration file, and you'll likely need to open some ports in your router's NAT configuration. You'll need to open the following ports depending on which client versions you want to be able to connect:
+If you want to accept connections from outside your local network, you'll need to set ExternalAddress to your public IP address in the configuration file, and you'll likely need to open some ports in your router's NAT configuration - specifically, all the TCP ports listed in PortConfiguration in config.json.
 
-    PSO PC           9100, 9300, 9420, 10000
-    PSO GC 1.0 JP    9000, 9421
-    PSO GC 1.1 JP    9001, 9421
-    PSO GC Ep3 JP    9003, 9421
-    PSO GC 1.0 US    9100, 9421
-    PSO GC Ep3 US    9103, 9421
-    PSO GC 1.0 EU    9200, 9421
-    PSO GC 1.1 EU    9201, 9421
-    PSO GC Ep3 EU    9203, 9421
-    PSO BB           9422, 11000, 12000, 12004, 12005, 12008
-
-For GC clients, you'll have to use newserv's built-in DNS server or set up your own DNS server as well. Remote players can connect to your server by entering your DNS server's IP address in their client's network configuration. If you use newserv's built-in DNS server, you'll also need to forward UDP port 53 to your newserv instance.
-
-### Using newserv as a proxy
-
-If you want to play online on remote servers rather than running your own server, newserv also includes a PSO proxy. Currently this works with PSO GC and may work with PC, but not with BB.
-
-Run newserv like `./newserv --proxy-destination=1.1.1.1` (replace the IP address appropriately for the server you want to connect to). This works for normal clients (using the connection parameters in config.json), as well as tapserver clients.
+For GC clients, you'll have to use newserv's built-in DNS server or set up your own DNS server as well. If you want external clients to be able to use your DNS server, you'll have to forward UDP port 53 to your newserv instance. Remote players can then connect to your server by entering your DNS server's IP address in their client's network configuration.

src/ChatCommands.cc

@@ -13,307 +13,11 @@
 #include "Client.hh"
 #include "SendCommands.hh"
 #include "Text.hh"
+#include "StaticGameData.hh"
 
 using namespace std;
 
 
-////////////////////////////////////////////////////////////////////////////////
-
-const vector<string> section_id_to_name({
-  "Viridia", "Greennill", "Skyly", "Bluefull", "Purplenum", "Pinkal", "Redria",
-  "Oran", "Yellowboze", "Whitill"});
-
-const unordered_map<string, uint8_t> name_to_section_id({
-  {"viridia", 0},
-  {"greennill", 1},
-  {"skyly", 2},
-  {"bluefull", 3},
-  {"purplenum", 4},
-  {"pinkal", 5},
-  {"redria", 6},
-  {"oran", 7},
-  {"yellowboze", 8},
-  {"whitill", 9}});
-
-const vector<string> lobby_event_to_name({
-  "none", "xmas", "none", "val", "easter", "hallo", "sonic", "newyear",
-  "summer", "white", "wedding", "fall", "s-spring", "s-summer", "spring"});
-
-const unordered_map<string, uint8_t> name_to_lobby_event({
-  {"none",     0},
-  {"xmas",     1},
-  {"val",      3},
-  {"easter",   4},
-  {"hallo",    5},
-  {"sonic",    6},
-  {"newyear",  7},
-  {"summer",   8},
-  {"white",    9},
-  {"wedding",  10},
-  {"fall",     11},
-  {"s-spring", 12},
-  {"s-summer", 13},
-  {"spring",   14},
-});
-
-const unordered_map<uint8_t, string> lobby_type_to_name({
-  {0x00, "normal"},
-  {0x0F, "inormal"},
-  {0x10, "ipc"},
-  {0x11, "iball"},
-  {0x67, "cave2u"},
-  {0xD4, "cave1"},
-  {0xE9, "planet"},
-  {0xEA, "clouds"},
-  {0xED, "cave"},
-  {0xEE, "jungle"},
-  {0xEF, "forest2-2"},
-  {0xF0, "forest2-1"},
-  {0xF1, "windpower"},
-  {0xF2, "overview"},
-  {0xF3, "seaside"},
-  {0xF4, "some?"},
-  {0xF5, "dmorgue"},
-  {0xF6, "caelum"},
-  {0xF8, "digital"},
-  {0xF9, "boss1"},
-  {0xFA, "boss2"},
-  {0xFB, "boss3"},
-  {0xFC, "dragon"},
-  {0xFD, "derolle"},
-  {0xFE, "volopt"},
-  {0xFF, "darkfalz"},
-});
-
-const unordered_map<string, uint8_t> name_to_lobby_type({
-  {"normal",    0x00},
-  {"inormal",   0x0F},
-  {"ipc",       0x10},
-  {"iball",     0x11},
-  {"cave1",     0xD4},
-  {"cave2u",    0x67},
-  {"dragon",    0xFC},
-  {"derolle",   0xFD},
-  {"volopt",    0xFE},
-  {"darkfalz",  0xFF},
-  {"planet",    0xE9},
-  {"clouds",    0xEA},
-  {"cave",      0xED},
-  {"jungle",    0xEE},
-  {"forest2-2", 0xEF},
-  {"forest2-1", 0xF0},
-  {"windpower", 0xF1},
-  {"overview",  0xF2},
-  {"seaside",   0xF3},
-  {"some?",     0xF4},
-  {"dmorgue",   0xF5},
-  {"caelum",    0xF6},
-  {"digital",   0xF8},
-  {"boss1",     0xF9},
-  {"boss2",     0xFA},
-  {"boss3",     0xFB},
-  {"knight",    0xFC},
-  {"sky",       0xFE},
-  {"morgue",    0xFF},
-});
-
-const vector<string> tech_id_to_name({
-  "foie", "gifoie", "rafoie",
-  "barta", "gibarta", "rabarta",
-  "zonde", "gizonde", "razonde",
-  "grants", "deband", "jellen", "zalure", "shifta",
-  "ryuker", "resta", "anti", "reverser", "megid"});
-
-const unordered_map<string, uint8_t> name_to_tech_id({
-  {"foie",     0},
-  {"gifoie",   1},
-  {"rafoie",   2},
-  {"barta",    3},
-  {"gibarta",  4},
-  {"rabarta",  5},
-  {"zonde",    6},
-  {"gizonde",  7},
-  {"razonde",  8},
-  {"grants",   9},
-  {"deband",   10},
-  {"jellen",   11},
-  {"zalure",   12},
-  {"shifta",   13},
-  {"ryuker",   14},
-  {"resta",    15},
-  {"anti",     16},
-  {"reverser", 17},
-  {"megid",    18},
-});
-
-const vector<string> npc_id_to_name({
-  "ninja", "rico", "sonic", "knuckles", "tails", "flowen", "elly"});
-
-const unordered_map<string, uint8_t> name_to_npc_id({
-  {"ninja", 0},
-  {"rico", 1},
-  {"sonic", 2},
-  {"knuckles", 3},
-  {"tails", 4},
-  {"flowen", 5},
-  {"elly", 6}});
-
-const string& name_for_section_id(uint8_t section_id) {
-  if (section_id < section_id_to_name.size()) {
-    return section_id_to_name[section_id];
-  } else {
-    static const string ret = "<Unknown section id>";
-    return ret;
-  }
-}
-
-u16string u16name_for_section_id(uint8_t section_id) {
-  return decode_sjis(name_for_section_id(section_id));
-}
-
-uint8_t section_id_for_name(const string& name) {
-  try {
-    return name_to_section_id.at(name);
-  } catch (const out_of_range&) { }
-  try {
-    uint64_t x = stoul(name);
-    if (x < section_id_to_name.size()) {
-      return x;
-    }
-  } catch (const invalid_argument&) {
-  } catch (const out_of_range&) { }
-  return 0xFF;
-}
-
-uint8_t section_id_for_name(const u16string& name) {
-  return section_id_for_name(encode_sjis(name));
-}
-
-const string& name_for_event(uint8_t event) {
-  if (event < lobby_event_to_name.size()) {
-    return lobby_event_to_name[event];
-  } else {
-    static const string ret = "<Unknown lobby event>";
-    return ret;
-  }
-}
-
-u16string u16name_for_event(uint8_t event) {
-  return decode_sjis(name_for_event(event));
-}
-
-uint8_t event_for_name(const string& name) {
-  try {
-    return name_to_lobby_event.at(name);
-  } catch (const out_of_range&) { }
-  try {
-    uint64_t x = stoul(name);
-    if (x < lobby_event_to_name.size()) {
-      return x;
-    }
-  } catch (const invalid_argument&) {
-  } catch (const out_of_range&) { }
-  return 0xFF;
-}
-
-uint8_t event_for_name(const u16string& name) {
-  return event_for_name(encode_sjis(name));
-}
-
-const string& name_for_lobby_type(uint8_t type) {
-  try {
-    return lobby_type_to_name.at(type);
-  } catch (const out_of_range&) {
-    static const string ret = "<Unknown lobby type>";
-    return ret;
-  }
-}
-
-u16string u16name_for_lobby_type(uint8_t type) {
-  return decode_sjis(name_for_lobby_type(type));
-}
-
-uint8_t lobby_type_for_name(const string& name) {
-  try {
-    return name_to_lobby_type.at(name);
-  } catch (const out_of_range&) { }
-  try {
-    uint64_t x = stoul(name);
-    if (x < lobby_type_to_name.size()) {
-      return x;
-    }
-  } catch (const invalid_argument&) {
-  } catch (const out_of_range&) { }
-  return 0x80;
-}
-
-uint8_t lobby_type_for_name(const u16string& name) {
-  return lobby_type_for_name(encode_sjis(name));
-}
-
-const string& name_for_technique(uint8_t tech) {
-  try {
-    return tech_id_to_name.at(tech);
-  } catch (const out_of_range&) {
-    static const string ret = "<Unknown technique>";
-    return ret;
-  }
-}
-
-u16string u16name_for_technique(uint8_t tech) {
-  return decode_sjis(name_for_technique(tech));
-}
-
-uint8_t technique_for_name(const string& name) {
-  try {
-    return name_to_tech_id.at(name);
-  } catch (const out_of_range&) { }
-  try {
-    uint64_t x = stoul(name);
-    if (x < tech_id_to_name.size()) {
-      return x;
-    }
-  } catch (const invalid_argument&) {
-  } catch (const out_of_range&) { }
-  return 0xFF;
-}
-
-uint8_t technique_for_name(const u16string& name) {
-  return technique_for_name(encode_sjis(name));
-}
-
-const string& name_for_npc(uint8_t npc) {
-  try {
-    return npc_id_to_name.at(npc);
-  } catch (const out_of_range&) {
-    static const string ret = "<Unknown NPC>";
-    return ret;
-  }
-}
-
-u16string u16name_for_npc(uint8_t npc) {
-  return decode_sjis(name_for_npc(npc));
-}
-
-uint8_t npc_for_name(const string& name) {
-  try {
-    return name_to_npc_id.at(name);
-  } catch (const out_of_range&) { }
-  try {
-    uint64_t x = stoul(name);
-    if (x < npc_id_to_name.size()) {
-      return x;
-    }
-  } catch (const invalid_argument&) {
-  } catch (const out_of_range&) { }
-  return 0xFF;
-}
-
-uint8_t npc_for_name(const u16string& name) {
-  return npc_for_name(encode_sjis(name));
-}
-
-
 
 ////////////////////////////////////////////////////////////////////////////////
 // Checks
@@ -322,15 +26,15 @@ uint8_t npc_for_name(const u16string& name) {
 
 class precondition_failed {
 public:
-  precondition_failed(const char16_t* user_msg) : user_msg(user_msg) { }
+  precondition_failed(const std::u16string& user_msg) : user_msg(user_msg) { }
   ~precondition_failed() = default;
 
-  const char16_t* what() const {
+  const std::u16string& what() const {
     return this->user_msg;
   }
 
 private:
-  const char16_t* user_msg;
+  std::u16string user_msg;
 };
 
 static void check_privileges(shared_ptr<Client> c, uint64_t mask) {
@@ -363,7 +67,7 @@ static void check_is_game(shared_ptr<Lobby> l, bool is_game) {
 }
 
 static void check_is_ep3(shared_ptr<Client> c, bool is_ep3) {
-  if (!!(c->flags & ClientFlag::Episode3Games) != is_ep3) {
+  if (!!(c->flags & Client::Flag::EPISODE_3) != is_ep3) {
     throw precondition_failed(is_ep3 ?
         u"$C6This command can only\nbe used in Episode 3." :
         u"$C6This command cannot\nbe used in Episode 3.");
@@ -371,7 +75,7 @@ static void check_is_ep3(shared_ptr<Client> c, bool is_ep3) {
 }
 
 static void check_cheats_enabled(shared_ptr<Lobby> l) {
-  if (!(l->flags & LobbyFlag::CheatsEnabled)) {
+  if (!(l->flags & Lobby::Flag::CHEATS_ENABLED)) {
     throw precondition_failed(u"$C6This command can\nonly be used in\ncheat mode.");
   }
 }
@@ -388,7 +92,7 @@ static void check_is_leader(shared_ptr<Lobby> l, shared_ptr<Client> c) {
 // Message commands
 
 static void command_lobby_info(shared_ptr<ServerState>, shared_ptr<Lobby> l,
-    shared_ptr<Client> c, const char16_t*) {
+    shared_ptr<Client> c, const std::u16string&) {
   // no preconditions - everyone can use this command
 
   if (!l) {
@@ -402,10 +106,11 @@ static void command_lobby_info(shared_ptr<ServerState>, shared_ptr<Lobby> l,
       level_string = string_printf("Levels: %d-%d", l->min_level + 1, l->max_level + 1);
     }
 
-    send_text_message_printf(c, "$C6Game ID: %08X\n%s\nSection ID: %s\nCheat mode: %s",
+    send_text_message_printf(c,
+        "$C6Game ID: %08X\n%s\nSection ID: %s\nCheat mode: %s",
         l->lobby_id, level_string.c_str(),
         name_for_section_id(l->section_id).c_str(),
-        (l->flags & LobbyFlag::CheatsEnabled) ? "on" : "off");
+        (l->flags & Lobby::Flag::CHEATS_ENABLED) ? "on" : "off");
 
   } else {
     size_t num_clients = l->count_clients();
@@ -416,20 +121,20 @@ static void command_lobby_info(shared_ptr<ServerState>, shared_ptr<Lobby> l,
 }
 
 static void command_ax(shared_ptr<ServerState>, shared_ptr<Lobby>,
-    shared_ptr<Client> c, const char16_t* args) {
-  check_privileges(c, Privilege::Announce);
+    shared_ptr<Client> c, const std::u16string& args) {
+  check_privileges(c, Privilege::ANNOUNCE);
   string message = encode_sjis(args);
   log(INFO, "[Client message from %010u] %s\n", c->license->serial_number, message.c_str());
 }
 
 static void command_announce(shared_ptr<ServerState> s, shared_ptr<Lobby>,
-    shared_ptr<Client> c, const char16_t* args) {
-  check_privileges(c, Privilege::Announce);
+    shared_ptr<Client> c, const std::u16string& args) {
+  check_privileges(c, Privilege::ANNOUNCE);
   send_text_message(s, args);
 }
 
 static void command_arrow(shared_ptr<ServerState>, shared_ptr<Lobby> l,
-    shared_ptr<Client> c, const char16_t* args) {
+    shared_ptr<Client> c, const std::u16string& args) {
   // no preconditions
   c->lobby_arrow_color = stoull(encode_sjis(args), nullptr, 0);
   if (!l->is_game()) {
@@ -441,16 +146,16 @@ static void command_arrow(shared_ptr<ServerState>, shared_ptr<Lobby> l,
 // Lobby commands
 
 static void command_cheat(shared_ptr<ServerState>, shared_ptr<Lobby> l,
-    shared_ptr<Client> c, const char16_t*) {
+    shared_ptr<Client> c, const std::u16string&) {
   check_is_game(l, true);
   check_is_leader(l, c);
 
-  l->flags ^= LobbyFlag::CheatsEnabled;
+  l->flags ^= Lobby::Flag::CHEATS_ENABLED;
   send_text_message_printf(l, "Cheat mode %s",
-      (l->flags & LobbyFlag::CheatsEnabled) ? "enabled" : "disabled");
+      (l->flags & Lobby::Flag::CHEATS_ENABLED) ? "enabled" : "disabled");
 
   // if cheat mode was disabled, turn off all the cheat features that were on
-  if (!(l->flags & LobbyFlag::CheatsEnabled)) {
+  if (!(l->flags & Lobby::Flag::CHEATS_ENABLED)) {
     for (size_t x = 0; x < l->max_clients; x++) {
       auto c = l->clients[x];
       if (!c) {
@@ -458,15 +163,16 @@ static void command_cheat(shared_ptr<ServerState>, shared_ptr<Lobby> l,
       }
       c->infinite_hp = false;
       c->infinite_tp = false;
+      c->switch_assist = false;
     }
-    memset(&l->next_drop_item, 0, sizeof(l->next_drop_item));
+    l->next_drop_item = PlayerInventoryItem();
   }
 }
 
 static void command_lobby_event(shared_ptr<ServerState>, shared_ptr<Lobby> l,
-    shared_ptr<Client> c, const char16_t* args) {
+    shared_ptr<Client> c, const std::u16string& args) {
   check_is_game(l, false);
-  check_privileges(c, Privilege::ChangeEvent);
+  check_privileges(c, Privilege::CHANGE_EVENT);
 
   uint8_t new_event = event_for_name(args);
   if (new_event == 0xFF) {
@@ -479,8 +185,8 @@ static void command_lobby_event(shared_ptr<ServerState>, shared_ptr<Lobby> l,
 }
 
 static void command_lobby_event_all(shared_ptr<ServerState> s, shared_ptr<Lobby>,
-    shared_ptr<Client> c, const char16_t* args) {
-  check_privileges(c, Privilege::ChangeEvent);
+    shared_ptr<Client> c, const std::u16string& args) {
+  check_privileges(c, Privilege::CHANGE_EVENT);
 
   uint8_t new_event = event_for_name(args);
   if (new_event == 0xFF) {
@@ -489,7 +195,7 @@ static void command_lobby_event_all(shared_ptr<ServerState> s, shared_ptr<Lobby>
   }
 
   for (auto l : s->all_lobbies()) {
-    if (l->is_game() || !(l->flags & LobbyFlag::Default)) {
+    if (l->is_game() || !(l->flags & Lobby::Flag::DEFAULT)) {
       continue;
     }
 
@@ -499,9 +205,9 @@ static void command_lobby_event_all(shared_ptr<ServerState> s, shared_ptr<Lobby>
 }
 
 static void command_lobby_type(shared_ptr<ServerState>, shared_ptr<Lobby> l,
-    shared_ptr<Client> c, const char16_t* args) {
+    shared_ptr<Client> c, const std::u16string& args) {
   check_is_game(l, false);
-  check_privileges(c, Privilege::ChangeEvent);
+  check_privileges(c, Privilege::CHANGE_EVENT);
 
   uint8_t new_type = lobby_type_for_name(args);
   if (new_type == 0x80) {
@@ -510,7 +216,7 @@ static void command_lobby_type(shared_ptr<ServerState>, shared_ptr<Lobby> l,
   }
 
   l->type = new_type;
-  if (l->type < ((l->flags & LobbyFlag::Episode3) ? 20 : 15)) {
+  if (l->type < ((l->flags & Lobby::Flag::EPISODE_3_ONLY) ? 20 : 15)) {
     l->type = l->block - 1;
   }
 
@@ -524,8 +230,21 @@ static void command_lobby_type(shared_ptr<ServerState>, shared_ptr<Lobby> l,
 ////////////////////////////////////////////////////////////////////////////////
 // Game commands
 
+static void command_secid(shared_ptr<ServerState>, shared_ptr<Lobby> l,
+    shared_ptr<Client> c, const std::u16string& args) {
+  check_is_game(l, false);
+
+  if (!args[0]) {
+    c->override_section_id = -1;
+    send_text_message(l, u"$C6Override section ID\nremoved");
+  } else {
+    c->override_section_id = section_id_for_name(args);
+    send_text_message(l, u"$C6Override section ID\nset");
+  }
+}
+
 static void command_password(shared_ptr<ServerState>, shared_ptr<Lobby> l,
-    shared_ptr<Client> c, const char16_t* args) {
+    shared_ptr<Client> c, const std::u16string& args) {
   check_is_game(l, true);
   check_is_leader(l, c);
 
@@ -534,7 +253,7 @@ static void command_password(shared_ptr<ServerState>, shared_ptr<Lobby> l,
     send_text_message(l, u"$C6Game unlocked");
 
   } else {
-    char16cpy(l->password, args, 0x10);
+    l->password = args;
     auto encoded = encode_sjis(l->password);
     send_text_message_printf(l, "$C6Game password:\n%s",
         encoded.c_str());
@@ -542,7 +261,7 @@ static void command_password(shared_ptr<ServerState>, shared_ptr<Lobby> l,
 }
 
 static void command_min_level(shared_ptr<ServerState>, shared_ptr<Lobby> l,
-    shared_ptr<Client> c, const char16_t* args) {
+    shared_ptr<Client> c, const std::u16string& args) {
   check_is_game(l, true);
   check_is_leader(l, c);
 
@@ -553,7 +272,7 @@ static void command_min_level(shared_ptr<ServerState>, shared_ptr<Lobby> l,
 }
 
 static void command_max_level(shared_ptr<ServerState>, shared_ptr<Lobby> l,
-    shared_ptr<Client> c, const char16_t* args) {
+    shared_ptr<Client> c, const std::u16string& args) {
   check_is_game(l, true);
   check_is_leader(l, c);
 
@@ -573,12 +292,12 @@ static void command_max_level(shared_ptr<ServerState>, shared_ptr<Lobby> l,
 // Character commands
 
 static void command_edit(shared_ptr<ServerState> s, shared_ptr<Lobby> l,
-    shared_ptr<Client> c, const char16_t* args) {
+    shared_ptr<Client> c, const std::u16string& args) {
   check_is_game(l, false);
   check_version(c, GameVersion::BB);
 
   string encoded_args = encode_sjis(args);
-  vector<string> tokens = split(encoded_args, L' ');
+  vector<string> tokens = split(encoded_args, ' ');
 
   if (tokens.size() < 3) {
     send_text_message(c, u"$C6Not enough arguments");
@@ -586,56 +305,57 @@ static void command_edit(shared_ptr<ServerState> s, shared_ptr<Lobby> l,
   }
 
   if (tokens[0] == "atp") {
-    c->player.disp.stats.atp = stoul(tokens[1]);
+    c->game_data.player()->disp.stats.atp = stoul(tokens[1]);
   } else if (tokens[0] == "mst") {
-    c->player.disp.stats.mst = stoul(tokens[1]);
+    c->game_data.player()->disp.stats.mst = stoul(tokens[1]);
   } else if (tokens[0] == "evp") {
-    c->player.disp.stats.evp = stoul(tokens[1]);
+    c->game_data.player()->disp.stats.evp = stoul(tokens[1]);
   } else if (tokens[0] == "hp") {
-    c->player.disp.stats.hp = stoul(tokens[1]);
+    c->game_data.player()->disp.stats.hp = stoul(tokens[1]);
   } else if (tokens[0] == "dfp") {
-    c->player.disp.stats.dfp = stoul(tokens[1]);
+    c->game_data.player()->disp.stats.dfp = stoul(tokens[1]);
   } else if (tokens[0] == "ata") {
-    c->player.disp.stats.ata = stoul(tokens[1]);
+    c->game_data.player()->disp.stats.ata = stoul(tokens[1]);
   } else if (tokens[0] == "lck") {
-    c->player.disp.stats.lck = stoul(tokens[1]);
+    c->game_data.player()->disp.stats.lck = stoul(tokens[1]);
   } else if (tokens[0] == "meseta") {
-    c->player.disp.meseta = stoul(tokens[1]);
+    c->game_data.player()->disp.meseta = stoul(tokens[1]);
   } else if (tokens[0] == "exp") {
-    c->player.disp.experience = stoul(tokens[1]);
+    c->game_data.player()->disp.experience = stoul(tokens[1]);
   } else if (tokens[0] == "level") {
-    c->player.disp.level = stoul(tokens[1]) - 1;
+    c->game_data.player()->disp.level = stoul(tokens[1]) - 1;
   } else if (tokens[0] == "namecolor") {
-    sscanf(tokens[1].c_str(), "%8X", &c->player.disp.name_color);
+    uint32_t new_color;
+    sscanf(tokens[1].c_str(), "%8X", &new_color);
+    c->game_data.player()->disp.name_color = new_color;
   } else if (tokens[0] == "secid") {
     uint8_t secid = section_id_for_name(decode_sjis(tokens[1]));
     if (secid == 0xFF) {
       send_text_message(c, u"$C6No such section ID.");
       return;
     } else {
-      c->player.disp.section_id = secid;
+      c->game_data.player()->disp.section_id = secid;
     }
   } else if (tokens[0] == "name") {
-    decode_sjis(c->player.disp.name, tokens[1].c_str(), 0x10);
-    add_language_marker_inplace(c->player.disp.name, u'J', 0x10);
+    c->game_data.player()->disp.name = add_language_marker(tokens[1], 'J');
   } else if (tokens[0] == "npc") {
     if (tokens[1] == "none") {
-      c->player.disp.extra_model = 0;
-      c->player.disp.v2_flags &= 0xFD;
+      c->game_data.player()->disp.extra_model = 0;
+      c->game_data.player()->disp.v2_flags &= 0xFD;
     } else {
       uint8_t npc = npc_for_name(decode_sjis(tokens[1]));
       if (npc == 0xFF) {
         send_text_message(c, u"$C6No such NPC.");
         return;
       }
-      c->player.disp.extra_model = npc;
-      c->player.disp.v2_flags |= 0x02;
+      c->game_data.player()->disp.extra_model = npc;
+      c->game_data.player()->disp.v2_flags |= 0x02;
     }
   } else if ((tokens[0] == "tech") && (tokens.size() > 2)) {
     uint8_t level = stoul(tokens[2]) - 1;
     if (tokens[1] == "all") {
       for (size_t x = 0; x < 0x14; x++) {
-        c->player.disp.technique_levels[x] = level;
+        c->game_data.player()->disp.technique_levels.data()[x] = level;
       }
     } else {
       uint8_t tech_id = technique_for_name(decode_sjis(tokens[1]));
@@ -643,7 +363,7 @@ static void command_edit(shared_ptr<ServerState> s, shared_ptr<Lobby> l,
         send_text_message(c, u"$C6No such technique.");
         return;
       }
-      c->player.disp.technique_levels[tech_id] = level;
+      c->game_data.player()->disp.technique_levels.data()[tech_id] = level;
     }
   } else {
     send_text_message(c, u"$C6Unknown field.");
@@ -656,16 +376,17 @@ static void command_edit(shared_ptr<ServerState> s, shared_ptr<Lobby> l,
   s->send_lobby_join_notifications(l, c);
 }
 
-static void command_change_bank(shared_ptr<ServerState>, shared_ptr<Lobby>,
-    shared_ptr<Client> c, const char16_t*) {
+// TODO: implement this
+// TODO: make sure the bank name is filesystem-safe
+/* static void command_change_bank(shared_ptr<ServerState>, shared_ptr<Lobby>,
+    shared_ptr<Client> c, const std::u16string&) {
   check_version(c, GameVersion::BB);
 
-  // TODO: implement this
-  // TODO: make sure the bank name is filesystem-safe
-}
+  TODO
+} */
 
 static void command_convert_char_to_bb(shared_ptr<ServerState> s,
-    shared_ptr<Lobby> l, shared_ptr<Client> c, const char16_t* args) {
+    shared_ptr<Lobby> l, shared_ptr<Client> c, const std::u16string& args) {
   check_is_game(l, false);
   check_not_version(c, GameVersion::BB);
 
@@ -699,53 +420,66 @@ static void command_convert_char_to_bb(shared_ptr<ServerState> s,
 ////////////////////////////////////////////////////////////////////////////////
 // Administration commands
 
-static void command_silence(shared_ptr<ServerState> s, shared_ptr<Lobby> l,
-    shared_ptr<Client> c, const char16_t* args) {
-  check_privileges(c, Privilege::SilenceUser);
+static string name_for_client(shared_ptr<Client> c) {
+  auto player = c->game_data.player(false);
+  if (player.get()) {
+    return encode_sjis(player->disp.name);
+  }
 
-  auto target = s->find_client(args);
+  if (c->license.get()) {
+    return string_printf("SN:%" PRIu32, c->license->serial_number);
+  }
+
+  return "Player";
+}
+
+static void command_silence(shared_ptr<ServerState> s, shared_ptr<Lobby> l,
+    shared_ptr<Client> c, const std::u16string& args) {
+  check_privileges(c, Privilege::SILENCE_USER);
+
+  auto target = s->find_client(&args);
   if (!target->license) {
     // this should be impossible, but I'll bet it's not actually
     send_text_message(c, u"$C6Client not logged in");
     return;
   }
 
-  if (target->license->privileges & Privilege::Moderator) {
+  if (target->license->privileges & Privilege::MODERATOR) {
     send_text_message(c, u"$C6You do not have\nsufficient privileges.");
     return;
   }
 
   target->can_chat = !target->can_chat;
-  string target_name_sjis = encode_sjis(target->player.disp.name);
-  send_text_message_printf(l, "$C6%s %ssilenced", target_name_sjis.c_str(),
+  string target_name = name_for_client(target);
+  send_text_message_printf(l, "$C6%s %ssilenced", target_name.c_str(),
       target->can_chat ? "un" : "");
 }
 
 static void command_kick(shared_ptr<ServerState> s, shared_ptr<Lobby> l,
-    shared_ptr<Client> c, const char16_t* args) {
-  check_privileges(c, Privilege::KickUser);
+    shared_ptr<Client> c, const std::u16string& args) {
+  check_privileges(c, Privilege::KICK_USER);
 
-  auto target = s->find_client(args);
+  auto target = s->find_client(&args);
   if (!target->license) {
-    // this should be impossible, but I'll bet it's not actually
+    // This should be impossible, but I'll bet it's not actually
     send_text_message(c, u"$C6Client not logged in");
     return;
   }
 
-  if (target->license->privileges & Privilege::Moderator) {
+  if (target->license->privileges & Privilege::MODERATOR) {
     send_text_message(c, u"$C6You do not have\nsufficient privileges.");
     return;
   }
 
   send_message_box(target, u"$C6You were kicked off by a moderator.");
   target->should_disconnect = true;
-  string target_name_sjis = encode_sjis(target->player.disp.name);
-  send_text_message_printf(l, "$C6%s kicked off", target_name_sjis.c_str());
+  string target_name = name_for_client(target);
+  send_text_message_printf(l, "$C6%s kicked off", target_name.c_str());
 }
 
 static void command_ban(shared_ptr<ServerState> s, shared_ptr<Lobby> l,
-    shared_ptr<Client> c, const char16_t* args) {
-  check_privileges(c, Privilege::BanUser);
+    shared_ptr<Client> c, const std::u16string& args) {
+  check_privileges(c, Privilege::BAN_USER);
 
   u16string args_str(args);
   size_t space_pos = args_str.find(L' ');
@@ -754,14 +488,15 @@ static void command_ban(shared_ptr<ServerState> s, shared_ptr<Lobby> l,
     return;
   }
 
-  auto target = s->find_client(args_str.data() + space_pos + 1);
+  u16string identifier = args_str.substr(space_pos + 1);
+  auto target = s->find_client(&identifier);
   if (!target->license) {
-    // this should be impossible, but I'll bet it's not actually
+    // This should be impossible, but I'll bet it's not actually
     send_text_message(c, u"$C6Client not logged in");
     return;
   }
 
-  if (target->license->privileges & Privilege::BanUser) {
+  if (target->license->privileges & Privilege::BAN_USER) {
     send_text_message(c, u"$C6You do not have\nsufficient privileges.");
     return;
   }
@@ -789,15 +524,15 @@ static void command_ban(shared_ptr<ServerState> s, shared_ptr<Lobby> l,
   s->license_manager->ban_until(target->license->serial_number, now() + usecs);
   send_message_box(target, u"$C6You were banned by a moderator.");
   target->should_disconnect = true;
-  auto encoded_name = encode_sjis(target->player.disp.name);
-  send_text_message_printf(l, "$C6%s banned", encoded_name.c_str());
+  string target_name = name_for_client(target);
+  send_text_message_printf(l, "$C6%s banned", target_name.c_str());
 }
 
 ////////////////////////////////////////////////////////////////////////////////
 // Cheat commands
 
 static void command_warp(shared_ptr<ServerState>, shared_ptr<Lobby> l,
-    shared_ptr<Client> c, const char16_t* args) {
+    shared_ptr<Client> c, const std::u16string& args) {
   check_is_game(l, true);
   check_cheats_enabled(l);
 
@@ -825,8 +560,58 @@ static void command_warp(shared_ptr<ServerState>, shared_ptr<Lobby> l,
   send_warp(c, area);
 }
 
+static void command_next(shared_ptr<ServerState>, shared_ptr<Lobby> l,
+    shared_ptr<Client> c, const std::u16string&) {
+  check_is_game(l, true);
+  check_cheats_enabled(l);
+
+  if (!l->episode || (l->episode > 3)) {
+    return;
+  }
+
+  uint8_t new_area = c->area + 1;
+  if (((l->episode == 1) && (new_area > 17)) ||
+      ((l->episode == 2) && (new_area > 17)) ||
+      ((l->episode == 3) && (new_area > 10))) {
+    new_area = 0;
+  }
+
+  send_warp(c, new_area);
+}
+
+static void command_what(shared_ptr<ServerState>, shared_ptr<Lobby> l,
+    shared_ptr<Client> c, const std::u16string&) {
+  check_is_game(l, true);
+  if (!l->episode || (l->episode > 3)) {
+    return;
+  }
+
+  float min_dist2 = 0.0f;
+  uint32_t nearest_item_id = 0xFFFFFFFF;
+  for (const auto& it : l->item_id_to_floor_item) {
+    if (it.second.area != c->area) {
+      continue;
+    }
+    float dx = it.second.x - c->x;
+    float dz = it.second.z - c->z;
+    float dist2 = (dx * dx) + (dz * dz);
+    if ((nearest_item_id == 0xFFFFFFFF) || (dist2 < min_dist2)) {
+      nearest_item_id = it.first;
+      min_dist2 = dist2;
+    }
+  }
+
+  if (nearest_item_id == 0xFFFFFFFF) {
+    send_text_message(c, u"No items are near you");
+  } else {
+    const auto& item = l->item_id_to_floor_item.at(nearest_item_id);
+    string name = name_for_item(item.inv_item.data, true);
+    send_text_message(c, decode_sjis(name));
+  }
+}
+
 static void command_song(shared_ptr<ServerState>, shared_ptr<Lobby>,
-    shared_ptr<Client> c, const char16_t* args) {
+    shared_ptr<Client> c, const std::u16string& args) {
   check_is_ep3(c, true);
 
   uint32_t song = stoul(encode_sjis(args), nullptr, 0);
@@ -834,7 +619,7 @@ static void command_song(shared_ptr<ServerState>, shared_ptr<Lobby>,
 }
 
 static void command_infinite_hp(shared_ptr<ServerState>, shared_ptr<Lobby> l,
-    shared_ptr<Client> c, const char16_t*) {
+    shared_ptr<Client> c, const std::u16string&) {
   check_is_game(l, true);
   check_cheats_enabled(l);
 
@@ -843,7 +628,7 @@ static void command_infinite_hp(shared_ptr<ServerState>, shared_ptr<Lobby> l,
 }
 
 static void command_infinite_tp(shared_ptr<ServerState>, shared_ptr<Lobby> l,
-    shared_ptr<Client> c, const char16_t*) {
+    shared_ptr<Client> c, const std::u16string&) {
   check_is_game(l, true);
   check_cheats_enabled(l);
 
@@ -851,8 +636,17 @@ static void command_infinite_tp(shared_ptr<ServerState>, shared_ptr<Lobby> l,
   send_text_message_printf(c, "$C6Infinite TP %s", c->infinite_tp ? "enabled" : "disabled");
 }
 
+static void command_switch_assist(shared_ptr<ServerState>, shared_ptr<Lobby> l,
+    shared_ptr<Client> c, const std::u16string&) {
+  check_is_game(l, true);
+  check_cheats_enabled(l);
+
+  c->switch_assist = !c->switch_assist;
+  send_text_message_printf(c, "$C6Switch assist %s", c->switch_assist ? "enabled" : "disabled");
+}
+
 static void command_item(shared_ptr<ServerState>, shared_ptr<Lobby> l,
-    shared_ptr<Client> c, const char16_t* args) {
+    shared_ptr<Client> c, const std::u16string& args) {
   check_is_game(l, true);
   check_cheats_enabled(l);
 
@@ -867,12 +661,11 @@ static void command_item(shared_ptr<ServerState>, shared_ptr<Lobby> l,
   }
 
   ItemData item_data;
-  memset(&item_data, 0, sizeof(item_data));
-  if (data.size() < 12) {
-    memcpy(&l->next_drop_item.data.item_data1, data.data(), data.size());
+  if (data.size() <= 12) {
+    memcpy(&l->next_drop_item.data.data1, data.data(), data.size());
   } else {
-    memcpy(&l->next_drop_item.data.item_data1, data.data(), 12);
-    memcpy(&l->next_drop_item.data.item_data2, data.data() + 12, 12 - data.size());
+    memcpy(&l->next_drop_item.data.data1, data.data(), 12);
+    memcpy(&l->next_drop_item.data.data2, data.data() + 12, 12 - data.size());
   }
 
   send_text_message(c, u"$C6Next drop chosen.");
@@ -883,48 +676,47 @@ static void command_item(shared_ptr<ServerState>, shared_ptr<Lobby> l,
 ////////////////////////////////////////////////////////////////////////////////
 
 typedef void (*handler_t)(shared_ptr<ServerState> s, shared_ptr<Lobby> l,
-    shared_ptr<Client> c, const char16_t* args);
+    shared_ptr<Client> c, const std::u16string& args);
 struct ChatCommandDefinition {
   handler_t handler;
-  const char16_t* usage;
+  u16string usage;
 };
 
 static const unordered_map<u16string, ChatCommandDefinition> chat_commands({
     // TODO: implement command_help and actually use the usage strings here
-    {u"allevent"  , {command_lobby_event_all   , u"Usage:\nallevent <name/ID>"}},
-    {u"ann"       , {command_announce          , u"Usage:\nann <message>"}},
-    {u"arrow"     , {command_arrow             , u"Usage:\narrow <color>"}},
-    {u"ax"        , {command_ax                , u"Usage:\nax <message>"}},
-    {u"ban"       , {command_ban               , u"Usage:\nban <name-or-number>"}},
-    {u"bbchar"    , {command_convert_char_to_bb, u"Usage:\nbbchar <user> <pass> <1-4>"}},
-    {u"changebank", {command_change_bank       , u"Usage:\nchangebank <bank name>"}},
-    {u"cheat"     , {command_cheat             , u"Usage:\ncheat"}},
-    {u"edit"      , {command_edit              , u"Usage:\nedit <stat> <value>"}},
-    {u"event"     , {command_lobby_event       , u"Usage:\nevent <name>"}},
-    {u"infhp"     , {command_infinite_hp       , u"Usage:\ninfhp"}},
-    {u"inftp"     , {command_infinite_tp       , u"Usage:\ninftp"}},
-    {u"item"      , {command_item              , u"Usage:\nitem <item-code>"}},
-    {u"kick"      , {command_kick              , u"Usage:\nkick <name-or-number>"}},
-    {u"li"        , {command_lobby_info        , u"Usage:\nli"}},
-    {u"maxlevel"  , {command_max_level         , u"Usage:\nmax_level <level>"}},
-    {u"minlevel"  , {command_min_level         , u"Usage:\nmin_level <level>"}},
-    {u"password"  , {command_password          , u"Usage:\nlock [password]\nomit password to\nunlock game"}},
-    {u"silence"   , {command_silence           , u"Usage:\nsilence <name-or-number>"}},
-    {u"song"      , {command_song              , u"Usage:\nsong <song-number>"}},
-    {u"type"      , {command_lobby_type        , u"Usage:\ntype <name>"}},
-    {u"warp"      , {command_warp              , u"Usage:\nwarp <area-number>"}},
+    {u"$allevent"  , {command_lobby_event_all   , u"Usage:\nallevent <name/ID>"}},
+    {u"$ann"       , {command_announce          , u"Usage:\nann <message>"}},
+    {u"$arrow"     , {command_arrow             , u"Usage:\narrow <color>"}},
+    {u"$ax"        , {command_ax                , u"Usage:\nax <message>"}},
+    {u"$ban"       , {command_ban               , u"Usage:\nban <name-or-number>"}},
+    {u"$bbchar"    , {command_convert_char_to_bb, u"Usage:\nbbchar <user> <pass> <1-4>"}},
+    // {u"$bank",       {command_bank              , u"Usage:\nbank <bank name>"}},
+    {u"$cheat"     , {command_cheat             , u"Usage:\ncheat"}},
+    {u"$edit"      , {command_edit              , u"Usage:\nedit <stat> <value>"}},
+    {u"$event"     , {command_lobby_event       , u"Usage:\nevent <name>"}},
+    {u"$infhp"     , {command_infinite_hp       , u"Usage:\ninfhp"}},
+    {u"$inftp"     , {command_infinite_tp       , u"Usage:\ninftp"}},
+    {u"$item"      , {command_item              , u"Usage:\nitem <item-code>"}},
+    {u"$kick"      , {command_kick              , u"Usage:\nkick <name-or-number>"}},
+    {u"$li"        , {command_lobby_info        , u"Usage:\nli"}},
+    {u"$maxlevel"  , {command_max_level         , u"Usage:\nmax_level <level>"}},
+    {u"$minlevel"  , {command_min_level         , u"Usage:\nmin_level <level>"}},
+    {u"$next"      , {command_next              , u"Usage:\nnext"}},
+    {u"$password"  , {command_password          , u"Usage:\nlock [password]\nomit password to\nunlock game"}},
+    {u"$secid"     , {command_secid             , u"Usage:\nsecid [section ID]\nomit section ID to\nrevert to normal"}},
+    {u"$silence"   , {command_silence           , u"Usage:\nsilence <name-or-number>"}},
+    {u"$song"      , {command_song              , u"Usage:\nsong <song-number>"}},
+    {u"$swa"       , {command_switch_assist     , u"Usage:\nswa"}},
+    {u"$type"      , {command_lobby_type        , u"Usage:\ntype <name>"}},
+    {u"$warp"      , {command_warp              , u"Usage:\nwarp <area-number>"}},
+    {u"$what"      , {command_what              , u"Usage:\nwhat"}},
 });
 
-// this function is called every time any player sends a chat beginning with a dollar sign.
-// It is this function's responsibility to see if the chat is a command, and to
-// execute the command and block the chat if it is.
+// This function is called every time any player sends a chat beginning with a
+// dollar sign. It is this function's responsibility to see if the chat is a
+// command, and to execute the command and block the chat if it is.
 void process_chat_command(std::shared_ptr<ServerState> s, std::shared_ptr<Lobby> l,
-    std::shared_ptr<Client> c, const char16_t* text) {
-
-  // remove the chat command marker
-  if (text[0] == u'$') {
-    text++;
-  }
+    std::shared_ptr<Client> c, const std::u16string& text) {
 
   u16string command_name;
   u16string text_str(text);

src/ChatCommands.hh

@@ -9,30 +9,5 @@
 #include "Lobby.hh"
 #include "Client.hh"
 
-const std::string& name_for_section_id(uint8_t section_id);
-std::u16string u16name_for_section_id(uint8_t section_id);
-uint8_t section_id_for_name(const std::string& name);
-uint8_t section_id_for_name(const std::u16string& name);
-
-const std::string& name_for_event(uint8_t event);
-std::u16string u16name_for_event(uint8_t event);
-uint8_t event_for_name(const std::string& name);
-uint8_t event_for_name(const std::u16string& name);
-
-const std::string& name_for_lobby_type(uint8_t type);
-std::u16string u16name_for_lobby_type(uint8_t type);
-uint8_t lobby_type_for_name(const std::string& name);
-uint8_t lobby_type_for_name(const std::u16string& name);
-
-const std::string& name_for_technique(uint8_t tech);
-std::u16string u16name_for_technique(uint8_t tech);
-uint8_t technique_for_name(const std::string& name);
-uint8_t technique_for_name(const std::u16string& name);
-
-const std::string& name_for_npc(uint8_t npc);
-std::u16string u16name_for_npc(uint8_t npc);
-uint8_t npc_for_name(const std::string& name);
-uint8_t npc_for_name(const std::u16string& name);
-
 void process_chat_command(std::shared_ptr<ServerState> s, std::shared_ptr<Lobby> l,
-    std::shared_ptr<Client> c, const char16_t* text);
+    std::shared_ptr<Client> c, const std::u16string& text);

src/Client.cc

@@ -15,7 +15,7 @@ using namespace std;
 
 
 
-static const uint64_t CLIENT_CONFIG_MAGIC = 0x492A890E82AC9839;
+const uint64_t CLIENT_CONFIG_MAGIC = 0x492A890E82AC9839;
 
 
 
@@ -28,18 +28,24 @@ Client::Client(
     bev(bev),
     server_behavior(server_behavior),
     should_disconnect(false),
+    proxy_destination_address(0),
+    proxy_destination_port(0),
     play_time_begin(now()),
     last_recv_time(this->play_time_begin),
     last_send_time(0),
+    x(0.0f),
+    z(0.0f),
     area(0),
     lobby_id(0),
     lobby_client_id(0),
     lobby_arrow_color(0),
     next_exp_value(0),
+    override_section_id(-1),
     infinite_hp(false),
     infinite_tp(false),
+    switch_assist(false),
     can_chat(true) {
-
+  this->last_switch_enabled_command.subcommand = 0;
   int fd = bufferevent_getfd(this->bev);
   if (fd < 0) {
     this->is_virtual_connection = true;
@@ -52,32 +58,30 @@ Client::Client(
   memset(&this->next_connection_addr, 0, sizeof(this->next_connection_addr));
 }
 
-bool Client::send(string&& data) {
-  if (!this->bev) {
-    return false;
+void Client::set_license(shared_ptr<const License> l) {
+  this->license = l;
+  this->game_data.serial_number = this->license->serial_number;
+  if (this->version == GameVersion::BB) {
+    this->game_data.bb_username = this->license->username;
   }
-
-  if (this->crypt_out.get()) {
-    this->crypt_out->encrypt(data.data(), data.size());
-  }
-
-  struct evbuffer* buf = bufferevent_get_output(this->bev);
-  evbuffer_add(buf, data.data(), data.size());
-  return true;
 }
 
 ClientConfig Client::export_config() const {
   ClientConfig cc;
   cc.magic = CLIENT_CONFIG_MAGIC;
+  cc.flags = this->flags;
+  cc.proxy_destination_address = this->proxy_destination_address;
+  cc.proxy_destination_port = this->proxy_destination_port;
+  cc.unused.clear(0xFF);
+  return cc;
+}
+
+ClientConfigBB Client::export_config_bb() const {
+  ClientConfigBB cc;
+  cc.cfg = this->export_config();
   cc.bb_game_state = this->bb_game_state;
   cc.bb_player_index = this->bb_player_index;
-  cc.flags = this->flags;
-  for (size_t x = 0; x < 5; x++) {
-    cc.unused[x] = 0xFFFFFFFF;
-  }
-  for (size_t x = 0; x < 2; x++) {
-    cc.unused_bb_only[x] = 0xFFFFFFFF;
-  }
+  cc.unused.clear(0xFF);
   return cc;
 }
 
@@ -85,7 +89,13 @@ void Client::import_config(const ClientConfig& cc) {
   if (cc.magic != CLIENT_CONFIG_MAGIC) {
     throw invalid_argument("invalid client config");
   }
+  this->flags = cc.flags;
+  this->proxy_destination_address = cc.proxy_destination_address;
+  this->proxy_destination_port = cc.proxy_destination_port;
+}
+
+void Client::import_config(const ClientConfigBB& cc) {
+  this->import_config(cc.cfg);
   this->bb_game_state = cc.bb_game_state;
   this->bb_player_index = cc.bb_player_index;
-  this->flags = cc.flags;
 }

src/Client.hh

@@ -8,26 +8,51 @@
 #include "Player.hh"
 #include "PSOEncryption.hh"
 
+#include "Text.hh"
+#include "PSOProtocol.hh"
+#include "CommandFormats.hh"
 
 
-enum class ServerBehavior {
-  SplitReconnect = 0,
-  LoginServer,
-  LobbyServer,
-  DataServerBB,
-  PatchServer,
-};
 
-struct ClientConfig {
-  uint64_t magic;
-  uint8_t bb_game_state;
-  uint8_t bb_player_index;
-  uint16_t flags;
-  uint32_t unused[5];
-  uint32_t unused_bb_only[2];
-} __attribute__((packed));
+extern const uint64_t CLIENT_CONFIG_MAGIC;
+
+
 
 struct Client {
+  enum Flag {
+    // For patch server clients, client is Blue Burst rather than PC
+    BB_PATCH = 0x0001,
+    // After joining a lobby, client will no longer send D6 commands when they
+    // close message boxes
+    NO_MESSAGE_BOX_CLOSE_CONFIRMATION_AFTER_LOBBY_JOIN = 0x0002,
+    // Client has the above flag and has already joined a lobby, or is Blue Burst
+    // (BB never sends D6 commands)
+    NO_MESSAGE_BOX_CLOSE_CONFIRMATION = 0x0004,
+    // Client is Episode 3, should be able to see CARD lobbies, and should only be
+    // able to see/join games with the IS_EPISODE_3 flag
+    EPISODE_3 = 0x0008,
+    // Client is DC v1 (disables some features)
+    DCV1 = 0x0010,
+    // Client is loading into a game
+    LOADING = 0x0020,
+    // Client is loading a quest
+    LOADING_QUEST = 0x0040,
+    // Client is in the information menu (login server only)
+    IN_INFORMATION_MENU = 0x0080,
+    // Client is at the welcome message (login server only)
+    AT_WELCOME_MESSAGE = 0x0100,
+
+    // Note: There isn't a good way to detect Episode 3 until the player data is
+    // sent (via a 61 command), so the IS_EPISODE_3 flag is set in that handler
+    DEFAULT_V1 = DCV1,
+    DEFAULT_V2_DC = 0x0000,
+    DEFAULT_V2_PC = NO_MESSAGE_BOX_CLOSE_CONFIRMATION,
+    DEFAULT_V3_GC = 0x0000,
+    DEFAULT_V3_GC_PLUS = NO_MESSAGE_BOX_CLOSE_CONFIRMATION_AFTER_LOBBY_JOIN,
+    DEFAULT_V3_GC_EP3 = NO_MESSAGE_BOX_CLOSE_CONFIRMATION_AFTER_LOBBY_JOIN | EPISODE_3,
+    DEFAULT_V4_BB = NO_MESSAGE_BOX_CLOSE_CONFIRMATION_AFTER_LOBBY_JOIN | NO_MESSAGE_BOX_CLOSE_CONFIRMATION,
+  };
+
   // License & account
   std::shared_ptr<const License> license;
   GameVersion version;
@@ -40,8 +65,8 @@ struct Client {
   uint16_t flags;
 
   // Encryption
-  std::unique_ptr<PSOEncryption> crypt_in;
-  std::unique_ptr<PSOEncryption> crypt_out;
+  std::shared_ptr<PSOEncryption> crypt_in;
+  std::shared_ptr<PSOEncryption> crypt_out;
 
   // Network
   struct sockaddr_storage local_addr;
@@ -51,24 +76,30 @@ struct Client {
   ServerBehavior server_behavior;
   bool is_virtual_connection;
   bool should_disconnect;
-  std::string recv_buffer;
+  uint32_t proxy_destination_address;
+  uint16_t proxy_destination_port;
 
-  // timing & menus
+  // Timing & menus
   uint64_t play_time_begin; // time of connection (used for incrementing play time on BB)
   uint64_t last_recv_time; // time of last data received
   uint64_t last_send_time; // time of last data sent
 
-  // lobby/positioning
+  // Lobby/positioning
+  float x;
+  float z;
   uint32_t area; // which area is the client in?
   uint32_t lobby_id; // which lobby is this person in?
   uint8_t lobby_client_id; // which client number is this person?
   uint8_t lobby_arrow_color; // lobby arrow color ID
-  Player player;
+  ClientGameData game_data;
 
-  // miscellaneous (used by chat commands)
+  // Miscellaneous (used by chat commands)
   uint32_t next_exp_value; // next EXP value to give
+  int16_t override_section_id; // valid if >= 0
   bool infinite_hp; // cheats enabled
   bool infinite_tp; // cheats enabled
+  bool switch_assist; // cheats enabled
+  G_SwitchStateChanged_6x05 last_switch_enabled_command;
   bool can_chat;
   std::string pending_bb_save_username;
   uint8_t pending_bb_save_player_index;
@@ -76,9 +107,10 @@ struct Client {
   Client(struct bufferevent* bev, GameVersion version,
       ServerBehavior server_behavior);
 
-  // adds data to the client's output buffer, encrypting it first
-  bool send(std::string&& data);
+  void set_license(std::shared_ptr<const License> l);
 
   ClientConfig export_config() const;
+  ClientConfigBB export_config_bb() const;
   void import_config(const ClientConfig& cc);
+  void import_config(const ClientConfigBB& cc);
 };

src/Compression.cc

@@ -13,18 +13,15 @@ using namespace std;
 
 
 struct prs_compress_ctx {
-  unsigned char bitpos;
+  uint8_t bitpos;
   std::string forward_log;
   std::string output;
 
-  prs_compress_ctx() : bitpos(0) { }
+  prs_compress_ctx() : bitpos(0), forward_log("\0", 1) { }
 
   string finish() {
     this->put_control_bit(0);
     this->put_control_bit(1);
-    if (this->bitpos != 0) {
-      this->forward_log[0] = ((this->forward_log[0] << this->bitpos) >> 8);
-    }
     this->put_static_data(0);
     this->put_static_data(0);
     this->output += this->forward_log;
@@ -33,8 +30,9 @@ struct prs_compress_ctx {
   }
 
   void put_control_bit_nosave(bool bit) {
-    this->forward_log[0] = this->forward_log[0] >> 1;
-    this->forward_log[0] |= ((!!bit) << 7);
+    if (bit) {
+      this->forward_log[0] |= 1 << this->bitpos;
+    }
     this->bitpos++;
   }
 
@@ -53,7 +51,7 @@ struct prs_compress_ctx {
   }
 
   void put_static_data(uint8_t data) {
-    this->forward_log += static_cast<char>(data);
+    this->forward_log.push_back(static_cast<char>(data));
   }
 
   void raw_byte(uint8_t value) {
@@ -108,10 +106,10 @@ string prs_compress(const string& data) {
     // look for a chunk of data in history matching what's at the current offset
     ssize_t best_offset = 0;
     ssize_t best_size = 0;
-    for (ssize_t this_offset = -3;
-         (this_offset + data_ssize >= 0) &&
-         (this_offset > -0x1FF0) &&
-         (best_size < 255);
+    for (ssize_t this_offset = -3; // min copy size is 3 bytes
+         (this_offset + read_offset >= 0) && // don't go before the beginning
+         (this_offset > -0x1FF0) && // max offset is -0x1FF0
+         (best_size < 255); // max size is 0xFF bytes
          this_offset--) {
 
       // for this offset, expand the match as much as possible

src/FileContentsCache.cc

@@ -12,7 +12,15 @@ FileContentsCache::File::File(const string& name, shared_ptr<const string> conte
     uint64_t load_time) : name(name), contents(contents), load_time(load_time) { }
 
 shared_ptr<const string> FileContentsCache::get(const std::string& name) {
+  return this->get(name, [name]() -> string { return load_file(name); });
+}
 
+shared_ptr<const string> FileContentsCache::get(const char* name) {
+  return this->get(string(name));
+}
+
+shared_ptr<const string> FileContentsCache::get(const std::string& name,
+    std::function<std::string()> generate) {
   uint64_t t = now();
   try {
     auto& entry = this->name_to_file.at(name);
@@ -21,7 +29,7 @@ shared_ptr<const string> FileContentsCache::get(const std::string& name) {
     }
   } catch (const out_of_range& e) { }
 
-  shared_ptr<const string> contents(new string(load_file(name)));
+  shared_ptr<const string> contents(new string(generate()));
   this->name_to_file.erase(name);
   this->name_to_file.emplace(piecewise_construct, forward_as_tuple(name),
       forward_as_tuple(name, contents, t));
@@ -29,6 +37,7 @@ shared_ptr<const string> FileContentsCache::get(const std::string& name) {
   return contents;
 }
 
-shared_ptr<const string> FileContentsCache::get(const char* name) {
-  return this->get(string(name));
+shared_ptr<const string> FileContentsCache::get(const char* name,
+    std::function<std::string()> generate) {
+  return this->get(string(name), generate);
 }

src/FileContentsCache.hh

@@ -3,6 +3,7 @@
 #include <memory>
 #include <string>
 #include <unordered_map>
+#include <functional>
 
 using namespace std;
 
@@ -35,6 +36,11 @@ public:
   std::shared_ptr<const std::string> get(const std::string& name);
   std::shared_ptr<const std::string> get(const char* name);
 
+  std::shared_ptr<const std::string> get(
+      const std::string& name, std::function<std::string()> generate);
+  std::shared_ptr<const std::string> get(
+      const char* name, std::function<std::string()> generate);
+
 private:
   std::unordered_map<std::string, File> name_to_file;
 };

src/IPStackSimulator.cc

@@ -27,6 +27,7 @@ using namespace std;
 
 
 static const size_t DEFAULT_RESEND_PUSH_USECS = 200000; // 200ms
+PrefixedLogger IPStackSimulator::log("[IPStackSimulator] ");
 
 
 
@@ -77,11 +78,9 @@ string IPStackSimulator::str_for_tcp_connection(shared_ptr<const IPClient> c,
 IPStackSimulator::IPStackSimulator(
     std::shared_ptr<struct event_base> base,
     std::shared_ptr<Server> game_server,
-    std::shared_ptr<ProxyServer> proxy_server,
     std::shared_ptr<ServerState> state)
   : base(base),
     game_server(game_server),
-    proxy_server(proxy_server),
     state(state),
     pcap_text_log_file(state->ip_stack_debug ? fopen("IPStackSimulator-Log.txt", "wt") : nullptr) {
   memset(this->host_mac_address_bytes, 0x90, 6);
@@ -123,7 +122,7 @@ void IPStackSimulator::add_socket(int fd) {
 
 
 uint32_t IPStackSimulator::connect_address_for_remote_address(uint32_t remote_addr) {
-  // Use and address not on the same subnet as the client, so that PSO Plus and
+  // Use an address not on the same subnet as the client, so that PSO Plus and
   // Episode III will think they're talking to a remote network and won't reject
   // the connection.
   if ((remote_addr & 0xFF000000) != 0x23000000) {
@@ -164,7 +163,7 @@ void IPStackSimulator::dispatch_on_listen_accept(
 void IPStackSimulator::on_listen_accept(struct evconnlistener* listener,
     evutil_socket_t fd, struct sockaddr*, int) {
   int listen_fd = evconnlistener_get_fd(listener);
-  log(INFO, "[IPStackSimulator] Client fd %d connected via fd %d",
+  this->log(INFO, "Client fd %d connected via fd %d",
       fd, listen_fd);
 
   struct bufferevent *bev = bufferevent_socket_new(this->base.get(), fd,
@@ -185,7 +184,7 @@ void IPStackSimulator::dispatch_on_listen_error(
 
 void IPStackSimulator::on_listen_error(struct evconnlistener* listener) {
   int err = EVUTIL_SOCKET_ERROR();
-  log(ERROR, "[IPStackSimulator] Failure on listening socket %d: %d (%s)",
+  this->log(ERROR, "Failure on listening socket %d: %d (%s)",
       evconnlistener_get_fd(listener), err, evutil_socket_error_to_string(err));
   event_base_loopexit(this->base.get(), nullptr);
 }
@@ -205,7 +204,7 @@ void IPStackSimulator::on_client_input(struct bufferevent* bev) {
     c = this->bev_to_client.at(bev);
   } catch (const out_of_range&) {
     size_t bytes = evbuffer_get_length(buf);
-    log(ERROR, "[IPStackSimulator] Ignoring data received from unregistered client (0x%zX bytes)",
+    this->log(ERROR, "Ignoring data received from unregistered client (0x%zX bytes)",
         bytes);
     evbuffer_drain(buf, bytes);
     return;
@@ -225,8 +224,10 @@ void IPStackSimulator::on_client_input(struct bufferevent* bev) {
     try {
       this->on_client_frame(c, frame);
     } catch (const exception& e) {
-      log(WARNING, "[IPStackSimulator] Failed to process client frame: %s", e.what());
-      print_data(stderr, frame);
+      if (this->state->ip_stack_debug) {
+        this->log(WARNING, "Failed to process client frame: %s", e.what());
+        print_data(stderr, frame);
+      }
     }
   }
 }
@@ -239,11 +240,11 @@ void IPStackSimulator::on_client_error(struct bufferevent* bev,
     short events) {
   if (events & BEV_EVENT_ERROR) {
     int err = EVUTIL_SOCKET_ERROR();
-    log(WARNING, "[IPStackSimulator] Client caused error %d (%s)", err,
+    this->log(WARNING, "Client caused error %d (%s)", err,
         evutil_socket_error_to_string(err));
   }
   if (events & (BEV_EVENT_EOF | BEV_EVENT_ERROR)) {
-    log(INFO, "[IPStackSimulator] Client fd %d disconnected",
+    this->log(INFO, "Client fd %d disconnected",
         bufferevent_getfd(bev));
 
     this->bev_to_client.erase(bev);
@@ -256,7 +257,7 @@ void IPStackSimulator::on_client_frame(
     shared_ptr<IPClient> c, const string& frame) {
   if (this->state->ip_stack_debug) {
     fputc('\n', stderr);
-    log(INFO, "[IPStackSimulator] Client sent frame");
+    this->log(INFO, "Client sent frame");
     print_data(stderr, frame);
   }
   this->log_frame(frame);
@@ -264,7 +265,7 @@ void IPStackSimulator::on_client_frame(
   FrameInfo fi(frame);
   if (this->state->ip_stack_debug) {
     string fi_header = fi.header_str();
-    log(INFO, "[IPStackSimulator] Frame header: %s", fi_header.c_str());
+    this->log(INFO, "Frame header: %s", fi_header.c_str());
   }
 
   if (fi.arp) {
@@ -373,7 +374,7 @@ void IPStackSimulator::on_client_arp_frame(
   evbuffer_add(out_buf, r_payload, sizeof(r_payload));
 
   if (this->state->ip_stack_debug) {
-    log(INFO, "[IPStackSimulator] Sending ARP response");
+    this->log(INFO, "Sending ARP response");
   }
 
   if (this->pcap_text_log_file) {
@@ -435,7 +436,7 @@ void IPStackSimulator::on_client_udp_frame(
 
   if (this->state->ip_stack_debug) {
     string remote_str = this->str_for_ipv4_netloc(fi.ipv4->src_addr, fi.udp->src_port);
-    log(INFO, "[IPStackSimulator] Sending DNS response to %s", remote_str.c_str());
+    this->log(INFO, "Sending DNS response to %s", remote_str.c_str());
   }
 
   uint16_t frame_size = sizeof(r_ether) + sizeof(r_ipv4) + sizeof(r_udp) + r_data.size();
@@ -482,7 +483,7 @@ uint64_t IPStackSimulator::tcp_conn_key_for_client_frame(const FrameInfo& fi) {
 void IPStackSimulator::on_client_tcp_frame(
     shared_ptr<IPClient> c, const FrameInfo& fi) {
   if (this->state->ip_stack_debug) {
-    log(INFO, "[IPStackSimulator] Client sent TCP frame (seq=%08" PRIX32 ", ack=%08" PRIX32 ")",
+    this->log(INFO, "Client sent TCP frame (seq=%08" PRIX32 ", ack=%08" PRIX32 ")",
         fi.tcp->seq_num.load(), fi.tcp->ack_num.load());
   }
 
@@ -544,7 +545,7 @@ void IPStackSimulator::on_client_tcp_frame(
     uint64_t key = this->tcp_conn_key_for_client_frame(fi);
     auto emplace_ret = c->tcp_connections.emplace(key, IPClient::TCPConnection());
     auto& conn = emplace_ret.first->second;
-    string conn_str = this->state->ip_stack_debug ? this->str_for_tcp_connection(c, conn) : "";
+    string conn_str;
 
     if (emplace_ret.second) {
       // Connection is new; initialize it
@@ -559,9 +560,14 @@ void IPStackSimulator::on_client_tcp_frame(
       conn.resend_push_usecs = DEFAULT_RESEND_PUSH_USECS;
       conn.awaiting_first_ack = true;
       conn.max_frame_size = max_frame_size;
+
+      conn_str = this->str_for_tcp_connection(c, conn);
       if (this->state->ip_stack_debug) {
-        log(INFO, "[IPStackSimulator] Client opened TCP connection %s (acked_server_seq=%08" PRIX32 ", next_client_seq=%08" PRIX32 ")",
+        this->log(INFO, "Client opened TCP connection %s (acked_server_seq=%08" PRIX32 ", next_client_seq=%08" PRIX32 ")",
             conn_str.c_str(), conn.acked_server_seq, conn.next_client_seq);
+      } else {
+        this->log(INFO, "Client opened TCP connection %s",
+            conn_str.c_str());
       }
 
     } else {
@@ -571,8 +577,9 @@ void IPStackSimulator::on_client_tcp_frame(
       }
       // TODO: We should check the syn/ack numbers here instead of just assuming
       // they're correct
+      conn_str = this->str_for_tcp_connection(c, conn);
       if (this->state->ip_stack_debug) {
-        log(INFO, "[IPStackSimulator] Client resent SYN for TCP connection %s",
+        this->log(INFO, "Client resent SYN for TCP connection %s",
             conn_str.c_str());
       }
     }
@@ -580,7 +587,7 @@ void IPStackSimulator::on_client_tcp_frame(
     // Send a SYN+ACK (send_tcp_frame always adds the ACK flag)
     this->send_tcp_frame(c, conn, TCPHeader::Flag::SYN);
     if (this->state->ip_stack_debug) {
-      log(INFO, "[IPStackSimulator] Sent SYN+ACK on %s (acked_server_seq=%08" PRIX32 ", next_client_seq=%08" PRIX32 ")",
+      this->log(INFO, "Sent SYN+ACK on %s (acked_server_seq=%08" PRIX32 ", next_client_seq=%08" PRIX32 ")",
           conn_str.c_str(), conn.acked_server_seq, conn.next_client_seq);
     }
 
@@ -597,7 +604,7 @@ void IPStackSimulator::on_client_tcp_frame(
 
     if (fi.tcp->flags & TCPHeader::Flag::ACK) {
       if (this->state->ip_stack_debug) {
-        log(INFO, "[IPStackSimulator] Client sent ACK %08" PRIX32, fi.tcp->ack_num.load());
+        this->log(INFO, "Client sent ACK %08" PRIX32, fi.tcp->ack_num.load());
       }
       if (conn->awaiting_first_ack) {
         if (fi.tcp->ack_num != conn->acked_server_seq + 1) {
@@ -609,7 +616,7 @@ void IPStackSimulator::on_client_tcp_frame(
       } else {
         if (seq_num_greater(fi.tcp->ack_num, conn->acked_server_seq)) {
           if (this->state->ip_stack_debug) {
-            log(INFO, "[IPStackSimulator] Advancing acked_server_seq from %08" PRIX32, conn->acked_server_seq);
+            this->log(INFO, "Advancing acked_server_seq from %08" PRIX32, conn->acked_server_seq);
           }
           uint32_t ack_delta = fi.tcp->ack_num - conn->acked_server_seq;
           size_t pending_bytes = evbuffer_get_length(conn->pending_data.get());
@@ -622,7 +629,7 @@ void IPStackSimulator::on_client_tcp_frame(
           conn->resend_push_usecs = DEFAULT_RESEND_PUSH_USECS;
 
           if (this->state->ip_stack_debug) {
-            log(INFO, "[IPStackSimulator] Removed %08" PRIX32 " bytes from pending buffer and advanced acked_server_seq to %08" PRIX32,
+            this->log(INFO, "Removed %08" PRIX32 " bytes from pending buffer and advanced acked_server_seq to %08" PRIX32,
                 ack_delta, conn->acked_server_seq);
           }
 
@@ -642,10 +649,8 @@ void IPStackSimulator::on_client_tcp_frame(
         throw runtime_error("client sent TCP FIN+RST");
       }
 
-      if (this->state->ip_stack_debug) {
-        string conn_str = this->str_for_tcp_connection(c, *conn);
-        log(INFO, "[IPStackSimulator] Client closed TCP connection %s", conn_str.c_str());
-      }
+      string conn_str = this->str_for_tcp_connection(c, *conn);
+      this->log(INFO, "Client closed TCP connection %s", conn_str.c_str());
 
       // TODO: Are we supposed to send a response to an RST? Here we do, and the
       // client probably just ignores it anyway
@@ -686,8 +691,8 @@ void IPStackSimulator::on_client_tcp_frame(
         // ignore it (but warn) and send an ACK later, and the client should
         // retransmit the lost data
         if (this->state->ip_stack_debug) {
-          log(WARNING,
-              "[IPStackSimulator] Client sent out-of-order sequence number (expected %08" PRIX32 ", received %08" PRIX32 ", 0x%zX data bytes)",
+          this->log(WARNING,
+              "Client sent out-of-order sequence number (expected %08" PRIX32 ", received %08" PRIX32 ", 0x%zX data bytes)",
               conn->next_client_seq, fi.tcp->seq_num.load(), fi.payload_size);
         }
         payload_skip_bytes = fi.payload_size;
@@ -703,10 +708,10 @@ void IPStackSimulator::on_client_tcp_frame(
 
         if (this->state->ip_stack_debug) {
           if (payload_skip_bytes) {
-            log(INFO, "[IPStackSimulator] Client sent data on TCP connection %s, overlapping existing ack'ed data (0x%zX bytes ignored)",
+            this->log(INFO, "Client sent data on TCP connection %s, overlapping existing ack'ed data (0x%zX bytes ignored)",
                 conn_str.c_str(), payload_skip_bytes);
           } else {
-            log(INFO, "[IPStackSimulator] Client sent data on TCP connection %s",
+            this->log(INFO, "Client sent data on TCP connection %s",
                 conn_str.c_str());
           }
           print_data(stderr, payload, payload_size);
@@ -725,7 +730,7 @@ void IPStackSimulator::on_client_tcp_frame(
       // Send an ACK
       this->send_tcp_frame(c, *conn);
       if (this->state->ip_stack_debug) {
-        log(INFO, "[IPStackSimulator] Sent PSH ACK on %s (acked_server_seq=%08" PRIX32 ", next_client_seq=%08" PRIX32 ", bytes_received=0x%zX)",
+        this->log(INFO, "Sent PSH ACK on %s (acked_server_seq=%08" PRIX32 ", next_client_seq=%08" PRIX32 ", bytes_received=0x%zX)",
             conn_str.c_str(), conn->acked_server_seq, conn->next_client_seq, conn->bytes_received);
       }
     }
@@ -743,13 +748,6 @@ void IPStackSimulator::open_server_connection(
     throw logic_error("server connection is already open");
   }
 
-  const PortConfiguration* port_config;
-  try {
-    port_config = &this->state->numbered_port_configuration.at(conn.server_port);
-  } catch (const out_of_range&) {
-    throw logic_error("client connected to port missing from configuration");
-  }
-
   struct bufferevent* bevs[2];
   bufferevent_pair_new(this->base.get(), 0, bevs);
 
@@ -762,16 +760,35 @@ void IPStackSimulator::open_server_connection(
   // Link the client to the server - the server sees this as a normal TCP
   // connection and treats it as if the client connected to one of its listening
   // sockets
-  if (this->game_server.get()) {
-    this->game_server->connect_client(bevs[1], c->ipv4_addr, conn.client_port,
-        port_config->version, port_config->behavior);
-  } else if (this->proxy_server.get()) {
-    this->proxy_server->connect_client(bevs[1]);
+  shared_ptr<const PortConfiguration> port_config;
+  try {
+    port_config = this->state->number_to_port_config.at(conn.server_port);
+  } catch (const out_of_range&) {
+    bufferevent_free(bevs[1]);
+    throw logic_error("client connected to port missing from configuration");
   }
 
   string conn_str = this->str_for_tcp_connection(c, conn);
-  log(INFO, "[IPStackSimulator] Connected TCP connection %s to game server",
-      conn_str.c_str());
+  if (port_config->behavior == ServerBehavior::PROXY_SERVER) {
+    if (!this->state->proxy_server.get()) {
+      this->log(ERROR, "TCP connection %s is to non-running proxy server",
+          conn_str.c_str());
+      flush_and_free_bufferevent(bevs[1]);
+    } else {
+      this->state->proxy_server->connect_client(bevs[1], conn.server_port);
+      this->log(INFO, "Connected TCP connection %s to proxy server",
+          conn_str.c_str());
+    }
+  } else if (this->game_server.get()) {
+    this->game_server->connect_client(bevs[1], c->ipv4_addr, conn.client_port,
+        port_config->version, port_config->behavior);
+    this->log(INFO, "Connected TCP connection %s to game server",
+        conn_str.c_str());
+  } else {
+    this->log(ERROR, "No server available for TCP connection %s",
+        conn_str.c_str());
+    flush_and_free_bufferevent(bevs[1]);
+  }
 }
 
 void IPStackSimulator::send_pending_push_frame(
@@ -784,7 +801,7 @@ void IPStackSimulator::send_pending_push_frame(
   size_t bytes_to_send = min<size_t>(pending_bytes, conn.max_frame_size);
 
   if (this->state->ip_stack_debug) {
-    log(INFO, "[IPStackSimulator] Sending PSH frame with seq_num %08" PRIX32 ", 0x%zX/0x%zX data bytes",
+    this->log(INFO, "Sending PSH frame with seq_num %08" PRIX32 ", 0x%zX/0x%zX data bytes",
         conn.acked_server_seq, bytes_to_send, pending_bytes);
   }
 
@@ -872,7 +889,7 @@ void IPStackSimulator::dispatch_on_resend_push(evutil_socket_t, short, void* ctx
   auto* conn = reinterpret_cast<IPClient::TCPConnection*>(ctx);
   auto c = conn->client.lock();
   if (!c.get()) {
-    log(WARNING, "[IPStackSimulator] Resend push event triggered for deleted client; ignoring");
+    IPStackSimulator::log(WARNING, "Resend push event triggered for deleted client; ignoring");
   } else {
     c->sim->on_resend_push(c, *conn);
   }
@@ -886,7 +903,7 @@ void IPStackSimulator::dispatch_on_server_input(struct bufferevent*, void* ctx)
   auto* conn = reinterpret_cast<IPClient::TCPConnection*>(ctx);
   auto c = conn->client.lock();
   if (!c.get()) {
-    log(WARNING, "[IPStackSimulator] Server input event triggered for deleted client; ignoring");
+    IPStackSimulator::log(WARNING, "Server input event triggered for deleted client; ignoring");
   } else {
     c->sim->on_server_input(c, *conn);
   }
@@ -895,7 +912,7 @@ void IPStackSimulator::dispatch_on_server_input(struct bufferevent*, void* ctx)
 void IPStackSimulator::on_server_input(shared_ptr<IPClient> c, IPClient::TCPConnection& conn) {
   struct evbuffer* buf = bufferevent_get_input(conn.server_bev.get());
   if (this->state->ip_stack_debug) {
-    log(INFO, "[IPStackSimulator] Server input event: 0x%zX bytes to read",
+    this->log(INFO, "Server input event: 0x%zX bytes to read",
         evbuffer_get_length(buf));
   }
 
@@ -908,7 +925,7 @@ void IPStackSimulator::dispatch_on_server_error(
   auto* conn = reinterpret_cast<IPClient::TCPConnection*>(ctx);
   auto c = conn->client.lock();
   if (!c.get()) {
-    log(WARNING, "[IPStackSimulator] Server error event triggered for deleted client; ignoring");
+    IPStackSimulator::log(WARNING, "Server error event triggered for deleted client; ignoring");
   } else {
     c->sim->on_server_error(c, *conn, events);
   }
@@ -918,7 +935,7 @@ void IPStackSimulator::on_server_error(
     shared_ptr<IPClient> c, IPClient::TCPConnection& conn, short events) {
   if (events & BEV_EVENT_ERROR) {
     int err = EVUTIL_SOCKET_ERROR();
-    log(WARNING, "[IPStackSimulator] Received error %d from virtual connection (%s)", err,
+    this->log(WARNING, "Received error %d from virtual connection (%s)", err,
         evutil_socket_error_to_string(err));
   }
   if (events & (BEV_EVENT_EOF | BEV_EVENT_ERROR)) {
@@ -929,6 +946,9 @@ void IPStackSimulator::on_server_error(
 
     // Delete the connection object (this also flushes and frees the server
     // virtual connection bufferevent)
+    string conn_str = this->str_for_tcp_connection(c, conn);
+    this->log(INFO, "Server closed TCP connection %s",
+        conn_str.c_str());
     c->tcp_connections.erase(this->tcp_conn_key_for_connection(conn));
   }
 }
@@ -938,7 +958,7 @@ void IPStackSimulator::on_server_error(
 void IPStackSimulator::log_frame(const string& data) const {
   if (this->pcap_text_log_file) {
     print_data(this->pcap_text_log_file, data, 0, nullptr,
-        PrintDataFlags::SkipSeparator);
+        PrintDataFlags::SKIP_SEPARATOR);
     fputc('\n', this->pcap_text_log_file);
     fflush(this->pcap_text_log_file);
   }

src/IPStackSimulator.hh

@@ -18,7 +18,6 @@ public:
   IPStackSimulator(
       std::shared_ptr<struct event_base> base,
       std::shared_ptr<Server> game_server,
-      std::shared_ptr<ProxyServer> proxy_server,
       std::shared_ptr<ServerState> state);
   ~IPStackSimulator();
 
@@ -30,9 +29,9 @@ public:
   static uint32_t connect_address_for_remote_address(uint32_t remote_addr);
 
 private:
+  static PrefixedLogger log;
   std::shared_ptr<struct event_base> base;
   std::shared_ptr<Server> game_server;
-  std::shared_ptr<ProxyServer> proxy_server;
   std::shared_ptr<ServerState> state;
 
   using unique_listener = std::unique_ptr<struct evconnlistener, void(*)(struct evconnlistener*)>;

src/Items.cc

@@ -142,23 +142,24 @@ using namespace std;
 ////////////////////////////////////////////////////////////////////////////////
 
 void player_use_item(shared_ptr<Client> c, size_t item_index) {
+  auto player = c->game_data.player();
 
   ssize_t equipped_weapon = -1;
   // ssize_t equipped_armor = -1;
   // ssize_t equipped_shield = -1;
   // ssize_t equipped_mag = -1;
-  for (size_t y = 0; y < c->player.inventory.num_items; y++) {
-    if (c->player.inventory.items[y].equip_flags & 0x0008) {
-      if (c->player.inventory.items[y].data.item_data1[0] == 0) {
+  for (size_t y = 0; y < c->game_data.player()->inventory.num_items; y++) {
+    if (c->game_data.player()->inventory.items[y].equip_flags & 0x0008) {
+      if (c->game_data.player()->inventory.items[y].data.data1[0] == 0) {
         equipped_weapon = y;
       }
-      // else if ((c->player.inventory.items[y].data.item_data1[0] == 1) &&
-      //            (c->player.inventory.items[y].data.item_data1[1] == 1)) {
+      // else if ((c->game_data.player()->inventory.items[y].data.data1[0] == 1) &&
+      //            (c->game_data.player()->inventory.items[y].data.data1[1] == 1)) {
       //   equipped_armor = y;
-      // } else if ((c->player.inventory.items[y].data.item_data1[0] == 1) &&
-      //            (c->player.inventory.items[y].data.item_data1[1] == 2)) {
+      // } else if ((c->game_data.player()->inventory.items[y].data.data1[0] == 1) &&
+      //            (c->game_data.player()->inventory.items[y].data.data1[1] == 2)) {
       //   equipped_shield = y;
-      // } else if (c->player.inventory.items[y].data.item_data1[0] == 2) {
+      // } else if (c->game_data.player()->inventory.items[y].data.data1[0] == 2) {
       //   equipped_mag = y;
       // }
     }
@@ -166,42 +167,42 @@ void player_use_item(shared_ptr<Client> c, size_t item_index) {
 
   bool should_delete_item = true;
 
-  auto& item = c->player.inventory.items[item_index];
-  if (item.data.item_data1w[0] == 0x0203) { // technique disk
-    c->player.disp.technique_levels[item.data.item_data1[4]] = item.data.item_data1[2];
+  auto& item = c->game_data.player()->inventory.items[item_index];
+  if (item.data.data1w[0] == 0x0203) { // technique disk
+    c->game_data.player()->disp.technique_levels.data()[item.data.data1[4]] = item.data.data1[2];
 
-  } else if (item.data.item_data1w[0] == 0x0A03) { // grinder
+  } else if (item.data.data1w[0] == 0x0A03) { // grinder
     if (equipped_weapon < 0) {
       throw invalid_argument("grinder used with no weapon equipped");
     }
-    if (item.data.item_data1[2] > 2) {
+    if (item.data.data1[2] > 2) {
       throw invalid_argument("incorrect grinder value");
     }
-    c->player.inventory.items[equipped_weapon].data.item_data1[3] += (item.data.item_data1[2] + 1);
+    c->game_data.player()->inventory.items[equipped_weapon].data.data1[3] += (item.data.data1[2] + 1);
     // TODO: we should check for max grind here
 
-  } else if (item.data.item_data1w[0] == 0x0B03) { // material
-    switch (item.data.item_data1[2]) {
+  } else if (item.data.data1w[0] == 0x0B03) { // material
+    switch (item.data.data1[2]) {
       case 0: // Power Material
-        c->player.disp.stats.atp += 2;
+        c->game_data.player()->disp.stats.atp += 2;
         break;
       case 1: // Mind Material
-        c->player.disp.stats.mst += 2;
+        c->game_data.player()->disp.stats.mst += 2;
         break;
       case 2: // Evade Material
-        c->player.disp.stats.evp += 2;
+        c->game_data.player()->disp.stats.evp += 2;
         break;
       case 3: // HP Material
-        c->player.inventory.hp_materials_used += 2;
+        c->game_data.player()->inventory.hp_materials_used += 2;
         break;
       case 4: // TP Material
-        c->player.inventory.tp_materials_used += 2;
+        c->game_data.player()->inventory.tp_materials_used += 2;
         break;
       case 5: // Def Material
-        c->player.disp.stats.dfp += 2;
+        c->game_data.player()->disp.stats.dfp += 2;
         break;
       case 6: // Luck Material
-        c->player.disp.stats.lck += 2;
+        c->game_data.player()->disp.stats.lck += 2;
         break;
       default:
         throw invalid_argument("unknown material used");
@@ -209,17 +210,17 @@ void player_use_item(shared_ptr<Client> c, size_t item_index) {
 
   } else {
     // default item action is to unwrap the item if it's a present
-    if ((item.data.item_data1[0] == 2) && (item.data.item_data2[2] & 0x40)) {
-      item.data.item_data2[2] &= 0xBF;
+    if ((item.data.data1[0] == 2) && (item.data.data2[2] & 0x40)) {
+      item.data.data2[2] &= 0xBF;
       should_delete_item = false;
-    } else if ((item.data.item_data1[0] != 2) && (item.data.item_data1[4] & 0x40)) {
-      item.data.item_data1[4] &= 0xBF;
+    } else if ((item.data.data1[0] != 2) && (item.data.data1[4] & 0x40)) {
+      item.data.data1[4] &= 0xBF;
       should_delete_item = false;
     }
   }
 
   if (should_delete_item) {
-    c->player.remove_item(item.data.item_id, 1, nullptr);
+    c->game_data.player()->remove_item(item.data.id, 1);
   }
 }
 
@@ -315,7 +316,6 @@ ItemData CommonItemCreator::create_drop_item(bool is_box, uint8_t episode,
   }
 
   ItemData item;
-  memset(&item, 0, sizeof(item));
 
   // picks a random non-rare item type, then gives it appropriate random stats
   // modify some of the constants in this section to change the system's
@@ -323,56 +323,56 @@ ItemData CommonItemCreator::create_drop_item(bool is_box, uint8_t episode,
   int32_t type = this->decide_item_type(is_box);
   switch (type) {
     case 0x00: // material
-      item.item_data1[0] = 0x03;
-      item.item_data1[1] = 0x0B;
-      item.item_data1[2] = random_int(0, 6);
+      item.data1[0] = 0x03;
+      item.data1[1] = 0x0B;
+      item.data1[2] = random_int(0, 6);
       break;
 
     case 0x01: // equipment
       switch (random_int(0, 3)) {
         case 0x00: // weapon
-          item.item_data1[1] = random_int(1, 12); // random normal class
-          item.item_data1[2] = difficulty + random_int(0, 2); // special type
-          if ((item.item_data1[1] > 0x09) && (item.item_data1[2] > 0x04)) {
-            item.item_data1[2] = 0x04; // no special classes above 4
+          item.data1[1] = random_int(1, 12); // random normal class
+          item.data1[2] = difficulty + random_int(0, 2); // special type
+          if ((item.data1[1] > 0x09) && (item.data1[2] > 0x04)) {
+            item.data1[2] = 0x04; // no special classes above 4
           }
-          item.item_data1[4] = 0x80; // untekked
-          if (item.item_data1[2] < 0x04) {
-            item.item_data1[4] |= random_int(0, 40); // give a special
+          item.data1[4] = 0x80; // untekked
+          if (item.data1[2] < 0x04) {
+            item.data1[4] |= random_int(0, 40); // give a special
           }
           for (size_t x = 0, y = 0; (x < 5) && (y < 3); x++) { // percentages
             if (random_int(0, 10) == 1) { // 1/11 chance of getting each type of percentage
-              item.item_data1[6 + (y * 2)] = x + 1;
-              item.item_data1[7 + (y * 2)] = random_int(0, 10) * 5;
+              item.data1[6 + (y * 2)] = x + 1;
+              item.data1[7 + (y * 2)] = random_int(0, 10) * 5;
               y++;
             }
           }
           break;
 
         case 0x01: // armor
-          item.item_data1[0] = 0x01;
-          item.item_data1[1] = 0x01;
-          item.item_data1[2] = (6 * difficulty) + random_int(0, ((area / 2) + 2) - 1); // standard type based on difficulty and area
-          if (item.item_data1[2] > 0x17) {
-            item.item_data1[2] = 0x17; // no standard types above 0x17
+          item.data1[0] = 0x01;
+          item.data1[1] = 0x01;
+          item.data1[2] = (6 * difficulty) + random_int(0, ((area / 2) + 2) - 1); // standard type based on difficulty and area
+          if (item.data1[2] > 0x17) {
+            item.data1[2] = 0x17; // no standard types above 0x17
           }
           if (random_int(0, 10) == 0) { // +/-
-            item.item_data1[4] = random_int(0, 5);
-            item.item_data1[6] = random_int(0, 2);
+            item.data1[4] = random_int(0, 5);
+            item.data1[6] = random_int(0, 2);
           }
-          item.item_data1[5] = random_int(0, 4); // slots
+          item.data1[5] = random_int(0, 4); // slots
           break;
 
         case 0x02: // shield
-          item.item_data1[0] = 0x01;
-          item.item_data1[1] = 0x02;
-          item.item_data1[2] = (5 * difficulty) + random_int(0, ((area / 2) + 2) - 1); // standard type based on difficulty and area
-          if (item.item_data1[2] > 0x14) {
-            item.item_data1[2] = 0x14; // no standard types above 0x14
+          item.data1[0] = 0x01;
+          item.data1[1] = 0x02;
+          item.data1[2] = (5 * difficulty) + random_int(0, ((area / 2) + 2) - 1); // standard type based on difficulty and area
+          if (item.data1[2] > 0x14) {
+            item.data1[2] = 0x14; // no standard types above 0x14
           }
           if (random_int(0, 10) == 0) { // +/-
-            item.item_data1[4] = random_int(0, 5);
-            item.item_data1[6] = random_int(0, 5);
+            item.data1[4] = random_int(0, 5);
+            item.data1[6] = random_int(0, 5);
           }
           break;
 
@@ -382,81 +382,81 @@ ItemData CommonItemCreator::create_drop_item(bool is_box, uint8_t episode,
           if (type == 0xFF) {
             throw out_of_range("no item dropped"); // 0xFF -> no item drops
           }
-          item.item_data1[0] = 0x01;
-          item.item_data1[1] = 0x03;
-          item.item_data1[2] = type;
+          item.data1[0] = 0x01;
+          item.data1[1] = 0x03;
+          item.data1[2] = type;
           break;
         }
       }
       break;
 
     case 0x02: // technique
-      item.item_data1[0] = 0x03;
-      item.item_data1[1] = 0x02;
-      item.item_data1[4] = random_int(0, 18); // tech type
-      if ((item.item_data1[4] != 14) && (item.item_data1[4] != 17)) { // if not ryuker or reverser, give it a level
-        if (item.item_data1[4] == 16) { // if not anti, give it a level between 1 and 30
+      item.data1[0] = 0x03;
+      item.data1[1] = 0x02;
+      item.data1[4] = random_int(0, 18); // tech type
+      if ((item.data1[4] != 14) && (item.data1[4] != 17)) { // if not ryuker or reverser, give it a level
+        if (item.data1[4] == 16) { // if not anti, give it a level between 1 and 30
           if (area > 3) {
-            item.item_data1[2] = difficulty + random_int(0, ((area - 1) / 2) - 1);
+            item.data1[2] = difficulty + random_int(0, ((area - 1) / 2) - 1);
           } else {
-            item.item_data1[2] = difficulty;
+            item.data1[2] = difficulty;
           }
-          if (item.item_data1[2] > 6) {
-            item.item_data1[2] = 6;
+          if (item.data1[2] > 6) {
+            item.data1[2] = 6;
           }
         } else {
-          item.item_data1[2] = (5 * difficulty) + random_int(0, ((area * 3) / 2) - 1); // else between 1 and 7
+          item.data1[2] = (5 * difficulty) + random_int(0, ((area * 3) / 2) - 1); // else between 1 and 7
         }
       }
       break;
 
     case 0x03: // scape doll
-      item.item_data1[0] = 0x03;
-      item.item_data1[1] = 0x09;
-      item.item_data1[2] = 0x00;
+      item.data1[0] = 0x03;
+      item.data1[1] = 0x09;
+      item.data1[2] = 0x00;
       break;
 
     case 0x04: // grinder
-      item.item_data1[0] = 0x03;
-      item.item_data1[1] = 0x0A;
-      item.item_data1[2] = random_int(0, 2); // mono, di, tri
+      item.data1[0] = 0x03;
+      item.data1[1] = 0x0A;
+      item.data1[2] = random_int(0, 2); // mono, di, tri
       break;
 
     case 0x05: // consumable
-      item.item_data1[0] = 0x03;
-      item.item_data1[5] = 0x01;
+      item.data1[0] = 0x03;
+      item.data1[5] = 0x01;
       switch (random_int(0, 2)) {
         case 0: // antidote / antiparalysis
-          item.item_data1[1] = 6;
-          item.item_data1[2] = random_int(0, 1);
+          item.data1[1] = 6;
+          item.data1[2] = random_int(0, 1);
           break;
 
         case 1: // telepipe / trap vision
-          item.item_data1[1] = 7 + random_int(0, 1);
+          item.data1[1] = 7 + random_int(0, 1);
           break;
 
         case 2: // sol / moon / star atomizer
-          item.item_data1[1] = 3 + random_int(0, 2);
+          item.data1[1] = 3 + random_int(0, 2);
           break;
       }
       break;
 
     case 0x06: // consumable
-      item.item_data1[0] = 0x03;
-      item.item_data1[5] = 0x01;
-      item.item_data1[1] = random_int(0, 1); // mate or fluid
+      item.data1[0] = 0x03;
+      item.data1[5] = 0x01;
+      item.data1[1] = random_int(0, 1); // mate or fluid
       if (difficulty == 0) {
-        item.item_data1[2] = random_int(0, 1); // only mono and di on normal
+        item.data1[2] = random_int(0, 1); // only mono and di on normal
       } else if (difficulty == 3) {
-        item.item_data1[2] = random_int(1, 2); // only di and tri on ultimate
+        item.data1[2] = random_int(1, 2); // only di and tri on ultimate
       } else {
-        item.item_data1[2] = random_int(0, 2); // else, any of the three
+        item.data1[2] = random_int(0, 2); // else, any of the three
       }
       break;
 
     case 0x07: // meseta
-      item.item_data1[0] = 0x04;
-      item.item_data2d = (90 * difficulty) + (random_int(0, 20) * (area * 2)); // meseta amount
+      item.data1[0] = 0x04;
+      item.data2d = (90 * difficulty) + (random_int(0, 20) * (area * 2)); // meseta amount
       break;
 
     default:
@@ -475,29 +475,28 @@ ItemData CommonItemCreator::create_shop_item(uint8_t difficulty,
   static const uint8_t max_anti_level[4] =  { 2,  4,  6,  7};
 
   ItemData item;
-  memset(&item, 0, sizeof(item));
 
-  item.item_data1[0] = item_type;
-  while (item.item_data1[0] == 2) {
-    item.item_data1[0] = rand() % 3;
+  item.data1[0] = item_type;
+  while (item.data1[0] == 2) {
+    item.data1[0] = rand() % 3;
   }
-  switch (item.item_data1[0]) {
+  switch (item.data1[0]) {
     case 0: { // weapon
-      item.item_data1[1] = (rand() % 12) + 1;
-      if (item.item_data1[1] > 9) {
-        item.item_data1[2] = difficulty;
+      item.data1[1] = (rand() % 12) + 1;
+      if (item.data1[1] > 9) {
+        item.data1[2] = difficulty;
       } else {
-        item.item_data1[2] = (rand() & 1) + difficulty;
+        item.data1[2] = (rand() & 1) + difficulty;
       }
 
-      item.item_data1[3] = rand() % 11;
-      item.item_data1[4] = rand() % 11;
+      item.data1[3] = rand() % 11;
+      item.data1[4] = rand() % 11;
 
       size_t num_percentages = 0;
       for (size_t x = 0; (x < 5) && (num_percentages < 3); x++) {
         if ((rand() % 4) == 1) {
-          item.item_data1[(num_percentages * 2) + 6] = x;
-          item.item_data1[(num_percentages * 2) + 7] = rand() % (max_percentages[difficulty] + 1);
+          item.data1[(num_percentages * 2) + 6] = x;
+          item.data1[(num_percentages * 2) + 7] = rand() % (max_percentages[difficulty] + 1);
           num_percentages++;
         }
       }
@@ -505,69 +504,69 @@ ItemData CommonItemCreator::create_shop_item(uint8_t difficulty,
     }
 
     case 1: // armor
-      item.item_data1[1] = 0;
-      while (item.item_data1[1] == 0) {
-        item.item_data1[1] = rand() & 3;
+      item.data1[1] = 0;
+      while (item.data1[1] == 0) {
+        item.data1[1] = rand() & 3;
       }
-      switch (item.item_data1[1]) {
+      switch (item.data1[1]) {
         case 1:
-          item.item_data1[2] = (rand() % 6) + (difficulty * 6);
-          item.item_data1[5] = rand() % 5;
+          item.data1[2] = (rand() % 6) + (difficulty * 6);
+          item.data1[5] = rand() % 5;
           break;
         case 2:
-          item.item_data2[2] = (rand() % 6) + (difficulty * 5);
-          *reinterpret_cast<short*>(&item.item_data1[6]) = (rand() % 9) - 4;
-          *reinterpret_cast<short*>(&item.item_data1[9]) = (rand() % 9) - 4;
+          item.data2[2] = (rand() % 6) + (difficulty * 5);
+          *reinterpret_cast<short*>(&item.data1[6]) = (rand() % 9) - 4;
+          *reinterpret_cast<short*>(&item.data1[9]) = (rand() % 9) - 4;
           break;
         case 3:
-          item.item_data2[2] = rand() % 0x3B;
-          *reinterpret_cast<short*>(&item.item_data1[7]) = (rand() % 5) - 4;
+          item.data2[2] = rand() % 0x3B;
+          *reinterpret_cast<short*>(&item.data1[7]) = (rand() % 5) - 4;
           break;
       }
       break;
 
     case 3: // tool
-      item.item_data1[1] = rand() % 12;
-      switch (item.item_data1[1]) {
+      item.data1[1] = rand() % 12;
+      switch (item.data1[1]) {
         case 0:
         case 1:
           if (difficulty == 0) {
-            item.item_data1[2] = 0;
+            item.data1[2] = 0;
           } else if (difficulty == 1) {
-            item.item_data1[2] = rand() % 2;
+            item.data1[2] = rand() % 2;
           } else if (difficulty == 2) {
-            item.item_data1[2] = (rand() % 2) + 1;
+            item.data1[2] = (rand() % 2) + 1;
           } else if (difficulty == 3) {
-            item.item_data1[2] = 2;
+            item.data1[2] = 2;
           }
           break;
 
         case 6:
-          item.item_data1[2] = rand() % 2;
+          item.data1[2] = rand() % 2;
           break;
 
         case 10:
-          item.item_data1[2] = rand() % 3;
+          item.data1[2] = rand() % 3;
           break;
 
         case 11:
-          item.item_data1[2] = rand() % 7;
+          item.data1[2] = rand() % 7;
           break;
       }
 
-      switch (item.item_data1[1]) {
+      switch (item.data1[1]) {
         case 2:
-          item.item_data1[4] = rand() % 19;
-          switch (item.item_data1[4]) {
+          item.data1[4] = rand() % 19;
+          switch (item.data1[4]) {
             case 14:
             case 17:
-              item.item_data1[2] = 0; // reverser & ryuker always level 1 
+              item.data1[2] = 0; // reverser & ryuker always level 1 
               break;
             case 16:
-              item.item_data1[2] = rand() % max_anti_level[difficulty];
+              item.data1[2] = rand() % max_anti_level[difficulty];
               break;
             default:
-              item.item_data1[2] = rand() % max_tech_level[difficulty];
+              item.data1[2] = rand() % max_tech_level[difficulty];
           }
           break;
         case 0:
@@ -579,7 +578,7 @@ ItemData CommonItemCreator::create_shop_item(uint8_t difficulty,
         case 7:
         case 8:
         case 16:
-          item.item_data1[5] = rand() % (max_quantity[difficulty] + 1);
+          item.data1[5] = rand() % (max_quantity[difficulty] + 1);
           break;
       }
   }

src/LevelTable.cc

@@ -10,7 +10,7 @@ using namespace std;
 
 
 
-LevelTable::LevelTable(const char* filename, bool compressed) {
+LevelTable::LevelTable(const string& filename, bool compressed) {
 
   string data = load_file(filename);
   if (compressed) {

src/LevelTable.hh

@@ -2,7 +2,22 @@
 
 #include <stdint.h>
 
-#include "Player.hh"
+#include <string>
+#include <phosg/Encoding.hh>
+
+
+
+struct PlayerStats {
+  le_uint16_t atp;
+  le_uint16_t mst;
+  le_uint16_t evp;
+  le_uint16_t hp;
+  le_uint16_t dfp;
+  le_uint16_t ata;
+  le_uint16_t lck;
+
+  PlayerStats() noexcept;
+} __attribute__((packed));
 
 // information on a single level for a single class
 struct LevelStats {
@@ -13,19 +28,19 @@ struct LevelStats {
   uint8_t dfp; // dfp to add on level up
   uint8_t ata; // ata to add on level up
   uint8_t unknown[2];
-  uint32_t experience; // EXP value of this level
+  le_uint32_t experience; // EXP value of this level
 
   void apply(PlayerStats& ps) const;
-};
+} __attribute__((packed));
 
 // level table format (PlyLevelTbl.prs)
 struct LevelTable {
   PlayerStats base_stats[12];
-  uint32_t unknown[12];
+  le_uint32_t unknown[12];
   LevelStats levels[12][200];
 
-  LevelTable(const char* filename, bool compressed);
+  LevelTable(const std::string& filename, bool compressed);
 
   const PlayerStats& base_stats_for_class(uint8_t char_class) const;
   const LevelStats& stats_for_level(uint8_t char_class, uint8_t level) const;
-};
+} __attribute__((packed));

src/License.cc

@@ -11,31 +11,23 @@ using namespace std;
 
 
 
-License::License() {
-  memset(this->username, 0, 20);
-  memset(this->bb_password, 0, 20);
-  this->serial_number = 0;
-  memset(this->access_key, 0, 16);
-  memset(this->gc_password, 0, 12);
-  this->privileges = 0;
-  this->ban_end_time = 0;
-}
+License::License() : serial_number(0), privileges(0), ban_end_time(0) { }
 
 string License::str() const {
   string ret = string_printf("License(serial_number=%" PRIu32, this->serial_number);
-  if (this->username[0]) {
+  if (!this->username.empty()) {
     ret += ", username=";
     ret += this->username;
   }
-  if (this->bb_password[0]) {
+  if (!this->bb_password.empty()) {
     ret += ", bb-password=";
     ret += this->bb_password;
   }
-  if (this->access_key[0]) {
+  if (!this->access_key.empty()) {
     ret += ", access-key=";
     ret += this->access_key;
   }
-  if (this->gc_password[0]) {
+  if (!this->gc_password.empty()) {
     ret += ", gc-password=";
     ret += this->gc_password;
   }
@@ -53,6 +45,12 @@ LicenseManager::LicenseManager(const string& filename) : filename(filename) {
     auto licenses = load_vector_file<License>(this->filename);
     for (const auto& read_license : licenses) {
       shared_ptr<License> license(new License(read_license));
+
+      // Before the temporary flag existed, licenses with root privileges would
+      // have the temporary flag set. To migrate these, explicitly unset the
+      // flag for all licenses loaded from the license file.
+      license->privileges &= ~Privilege::TEMPORARY;
+
       uint32_t serial_number = license->serial_number;
       this->bb_username_to_license.emplace(license->username, license);
       this->serial_number_to_license.emplace(serial_number, license);
@@ -67,19 +65,19 @@ LicenseManager::LicenseManager(const string& filename) : filename(filename) {
 void LicenseManager::save() const {
   auto f = fopen_unique(this->filename, "wb");
   for (const auto& it : this->serial_number_to_license) {
+    if (it.second->privileges & Privilege::TEMPORARY) {
+      continue;
+    }
     fwritex(f.get(), it.second.get(), sizeof(License));
   }
 }
 
 shared_ptr<const License> LicenseManager::verify_pc(uint32_t serial_number,
-    const char* access_key, const char* password) const {
+    const string& access_key) const {
   auto& license = this->serial_number_to_license.at(serial_number);
-  if (strncmp(license->access_key, access_key, 8)) {
+  if (!license->access_key.eq_n(access_key, 8)) {
     throw invalid_argument("incorrect access key");
   }
-  if (password && (strcmp(license->gc_password, password))) {
-    throw invalid_argument("incorrect password");
-  }
 
   if (license->ban_end_time && (license->ban_end_time >= now())) {
     throw invalid_argument("user is banned");
@@ -88,25 +86,36 @@ shared_ptr<const License> LicenseManager::verify_pc(uint32_t serial_number,
 }
 
 shared_ptr<const License> LicenseManager::verify_gc(uint32_t serial_number,
-    const char* access_key, const char* password) const {
+    const string& access_key) const {
   auto& license = this->serial_number_to_license.at(serial_number);
-  if (strncmp(license->access_key, access_key, 12)) {
+  if (!license->access_key.eq_n(access_key, 12)) {
     throw invalid_argument("incorrect access key");
   }
-  if (password && (strcmp(license->gc_password, password))) {
-    throw invalid_argument("incorrect password");
-  }
-
   if (license->ban_end_time && (license->ban_end_time >= now())) {
     throw invalid_argument("user is banned");
   }
   return license;
 }
 
-shared_ptr<const License> LicenseManager::verify_bb(const char* username,
-    const char* password) const {
+shared_ptr<const License> LicenseManager::verify_gc(uint32_t serial_number,
+    const string& access_key, const string& password) const {
+  auto& license = this->serial_number_to_license.at(serial_number);
+  if (!license->access_key.eq_n(access_key, 12)) {
+    throw invalid_argument("incorrect access key");
+  }
+  if (license->gc_password != password) {
+    throw invalid_argument("incorrect password");
+  }
+  if (license->ban_end_time && (license->ban_end_time >= now())) {
+    throw invalid_argument("user is banned");
+  }
+  return license;
+}
+
+shared_ptr<const License> LicenseManager::verify_bb(const string& username,
+    const string& password) const {
   auto& license = this->bb_username_to_license.at(username);
-  if (password && strcmp(license->bb_password, password)) {
+  if (license->bb_password != password) {
     throw invalid_argument("incorrect password");
   }
 
@@ -128,20 +137,18 @@ void LicenseManager::ban_until(uint32_t serial_number, uint64_t end_time) {
 void LicenseManager::add(shared_ptr<License> l) {
   uint32_t serial_number = l->serial_number;
   this->serial_number_to_license.emplace(serial_number, l);
-  if (l->username[0]) {
+  if (!l->username.empty()) {
     this->bb_username_to_license.emplace(l->username, l);
   }
-
   this->save();
 }
 
 void LicenseManager::remove(uint32_t serial_number) {
   auto l = this->serial_number_to_license.at(serial_number);
   this->serial_number_to_license.erase(l->serial_number);
-  if (l->username[0]) {
+  if (!l->username.empty()) {
     this->bb_username_to_license.erase(l->username);
   }
-
   this->save();
 }
 
@@ -154,33 +161,40 @@ vector<License> LicenseManager::snapshot() const {
 }
 
 
-shared_ptr<const License> LicenseManager::create_license_pc(
-    uint32_t serial_number,const char* access_key, const char* password) {
+
+shared_ptr<License> LicenseManager::create_license_pc(
+    uint32_t serial_number, const string& access_key, bool temporary) {
   shared_ptr<License> l(new License());
   l->serial_number = serial_number;
-  strncpy(l->access_key, access_key, 8);
-  if (password) {
-    strncpy(l->gc_password, password, 8);
+  l->access_key = access_key;
+  if (temporary) {
+    l->privileges |= Privilege::TEMPORARY;
   }
   return l;
 }
 
-shared_ptr<const License> LicenseManager::create_license_gc(
-    uint32_t serial_number, const char* access_key, const char* password) {
+shared_ptr<License> LicenseManager::create_license_gc(
+    uint32_t serial_number, const string& access_key, const string& password,
+    bool temporary) {
   shared_ptr<License> l(new License());
   l->serial_number = serial_number;
-  strncpy(l->access_key, access_key, 12);
-  if (password) {
-    strncpy(l->gc_password, password, 8);
+  l->access_key = access_key;
+  l->gc_password = password;
+  if (temporary) {
+    l->privileges |= Privilege::TEMPORARY;
   }
   return l;
 }
 
-shared_ptr<const License> LicenseManager::create_license_bb(
-    uint32_t serial_number, const char* username, const char* password) {
+shared_ptr<License> LicenseManager::create_license_bb(
+    uint32_t serial_number, const string& username, const string& password,
+    bool temporary) {
   shared_ptr<License> l(new License());
   l->serial_number = serial_number;
-  strncpy(l->username, username, 19);
-  strncpy(l->bb_password, password, 19);
+  l->username = username;
+  l->bb_password = password;
+  if (temporary) {
+    l->privileges |= Privilege::TEMPORARY;
+  }
   return l;
 }

src/License.hh

@@ -5,34 +5,38 @@
 #include <vector>
 #include <memory>
 
-enum Privilege {
-  KickUser        = 0x00000001,
-  BanUser         = 0x00000002,
-  SilenceUser     = 0x00000004,
-  ChangeLobbyInfo = 0x00000008,
-  ChangeEvent     = 0x00000010,
-  Announce        = 0x00000020,
-  FreeJoinGames   = 0x00000040,
-  UnlockGames     = 0x00000080,
+#include "Text.hh"
 
-  Moderator       = 0x00000007,
-  Administrator   = 0x0000003F,
-  Root            = 0xFFFFFFFF,
+enum Privilege {
+  KICK_USER         = 0x00000001,
+  BAN_USER          = 0x00000002,
+  SILENCE_USER      = 0x00000004,
+  CHANGE_LOBBY_INFO = 0x00000008,
+  CHANGE_EVENT      = 0x00000010,
+  ANNOUNCE          = 0x00000020,
+  FREE_JOIN_GAMES   = 0x00000040,
+  UNLOCK_GAMES      = 0x00000080,
+
+  MODERATOR         = 0x00000007,
+  ADMINISTRATOR     = 0x0000003F,
+  ROOT              = 0x7FFFFFFF,
+
+  TEMPORARY         = 0x80000000,
 };
 
 enum LicenseVerifyAction {
   BB = 0x00,
   GC = 0x01,
   PC = 0x02,
-  SerialNumber = 0x03,
+  SERIAL_NUMBER = 0x03,
 };
 
 struct License {
-  char username[20]; // BB username (max. 16 chars; should technically be Unicode)
-  char bb_password[20]; // BB password (max. 16 chars)
+  ptext<char, 0x14> username; // BB username (max. 16 chars; should technically be Unicode)
+  ptext<char, 0x14> bb_password; // BB password (max. 16 chars)
   uint32_t serial_number; // PC/GC serial number. MUST BE PRESENT FOR BB LICENSES TOO; this is also the player's guild card number.
-  char access_key[16]; // PC/GC access key. (to log in using PC on a GC license, just enter the first 8 characters of the GC access key)
-  char gc_password[12]; // GC password
+  ptext<char, 0x10> access_key; // PC/GC access key. (to log in using PC on a GC license, just enter the first 8 characters of the GC access key)
+  ptext<char, 0x0C> gc_password; // GC password
   uint32_t privileges; // privilege level
   uint64_t ban_end_time; // end time of ban (zero = not banned)
 
@@ -46,11 +50,13 @@ public:
   ~LicenseManager() = default;
 
   std::shared_ptr<const License> verify_pc(uint32_t serial_number,
-      const char* access_key, const char* password) const;
+      const std::string& access_key) const;
   std::shared_ptr<const License> verify_gc(uint32_t serial_number,
-      const char* access_key, const char* password) const;
-  std::shared_ptr<const License> verify_bb(const char* username,
-      const char* password) const;
+      const std::string& access_key) const;
+  std::shared_ptr<const License> verify_gc(uint32_t serial_number,
+      const std::string& access_key, const std::string& password) const;
+  std::shared_ptr<const License> verify_bb(const std::string& username,
+      const std::string& password) const;
   void ban_until(uint32_t serial_number, uint64_t seconds);
 
   size_t count() const;
@@ -59,12 +65,14 @@ public:
   void remove(uint32_t serial_number);
   std::vector<License> snapshot() const;
 
-  static std::shared_ptr<const License> create_license_pc(
-      uint32_t serial_number, const char* access_key, const char* password);
-  static std::shared_ptr<const License> create_license_gc(
-      uint32_t serial_number, const char* access_key, const char* password);
-  static std::shared_ptr<const License> create_license_bb(
-      uint32_t serial_number, const char* username, const char* password);
+  static std::shared_ptr<License> create_license_pc(
+      uint32_t serial_number, const std::string& access_key, bool temporary);
+  static std::shared_ptr<License> create_license_gc(
+      uint32_t serial_number, const std::string& access_key,
+      const std::string& password, bool temporary);
+  static std::shared_ptr<License> create_license_bb(
+      uint32_t serial_number, const std::string& username,
+      const std::string& password, bool temporary);
 
 protected:
   void save() const;

src/Lobby.cc

@@ -14,20 +14,12 @@ using namespace std;
 Lobby::Lobby() : lobby_id(0), min_level(0), max_level(0xFFFFFFFF),
     next_game_item_id(0), version(GameVersion::GC), section_id(0), episode(1),
     difficulty(0), mode(0), rare_seed(random_object<uint32_t>()), event(0),
-    block(0), type(0), leader_id(0), max_clients(12), flags(0),
-    loading_quest_id(0) {
+    block(0), type(0), leader_id(0), max_clients(12), flags(0) {
 
   for (size_t x = 0; x < 12; x++) {
     this->next_item_id[x] = 0;
   }
-  memset(&this->next_drop_item, 0, sizeof(this->next_drop_item));
-  memset(this->variations, 0, 0x20 * sizeof(this->variations[0]));
-  memset(this->password, 0, 36 * sizeof(this->password[0]));
-  memset(this->name, 0, 36 * sizeof(this->name[0]));
-}
-
-bool Lobby::is_game() const {
-  return this->flags & LobbyFlag::IsGame;
+  this->next_drop_item = PlayerInventoryItem();
 }
 
 void Lobby::reassign_leader_on_client_departure(size_t leaving_client_index) {
@@ -48,7 +40,7 @@ bool Lobby::any_client_loading() const {
     if (!this->clients[x].get()) {
       continue;
     }
-    if (this->clients[x]->flags & ClientFlag::Loading) {
+    if (this->clients[x]->flags & (Client::Flag::LOADING | Client::Flag::LOADING_QUEST)) {
       return true;
     }
   }
@@ -65,24 +57,37 @@ size_t Lobby::count_clients() const {
   return ret;
 }
 
-void Lobby::add_client(shared_ptr<Client> c) {
+void Lobby::add_client(shared_ptr<Client> c, bool reverse_indexes) {
   ssize_t index;
-  for (index = 0; index < max_clients; index++) {
-    if (!this->clients[index].get()) {
-      this->clients[index] = c;
-      break;
+  if (reverse_indexes) {
+    for (index = max_clients - 1; index >= 0; index--) {
+      if (!this->clients[index].get()) {
+        this->clients[index] = c;
+        break;
+      }
+    }
+    if (index < 0) {
+      throw out_of_range("no space left in lobby");
+    }
+  } else {
+    for (index = 0; index < max_clients; index++) {
+      if (!this->clients[index].get()) {
+        this->clients[index] = c;
+        break;
+      }
+    }
+    if (index >= max_clients) {
+      throw out_of_range("no space left in lobby");
     }
   }
-  if (index >= max_clients) {
-    throw out_of_range("no space left in lobby");
-  }
+
   c->lobby_client_id = index;
   c->lobby_id = this->lobby_id;
 
   // if there's no one else in the lobby, set the leader id as well
-  if (index == 0) {
-    for (index = 1; index < max_clients; index++) {
-      if (this->clients[index].get()) {
+  if (index == (max_clients - 1) * reverse_indexes) {
+    for (index = 0; index < max_clients; index++) {
+      if (this->clients[index].get() && this->clients[index] != c) {
         break;
       }
     }
@@ -90,6 +95,16 @@ void Lobby::add_client(shared_ptr<Client> c) {
       this->leader_id = c->lobby_client_id;
     }
   }
+
+  // If the lobby is a game, assign the inventory's item IDs
+  if (this->is_game()) {
+    auto& inv = c->game_data.player()->inventory;
+    size_t count = max<uint8_t>(inv.num_items, 30);
+    for (size_t x = 0; x < count; x++) {
+      inv.items[x].data.id = 0x00010000 + 0x00200000 * c->lobby_client_id + x;
+    }
+    c->game_data.player()->print_inventory(stderr);
+  }
 }
 
 void Lobby::remove_client(shared_ptr<Client> c) {
@@ -129,7 +144,7 @@ void Lobby::move_client_to_lobby(shared_ptr<Lobby> dest_lobby,
 
 
 
-shared_ptr<Client> Lobby::find_client(const char16_t* identifier,
+shared_ptr<Client> Lobby::find_client(const u16string* identifier,
     uint64_t serial_number) {
   for (size_t x = 0; x < this->max_clients; x++) {
     if (!this->clients[x]) {
@@ -139,7 +154,7 @@ shared_ptr<Client> Lobby::find_client(const char16_t* identifier,
         (this->clients[x]->license->serial_number == serial_number)) {
       return this->clients[x];
     }
-    if (identifier && !char16cmp(this->clients[x]->player.disp.name, identifier, 0x10)) {
+    if (identifier && (this->clients[x]->game_data.player()->disp.name == *identifier)) {
       return this->clients[x];
     }
   }
@@ -164,17 +179,22 @@ uint8_t Lobby::game_event_for_lobby_event(uint8_t lobby_event) {
 
 
 
-void Lobby::add_item(const PlayerInventoryItem& item) {
-  this->item_id_to_floor_item.emplace(item.data.item_id, item);
+void Lobby::add_item(const PlayerInventoryItem& item, uint8_t area, float x, float z) {
+  auto& fi = this->item_id_to_floor_item[item.data.id];
+  fi.inv_item = item;
+  fi.area = area;
+  fi.x = x;
+  fi.z = z;
 }
 
-void Lobby::remove_item(uint32_t item_id, PlayerInventoryItem* item) {
+PlayerInventoryItem Lobby::remove_item(uint32_t item_id) {
   auto item_it = this->item_id_to_floor_item.find(item_id);
   if (item_it == this->item_id_to_floor_item.end()) {
     throw out_of_range("item not present");
   }
-  *item = move(item_it->second);
+  PlayerInventoryItem ret = move(item_it->second.inv_item);
   this->item_id_to_floor_item.erase(item_it);
+  return ret;
 }
 
 uint32_t Lobby::generate_item_id(uint8_t client_id) {
@@ -183,9 +203,3 @@ uint32_t Lobby::generate_item_id(uint8_t client_id) {
   }
   return this->next_game_item_id++;
 }
-
-void Lobby::assign_item_ids_for_player(uint32_t client_id, PlayerInventory& inv) {
-  for (size_t x = 0; x < inv.num_items; x++) {
-    inv.items[x].data.item_id = this->generate_item_id(client_id);
-  }
-}

src/Lobby.hh

@@ -2,47 +2,60 @@
 
 #include <inttypes.h>
 
+#include <array>
 #include <vector>
+#include <string>
+#include <memory>
+#include <unordered_map>
+#include <phosg/Encoding.hh>
 
 #include "Client.hh"
 #include "Player.hh"
 #include "Map.hh"
 #include "RareItemSet.hh"
-
-enum LobbyFlag {
-  IsGame = 0x01,
-  CheatsEnabled = 0x02, // game only
-  Public = 0x04, // lobby only
-  Episode3 = 0x08, // lobby & game
-  QuestInProgress = 0x10, // game only
-  JoinableQuestInProgress = 0x20, // game only
-  Default = 0x40, // lobby only; not set for games and private lobbies
-  Persistent = 0x80, // if not set, lobby is deleted when empty
-};
+#include "Text.hh"
+#include "Quest.hh"
 
 struct Lobby {
+  enum Flag {
+    GAME = 0x01,
+    CHEATS_ENABLED = 0x02, // game only
+    PUBLIC = 0x04, // lobby only
+    EPISODE_3_ONLY = 0x08, // lobby & game
+    QUEST_IN_PROGRESS = 0x10, // game only
+    JOINABLE_QUEST_IN_PROGRESS = 0x20, // game only
+    DEFAULT = 0x40, // lobby only; not set for games and private lobbies
+    PERSISTENT = 0x80, // if not set, lobby is deleted when empty
+  };
+
   uint32_t lobby_id;
 
   uint32_t min_level;
   uint32_t max_level;
 
   // item info
+  struct FloorItem {
+    PlayerInventoryItem inv_item;
+    float x;
+    float z;
+    uint8_t area;
+  };
   std::vector<PSOEnemy> enemies;
   std::shared_ptr<const RareItemSet> rare_item_set;
-  uint32_t next_item_id[12];
+  std::array<uint32_t, 12> next_item_id;
   uint32_t next_game_item_id;
   PlayerInventoryItem next_drop_item;
-  std::unordered_map<uint32_t, PlayerInventoryItem> item_id_to_floor_item;
-  uint32_t variations[0x20];
+  std::unordered_map<uint32_t, FloorItem> item_id_to_floor_item;
+  parray<le_uint32_t, 0x20> variations;
 
   // game config
   GameVersion version;
   uint8_t section_id;
-  uint8_t episode;
+  uint8_t episode; // 1 = Ep1, 2 = Ep2, 3 = Ep4, 0xFF = Ep3
   uint8_t difficulty;
   uint8_t mode;
-  char16_t password[0x24];
-  char16_t name[0x24];
+  std::u16string password;
+  std::u16string name;
   uint32_t rare_seed;
 
   //EP3_GAME_CONFIG* ep3; // only present if this is an Episode 3 game
@@ -54,32 +67,33 @@ struct Lobby {
   uint8_t leader_id;
   uint8_t max_clients;
   uint32_t flags;
-  uint32_t loading_quest_id; // for use with joinable quests
-  std::shared_ptr<Client> clients[12];
+  std::shared_ptr<const Quest> loading_quest;
+  std::array<std::shared_ptr<Client>, 12> clients;
 
   Lobby();
 
-  bool is_game() const;
+  inline bool is_game() const {
+    return this->flags & Flag::GAME;
+  }
 
   void reassign_leader_on_client_departure(size_t leaving_client_id);
   size_t count_clients() const;
   bool any_client_loading() const;
 
-  void add_client(std::shared_ptr<Client> c);
+  void add_client(std::shared_ptr<Client> c, bool reverse_indexes = true);
   void remove_client(std::shared_ptr<Client> c);
 
   void move_client_to_lobby(std::shared_ptr<Lobby> dest_lobby,
       std::shared_ptr<Client> c);
 
-  std::shared_ptr<Client> find_client(const char16_t* identifier = nullptr,
+  std::shared_ptr<Client> find_client(
+      const std::u16string* identifier = nullptr,
       uint64_t serial_number = 0);
 
-  void add_item(const PlayerInventoryItem& item);
-  void remove_item(uint32_t item_id, PlayerInventoryItem* item);
+  void add_item(const PlayerInventoryItem& item, uint8_t area, float x, float z);
+  PlayerInventoryItem remove_item(uint32_t item_id);
   size_t find_item(uint32_t item_id);
   uint32_t generate_item_id(uint8_t client_id);
 
-  void assign_item_ids_for_player(uint32_t client_id, PlayerInventory& inv);
-
   static uint8_t game_event_for_lobby_event(uint8_t lobby_event);
 };

src/Main.cc

@@ -19,7 +19,6 @@
 #include "FileContentsCache.hh"
 #include "Text.hh"
 #include "ServerShell.hh"
-#include "ProxyShell.hh"
 #include "IPStackSimulator.hh"
 
 using namespace std;
@@ -27,33 +26,23 @@ using namespace std;
 
 
 FileContentsCache file_cache;
+bool use_terminal_colors = false;
 
 
 
-static const unordered_map<string, PortConfiguration> default_port_to_behavior({
-  {"gc-jp10",  {9000,  GameVersion::GC,    ServerBehavior::LoginServer}},
-  {"gc-jp11",  {9001,  GameVersion::GC,    ServerBehavior::LoginServer}},
-  {"gc-jp3",   {9003,  GameVersion::GC,    ServerBehavior::LoginServer}},
-  {"gc-us10",  {9100,  GameVersion::PC,    ServerBehavior::SplitReconnect}},
-  {"gc-us3",   {9103,  GameVersion::GC,    ServerBehavior::LoginServer}},
-  {"gc-eu10",  {9200,  GameVersion::GC,    ServerBehavior::LoginServer}},
-  {"gc-eu11",  {9201,  GameVersion::GC,    ServerBehavior::LoginServer}},
-  {"gc-eu3",   {9203,  GameVersion::GC,    ServerBehavior::LoginServer}},
-  {"pc-login", {9300,  GameVersion::PC,    ServerBehavior::LoginServer}},
-  {"pc-patch", {10000, GameVersion::Patch, ServerBehavior::PatchServer}},
-  {"bb-patch", {11000, GameVersion::Patch, ServerBehavior::PatchServer}},
-  {"bb-data",  {12000, GameVersion::BB,    ServerBehavior::DataServerBB}},
-
-  // these aren't hardcoded in any games; user can override them
-  {"bb-data1", {12004, GameVersion::BB,    ServerBehavior::DataServerBB}},
-  {"bb-data2", {12005, GameVersion::BB,    ServerBehavior::DataServerBB}},
-  {"bb-login", {12008, GameVersion::BB,    ServerBehavior::LoginServer}},
-  {"pc-lobby", {9420,  GameVersion::PC,    ServerBehavior::LobbyServer}},
-  {"gc-lobby", {9421,  GameVersion::GC,    ServerBehavior::LobbyServer}},
-  {"bb-lobby", {9422,  GameVersion::BB,    ServerBehavior::LobbyServer}},
-});
-
-
+static vector<PortConfiguration> parse_port_configuration(
+    shared_ptr<const JSONObject> json) {
+  vector<PortConfiguration> ret;
+  for (const auto& item_json_it : json->as_dict()) {
+    auto item_list = item_json_it.second->as_list();
+    PortConfiguration& pc = ret.emplace_back();
+    pc.name = item_json_it.first;
+    pc.port = item_list[0]->as_int();
+    pc.version = version_for_name(item_list[1]->as_string().c_str());
+    pc.behavior = server_behavior_for_name(item_list[2]->as_string().c_str());
+  }
+  return ret;
+}
 
 template <typename T>
 vector<T> parse_int_vector(shared_ptr<const JSONObject> o) {
@@ -83,8 +72,7 @@ void populate_state_from_config(shared_ptr<ServerState> s,
     }
   } catch (const out_of_range&) { }
 
-  // TODO: make this configurable
-  s->set_port_configuration(default_port_to_behavior);
+  s->set_port_configuration(parse_port_configuration(d.at("PortConfiguration")));
 
   auto enemy_categories = parse_int_vector<uint32_t>(d.at("CommonItemDropRates-Enemy"));
   auto box_categories = parse_int_vector<uint32_t>(d.at("CommonItemDropRates-Box"));
@@ -95,23 +83,96 @@ void populate_state_from_config(shared_ptr<ServerState> s,
   s->common_item_creator.reset(new CommonItemCreator(enemy_categories,
       box_categories, unit_types));
 
-  shared_ptr<vector<MenuItem>> information_menu(new vector<MenuItem>());
+  shared_ptr<vector<MenuItem>> information_menu_pc(new vector<MenuItem>());
+  shared_ptr<vector<MenuItem>> information_menu_gc(new vector<MenuItem>());
   shared_ptr<vector<u16string>> information_contents(new vector<u16string>());
 
-  information_menu->emplace_back(INFORMATION_MENU_GO_BACK, u"Go back",
+  information_menu_gc->emplace_back(INFORMATION_MENU_GO_BACK, u"Go back",
       u"Return to the\nmain menu", 0);
-
-  uint32_t item_id = 0;
-  for (const auto& item : d.at("InformationMenuContents")->as_list()) {
-    auto& v = item->as_list();
-    information_menu->emplace_back(item_id, decode_sjis(v.at(0)->as_string()),
-        decode_sjis(v.at(1)->as_string()), MenuItemFlag::RequiresMessageBoxes);
-    information_contents->emplace_back(decode_sjis(v.at(2)->as_string()));
-    item_id++;
+  {
+    uint32_t item_id = 0;
+    for (const auto& item : d.at("InformationMenuContents")->as_list()) {
+      auto& v = item->as_list();
+      information_menu_pc->emplace_back(item_id, decode_sjis(v.at(0)->as_string()),
+          decode_sjis(v.at(1)->as_string()), 0);
+      information_menu_gc->emplace_back(item_id, decode_sjis(v.at(0)->as_string()),
+          decode_sjis(v.at(1)->as_string()), MenuItem::Flag::REQUIRES_MESSAGE_BOXES);
+      information_contents->emplace_back(decode_sjis(v.at(2)->as_string()));
+      item_id++;
+    }
   }
-  s->information_menu = information_menu;
+  s->information_menu_pc = information_menu_pc;
+  s->information_menu_gc = information_menu_gc;
   s->information_contents = information_contents;
 
+  s->proxy_destinations_menu_pc.emplace_back(PROXY_DESTINATIONS_MENU_GO_BACK,
+      u"Go back", u"Return to the\nmain menu", 0);
+  s->proxy_destinations_menu_gc.emplace_back(PROXY_DESTINATIONS_MENU_GO_BACK,
+      u"Go back", u"Return to the\nmain menu", 0);
+  {
+    uint32_t item_id = 0;
+    for (const auto& item : d.at("ProxyDestinations-GC")->as_dict()) {
+      const string& netloc_str = item.second->as_string();
+      s->proxy_destinations_menu_gc.emplace_back(item_id, decode_sjis(item.first),
+          decode_sjis(netloc_str), 0);
+      s->proxy_destinations_gc.emplace_back(parse_netloc(netloc_str));
+      item_id++;
+    }
+  }
+  {
+    uint32_t item_id = 0;
+    for (const auto& item : d.at("ProxyDestinations-PC")->as_dict()) {
+      const string& netloc_str = item.second->as_string();
+      s->proxy_destinations_menu_pc.emplace_back(item_id, decode_sjis(item.first),
+          decode_sjis(netloc_str), 0);
+      s->proxy_destinations_pc.emplace_back(parse_netloc(netloc_str));
+      item_id++;
+    }
+  }
+  try {
+    const string& netloc_str = d.at("ProxyDestination-Patch")->as_string();
+    s->proxy_destination_patch = parse_netloc(netloc_str);
+    log(INFO, "Patch server proxy is enabled with destination %s", netloc_str.c_str());
+    for (auto& it : s->name_to_port_config) {
+      if (it.second->version == GameVersion::PATCH) {
+        it.second->behavior = ServerBehavior::PROXY_SERVER;
+      }
+    }
+  } catch (const out_of_range&) {
+    s->proxy_destination_patch.first = "";
+    s->proxy_destination_patch.second = 0;
+  }
+  try {
+    const string& netloc_str = d.at("ProxyDestination-BB")->as_string();
+    s->proxy_destination_bb = parse_netloc(netloc_str);
+    log(INFO, "BB proxy is enabled with destination %s", netloc_str.c_str());
+    for (auto& it : s->name_to_port_config) {
+      if (it.second->version == GameVersion::BB) {
+        it.second->behavior = ServerBehavior::PROXY_SERVER;
+      }
+    }
+  } catch (const out_of_range&) {
+    s->proxy_destination_bb.first = "";
+    s->proxy_destination_bb.second = 0;
+  }
+
+  s->main_menu.emplace_back(MAIN_MENU_GO_TO_LOBBY, u"Go to lobby",
+      u"Join the lobby", 0);
+  s->main_menu.emplace_back(MAIN_MENU_INFORMATION, u"Information",
+      u"View server information", MenuItem::Flag::REQUIRES_MESSAGE_BOXES);
+  if (!s->proxy_destinations_pc.empty()) {
+    s->main_menu.emplace_back(MAIN_MENU_PROXY_DESTINATIONS, u"Proxy server",
+        u"Connect to another\nserver", MenuItem::Flag::PC_ONLY);
+  }
+  if (!s->proxy_destinations_gc.empty()) {
+    s->main_menu.emplace_back(MAIN_MENU_PROXY_DESTINATIONS, u"Proxy server",
+        u"Connect to another\nserver", MenuItem::Flag::GC_ONLY);
+  }
+  s->main_menu.emplace_back(MAIN_MENU_DOWNLOAD_QUESTS, u"Download quests",
+      u"Download quests", 0);
+  s->main_menu.emplace_back(MAIN_MENU_DISCONNECT, u"Disconnect",
+      u"Disconnect", 0);
+
   try {
     s->welcome_message = decode_sjis(d.at("WelcomeMessage")->as_string());
   } catch (const out_of_range&) { }
@@ -161,24 +222,28 @@ void populate_state_from_config(shared_ptr<ServerState> s,
     s->allow_unregistered_users = true;
   }
 
-  {
-    string key_file_name = d.at("BlueBurstKeyFile")->as_string();
-    string key_file_contents = load_file("system/blueburst/keys/" + key_file_name + ".nsk");
-    if (key_file_contents.size() != sizeof(PSOBBEncryption::KeyFile)) {
-      log(WARNING, "Blue Burst key file is the wrong size (%zu bytes; should be %zu bytes)",
-          key_file_contents.size(), sizeof(PSOBBEncryption::KeyFile));
-      log(WARNING, "Ignoring key file; Blue Burst clients will not be able to connect");
+  for (const string& filename : list_directory("system/blueburst/keys")) {
+    if (!ends_with(filename, ".nsk")) {
+      continue;
+    }
+    string contents = load_file("system/blueburst/keys/" + filename);
+    if (contents.size() != sizeof(PSOBBEncryption::KeyFile)) {
+      log(WARNING, "Blue Burst key file %s is the wrong size (%zu bytes; should be %zu bytes)",
+          filename.c_str(), contents.size(), sizeof(PSOBBEncryption::KeyFile));
     } else {
-      memcpy(&s->default_key_file, key_file_contents.data(), sizeof(PSOBBEncryption::KeyFile));
-      log(INFO, "Loaded Blue Burst key file: %s", key_file_name.c_str());
+      shared_ptr<PSOBBEncryption::KeyFile> k(new PSOBBEncryption::KeyFile());
+      memcpy(k.get(), contents.data(), sizeof(PSOBBEncryption::KeyFile));
+      s->bb_private_keys.emplace_back(k);
+      log(INFO, "Loaded Blue Burst key file: %s", filename.c_str());
     }
   }
+  log(INFO, "%zu Blue Burst key file(s) loaded", s->bb_private_keys.size());
 
   try {
     bool run_shell = d.at("RunInteractiveShell")->as_bool();
     s->run_shell_behavior = run_shell ?
-        ServerState::RunShellBehavior::Always :
-        ServerState::RunShellBehavior::Never;
+        ServerState::RunShellBehavior::ALWAYS :
+        ServerState::RunShellBehavior::NEVER;
   } catch (const out_of_range&) { }
 }
 
@@ -214,26 +279,90 @@ void drop_privileges(const string& username) {
 
 
 
-int main(int argc, char* argv[]) {
-  string proxy_hostname;
-  int proxy_port = 0;
-  GameVersion proxy_version = GameVersion::GC;
+enum class Behavior {
+  RUN_SERVER = 0,
+  DECRYPT_DATA,
+  ENCRYPT_DATA,
+};
+
+enum class EncryptionType {
+  PC = 0,
+  GC,
+  BB,
+};
+
+int main(int argc, char** argv) {
+  Behavior behavior = Behavior::RUN_SERVER;
+  EncryptionType crypt_type = EncryptionType::PC;
+  string seed;
+  string key_file_name;
+  bool parse_data = false;
   for (int x = 1; x < argc; x++) {
-    if (!strncmp(argv[x], "--proxy-destination=", 20)) {
-      auto netloc = parse_netloc(&argv[x][20], 9100);
-      proxy_hostname = netloc.first;
-      proxy_port = netloc.second;
-
-    } else if (!strncmp(argv[x], "--proxy-version=", 16)) {
-      proxy_version = version_for_name(&argv[x][16]);
-
+    if (!strcmp(argv[x], "--decrypt-data")) {
+      behavior = Behavior::DECRYPT_DATA;
+    } else if (!strcmp(argv[x], "--encrypt-data")) {
+      behavior = Behavior::ENCRYPT_DATA;
+    } else if (!strcmp(argv[x], "--pc")) {
+      crypt_type = EncryptionType::PC;
+    } else if (!strcmp(argv[x], "--gc")) {
+      crypt_type = EncryptionType::GC;
+    } else if (!strcmp(argv[x], "--bb")) {
+      crypt_type = EncryptionType::BB;
+    } else if (!strncmp(argv[x], "--seed=", 7)) {
+      seed = &argv[x][7];
+    } else if (!strncmp(argv[x], "--key=", 6)) {
+      key_file_name = &argv[x][6];
+    } else if (!strcmp(argv[x], "--parse-data")) {
+      parse_data = true;
     } else {
       throw invalid_argument(string_printf("unknown option: %s", argv[x]));
     }
   }
 
+  if (behavior != Behavior::RUN_SERVER) {
+    shared_ptr<PSOEncryption> crypt;
+    if (crypt_type == EncryptionType::PC) {
+      crypt.reset(new PSOPCEncryption(stoul(seed, nullptr, 16)));
+    } else if (crypt_type == EncryptionType::GC) {
+      crypt.reset(new PSOGCEncryption(stoul(seed, nullptr, 16)));
+    } else if (crypt_type == EncryptionType::BB) {
+      seed = parse_data_string(seed);
+      auto key = load_object_file<PSOBBEncryption::KeyFile>(
+          "system/blueburst/keys/" + key_file_name + ".nsk");
+      crypt.reset(new PSOBBEncryption(key, seed.data(), seed.size()));
+    } else {
+      throw logic_error("invalid encryption type");
+    }
+
+    string data = read_all(stdin);
+    if (parse_data) {
+      data = parse_data_string(data);
+    }
+
+    if (behavior == Behavior::DECRYPT_DATA) {
+      crypt->decrypt(data.data(), data.size());
+    } else if (behavior == Behavior::ENCRYPT_DATA) {
+      crypt->encrypt(data.data(), data.size());
+    } else {
+      throw logic_error("invalid behavior");
+    }
+
+    if (isatty(fileno(stdout))) {
+      print_data(stdout, data);
+    } else {
+      fwritex(stdout, data);
+    }
+    fflush(stdout);
+
+    return 0;
+  }
+
   signal(SIGPIPE, SIG_IGN);
 
+  if (isatty(fileno(stderr))) {
+    use_terminal_colors = true;
+  }
+
   shared_ptr<ServerState> state(new ServerState());
 
   shared_ptr<struct event_base> base(event_base_new(), event_base_free);
@@ -249,6 +378,18 @@ int main(int argc, char* argv[]) {
   auto config_json = JSONObject::parse(load_file("system/config.json"));
   populate_state_from_config(state, config_json);
 
+  log(INFO, "Loading license list");
+  state->license_manager.reset(new LicenseManager("system/licenses.nsi"));
+
+  log(INFO, "Loading battle parameters");
+  state->battle_params.reset(new BattleParamTable("system/blueburst/BattleParamEntry"));
+
+  log(INFO, "Loading level table");
+  state->level_table.reset(new LevelTable("system/blueburst/PlyLevelTbl.prs", true));
+
+  log(INFO, "Collecting quest metadata");
+  state->quest_index.reset(new QuestIndex("system/quests"));
+
   shared_ptr<DNSServer> dns_server;
   if (state->dns_server_port) {
     log(INFO, "Starting DNS server");
@@ -259,44 +400,53 @@ int main(int argc, char* argv[]) {
     log(INFO, "DNS server is disabled");
   }
 
-  shared_ptr<ProxyServer> proxy_server;
   shared_ptr<Server> game_server;
-  if (proxy_port) {
-    log(INFO, "Starting proxy server");
-    sockaddr_storage proxy_destination_ss = make_sockaddr_storage(
-        proxy_hostname, proxy_port).first;
-    proxy_server.reset(new ProxyServer(base, proxy_destination_ss,
-        proxy_version));
-    proxy_server->listen(proxy_port);
-    if (proxy_version == GameVersion::BB) {
-      proxy_server->listen(proxy_port + 1);
+
+  log(INFO, "Opening sockets");
+  for (const auto& it : state->name_to_port_config) {
+    const auto& pc = it.second;
+    if (pc->behavior == ServerBehavior::PROXY_SERVER) {
+      if (!state->proxy_server.get()) {
+        log(INFO, "Starting proxy server");
+        state->proxy_server.reset(new ProxyServer(base, state));
+      }
+      if (state->proxy_server.get()) {
+        // For PC and GC, proxy sessions are dynamically created when a client
+        // picks a destination from the menu. For patch and BB clients, there's
+        // no way to ask the client which destination they want, so only one
+        // destination is supported, and we have to manually specify the
+        // destination netloc here.
+        if (pc->version == GameVersion::PATCH) {
+          struct sockaddr_storage ss = make_sockaddr_storage(
+              state->proxy_destination_patch.first,
+              state->proxy_destination_patch.second).first;
+          state->proxy_server->listen(pc->port, pc->version, &ss);
+        } else if (pc->version == GameVersion::BB) {
+          struct sockaddr_storage ss = make_sockaddr_storage(
+              state->proxy_destination_bb.first,
+              state->proxy_destination_bb.second).first;
+          state->proxy_server->listen(pc->port, pc->version, &ss);
+        } else {
+          state->proxy_server->listen(pc->port, pc->version);
+        }
+      }
+    } else {
+      if (!game_server.get()) {
+        log(INFO, "Starting game server");
+        game_server.reset(new Server(base, state));
+      }
+      string name = string_printf("%s (%s, %s) on port %hu",
+          pc->name.c_str(), name_for_version(pc->version),
+          name_for_server_behavior(pc->behavior), pc->port);
+      game_server->listen(name, "", pc->port, pc->version, pc->behavior);
     }
-
-  } else {
-    log(INFO, "Starting game server");
-    game_server.reset(new Server(base, state));
-    for (const auto& it : state->named_port_configuration) {
-      game_server->listen("", it.second.port, it.second.version, it.second.behavior);
-    }
-
-    log(INFO, "Loading license list");
-    state->license_manager.reset(new LicenseManager("system/licenses.nsi"));
-
-    log(INFO, "Loading battle parameters");
-    state->battle_params.reset(new BattleParamTable("system/blueburst/BattleParamEntry"));
-
-    log(INFO, "Loading level table");
-    state->level_table.reset(new LevelTable("system/blueburst/PlyLevelTbl.prs", true));
-
-    log(INFO, "Collecting quest metadata");
-    state->quest_index.reset(new QuestIndex("system/quests"));
   }
 
   shared_ptr<IPStackSimulator> ip_stack_simulator;
   if (!state->ip_stack_addresses.empty()) {
     log(INFO, "Starting IP stack simulator");
     ip_stack_simulator.reset(new IPStackSimulator(
-        base, game_server, proxy_server, state));
+        base, game_server, state));
     for (const auto& it : state->ip_stack_addresses) {
       auto netloc = parse_netloc(it);
       ip_stack_simulator->listen(netloc.first, netloc.second);
@@ -308,24 +458,20 @@ int main(int argc, char* argv[]) {
     drop_privileges(state->username);
   }
 
-  bool should_run_shell = (state->run_shell_behavior == ServerState::RunShellBehavior::Always);
-  if (state->run_shell_behavior == ServerState::RunShellBehavior::Default) {
+  bool should_run_shell = (state->run_shell_behavior == ServerState::RunShellBehavior::ALWAYS);
+  if (state->run_shell_behavior == ServerState::RunShellBehavior::DEFAULT) {
     should_run_shell = isatty(fileno(stdin));
   }
 
   shared_ptr<Shell> shell;
   if (should_run_shell) {
-    log(INFO, "Starting interactive shell");
-    if (proxy_port) {
-      shell.reset(new ProxyShell(base, state, proxy_server));
-    } else {
-      shell.reset(new ServerShell(base, state));
-    }
+    shell.reset(new ServerShell(base, state));
   }
 
   log(INFO, "Ready");
   event_base_dispatch(base.get());
 
   log(INFO, "Normal shutdown");
+  state->proxy_server.reset(); // Break reference cycle
   return 0;
 }

src/Map.cc

@@ -74,7 +74,7 @@ struct EnemyEntry {
   uint32_t reserved14;
   uint32_t skin;
   uint32_t reserved15;
-};
+} __attribute__((packed));
 
 static vector<PSOEnemy> parse_map(uint8_t episode, uint8_t difficulty,
     const BattleParams* battle_params, const EnemyEntry* map,
@@ -433,7 +433,7 @@ static vector<PSOEnemy> parse_map(uint8_t episode, uint8_t difficulty,
   return enemies;
 }
 
-vector<PSOEnemy> load_map(const char* filename, uint8_t episode,
+vector<PSOEnemy> load_map(const std::string& filename, uint8_t episode,
     uint8_t difficulty, const BattleParams* battle_params, bool alt_enemies) {
   shared_ptr<const string> data = file_cache.get(filename);
   const EnemyEntry* entries = reinterpret_cast<const EnemyEntry*>(data->data());

src/Map.hh

@@ -3,6 +3,7 @@
 #include <inttypes.h>
 
 #include <vector>
+#include <string>
 
 
 
@@ -10,14 +11,14 @@ struct BattleParams {
     uint16_t atp; // attack power
     uint16_t psv; // perseverance (intelligence?)
     uint16_t evp; // evasion
-    uint16_t hp; // hit points
+    uint16_t hp;  // hit points
     uint16_t dfp; // defense
     uint16_t ata; // accuracy
     uint16_t lck; // luck
-    uint8_t unknown[14];
+    uint8_t unknown_a1[0x0E];
     uint32_t experience;
     uint32_t difficulty;
-};
+} __attribute__((packed));
 
 struct BattleParamTable {
   BattleParams entries[2][3][4][0x60]; // online/offline, episode, difficulty, monster type
@@ -28,13 +29,13 @@ struct BattleParamTable {
       uint8_t monster_type) const;
   const BattleParams* get_subtable(bool solo, uint8_t episode,
       uint8_t difficulty) const;
-};
+} __attribute__((packed));
 
 
 
 struct BattleParamIndex {
   BattleParamTable table_for_episode[3];
-};
+} __attribute__((packed));
 
 // an enemy entry as loaded by the game
 struct PSOEnemy {
@@ -46,7 +47,7 @@ struct PSOEnemy {
 
   PSOEnemy();
   PSOEnemy(uint32_t experience, uint32_t rt_index);
-};
+} __attribute__((packed));
 
-std::vector<PSOEnemy> load_map(const char* filename, uint8_t episode,
+std::vector<PSOEnemy> load_map(const std::string& filename, uint8_t episode,
     uint8_t difficulty, const BattleParams* bp, bool alt_enemies);

src/Menu.hh

@@ -6,16 +6,37 @@
 
 
 
-enum MenuItemFlag {
-  InvisibleOnDC = 0x01,
-  InvisibleOnPC = 0x02,
-  InvisibleOnGC = 0x04,
-  InvisibleOnGCEpisode3 = 0x08,
-  InvisibleOnBB = 0x10,
-  RequiresMessageBoxes = 0x20,
-};
+#define MAIN_MENU_ID                     0x11000011
+#define INFORMATION_MENU_ID              0x22000022
+#define LOBBY_MENU_ID                    0x33000033
+#define GAME_MENU_ID                     0x44000044
+#define QUEST_MENU_ID                    0x55000055
+#define QUEST_FILTER_MENU_ID             0x66000066
+#define PROXY_DESTINATIONS_MENU_ID       0x77000077
+
+#define MAIN_MENU_GO_TO_LOBBY            0x11AAAA11
+#define MAIN_MENU_INFORMATION            0x11BBBB11
+#define MAIN_MENU_DOWNLOAD_QUESTS        0x11CCCC11
+#define MAIN_MENU_PROXY_DESTINATIONS     0x11DDDD11
+#define MAIN_MENU_DISCONNECT             0x11EEEE11
+#define INFORMATION_MENU_GO_BACK         0x22FFFF22
+#define PROXY_DESTINATIONS_MENU_GO_BACK  0x77FFFF77
+
+
 
 struct MenuItem {
+  enum Flag {
+    INVISIBLE_ON_DC = 0x01,
+    INVISIBLE_ON_PC = 0x02,
+    INVISIBLE_ON_GC = 0x04,
+    INVISIBLE_ON_BB = 0x08,
+    DC_ONLY = INVISIBLE_ON_PC | INVISIBLE_ON_GC | INVISIBLE_ON_BB,
+    PC_ONLY = INVISIBLE_ON_DC | INVISIBLE_ON_GC | INVISIBLE_ON_BB,
+    GC_ONLY = INVISIBLE_ON_DC | INVISIBLE_ON_PC | INVISIBLE_ON_BB,
+    BB_ONLY = INVISIBLE_ON_DC | INVISIBLE_ON_PC | INVISIBLE_ON_GC,
+    REQUIRES_MESSAGE_BOXES = 0x10,
+  };
+
   uint32_t item_id;
   std::u16string name;
   std::u16string description;

src/PSOEncryption.cc

@@ -16,9 +16,9 @@ using namespace std;
 
 
 
-// most ciphers used by pso are symmetric; alias decrypt to encrypt by default
-void PSOEncryption::decrypt(void* data, size_t size) {
-  this->encrypt(data, size);
+// Most ciphers used by PSO are symmetric; alias decrypt to encrypt by default
+void PSOEncryption::decrypt(void* data, size_t size, bool advance) {
+  this->encrypt(data, size, advance);
 }
 
 
@@ -30,8 +30,7 @@ void PSOPCEncryption::update_stream() {
   eax = edi;
   while (edx > 0) {
     esi = this->stream[eax + 0x1F];
-    ebp = this->stream[eax];
-    ebp = ebp - esi;
+    ebp = this->stream[eax] - esi;
     this->stream[eax] = ebp;
     eax++;
     edx--;
@@ -41,8 +40,7 @@ void PSOPCEncryption::update_stream() {
   eax = edi;
   while (edx > 0) {
     esi = this->stream[eax - 0x18];
-    ebp = this->stream[eax];
-    ebp = ebp - esi;
+    ebp = this->stream[eax] - esi;
     this->stream[eax] = ebp;
     eax++;
     edx--;
@@ -71,23 +69,30 @@ PSOPCEncryption::PSOPCEncryption(uint32_t seed) : offset(1) {
   }
 }
 
-uint32_t PSOPCEncryption::next() {
+uint32_t PSOPCEncryption::next(bool advance) {
   if (this->offset == PC_STREAM_LENGTH) {
     this->update_stream();
     this->offset = 1;
   }
-  return this->stream[this->offset++];
+  uint32_t ret = this->stream[this->offset];
+  if (advance) {
+    this->offset++;
+  }
+  return ret;
 }
 
-void PSOPCEncryption::encrypt(void* vdata, size_t size) {
+void PSOPCEncryption::encrypt(void* vdata, size_t size, bool advance) {
   if (size & 3) {
     throw invalid_argument("size must be a multiple of 4");
   }
+  if (!advance && (size != 4)) {
+    throw logic_error("cannot peek-encrypt/decrypt with size > 4");
+  }
   size >>= 2;
 
-  uint32_t* data = reinterpret_cast<uint32_t*>(vdata);
+  le_uint32_t* data = reinterpret_cast<le_uint32_t*>(vdata);
   for (size_t x = 0; x < size; x++) {
-    data[x] ^= this->next();
+    data[x] ^= this->next(advance);
   }
 }
 
@@ -110,12 +115,15 @@ void PSOGCEncryption::update_stream() {
   this->offset = 0;
 }
 
-uint32_t PSOGCEncryption::next() {
-  this->offset++;
+uint32_t PSOGCEncryption::next(bool advance) {
   if (this->offset == GC_STREAM_LENGTH) {
     this->update_stream();
   }
-  return this->stream[this->offset];
+  uint32_t ret = this->stream[this->offset];
+  if (advance) {
+    this->offset++;
+  }
+  return ret;
 }
 
 PSOGCEncryption::PSOGCEncryption(uint32_t seed) : offset(0) {
@@ -145,102 +153,161 @@ PSOGCEncryption::PSOGCEncryption(uint32_t seed) : offset(0) {
     this->stream[this->offset++] = (this->stream[source3++] ^ (((this->stream[source1++] << 23) & 0xFF800000) ^ ((this->stream[source2++] >> 9) & 0x007FFFFF)));
   }
 
-  for (size_t x = 0; x < 3; x++) {
+  for (size_t x = 0; x < 4; x++) {
     this->update_stream();
   }
-  this->offset = GC_STREAM_LENGTH - 1;
 }
 
-void PSOGCEncryption::encrypt(void* vdata, size_t size) {
+void PSOGCEncryption::encrypt(void* vdata, size_t size, bool advance) {
   if (size & 3) {
     throw invalid_argument("size must be a multiple of 4");
   }
+  if (!advance && (size != 4)) {
+    throw logic_error("cannot peek-encrypt/decrypt with size > 4");
+  }
   size >>= 2;
 
-  uint32_t* data = reinterpret_cast<uint32_t*>(vdata);
+  le_uint32_t* data = reinterpret_cast<le_uint32_t*>(vdata);
   for (size_t x = 0; x < size; x++) {
-    data[x] ^= this->next();
+    data[x] ^= this->next(advance);
   }
 }
 
 
 
-void PSOBBEncryption::decrypt(void* vdata, size_t size) {
-  if (size & 7) {
-    throw invalid_argument("size must be a multiple of 8");
-  }
-  size >>= 3;
+void PSOBBEncryption::decrypt(void* vdata, size_t size, bool advance) {
+  if (this->subtype != Subtype::JSD1) {
+    if (size & 7) {
+      throw invalid_argument("size must be a multiple of 8");
+    }
+    size_t num_dwords = size >> 2;
+    le_uint32_t* dwords = reinterpret_cast<le_uint32_t*>(vdata);
+    uint32_t edx, ebx, ebp, esi, edi;
 
-  uint32_t* data = reinterpret_cast<uint32_t*>(vdata);
-  uint32_t edx, ebx, ebp, esi, edi;
+    edx = 0;
+    while (edx < num_dwords) {
+      ebx = dwords[edx];
+      ebx = ebx ^ this->stream[5];
+      ebp = ((this->stream[(ebx >> 0x18) + 0x12] +
+              this->stream[((ebx >> 0x10) & 0xFF) + 0x112]) ^
+             this->stream[((ebx >> 0x8) & 0xFF) + 0x212]) +
+            this->stream[(ebx & 0xFF) + 0x312];
+      ebp = ebp ^ this->stream[4];
+      ebp ^= dwords[edx + 1];
+      edi = ((this->stream[(ebp >> 0x18) + 0x12] +
+              this->stream[((ebp >> 0x10) & 0xFF) + 0x112]) ^
+             this->stream[((ebp >> 0x8) & 0xFF) + 0x212]) +
+            this->stream[(ebp & 0xFF) + 0x312];
+      edi = edi ^ this->stream[3];
+      ebx = ebx ^ edi;
+      esi = ((this->stream[(ebx >> 0x18) + 0x12] +
+              this->stream[((ebx >> 0x10) & 0xFF) + 0x112]) ^
+             this->stream[((ebx >> 0x8) & 0xFF) + 0x212]) +
+            this->stream[(ebx & 0xFF) + 0x312];
+      ebp = ebp ^ esi ^ this->stream[2];
+      edi = ((this->stream[(ebp >> 0x18) + 0x12] +
+              this->stream[((ebp >> 0x10) & 0xFF) + 0x112]) ^
+             this->stream[((ebp >> 0x8) & 0xFF) + 0x212]) +
+            this->stream[(ebp & 0xFF) + 0x312];
+      edi = edi ^ this->stream[1];
+      ebp = ebp ^ this->stream[0];
+      ebx = ebx ^ edi;
+      dwords[edx] = ebp;
+      dwords[edx + 1] = ebx;
+      edx += 2;
+    }
 
-  edx = 0;
-  while (edx < size) {
-    ebx = data[edx];
-    ebx = ebx ^ this->stream[5];
-    ebp = ((this->stream[(ebx >> 0x18) + 0x12]+this->stream[((ebx >> 0x10)& 0xff) + 0x112])
-      ^ this->stream[((ebx >> 0x8)& 0xff) + 0x212]) + this->stream[(ebx & 0xff) + 0x312];
-    ebp = ebp ^ this->stream[4];
-    ebp ^= data[edx + 1];
-    edi = ((this->stream[(ebp >> 0x18) + 0x12]+this->stream[((ebp >> 0x10)& 0xff) + 0x112])
-      ^ this->stream[((ebp >> 0x8)& 0xff) + 0x212]) + this->stream[(ebp & 0xff) + 0x312];
-    edi = edi ^ this->stream[3];
-    ebx = ebx ^ edi;
-    esi = ((this->stream[(ebx >> 0x18) + 0x12]+this->stream[((ebx >> 0x10)& 0xff) + 0x112])
-      ^ this->stream[((ebx >> 0x8)& 0xff) + 0x212]) + this->stream[(ebx & 0xff) + 0x312];
-    ebp = ebp ^ esi ^ this->stream[2];
-    edi = ((this->stream[(ebp >> 0x18) + 0x12]+this->stream[((ebp >> 0x10)& 0xff) + 0x112])
-      ^ this->stream[((ebp >> 0x8)& 0xff) + 0x212]) + this->stream[(ebp & 0xff) + 0x312];
-    edi = edi ^ this->stream[1];
-    ebp = ebp ^ this->stream[0];
-    ebx = ebx ^ edi;
-    data[edx] = ebp;
-    data[edx + 1] = ebx;
-    edx += 2;
+  } else { // subtype == Subtype::JSD1
+    if (!advance && (size > 0x100)) {
+      throw logic_error("JSD1 can only peek-decrypt up to 0x100 bytes");
+    }
+    uint8_t* bytes = reinterpret_cast<uint8_t*>(vdata);
+    for (size_t z = 0; z < size; z += 2) {
+      uint8_t a = bytes[z];
+      uint8_t b = bytes[z + 1];
+      bytes[z] = (a & 0x55) | (b & 0xAA);
+      bytes[z + 1] = (a & 0xAA) | (b & 0x55);
+    }
+    uint8_t* stream_bytes = reinterpret_cast<uint8_t*>(this->stream.data());
+    for (size_t z = 0; z < size; z++) {
+      bytes[z] ^= stream_bytes[this->jsd1_stream_offset];
+      if (advance) {
+        stream_bytes[this->jsd1_stream_offset] -= bytes[z];
+      }
+      this->jsd1_stream_offset++;
+    }
+    if (!advance) {
+      this->jsd1_stream_offset -= size;
+    }
   }
 }
 
-void PSOBBEncryption::encrypt(void* vdata, size_t size) {
-  if (size & 7) {
-    throw invalid_argument("size must be a multiple of 8");
-  }
-  size >>= 3;
+void PSOBBEncryption::encrypt(void* vdata, size_t size, bool advance) {
+  if (this->subtype != Subtype::JSD1) {
+    if (size & 7) {
+      throw invalid_argument("size must be a multiple of 8");
+    }
 
-  uint8_t* data = reinterpret_cast<uint8_t*>(vdata);
-  uint32_t edx, ebx, ebp, esi, edi;
+    size_t num_dwords = size >> 2;
+    le_uint32_t* data = reinterpret_cast<le_uint32_t*>(vdata);
+    uint32_t edx, ebx, ebp, esi, edi;
 
-  edx = 0;
-  while (edx < size) {
-    ebx = data[edx];
-    ebx = ebx ^ this->stream[0];
-    ebp = ((this->stream[(ebx >> 0x18) + 0x12]+this->stream[((ebx >> 0x10)& 0xff) + 0x112])
-      ^ this->stream[((ebx >> 0x8)& 0xff) + 0x212]) + this->stream[(ebx & 0xff) + 0x312];
-    ebp = ebp ^ this->stream[1];
-    ebp ^= data[edx + 1];
-    edi = ((this->stream[(ebp >> 0x18) + 0x12]+this->stream[((ebp >> 0x10)& 0xff) + 0x112])
-      ^ this->stream[((ebp >> 0x8)& 0xff) + 0x212]) + this->stream[(ebp & 0xff) + 0x312];
-    edi = edi ^ this->stream[2];
-    ebx = ebx ^ edi;
-    esi = ((this->stream[(ebx >> 0x18) + 0x12]+this->stream[((ebx >> 0x10)& 0xff) + 0x112])
-      ^ this->stream[((ebx >> 0x8)& 0xff) + 0x212]) + this->stream[(ebx & 0xff) + 0x312];
-    ebp = ebp ^ esi ^ this->stream[3];
-    edi = ((this->stream[(ebp >> 0x18) + 0x12]+this->stream[((ebp >> 0x10)& 0xff) + 0x112])
-      ^ this->stream[((ebp >> 0x8)& 0xff) + 0x212]) + this->stream[(ebp & 0xff) + 0x312];
-    edi = edi ^ this->stream[4];
-    ebp = ebp ^ this->stream[5];
-    ebx = ebx ^ edi;
-    data[edx] = ebp;
-    data[edx + 1] = ebx;
-    edx += 2;
+    edx = 0;
+    while (edx < num_dwords) {
+      ebx = data[edx] ^ this->stream[0];
+      ebp = ((this->stream[(ebx >> 0x18) + 0x12] + this->stream[((ebx >> 0x10) & 0xFF) + 0x112])
+          ^ this->stream[((ebx >> 0x8) & 0xFF) + 0x212]) + this->stream[(ebx & 0xFF) + 0x312];
+      ebp = ebp ^ this->stream[1];
+      ebp ^= data[edx + 1];
+      edi = ((this->stream[(ebp >> 0x18) + 0x12] + this->stream[((ebp >> 0x10) & 0xFF) + 0x112])
+          ^ this->stream[((ebp >> 0x8) & 0xFF) + 0x212]) + this->stream[(ebp & 0xFF) + 0x312];
+      edi = edi ^ this->stream[2];
+      ebx = ebx ^ edi;
+      esi = ((this->stream[(ebx >> 0x18) + 0x12] + this->stream[((ebx >> 0x10) & 0xFF) + 0x112])
+          ^ this->stream[((ebx >> 0x8) & 0xFF) + 0x212]) + this->stream[(ebx & 0xFF) + 0x312];
+      ebp = ebp ^ esi ^ this->stream[3];
+      edi = ((this->stream[(ebp >> 0x18) + 0x12] + this->stream[((ebp >> 0x10) & 0xFF) + 0x112])
+          ^ this->stream[((ebp >> 0x8) & 0xFF) + 0x212]) + this->stream[(ebp & 0xFF) + 0x312];
+      edi = edi ^ this->stream[4];
+      ebp = ebp ^ this->stream[5];
+      ebx = ebx ^ edi;
+      data[edx] = ebp;
+      data[edx + 1] = ebx;
+      edx += 2;
+    }
+
+  } else { // subtype == Subtype::JSD1
+    if (!advance && (size > 0x100)) {
+      throw logic_error("JSD1 can only peek-encrypt up to 0x100 bytes");
+    }
+    uint8_t* bytes = reinterpret_cast<uint8_t*>(vdata);
+    uint8_t* stream_bytes = reinterpret_cast<uint8_t*>(this->stream.data());
+    for (size_t z = 0; z < size; z++) {
+      uint8_t v = bytes[z];
+      bytes[z] = v ^ stream_bytes[this->jsd1_stream_offset];
+      if (advance) {
+        stream_bytes[this->jsd1_stream_offset] -= v;
+      }
+      this->jsd1_stream_offset++;
+    }
+    if (!advance) {
+      this->jsd1_stream_offset -= size;
+    }
+    for (size_t z = 0; z < size; z += 2) {
+      uint8_t a = bytes[z];
+      uint8_t b = bytes[z + 1];
+      bytes[z] = (a & 0x55) | (b & 0xAA);
+      bytes[z + 1] = (a & 0xAA) | (b & 0x55);
+    }
   }
 }
 
-PSOBBEncryption::PSOBBEncryption(const KeyFile& key, const void* original_seed,
-    size_t seed_size) : offset(0) {
-  if (seed_size % 3) {
-    throw invalid_argument("seed size must be divisible by 3");
-  }
+PSOBBEncryption::PSOBBEncryption(
+    const KeyFile& key, const void* original_seed, size_t seed_size)
+  : subtype(key.subtype), jsd1_stream_offset(0) {
 
+  // Note: This part is done in the 03 command handler in the BB client, and
+  // isn't actually part of the encryption library. (Why did they do this?)
   string seed;
   const uint8_t* original_seed_data = reinterpret_cast<const uint8_t*>(
       original_seed);
@@ -250,157 +317,75 @@ PSOBBEncryption::PSOBBEncryption(const KeyFile& key, const void* original_seed,
     seed.push_back(original_seed_data[x + 2] ^ 0x18);
   }
 
-  memcpy(this->stream, &key, sizeof(key));
-
-  this->postprocess_initial_stream(seed);
-}
-
-void PSOBBEncryption::postprocess_initial_stream(const string& seed) {
-  uint32_t eax, ecx, edx, ebx, ebp, esi, edi, ou, x;
-
-  ecx = 0;
-  ebx = 0;
-
-  while (ebx < (seed.size() / 4)) {
-    ebp = static_cast<uint32_t>(seed[ecx]) << 0x18;
-    eax = ecx + 1;
-    edx = eax % seed.size();
-    eax = (static_cast<uint32_t>(seed[edx]) << 0x10) & 0xFF0000;
-    ebp = (ebp | eax) & 0xffff00ff;
-    eax = ecx + 2;
-    edx = eax % seed.size();
-    eax = (static_cast<uint32_t>(seed[edx]) << 0x08) & 0xFF00;
-    ebp = (ebp | eax) & 0xffffff00;
-    eax = ecx + 3;
-    ecx = ecx + 4;
-    edx = eax % seed.size();
-    eax = static_cast<uint32_t>(seed[edx]);
-    ebp = ebp | eax;
-    eax = ecx;
-    edx = eax % seed.size();
-    this->stream[ebx] ^= ebp;
-    ecx = edx;
-    ebx++;
-  }
-
-  ebp = 0;
-  esi = 0;
-  ecx = 0;
-  edi = 0;
-  ebx = 0;
-  edx = 0x48;
-
-  while (edi < edx) {
-    esi = esi ^ this->stream[0];
-    eax = esi >> 0x18;
-    ebx = (esi >> 0x10) & 0xff;
-    eax = this->stream[eax + 0x12] + this->stream[ebx + 0x112];
-    ebx = (esi >> 8) & 0xFF;
-    eax = eax ^ this->stream[ebx + 0x212];
-    ebx = esi & 0xFF;
-    eax = eax + this->stream[ebx + 0x312];
-
-    eax = eax ^ this->stream[1];
-    ecx = ecx ^ eax;
-    ebx = ecx >> 0x18;
-    eax = (ecx >> 0x10) & 0xFF;
-    ebx = this->stream[ebx + 0x12] + this->stream[eax + 0x112];
-    eax = (ecx >> 8) & 0xFF;
-    ebx = ebx ^ this->stream[eax + 0x212];
-    eax = ecx & 0xFF;
-    ebx = ebx + this->stream[eax + 0x312];
-
-    for (x = 0; x <= 5; x++) {
-      ebx = ebx ^ this->stream[(x * 2) + 2];
-      esi = esi ^ ebx;
-      ebx = esi >> 0x18;
-      eax = (esi >> 0x10) & 0xFF;
-      ebx = this->stream[ebx + 0x12] + this->stream[eax + 0x112];
-      eax = (esi >> 8) & 0xFF;
-      ebx = ebx ^ this->stream[eax + 0x212];
-      eax = esi & 0xFF;
-      ebx = ebx + this->stream[eax + 0x312];
-
-      ebx = ebx ^ this->stream[(x * 2) + 3];
-      ecx = ecx ^ ebx;
-      ebx = ecx >> 0x18;
-      eax = (ecx >> 0x10) & 0xFF;
-      ebx = this->stream[ebx + 0x12] + this->stream[eax + 0x112];
-      eax = (ecx >> 8) & 0xFF;
-      ebx = ebx ^ this->stream[eax + 0x212];
-      eax = ecx & 0xff;
-      ebx = ebx + this->stream[eax + 0x312];
+  if (this->subtype != Subtype::JSD1) {
+    if (seed_size % 3) {
+      throw invalid_argument("seed size must be divisible by 3");
     }
 
-    ebx = ebx ^ this->stream[14];
-    esi = esi ^ ebx;
-    eax = esi >> 0x18;
-    ebx = (esi >> 0x10) & 0xFF;
-    eax = this->stream[eax + 0x12] + this->stream[ebx + 0x112];
-    ebx = (esi >> 8) & 0xff;
-    eax = eax ^ this->stream[ebx + 0x212];
-    ebx = esi & 0xff;
-    eax = eax + this->stream[ebx + 0x312];
+    this->stream.resize(BB_STREAM_LENGTH, 0);
 
-    eax = eax ^ this->stream[15];
-    eax = ecx ^ eax;
-    ecx = eax >> 0x18;
-    ebx = (eax >> 0x10) & 0xFF;
-    ecx = this->stream[ecx + 0x12] + this->stream[ebx + 0x112];
-    ebx = (eax >> 8) & 0xFF;
-    ecx = ecx ^ this->stream[ebx + 0x212];
-    ebx = eax & 0xFF;
-    ecx = ecx + this->stream[ebx + 0x312];
+    if (key.subtype == Subtype::MOCB1) {
+      for (size_t x = 0; x < 0x12; x++) {
+        uint8_t a = key.initial_keys[4 * x + 0];
+        uint8_t b = key.initial_keys[4 * x + 1];
+        uint8_t c = key.initial_keys[4 * x + 2];
+        uint8_t d = key.initial_keys[4 * x + 3];
+        this->stream[x] = ((a ^ d) << 24) | ((b ^ c) << 16) | (a << 8) | b;
+      }
+      memcpy(this->stream.data() + 0x12, &key.private_keys, sizeof(key.private_keys));
 
-    ecx = ecx ^ this->stream[16];
-    ecx = ecx ^ esi;
-    esi = this->stream[17];
-    esi = esi ^ eax;
-    this->stream[(edi / 4)] = esi;
-    this->stream[(edi / 4)+1] = ecx;
-    edi = edi + 8;
-  }
+    } else {
+      memcpy(this->stream.data(), &key, sizeof(key));
+    }
 
-  eax = 0;
-  edx = 0;
-  ou = 0;
-  while (ou < 0x1000) {
-    edi = 0x48;
-    edx = 0x448;
+    // This block was formerly postprocess_initial_stream
+    {
+      uint32_t eax, ecx, edx, ebx, ebp, esi, edi, ou, x;
 
-    while (edi < edx) {
-      esi = esi ^ this->stream[0];
-      eax = esi >> 0x18;
-      ebx = (esi >> 0x10) & 0xff;
-      eax = this->stream[eax + 0x12] + this->stream[ebx + 0x112];
-      ebx = (esi >> 8) & 0xFF;
-      eax = eax ^ this->stream[ebx + 0x212];
-      ebx = esi & 0xFF;
-      eax = eax + this->stream[ebx + 0x312];
+      ecx = 0;
+      ebx = 0;
 
-      eax = eax ^ this->stream[1];
-      ecx = ecx ^ eax;
-      ebx = ecx >> 0x18;
-      eax = (ecx >> 0x10) & 0xFF;
-      ebx = this->stream[ebx + 0x12] + this->stream[eax + 0x112];
-      eax = (ecx >> 8) & 0xFF;
-      ebx = ebx ^ this->stream[eax + 0x212];
-      eax = ecx & 0xFF;
-      ebx = ebx + this->stream[eax + 0x312];
+      while (ebx < 0x12) {
+        ebp = static_cast<uint32_t>(seed[ecx]) << 0x18;
+        eax = ecx + 1;
+        edx = eax % seed.size();
+        eax = (static_cast<uint32_t>(seed[edx]) << 0x10) & 0x00FF0000;
+        ebp = (ebp | eax) & 0xFFFF00FF;
+        eax = ecx + 2;
+        edx = eax % seed.size();
+        eax = (static_cast<uint32_t>(seed[edx]) << 0x08) & 0x0000FF00;
+        ebp = (ebp | eax) & 0xFFFFFF00;
+        eax = ecx + 3;
+        ecx = ecx + 4;
+        edx = eax % seed.size();
+        eax = static_cast<uint32_t>(seed[edx]) & 0x000000FF;
+        ebp = ebp | eax;
+        eax = ecx;
+        edx = eax % seed.size();
+        this->stream[ebx] ^= ebp;
+        ecx = edx;
+        ebx++;
+      }
 
-      for (x = 0; x <= 5; x++) {
-        ebx = ebx ^ this->stream[(x * 2) + 2];
-        esi = esi ^ ebx;
-        ebx = esi >> 0x18;
-        eax = (esi >> 0x10) & 0xFF;
-        ebx = this->stream[ebx + 0x12] + this->stream[eax + 0x112];
-        eax = (esi >> 8) & 0xFF;
-        ebx = ebx ^ this->stream[eax + 0x212];
-        eax = esi & 0xFF;
-        ebx = ebx + this->stream[eax + 0x312];
+      ebp = 0;
+      esi = 0;
+      ecx = 0;
+      edi = 0;
+      ebx = 0;
+      edx = 0x48;
 
-        ebx = ebx ^ this->stream[(x * 2) + 3];
-        ecx = ecx ^ ebx;
+      while (edi < edx) {
+        esi = esi ^ this->stream[0];
+        eax = esi >> 0x18;
+        ebx = (esi >> 0x10) & 0xFF;
+        eax = this->stream[eax + 0x12] + this->stream[ebx + 0x112];
+        ebx = (esi >> 8) & 0xFF;
+        eax = eax ^ this->stream[ebx + 0x212];
+        ebx = esi & 0xFF;
+        eax = eax + this->stream[ebx + 0x312];
+
+        eax = eax ^ this->stream[1];
+        ecx = ecx ^ eax;
         ebx = ecx >> 0x18;
         eax = (ecx >> 0x10) & 0xFF;
         ebx = this->stream[ebx + 0x12] + this->stream[eax + 0x112];
@@ -408,36 +393,229 @@ void PSOBBEncryption::postprocess_initial_stream(const string& seed) {
         ebx = ebx ^ this->stream[eax + 0x212];
         eax = ecx & 0xFF;
         ebx = ebx + this->stream[eax + 0x312];
+
+        for (x = 0; x <= 5; x++) {
+          ebx = ebx ^ this->stream[(x * 2) + 2];
+          esi = esi ^ ebx;
+          ebx = esi >> 0x18;
+          eax = (esi >> 0x10) & 0xFF;
+          ebx = this->stream[ebx + 0x12] + this->stream[eax + 0x112];
+          eax = (esi >> 8) & 0xFF;
+          ebx = ebx ^ this->stream[eax + 0x212];
+          eax = esi & 0xFF;
+          ebx = ebx + this->stream[eax + 0x312];
+
+          ebx = ebx ^ this->stream[(x * 2) + 3];
+          ecx = ecx ^ ebx;
+          ebx = ecx >> 0x18;
+          eax = (ecx >> 0x10) & 0xFF;
+          ebx = this->stream[ebx + 0x12] + this->stream[eax + 0x112];
+          eax = (ecx >> 8) & 0xFF;
+          ebx = ebx ^ this->stream[eax + 0x212];
+          eax = ecx & 0xFF;
+          ebx = ebx + this->stream[eax + 0x312];
+        }
+
+        ebx = ebx ^ this->stream[14];
+        esi = esi ^ ebx;
+        eax = esi >> 0x18;
+        ebx = (esi >> 0x10) & 0xFF;
+        eax = this->stream[eax + 0x12] + this->stream[ebx + 0x112];
+        ebx = (esi >> 8) & 0xFF;
+        eax = eax ^ this->stream[ebx + 0x212];
+        ebx = esi & 0xFF;
+        eax = eax + this->stream[ebx + 0x312];
+
+        eax = eax ^ this->stream[15];
+        eax = ecx ^ eax;
+        ecx = eax >> 0x18;
+        ebx = (eax >> 0x10) & 0xFF;
+        ecx = this->stream[ecx + 0x12] + this->stream[ebx + 0x112];
+        ebx = (eax >> 8) & 0xFF;
+        ecx = ecx ^ this->stream[ebx + 0x212];
+        ebx = eax & 0xFF;
+        ecx = ecx + this->stream[ebx + 0x312];
+
+        ecx = ecx ^ this->stream[16];
+        ecx = ecx ^ esi;
+        esi = this->stream[17];
+        esi = esi ^ eax;
+        this->stream[(edi / 4)] = esi;
+        this->stream[(edi / 4)+1] = ecx;
+        edi = edi + 8;
       }
 
-      ebx = ebx ^ this->stream[14];
-      esi = esi ^ ebx;
-      eax = esi >> 0x18;
-      ebx = (esi >> 0x10) & 0xFF;
-      eax = this->stream[eax + 0x12] + this->stream[ebx + 0x112];
-      ebx = (esi >> 8) & 0xFF;
-      eax = eax ^ this->stream[ebx + 0x212];
-      ebx = esi & 0xFF;
-      eax = eax + this->stream[ebx + 0x312];
+      eax = 0;
+      edx = 0;
+      ou = 0;
+      while (ou < 0x1000) {
+        edi = 0x48;
+        edx = 0x448;
 
-      eax = eax ^ this->stream[15];
-      eax = ecx ^ eax;
-      ecx = eax >> 0x18;
-      ebx = (eax >> 0x10) & 0xFF;
-      ecx = this->stream[ecx + 0x12] + this->stream[ebx + 0x112];
-      ebx = (eax >> 8) & 0xFF;
-      ecx = ecx ^ this->stream[ebx + 0x212];
-      ebx = eax & 0xFF;
-      ecx = ecx + this->stream[ebx + 0x312];
+        while (edi < edx) {
+          esi = esi ^ this->stream[0];
+          eax = esi >> 0x18;
+          ebx = (esi >> 0x10) & 0xFF;
+          eax = this->stream[eax + 0x12] + this->stream[ebx + 0x112];
+          ebx = (esi >> 8) & 0xFF;
+          eax = eax ^ this->stream[ebx + 0x212];
+          ebx = esi & 0xFF;
+          eax = eax + this->stream[ebx + 0x312];
 
-      ecx = ecx ^ this->stream[16];
-      ecx = ecx ^ esi;
-      esi = this->stream[17];
-      esi = esi ^ eax;
-      this->stream[(ou / 4) + (edi / 4)] = esi;
-      this->stream[(ou / 4) + (edi / 4) + 1] = ecx;
-      edi = edi + 8;
+          eax = eax ^ this->stream[1];
+          ecx = ecx ^ eax;
+          ebx = ecx >> 0x18;
+          eax = (ecx >> 0x10) & 0xFF;
+          ebx = this->stream[ebx + 0x12] + this->stream[eax + 0x112];
+          eax = (ecx >> 8) & 0xFF;
+          ebx = ebx ^ this->stream[eax + 0x212];
+          eax = ecx & 0xFF;
+          ebx = ebx + this->stream[eax + 0x312];
+
+          for (x = 0; x <= 5; x++) {
+            ebx = ebx ^ this->stream[(x * 2) + 2];
+            esi = esi ^ ebx;
+            ebx = esi >> 0x18;
+            eax = (esi >> 0x10) & 0xFF;
+            ebx = this->stream[ebx + 0x12] + this->stream[eax + 0x112];
+            eax = (esi >> 8) & 0xFF;
+            ebx = ebx ^ this->stream[eax + 0x212];
+            eax = esi & 0xFF;
+            ebx = ebx + this->stream[eax + 0x312];
+
+            ebx = ebx ^ this->stream[(x * 2) + 3];
+            ecx = ecx ^ ebx;
+            ebx = ecx >> 0x18;
+            eax = (ecx >> 0x10) & 0xFF;
+            ebx = this->stream[ebx + 0x12] + this->stream[eax + 0x112];
+            eax = (ecx >> 8) & 0xFF;
+            ebx = ebx ^ this->stream[eax + 0x212];
+            eax = ecx & 0xFF;
+            ebx = ebx + this->stream[eax + 0x312];
+          }
+
+          ebx = ebx ^ this->stream[14];
+          esi = esi ^ ebx;
+          eax = esi >> 0x18;
+          ebx = (esi >> 0x10) & 0xFF;
+          eax = this->stream[eax + 0x12] + this->stream[ebx + 0x112];
+          ebx = (esi >> 8) & 0xFF;
+          eax = eax ^ this->stream[ebx + 0x212];
+          ebx = esi & 0xFF;
+          eax = eax + this->stream[ebx + 0x312];
+
+          eax = eax ^ this->stream[15];
+          eax = ecx ^ eax;
+          ecx = eax >> 0x18;
+          ebx = (eax >> 0x10) & 0xFF;
+          ecx = this->stream[ecx + 0x12] + this->stream[ebx + 0x112];
+          ebx = (eax >> 8) & 0xFF;
+          ecx = ecx ^ this->stream[ebx + 0x212];
+          ebx = eax & 0xFF;
+          ecx = ecx + this->stream[ebx + 0x312];
+
+          ecx = ecx ^ this->stream[16];
+          ecx = ecx ^ esi;
+          esi = this->stream[17];
+          esi = esi ^ eax;
+          this->stream[(ou / 4) + (edi / 4)] = esi;
+          this->stream[(ou / 4) + (edi / 4) + 1] = ecx;
+          edi = edi + 8;
+        }
+        ou = ou + 0x400;
+      }
+    }
+
+  } else { // subtype == Subtype::JSD1
+    this->stream.resize(0x40);
+    uint8_t* stream_bytes = reinterpret_cast<uint8_t*>(this->stream.data());
+    size_t seed_offset = 0;
+    for (size_t z = 0; z < 0x100; z++) {
+      stream_bytes[z] = (z + seed[seed_offset]) ^ (static_cast<uint8_t>(seed[seed_offset]) >> 1);
+      seed_offset = (seed_offset + 1) % seed.size();
     }
-    ou = ou + 0x400;
   }
 }
+
+
+
+PSOBBMultiKeyDetectorEncryption::PSOBBMultiKeyDetectorEncryption(
+    const vector<shared_ptr<const PSOBBEncryption::KeyFile>>& possible_keys,
+    const string& expected_first_data,
+    const void* seed,
+    size_t seed_size)
+  : possible_keys(possible_keys),
+    expected_first_data(expected_first_data),
+    seed(reinterpret_cast<const char*>(seed), seed_size) { }
+
+void PSOBBMultiKeyDetectorEncryption::encrypt(void* data, size_t size, bool advance) {
+  if (!this->active_crypt.get()) {
+    throw logic_error("PSOBB multi-key encryption requires client input first");
+  }
+  this->active_crypt->encrypt(data, size, advance);
+}
+
+void PSOBBMultiKeyDetectorEncryption::decrypt(void* data, size_t size, bool advance) {
+  if (!this->active_crypt.get()) {
+    if (size != this->expected_first_data.size()) {
+      throw logic_error("initial decryption size does not match expected first data size");
+    }
+
+    for (const auto& key : this->possible_keys) {
+      this->active_key = key;
+      this->active_crypt.reset(new PSOBBEncryption(
+          *this->active_key, this->seed.data(), this->seed.size()));
+      string test_data(reinterpret_cast<const char*>(data), size);
+      this->active_crypt->decrypt(test_data.data(), test_data.size(), false);
+      if (test_data == this->expected_first_data) {
+        break;
+      }
+      this->active_key.reset();
+      this->active_crypt.reset();
+    }
+    if (!this->active_crypt.get()) {
+      throw runtime_error("none of the registered private keys are valid for this client");
+    }
+  }
+  this->active_crypt->decrypt(data, size, advance);
+}
+
+
+
+PSOBBMultiKeyImitatorEncryption::PSOBBMultiKeyImitatorEncryption(
+    shared_ptr<const PSOBBMultiKeyDetectorEncryption> detector_crypt,
+    const void* seed,
+    size_t seed_size,
+    bool jsd1_use_detector_seed)
+  : detector_crypt(detector_crypt),
+    seed(reinterpret_cast<const char*>(seed), seed_size),
+    jsd1_use_detector_seed(jsd1_use_detector_seed) { }
+
+void PSOBBMultiKeyImitatorEncryption::encrypt(void* data, size_t size, bool advance) {
+  this->ensure_crypt()->encrypt(data, size, advance);
+}
+
+void PSOBBMultiKeyImitatorEncryption::decrypt(void* data, size_t size, bool advance) {
+  this->ensure_crypt()->decrypt(data, size, advance);
+}
+
+shared_ptr<PSOBBEncryption> PSOBBMultiKeyImitatorEncryption::ensure_crypt() {
+  if (!this->active_crypt.get()) {
+    auto key = this->detector_crypt->get_active_key();
+    if (!key.get()) {
+      throw logic_error("server crypt cannot be initialized because client crypt is not ready");
+    }
+    // Hack: JSD1 uses the client seed for both ends of the connection and
+    // ignores the server seed (though each end has its own state after that).
+    // To handle this, we use the other crypt's seed if the type is JSD1.
+    if ((key->subtype == PSOBBEncryption::Subtype::JSD1) && this->jsd1_use_detector_seed) {
+      const auto& detector_seed = this->detector_crypt->get_seed();
+      this->active_crypt.reset(new PSOBBEncryption(
+          *key, detector_seed.data(), detector_seed.size()));
+    } else {
+      this->active_crypt.reset(new PSOBBEncryption(
+          *key, this->seed.data(), this->seed.size()));
+    }
+  }
+  return this->active_crypt;
+}

src/PSOEncryption.hh

@@ -3,11 +3,13 @@
 #include <inttypes.h>
 #include <stddef.h>
 
+#include <memory>
 #include <string>
+#include <vector>
 
 
 
-#define PC_STREAM_LENGTH 57
+#define PC_STREAM_LENGTH 56
 #define GC_STREAM_LENGTH 521
 #define BB_STREAM_LENGTH 1042
 
@@ -15,8 +17,15 @@ class PSOEncryption {
 public:
   virtual ~PSOEncryption() = default;
 
-  virtual void encrypt(void* data, size_t size) = 0;
-  virtual void decrypt(void* data, size_t size);
+  virtual void encrypt(void* data, size_t size, bool advance = true) = 0;
+  virtual void decrypt(void* data, size_t size, bool advance = true);
+
+  inline void encrypt(std::string& data, bool advance = true) {
+    this->encrypt(data.data(), data.size(), advance);
+  }
+  inline void decrypt(std::string& data, bool advance = true) {
+    this->decrypt(data.data(), data.size(), advance);
+  }
 
 protected:
   PSOEncryption() = default;
@@ -26,25 +35,25 @@ class PSOPCEncryption : public PSOEncryption {
 public:
   explicit PSOPCEncryption(uint32_t seed);
 
-  virtual void encrypt(void* data, size_t size);
+  virtual void encrypt(void* data, size_t size, bool advance = true);
 
 protected:
   void update_stream();
-  uint32_t next();
+  uint32_t next(bool advance = true);
 
-  uint32_t stream[PC_STREAM_LENGTH];
-  uint16_t offset;
+  uint32_t stream[PC_STREAM_LENGTH + 1];
+  uint8_t offset;
 };
 
 class PSOGCEncryption : public PSOEncryption {
 public:
   explicit PSOGCEncryption(uint32_t key);
 
-  virtual void encrypt(void* data, size_t size);
+  virtual void encrypt(void* data, size_t size, bool advance = true);
 
 protected:
   void update_stream();
-  uint32_t next();
+  uint32_t next(bool advance = true);
 
   uint32_t stream[GC_STREAM_LENGTH];
   uint16_t offset;
@@ -52,23 +61,77 @@ protected:
 
 class PSOBBEncryption : public PSOEncryption {
 public:
-  struct KeyFile {
-    uint32_t initial_keys[18];
-    uint32_t private_keys[1024];
+  enum Subtype : uint8_t {
+    STANDARD = 0x00,
+    MOCB1 = 0x01,
+    JSD1 = 0x02,
   };
 
+  struct KeyFile {
+    // initial_keys are actually a stream of uint32_ts, but we treat them as
+    // bytes for code simplicity
+    uint8_t initial_keys[0x12 * 4];
+    uint32_t private_keys[0x400];
+    Subtype subtype;
+  } __attribute__((packed));
+
   PSOBBEncryption(const KeyFile& key, const void* seed, size_t seed_size);
 
-  virtual void encrypt(void* data, size_t size);
-  virtual void decrypt(void* data, size_t size);
+  virtual void encrypt(void* data, size_t size, bool advance = true);
+  virtual void decrypt(void* data, size_t size, bool advance = true);
 
 protected:
-  PSOBBEncryption() = default;
-
-  void postprocess_initial_stream(const std::string& seed);
-
-  void update_stream();
-
-  uint32_t stream[BB_STREAM_LENGTH];
-  uint16_t offset;
+  Subtype subtype;
+  std::vector<uint32_t> stream;
+  uint8_t jsd1_stream_offset;
+};
+
+// The following classes provide support for multiple PSOBB private keys, and
+// the ability to automatically detect which key the client is using based on
+// the first 8 bytes they send.
+
+class PSOBBMultiKeyDetectorEncryption : public PSOEncryption {
+public:
+  PSOBBMultiKeyDetectorEncryption(
+      const std::vector<std::shared_ptr<const PSOBBEncryption::KeyFile>>& possible_keys,
+      const std::string& expected_first_data,
+      const void* seed,
+      size_t seed_size);
+
+  virtual void encrypt(void* data, size_t size, bool advance = true);
+  virtual void decrypt(void* data, size_t size, bool advance = true);
+
+  inline std::shared_ptr<const PSOBBEncryption::KeyFile> get_active_key() const {
+    return this->active_key;
+  }
+  inline const std::string& get_seed() const {
+    return this->seed;
+  }
+
+protected:
+  std::vector<std::shared_ptr<const PSOBBEncryption::KeyFile>> possible_keys;
+  std::shared_ptr<const PSOBBEncryption::KeyFile> active_key;
+  std::shared_ptr<PSOBBEncryption> active_crypt;
+  std::string expected_first_data;
+  std::string seed;
+};
+
+class PSOBBMultiKeyImitatorEncryption : public PSOEncryption {
+public:
+  PSOBBMultiKeyImitatorEncryption(
+      std::shared_ptr<const PSOBBMultiKeyDetectorEncryption> client_crypt,
+      const void* seed,
+      size_t seed_size,
+      bool jsd1_use_detector_seed);
+
+  virtual void encrypt(void* data, size_t size, bool advance = true);
+  virtual void decrypt(void* data, size_t size, bool advance = true);
+
+protected:
+  std::shared_ptr<PSOBBEncryption> ensure_crypt();
+
+  std::shared_ptr<const PSOBBMultiKeyDetectorEncryption> detector_crypt;
+  std::shared_ptr<PSOBBEncryption> active_crypt;
+  std::string seed;
+  bool jsd1_use_detector_seed;
 };

src/PSOProtocol.cc

@@ -1,50 +1,244 @@
 #include "PSOProtocol.hh"
 
+#include <event2/buffer.h>
+
 #include <stdexcept>
+#include <phosg/Strings.hh>
+
+#include "Text.hh"
 
 using namespace std;
 
 
 
+extern bool use_terminal_colors;
+
+
+
+PSOCommandHeader::PSOCommandHeader() {
+  this->bb.size = 0;
+  this->bb.command = 0;
+  this->bb.flag = 0;
+}
+
 uint16_t PSOCommandHeader::command(GameVersion version) const {
   switch (version) {
     case GameVersion::DC:
+      return this->dc.command;
     case GameVersion::GC:
-      return reinterpret_cast<const PSOCommandHeaderDCGC*>(this)->command;
+      return this->gc.command;
     case GameVersion::PC:
-    case GameVersion::Patch:
-      return reinterpret_cast<const PSOCommandHeaderPC*>(this)->command;
+    case GameVersion::PATCH:
+      return this->pc.command;
     case GameVersion::BB:
-      return reinterpret_cast<const PSOCommandHeaderBB*>(this)->command;
+      return this->bb.command;
+    default:
+      throw logic_error("unknown game version");
+  }
+}
+
+void PSOCommandHeader::set_command(GameVersion version, uint16_t command) {
+  switch (version) {
+    case GameVersion::DC:
+      this->dc.command = command;
+      break;
+    case GameVersion::GC:
+      this->gc.command = command;
+      break;
+    case GameVersion::PC:
+    case GameVersion::PATCH:
+      this->pc.command = command;
+      break;
+    case GameVersion::BB:
+      this->bb.command = command;
+      break;
+    default:
+      throw logic_error("unknown game version");
   }
-  throw logic_error("unknown game version");
 }
 
 uint16_t PSOCommandHeader::size(GameVersion version) const {
   switch (version) {
     case GameVersion::DC:
+      return this->dc.size;
     case GameVersion::GC:
-      return reinterpret_cast<const PSOCommandHeaderDCGC*>(this)->size;
+      return this->gc.size;
     case GameVersion::PC:
-    case GameVersion::Patch:
-      return reinterpret_cast<const PSOCommandHeaderPC*>(this)->size;
+    case GameVersion::PATCH:
+      return this->pc.size;
     case GameVersion::BB:
-      return reinterpret_cast<const PSOCommandHeaderBB*>(this)->size;
+      return this->bb.size;
+    default:
+      throw logic_error("unknown game version");
+  }
+}
+
+void PSOCommandHeader::set_size(GameVersion version, uint32_t size) {
+  switch (version) {
+    case GameVersion::DC:
+      this->dc.size = size;
+      break;
+    case GameVersion::GC:
+      this->gc.size = size;
+      break;
+    case GameVersion::PC:
+    case GameVersion::PATCH:
+      this->pc.size = size;
+      break;
+    case GameVersion::BB:
+      this->bb.size = size;
+      break;
+    default:
+      throw logic_error("unknown game version");
   }
-  throw logic_error("unknown game version");
 }
 
 uint32_t PSOCommandHeader::flag(GameVersion version) const {
   switch (version) {
     case GameVersion::DC:
+      return this->dc.flag;
     case GameVersion::GC:
-      return reinterpret_cast<const PSOCommandHeaderDCGC*>(this)->flag;
+      return this->gc.flag;
     case GameVersion::PC:
-    case GameVersion::Patch:
-      return reinterpret_cast<const PSOCommandHeaderPC*>(this)->flag;
+    case GameVersion::PATCH:
+      return this->pc.flag;
     case GameVersion::BB:
-      return reinterpret_cast<const PSOCommandHeaderBB*>(this)->flag;
+      return this->bb.flag;
+    default:
+      throw logic_error("unknown game version");
   }
-  throw logic_error("unknown game version");
 }
 
+void PSOCommandHeader::set_flag(GameVersion version, uint32_t flag) {
+  switch (version) {
+    case GameVersion::DC:
+      this->dc.flag = flag;
+      break;
+    case GameVersion::GC:
+      this->gc.flag = flag;
+      break;
+    case GameVersion::PC:
+    case GameVersion::PATCH:
+      this->pc.flag = flag;
+      break;
+    case GameVersion::BB:
+      this->bb.flag = flag;
+      break;
+    default:
+      throw logic_error("unknown game version");
+  }
+}
+
+
+
+void for_each_received_command(
+    struct bufferevent* bev,
+    GameVersion version,
+    PSOEncryption* crypt,
+    function<void(uint16_t, uint16_t, string&)> fn) {
+  struct evbuffer* buf = bufferevent_get_input(bev);
+
+  size_t header_size = (version == GameVersion::BB) ? 8 : 4;
+  for (;;) {
+    PSOCommandHeader header;
+    if (evbuffer_copyout(buf, &header, header_size)
+        < static_cast<ssize_t>(header_size)) {
+      break;
+    }
+
+    if (crypt) {
+      crypt->decrypt(&header, header_size, false);
+    }
+
+    size_t command_logical_size = header.size(version);
+
+    // If encryption is enabled, BB pads commands to 8-byte boundaries, and this
+    // is not reflected in the size field. This logic does not occur if
+    // encryption is not yet enabled.
+    size_t command_physical_size = (crypt && (version == GameVersion::BB))
+        ? ((command_logical_size + header_size - 1) & ~(header_size - 1))
+        : command_logical_size;
+    if (evbuffer_get_length(buf) < command_physical_size) {
+      break;
+    }
+
+    // If we get here, then there is a full command in the buffer. Some
+    // encryption algorithms' advancement depends on the decrypted data, so we
+    // have to actually decrypt the header again (with advance=true) to keep
+    // them in a consistent state.
+
+    string header_data(header_size, '\0');
+    if (evbuffer_remove(buf, header_data.data(), header_data.size())
+        < static_cast<ssize_t>(header_data.size())) {
+      throw logic_error("enough bytes available, but could not remove them");
+    }
+
+    string command_data(command_physical_size - header_size, '\0');
+    if (evbuffer_remove(buf, command_data.data(), command_data.size())
+        < static_cast<ssize_t>(command_data.size())) {
+      throw logic_error("enough bytes available, but could not remove them");
+    }
+
+    if (crypt) {
+      crypt->decrypt(header_data.data(), header_data.size());
+      crypt->decrypt(command_data.data(), command_data.size());
+    }
+    command_data.resize(command_logical_size - header_size);
+
+    fn(header.command(version), header.flag(version), command_data);
+  }
+}
+
+void print_received_command(
+    uint16_t command,
+    uint32_t flag,
+    const void* data,
+    size_t size,
+    GameVersion version,
+    const char* name,
+    TerminalFormat color) {
+  if (use_terminal_colors) {
+    print_color_escape(stderr, color, TerminalFormat::BOLD, TerminalFormat::END);
+  }
+
+  string name_token;
+  if (name && name[0]) {
+    name_token = string(" from ") + name;
+  }
+  log(INFO, "Received%s (version=%s command=%04hX flag=%08X)",
+      name_token.c_str(), name_for_version(version), command, flag);
+
+  PSOCommandHeader header;
+  size_t header_size = header.header_size(version);
+  header.set_command(version, command);
+  header.set_flag(version, flag);
+  header.set_size(version, size + header_size);
+
+  // TODO: This is unnecessarily slow. It'd be nice to have a print_data_v() so
+  // we don't have to copy data around here.
+  StringWriter w;
+  w.write(&header, header_size);
+  w.write(data, size);
+
+  print_data(stderr, w.str());
+
+  if (use_terminal_colors) {
+    print_color_escape(stderr, TerminalFormat::NORMAL, TerminalFormat::END);
+  }
+}
+
+void check_size_v(size_t size, size_t min_size, size_t max_size) {
+  if (size < min_size) {
+    throw std::runtime_error(string_printf(
+        "command too small (expected at least 0x%zX bytes, received 0x%zX bytes)",
+        min_size, size));
+  }
+  if (max_size < min_size) {
+    max_size = min_size;
+  }
+  if (size > max_size) {
+    throw std::runtime_error(string_printf(
+        "command too large (expected at most 0x%zX bytes, received 0x%zX bytes)",
+        max_size, size));
+  }
+}

src/PSOProtocol.hh

@@ -1,26 +1,31 @@
 #pragma once
 
 #include <inttypes.h>
+#include <event2/bufferevent.h>
+
+#include <functional>
+#include <phosg/Strings.hh>
 
 #include "Version.hh"
+#include "PSOEncryption.hh"
 
 struct PSOCommandHeaderPC {
   uint16_t size;
   uint8_t command;
   uint8_t flag;
-};
+} __attribute__((packed));
 
 struct PSOCommandHeaderDCGC {
   uint8_t command;
   uint8_t flag;
   uint16_t size;
-};
+} __attribute__((packed));
 
 struct PSOCommandHeaderBB {
   uint16_t size;
   uint16_t command;
   uint32_t flag;
-};
+} __attribute__((packed));
 
 union PSOCommandHeader {
   PSOCommandHeaderDCGC dc;
@@ -29,12 +34,74 @@ union PSOCommandHeader {
   PSOCommandHeaderBB bb;
 
   uint16_t command(GameVersion version) const;
+  void set_command(GameVersion version, uint16_t command);
   uint16_t size(GameVersion version) const;
+  void set_size(GameVersion version, uint32_t size);
   uint32_t flag(GameVersion version) const;
-};
+  void set_flag(GameVersion version, uint32_t flag);
+  static inline size_t header_size(GameVersion version) {
+    return (version == GameVersion::BB) ? 8 : 4;
+  }
+
+  PSOCommandHeader();
+} __attribute__((packed));
 
 union PSOSubcommand {
   uint8_t byte[4];
   uint16_t word[2];
   uint32_t dword;
-};
+} __attribute__((packed));
+
+void for_each_received_command(
+    struct bufferevent* bev,
+    GameVersion version,
+    PSOEncryption* crypt,
+    std::function<void(uint16_t, uint16_t, std::string&)> fn);
+
+void print_received_command(
+    uint16_t command,
+    uint32_t flag,
+    const void* data,
+    size_t size,
+    GameVersion version,
+    const char* name = nullptr,
+    TerminalFormat color = TerminalFormat::FG_GREEN);
+
+// This function is used in a lot of places to check received command sizes and
+// cast them to the appropriate type
+template <typename T>
+const T& check_size_t(
+    const std::string& data,
+    size_t min_size = sizeof(T),
+    size_t max_size = sizeof(T)) {
+  if (data.size() < min_size) {
+    throw std::runtime_error(string_printf(
+        "command too small (expected at least 0x%zX bytes, received 0x%zX bytes)",
+        min_size, data.size()));
+  }
+  if (data.size() > max_size) {
+    throw std::runtime_error(string_printf(
+        "command too large (expected at most 0x%zX bytes, received 0x%zX bytes)",
+        max_size, data.size()));
+  }
+  return *reinterpret_cast<const T*>(data.data());
+}
+template <typename T>
+T& check_size_t(
+    std::string& data,
+    size_t min_size = sizeof(T),
+    size_t max_size = sizeof(T)) {
+  if (data.size() < min_size) {
+    throw std::runtime_error(string_printf(
+        "command too small (expected at least 0x%zX bytes, received 0x%zX bytes)",
+        min_size, data.size()));
+  }
+  if (data.size() > max_size) {
+    throw std::runtime_error(string_printf(
+        "command too large (expected at most 0x%zX bytes, received 0x%zX bytes)",
+        max_size, data.size()));
+  }
+  return *reinterpret_cast<T*>(data.data());
+}
+
+void check_size_v(size_t size, size_t min_size, size_t max_size = 0);

src/Player.hh

@@ -7,49 +7,51 @@
 #include <vector>
 #include <phosg/Encoding.hh>
 
+#include "LevelTable.hh"
 #include "Version.hh"
+#include "Text.hh"
 
 
 
-// raw item data
 struct ItemData {
   union {
-    uint8_t item_data1[12];
-    uint16_t item_data1w[6];
-    uint32_t item_data1d[3];
-  };
-  uint32_t item_id;
+    uint8_t data1[12];
+    le_uint16_t data1w[6];
+    le_uint32_t data1d[3];
+  } __attribute__((packed));
+  le_uint32_t id;
   union {
-    uint8_t item_data2[4];
-    uint16_t item_data2w[2];
-    uint32_t item_data2d;
-  };
+    uint8_t data2[4];
+    le_uint16_t data2w[2];
+    le_uint32_t data2d;
+  } __attribute__((packed));
+
+  ItemData();
 
   uint32_t primary_identifier() const;
-};
+} __attribute__((packed));
 
 struct PlayerBankItem;
 
-// an item in a player's inventory
 struct PlayerInventoryItem {
-  uint16_t equip_flags;
-  uint16_t tech_flag;
-  uint32_t game_flags;
+  le_uint16_t equip_flags;
+  le_uint16_t tech_flag;
+  le_uint32_t game_flags;
   ItemData data;
 
-  PlayerBankItem to_bank_item() const;
-};
+  PlayerInventoryItem();
+  PlayerInventoryItem(const PlayerBankItem&);
+} __attribute__((packed));
 
-// an item in a player's bank
 struct PlayerBankItem {
   ItemData data;
-  uint16_t amount;
-  uint16_t show_flags;
+  le_uint16_t amount;
+  le_uint16_t show_flags;
 
-  PlayerInventoryItem to_inventory_item() const;
-};
+  PlayerBankItem();
+  PlayerBankItem(const PlayerInventoryItem&);
+} __attribute__((packed));
 
-// a player's inventory (remarkably, the format is the same in all versions of PSO)
 struct PlayerInventory {
   uint8_t num_items;
   uint8_t hp_materials_used;
@@ -57,380 +59,403 @@ struct PlayerInventory {
   uint8_t language;
   PlayerInventoryItem items[30];
 
-  size_t find_item(uint32_t item_id);
-};
+  PlayerInventory();
+
+  size_t find_item(uint32_t item_id);
+} __attribute__((packed));
 
-// a player's bank
 struct PlayerBank {
-  uint32_t num_items;
-  uint32_t meseta;
+  le_uint32_t num_items;
+  le_uint32_t meseta;
   PlayerBankItem items[200];
 
   void load(const std::string& filename);
   void save(const std::string& filename) const;
 
-  bool switch_with_file(const char* save_filename, const char* load_filename);
+  bool switch_with_file(const std::string& save_filename,
+      const std::string& load_filename);
 
   void add_item(const PlayerBankItem& item);
-  void remove_item(uint32_t item_id, uint32_t amount, PlayerBankItem* item);
+  PlayerBankItem remove_item(uint32_t item_id, uint32_t amount);
   size_t find_item(uint32_t item_id);
-};
+} __attribute__((packed));
 
 
 
-// simple player stats
-struct PlayerStats {
-  uint16_t atp;
-  uint16_t mst;
-  uint16_t evp;
-  uint16_t hp;
-  uint16_t dfp;
-  uint16_t ata;
-  uint16_t lck;
-};
-
 struct PlayerDispDataBB;
 
 // PC/GC player appearance and stats data
 struct PlayerDispDataPCGC { // 0xD0 in size
   PlayerStats stats;
-  uint16_t unknown1;
-  uint32_t unknown2[2];
-  uint32_t level;
-  uint32_t experience;
-  uint32_t meseta;
-  char name[16];
-  uint32_t unknown3[2];
-  uint32_t name_color;
+  parray<uint8_t, 0x0A> unknown_a1;
+  le_uint32_t level;
+  le_uint32_t experience;
+  le_uint32_t meseta;
+  ptext<char, 0x10> name;
+  uint64_t unknown_a2;
+  le_uint32_t name_color;
   uint8_t extra_model;
-  uint8_t unused[15];
-  uint32_t name_color_checksum;
+  parray<uint8_t, 0x0F> unused;
+  le_uint32_t name_color_checksum;
   uint8_t section_id;
   uint8_t char_class;
   uint8_t v2_flags;
   uint8_t version;
-  uint32_t v1_flags;
-  uint16_t costume;
-  uint16_t skin;
-  uint16_t face;
-  uint16_t head;
-  uint16_t hair;
-  uint16_t hair_r;
-  uint16_t hair_g;
-  uint16_t hair_b;
-  float proportion_x;
-  float proportion_y;
-  uint8_t config[0x48];
-  uint8_t technique_levels[0x14];
+  le_uint32_t v1_flags;
+  le_uint16_t costume;
+  le_uint16_t skin;
+  le_uint16_t face;
+  le_uint16_t head;
+  le_uint16_t hair;
+  le_uint16_t hair_r;
+  le_uint16_t hair_g;
+  le_uint16_t hair_b;
+  le_float proportion_x;
+  le_float proportion_y;
+  parray<uint8_t, 0x48> config;
+  parray<uint8_t, 0x14> technique_levels;
 
+  // Note: This struct has a default constructor because it's used in a command
+  // that has a fixed-size array. If we didn't define this constructor, the
+  // trivial fields in that array's members would be uninitialized, and we could
+  // send uninitialized memory to the client.
+  PlayerDispDataPCGC() noexcept;
+
+  void enforce_pc_limits();
   PlayerDispDataBB to_bb() const;
-};
+} __attribute__((packed));
 
 // BB player preview format
 struct PlayerDispDataBBPreview {
-  uint32_t experience;
-  uint32_t level;
-  char guild_card[16];
-  uint32_t unknown3[2];
-  uint32_t name_color;
+  le_uint32_t experience;
+  le_uint32_t level;
+  ptext<char, 0x10> guild_card;
+  uint64_t unknown_a2;
+  le_uint32_t name_color;
   uint8_t extra_model;
-  uint8_t unused[15];
-  uint32_t name_color_checksum;
+  parray<uint8_t, 0x0F> unused;
+  le_uint32_t name_color_checksum;
   uint8_t section_id;
   uint8_t char_class;
   uint8_t v2_flags;
   uint8_t version;
-  uint32_t v1_flags;
-  uint16_t costume;
-  uint16_t skin;
-  uint16_t face;
-  uint16_t head;
-  uint16_t hair;
-  uint16_t hair_r;
-  uint16_t hair_g;
-  uint16_t hair_b;
-  float proportion_x;
-  float proportion_y;
-  char16_t name[16];
+  le_uint32_t v1_flags;
+  le_uint16_t costume;
+  le_uint16_t skin;
+  le_uint16_t face;
+  le_uint16_t head;
+  le_uint16_t hair;
+  le_uint16_t hair_r;
+  le_uint16_t hair_g;
+  le_uint16_t hair_b;
+  le_float proportion_x;
+  le_float proportion_y;
+  ptext<char16_t, 0x10> name;
   uint32_t play_time;
-};
+
+  PlayerDispDataBBPreview() noexcept;
+} __attribute__((packed));
 
 // BB player appearance and stats data
 struct PlayerDispDataBB {
   PlayerStats stats;
-  uint16_t unknown1;
-  uint32_t unknown2[2];
-  uint32_t level;
-  uint32_t experience;
-  uint32_t meseta;
-  char guild_card[16];
-  uint32_t unknown3[2];
-  uint32_t name_color;
+  parray<uint8_t, 0x0A> unknown_a1;
+  le_uint32_t level;
+  le_uint32_t experience;
+  le_uint32_t meseta;
+  ptext<char, 0x10> guild_card;
+  uint64_t unknown_a2;
+  le_uint32_t name_color;
   uint8_t extra_model;
-  uint8_t unused[11];
-  uint32_t play_time; // not actually a game field; used only by my server
-  uint32_t name_color_checksum;
+  parray<uint8_t, 0x0F> unused;
+  le_uint32_t name_color_checksum;
   uint8_t section_id;
   uint8_t char_class;
   uint8_t v2_flags;
   uint8_t version;
-  uint32_t v1_flags;
-  uint16_t costume;
-  uint16_t skin;
-  uint16_t face;
-  uint16_t head;
-  uint16_t hair;
-  uint16_t hair_r;
-  uint16_t hair_g;
-  uint16_t hair_b;
-  float proportion_x;
-  float proportion_y;
-  char16_t name[0x10];
-  uint8_t config[0xE8];
-  uint8_t technique_levels[0x14];
+  le_uint32_t v1_flags;
+  le_uint16_t costume;
+  le_uint16_t skin;
+  le_uint16_t face;
+  le_uint16_t head;
+  le_uint16_t hair;
+  le_uint16_t hair_r;
+  le_uint16_t hair_g;
+  le_uint16_t hair_b;
+  le_float proportion_x;
+  le_float proportion_y;
+  ptext<char16_t, 0x10> name;
+  parray<uint8_t, 0xE8> config;
+  parray<uint8_t, 0x14> technique_levels;
 
+  PlayerDispDataBB() noexcept;
+
+  inline void enforce_pc_limits() { }
   PlayerDispDataPCGC to_pcgc() const;
   PlayerDispDataBBPreview to_preview() const;
   void apply_preview(const PlayerDispDataBBPreview&);
-};
+} __attribute__((packed));
 
 
 
 struct GuildCardGC {
-  uint32_t player_tag;
-  uint32_t serial_number;
-  char name[0x18];
-  char desc[0x6C];
+  le_uint32_t player_tag;
+  le_uint32_t serial_number;
+  ptext<char, 0x18> name;
+  ptext<char, 0x6C> desc;
   uint8_t reserved1; // should be 1
   uint8_t reserved2; // should be 1
   uint8_t section_id;
   uint8_t char_class;
-};
+
+  GuildCardGC() noexcept;
+} __attribute__((packed));
 
 // BB guild card format
 struct GuildCardBB {
-  uint32_t serial_number;
-  char16_t name[0x18];
-  char16_t teamname[0x10];
-  char16_t desc[0x58];
+  le_uint32_t serial_number;
+  ptext<char16_t, 0x18> name;
+  ptext<char16_t, 0x10> teamname;
+  ptext<char16_t, 0x58> desc;
   uint8_t reserved1; // should be 1
   uint8_t reserved2; // should be 1
   uint8_t section_id;
   uint8_t char_class;
-};
+
+  GuildCardBB() noexcept;
+} __attribute__((packed));
 
 // an entry in the BB guild card file
 struct GuildCardEntryBB {
   GuildCardBB data;
-  uint8_t unknown[0xB4];
-};
+  parray<uint8_t, 0xB4> unknown;
+} __attribute__((packed));
 
 // the format of the BB guild card file
 struct GuildCardFileBB {
-  uint8_t unknown[0x1F84];
+  parray<uint8_t, 0x1F84> unknown_a1;
   GuildCardEntryBB entry[0x0068]; // that's 104 of them in decimal
-  uint8_t unknown2[0x01AC];
-};
+  parray<uint8_t, 0x01AC> unknown_a2;
+} __attribute__((packed));
 
-// PSOBB key config and team info
 struct KeyAndTeamConfigBB {
-  uint8_t unknown[0x0114];         // 0000
-  uint8_t key_config[0x016C];      // 0114
-  uint8_t joystick_config[0x0038]; // 0280
-  uint32_t serial_number;          // 02B8
-  uint32_t team_id;                // 02BC
-  uint32_t team_info[2];           // 02C0
-  uint16_t team_privilege_level;   // 02C8
-  uint16_t reserved;               // 02CA
-  char16_t team_name[0x0010];       // 02CC
-  uint8_t team_flag[0x0800];       // 02EC
-  uint32_t team_rewards[2];        // 0AEC
-};
-
-// BB account data
-struct PlayerAccountDataBB {
-  uint8_t symbol_chats[0x04E0];
-  KeyAndTeamConfigBB key_config;
-  GuildCardFileBB guild_cards;
-  uint32_t options;
-  uint8_t shortcuts[0x0A40]; // chat shortcuts (@1FB4 in E7 command)
-};
+  parray<uint8_t, 0x0114> unknown_a1;      // 0000
+  parray<uint8_t, 0x016C> key_config;      // 0114
+  parray<uint8_t, 0x0038> joystick_config; // 0280
+  le_uint32_t serial_number;               // 02B8
+  le_uint32_t team_id;                     // 02BC
+  le_uint64_t team_info;                   // 02C0
+  le_uint16_t team_privilege_level;        // 02C8
+  le_uint16_t reserved;                    // 02CA
+  ptext<char16_t, 0x0010> team_name;       // 02CC
+  parray<uint8_t, 0x0800> team_flag;       // 02EC
+  le_uint32_t team_rewards;                // 0AEC
+} __attribute__((packed));
 
 
 
 struct PlayerLobbyDataPC {
-  uint32_t player_tag;
-  uint32_t guild_card;
+  le_uint32_t player_tag;
+  le_uint32_t guild_card;
   be_uint32_t ip_address;
-  uint32_t client_id;
-  char16_t name[16];
-};
+  le_uint32_t client_id;
+  ptext<char16_t, 0x10> name;
+
+  PlayerLobbyDataPC() noexcept;
+} __attribute__((packed));
 
 struct PlayerLobbyDataGC {
-  uint32_t player_tag;
-  uint32_t guild_card;
+  le_uint32_t player_tag;
+  le_uint32_t guild_card;
   be_uint32_t ip_address;
-  uint32_t client_id;
-  char name[16];
-};
+  le_uint32_t client_id;
+  ptext<char, 0x10> name;
+
+  PlayerLobbyDataGC() noexcept;
+} __attribute__((packed));
 
 struct PlayerLobbyDataBB {
-  uint32_t player_tag;
-  uint32_t guild_card;
-  uint32_t unknown1[5];
-  uint32_t client_id;
-  char16_t name[16];
-  uint32_t unknown2;
-};
+  le_uint32_t player_tag;
+  le_uint32_t guild_card;
+  be_uint32_t ip_address; // Guess - the official builds didn't use this, but all other versions have it
+  parray<uint8_t, 0x10> unknown_a1;
+  le_uint32_t client_id;
+  ptext<char16_t, 0x10> name;
+  le_uint32_t unknown2;
+
+  PlayerLobbyDataBB() noexcept;
+} __attribute__((packed));
 
 
 
-struct PSOPlayerDataPC { // for command 0x61
+struct PSOPlayerDataPC { // For command 61
   PlayerInventory inventory;
   PlayerDispDataPCGC disp;
-};
+} __attribute__((packed));
 
-struct PSOPlayerDataGC { // for command 0x61
+struct PSOPlayerDataGC { // For command 61
   PlayerInventory inventory;
   PlayerDispDataPCGC disp;
-  char unknown[0x134];
-  char info_board[0xAC];
-  uint32_t blocked[0x1E];
-  uint32_t auto_reply_enabled;
+  parray<uint8_t, 0x134> unknown;
+  ptext<char, 0xAC> info_board;
+  parray<le_uint32_t, 0x1E> blocked_senders;
+  le_uint32_t auto_reply_enabled;
   char auto_reply[0];
-};
+} __attribute__((packed));
 
-struct PSOPlayerDataBB { // for command 0x61
+struct PSOPlayerDataBB { // For command 61
   PlayerInventory inventory;
   PlayerDispDataBB disp;
-  char unused[0x174];
-  char16_t info_board[0xAC];
-  uint32_t blocked[0x1E];
-  uint32_t auto_reply_enabled;
+  ptext<char, 0x174> unused;
+  ptext<char16_t, 0xAC> info_board;
+  parray<le_uint32_t, 0x1E> blocked_senders;
+  le_uint32_t auto_reply_enabled;
   char16_t auto_reply[0];
-};
+} __attribute__((packed));
 
-// PC/GC lobby player data (used in lobby/game join commands)
-struct PlayerLobbyJoinDataPCGC {
-  PlayerInventory inventory;
-  PlayerDispDataPCGC disp;
-};
-
-// BB lobby player data (used in lobby/game join commands)
-struct PlayerLobbyJoinDataBB {
-  PlayerInventory inventory;
-  PlayerDispDataBB disp;
-};
-
-// complete BB player data format (used in E7 command)
-struct PlayerBB {
-  PlayerInventory inventory;        // 0000 // player
-  PlayerDispDataBB disp;            // 034C // player
-  uint8_t unknown[0x0010];          // 04DC //
-  uint32_t option_flags;            // 04EC // account
-  uint8_t quest_data1[0x0208];      // 04F0 // player
-  PlayerBank bank;                  // 06F8 // player
-  uint32_t serial_number;           // 19C0 // player
-  char16_t name[0x18];              // 19C4 // player
-  char16_t team_name[0x10];         // 19C4 // player
-  char16_t guild_card_desc[0x58];   // 1A14 // player
-  uint8_t reserved1;                // 1AC4 // player
-  uint8_t reserved2;                // 1AC5 // player
-  uint8_t section_id;               // 1AC6 // player
-  uint8_t char_class;               // 1AC7 // player
-  uint32_t unknown3;                // 1AC8 //
-  uint8_t symbol_chats[0x04E0];     // 1ACC // account
-  uint8_t shortcuts[0x0A40];        // 1FAC // account
-  char16_t auto_reply[0x00AC];      // 29EC // player
-  char16_t info_board[0x00AC];      // 2B44 // player
-  uint8_t unknown5[0x001C];         // 2C9C //
-  uint8_t challenge_data[0x0140];   // 2CB8 // player
-  uint8_t tech_menu_config[0x0028]; // 2DF8 // player
-  uint8_t unknown6[0x002C];         // 2E20 //
-  uint8_t quest_data2[0x0058];      // 2E4C // player
-  KeyAndTeamConfigBB key_config;    // 2EA4 // account
-};                                  // total size: 39A0
+struct PlayerBB { // Used in 00E7 command
+  PlayerInventory inventory;                // 0000 // player
+  PlayerDispDataBB disp;                    // 034C // player
+  parray<uint8_t, 0x0010> unknown;          // 04DC //
+  le_uint32_t option_flags;                 // 04EC // account
+  parray<uint8_t, 0x0208> quest_data1;      // 04F0 // player
+  PlayerBank bank;                          // 06F8 // player
+  le_uint32_t serial_number;                // 19C0 // player
+  ptext<char16_t, 0x18> name;               // 19C4 // player
+  ptext<char16_t, 0x10> team_name;          // 19C4 // player
+  ptext<char16_t, 0x58> guild_card_desc;    // 1A14 // player
+  uint8_t reserved1;                        // 1AC4 // player
+  uint8_t reserved2;                        // 1AC5 // player
+  uint8_t section_id;                       // 1AC6 // player
+  uint8_t char_class;                       // 1AC7 // player
+  le_uint32_t unknown3;                     // 1AC8 //
+  parray<uint8_t, 0x04E0> symbol_chats;     // 1ACC // account
+  parray<uint8_t, 0x0A40> shortcuts;        // 1FAC // account
+  ptext<char16_t, 0x00AC> auto_reply;       // 29EC // player
+  ptext<char16_t, 0x00AC> info_board;       // 2B44 // player
+  parray<uint8_t, 0x001C> unknown5;         // 2C9C //
+  parray<uint8_t, 0x0140> challenge_data;   // 2CB8 // player
+  parray<uint8_t, 0x0028> tech_menu_config; // 2DF8 // player
+  parray<uint8_t, 0x002C> unknown6;         // 2E20 //
+  parray<uint8_t, 0x0058> quest_data2;      // 2E4C // player
+  KeyAndTeamConfigBB key_config;            // 2EA4 // account
+  le_uint32_t unused;
+} __attribute__((packed));                  // total size: 39A0
 
 
 
-struct SavedPlayerBB { // .nsc file format
-  char signature[0x40];
+struct SavedPlayerDataBB { // .nsc file format
+  ptext<char, 0x40>       signature;
   PlayerDispDataBBPreview preview;
-
-  char16_t          auto_reply[0x00AC];
-  PlayerBank        bank;
-  uint8_t           challenge_data[0x0140];
-  PlayerDispDataBB  disp;
-  char16_t          guild_card_desc[0x58];
-  char16_t          info_board[0x00AC];
-  PlayerInventory   inventory;
-  uint8_t           quest_data1[0x0208];
-  uint8_t           quest_data2[0x0058];
-  uint8_t           tech_menu_config[0x0028];
-};
-
-struct SavedAccountBB { // .nsa file format
-  char signature[0x40];
-  uint32_t            blocked[0x001E];
-  GuildCardFileBB     guild_cards;
-  KeyAndTeamConfigBB  key_config;
-  uint32_t            option_flags;
-  uint8_t             shortcuts[0x0A40];
-  uint8_t             symbol_chats[0x04E0];
-  char16_t            team_name[0x0010];
-};
-
-// complete player info stored by the server
-struct Player {
-  uint32_t            loaded_from_shipgate_time;
-
-  char16_t            auto_reply[0x00AC];       // player
-  PlayerBank          bank;                     // player
-  char                bank_name[0x20];
-  uint32_t            blocked[0x001E];          // account
-  uint8_t             challenge_data[0x0140];   // player
-  PlayerDispDataBB    disp;                     // player
-  uint8_t             ep3_config[0x2408];
-  char16_t            guild_card_desc[0x58];    // player
-  GuildCardFileBB     guild_cards;              // account
-  PlayerInventoryItem identify_result;
-  char16_t            info_board[0x00AC];       // player
-  PlayerInventory     inventory;                // player
-  KeyAndTeamConfigBB  key_config;               // account
-  uint32_t            option_flags;             // account
-  uint8_t             quest_data1[0x0208];      // player
-  uint8_t             quest_data2[0x0058];      // player
-  uint32_t            serial_number;
-  std::vector<ItemData> current_shop_contents;
-  uint8_t             shortcuts[0x0A40];        // account
-  uint8_t             symbol_chats[0x04E0];     // account
-  char16_t            team_name[0x0010];        // account
-  uint8_t             tech_menu_config[0x0028]; // player
-
-  void load_player_data(const std::string& filename);
-  void save_player_data(const std::string& filename) const;
-
-  void load_account_data(const std::string& filename);
-  void save_account_data(const std::string& filename) const;
-
-  void import(const PSOPlayerDataPC& pd);
-  void import(const PSOPlayerDataGC& pd);
-  void import(const PSOPlayerDataBB& pd);
-  PlayerLobbyJoinDataPCGC export_lobby_data_pc() const;
-  PlayerLobbyJoinDataPCGC export_lobby_data_gc() const;
-  PlayerLobbyJoinDataBB export_lobby_data_bb() const;
-  PlayerBB export_bb_player_data() const;
+  ptext<char16_t, 0x00AC> auto_reply;
+  PlayerBank              bank;
+  parray<uint8_t, 0x0140> challenge_data;
+  PlayerDispDataBB        disp;
+  ptext<char16_t, 0x0058> guild_card_desc;
+  ptext<char16_t, 0x00AC> info_board;
+  PlayerInventory         inventory;
+  parray<uint8_t, 0x0208> quest_data1;
+  parray<uint8_t, 0x0058> quest_data2;
+  parray<uint8_t, 0x0028> tech_menu_config;
 
   void add_item(const PlayerInventoryItem& item);
-  void remove_item(uint32_t item_id, uint32_t amount, PlayerInventoryItem* item);
-  size_t find_item(uint32_t item_id);
+  PlayerInventoryItem remove_item(uint32_t item_id, uint32_t amount);
+
+  void print_inventory(FILE* stream) const;
+} __attribute__((packed));
+
+struct SavedAccountDataBB { // .nsa file format
+  ptext<char, 0x40>           signature;
+  parray<le_uint32_t, 0x001E> blocked_senders;
+  GuildCardFileBB             guild_cards;
+  KeyAndTeamConfigBB          key_config;
+  le_uint32_t                 unused;
+  le_uint32_t                 option_flags;
+  parray<uint8_t, 0x0A40>     shortcuts;
+  parray<uint8_t, 0x04E0>     symbol_chats;
+  ptext<char16_t, 0x0010>     team_name;
+} __attribute__((packed));
+
+class ClientGameData {
+private:
+  std::shared_ptr<SavedAccountDataBB> account_data;
+  std::shared_ptr<SavedPlayerDataBB> player_data;
+
+public:
+  uint32_t serial_number;
+
+  // The following fields are not saved, and are only used in certain situations
+
+  // Null unless the client is Episode 3 and has sent its config already
+  std::shared_ptr<parray<uint8_t, 0x2408>> ep3_config;
+
+  // These are only used if the client is BB
+  std::string bb_username;
+  size_t bb_player_index;
+  PlayerInventoryItem identify_result;
+  std::vector<ItemData> shop_contents;
+
+  ClientGameData() : serial_number(0), bb_player_index(0) { }
+  ~ClientGameData();
+
+  std::shared_ptr<SavedAccountDataBB> account(bool should_load = true);
+  std::shared_ptr<SavedPlayerDataBB> player(bool should_load = true);
+  std::shared_ptr<const SavedAccountDataBB> account() const;
+  std::shared_ptr<const SavedPlayerDataBB> player() const;
+
+  std::string account_data_filename() const;
+  std::string player_data_filename() const;
+  static std::string player_template_filename(uint8_t char_class);
+
+  void create_player(
+      const PlayerDispDataBBPreview& preview,
+      std::shared_ptr<const LevelTable> level_table);
+
+  void load_account_data();
+  void save_account_data() const;
+  void load_player_data();
+  void save_player_data() const;
+
+  void import_player(const PSOPlayerDataPC& pd);
+  void import_player(const PSOPlayerDataGC& pd);
+  void import_player(const PSOPlayerDataBB& pd);
+  // Note: this function is not const because it can cause player and account
+  // data to be loaded
+  PlayerBB export_player_bb();
 };
 
 
 
 uint32_t compute_guild_card_checksum(const void* data, size_t size);
 
-std::string filename_for_player_bb(const std::string& username, uint8_t player_index);
-std::string filename_for_bank_bb(const std::string& username, const char* bank_name);
-std::string filename_for_class_template_bb(uint8_t char_class);
-std::string filename_for_account_bb(const std::string& username);
+
+
+template <typename DestT, typename SrcT = DestT>
+DestT convert_player_disp_data(const SrcT&) {
+  static_assert(always_false<DestT, SrcT>::v,
+      "unspecialized strcpy_t should never be called");
+}
+
+template <>
+inline PlayerDispDataPCGC convert_player_disp_data<PlayerDispDataPCGC>(
+    const PlayerDispDataPCGC& src) {
+  return src;
+}
+
+template <>
+inline PlayerDispDataPCGC convert_player_disp_data<PlayerDispDataPCGC, PlayerDispDataBB>(
+    const PlayerDispDataBB& src) {
+  return src.to_pcgc();
+}
+
+template <>
+inline PlayerDispDataBB convert_player_disp_data<PlayerDispDataBB, PlayerDispDataPCGC>(
+    const PlayerDispDataPCGC& src) {
+  return src.to_bb();
+}
+
+template <>
+inline PlayerDispDataBB convert_player_disp_data<PlayerDispDataBB>(
+    const PlayerDispDataBB& src) {
+  return src;
+}

src/ProxyCommands.hh

@@ -0,0 +1,18 @@
+#pragma once
+
+#include <stdint.h>
+
+#include <string>
+
+#include "ServerState.hh"
+#include "ProxyServer.hh"
+
+
+
+void process_proxy_command(
+    std::shared_ptr<ServerState> s,
+    ProxyServer::LinkedSession& session,
+    bool from_server,
+    uint16_t command,
+    uint32_t flag,
+    std::string& data);

src/ProxyServer.hh

@@ -5,12 +5,15 @@
 #include <map>
 #include <unordered_map>
 #include <unordered_set>
+#include <functional>
 #include <vector>
 #include <string>
 #include <memory>
+#include <phosg/Filesystem.hh>
 
 #include "PSOEncryption.hh"
 #include "PSOProtocol.hh"
+#include "ServerState.hh"
 
 
 
@@ -19,74 +22,217 @@ public:
   ProxyServer() = delete;
   ProxyServer(const ProxyServer&) = delete;
   ProxyServer(ProxyServer&&) = delete;
-  ProxyServer(std::shared_ptr<struct event_base> base,
-      const struct sockaddr_storage& initial_destination, GameVersion version);
+  ProxyServer(
+      std::shared_ptr<struct event_base> base,
+      std::shared_ptr<ServerState> state);
   virtual ~ProxyServer() = default;
 
-  void listen(int port);
+  void listen(uint16_t port, GameVersion version,
+      const struct sockaddr_storage* default_destination = nullptr);
 
-  void connect_client(struct bufferevent* bev);
+  void connect_client(struct bufferevent* bev, uint16_t server_port);
 
-  void send_to_client(const std::string& data);
-  void send_to_server(const std::string& data);
+  struct LinkedSession {
+    ProxyServer* server;
+    uint64_t id;
+    std::string client_name;
+    std::string server_name;
+    PrefixedLogger log;
 
-  void set_save_quests(bool save_quests);
+    std::unique_ptr<struct event, void(*)(struct event*)> timeout_event;
+
+    std::shared_ptr<const License> license;
+
+    std::unique_ptr<struct bufferevent, void(*)(struct bufferevent*)> client_bev;
+    std::unique_ptr<struct bufferevent, void(*)(struct bufferevent*)> server_bev;
+    uint16_t local_port;
+    struct sockaddr_storage next_destination;
+
+    uint8_t prev_server_command_bytes[6];
+
+    GameVersion version;
+    uint32_t sub_version;
+    std::string character_name;
+    C_Login_BB_93 login_command_bb;
+
+    uint32_t remote_guild_card_number;
+    parray<uint8_t, 0x20> remote_client_config_data;
+    ClientConfigBB newserv_client_config;
+    bool suppress_newserv_commands;
+    bool enable_chat_filter;
+    bool enable_switch_assist;
+    bool save_files;
+    int64_t function_call_return_value; // -1 = don't block function calls
+    G_SwitchStateChanged_6x05 last_switch_enabled_command;
+    int16_t override_section_id;
+    int16_t override_lobby_event;
+    int16_t override_lobby_number;
+
+    struct LobbyPlayer {
+      uint32_t guild_card_number;
+      std::string name;
+      LobbyPlayer() : guild_card_number(0) { }
+    };
+    std::vector<LobbyPlayer> lobby_players;
+    size_t lobby_client_id;
+
+    std::shared_ptr<PSOEncryption> client_input_crypt;
+    std::shared_ptr<PSOEncryption> client_output_crypt;
+    std::shared_ptr<PSOEncryption> server_input_crypt;
+    std::shared_ptr<PSOEncryption> server_output_crypt;
+    std::shared_ptr<PSOBBMultiKeyDetectorEncryption> detector_crypt;
+
+    struct SavingFile {
+      std::string basename;
+      std::string output_filename;
+      uint32_t remaining_bytes;
+      std::unique_ptr<FILE, std::function<void(FILE*)>> f;
+
+      SavingFile(
+          const std::string& basename,
+          const std::string& output_filename,
+          uint32_t remaining_bytes);
+    };
+    std::unordered_map<std::string, SavingFile> saving_files;
+
+    // TODO: This first constructor should be private
+    LinkedSession(
+        ProxyServer* server,
+        uint64_t id,
+        uint16_t local_port,
+        GameVersion version);
+    LinkedSession(
+        ProxyServer* server,
+        uint16_t local_port,
+        GameVersion version,
+        std::shared_ptr<const License> license,
+        const ClientConfigBB& newserv_client_config);
+    LinkedSession(
+        ProxyServer* server,
+        uint16_t local_port,
+        GameVersion version,
+        std::shared_ptr<const License> license,
+        const struct sockaddr_storage& next_destination);
+    LinkedSession(
+        ProxyServer* server,
+        uint64_t id,
+        uint16_t local_port,
+        GameVersion version,
+        const struct sockaddr_storage& next_destination);
+
+    void resume(
+        std::unique_ptr<struct bufferevent, void(*)(struct bufferevent*)>&& client_bev,
+        std::shared_ptr<PSOEncryption> client_input_crypt,
+        std::shared_ptr<PSOEncryption> client_output_crypt,
+        std::shared_ptr<PSOBBMultiKeyDetectorEncryption> detector_crypt,
+        uint32_t sub_version,
+        const std::string& character_name);
+    void resume(
+        std::unique_ptr<struct bufferevent, void(*)(struct bufferevent*)>&& client_bev,
+        std::shared_ptr<PSOEncryption> client_input_crypt,
+        std::shared_ptr<PSOEncryption> client_output_crypt,
+        std::shared_ptr<PSOBBMultiKeyDetectorEncryption> detector_crypt,
+        C_Login_BB_93 login_command_bb);
+    void resume(struct bufferevent* client_bev);
+    void resume_inner(
+        std::unique_ptr<struct bufferevent, void(*)(struct bufferevent*)>&& client_bev,
+        std::shared_ptr<PSOEncryption> client_input_crypt,
+        std::shared_ptr<PSOEncryption> client_output_crypt,
+        std::shared_ptr<PSOBBMultiKeyDetectorEncryption> detector_crypt);
+    void connect();
+
+    static void dispatch_on_client_input(struct bufferevent* bev, void* ctx);
+    static void dispatch_on_client_error(struct bufferevent* bev, short events,
+        void* ctx);
+    static void dispatch_on_server_input(struct bufferevent* bev, void* ctx);
+    static void dispatch_on_server_error(struct bufferevent* bev, short events,
+        void* ctx);
+    static void dispatch_on_timeout(evutil_socket_t fd, short what, void* ctx);
+    void on_client_input();
+    void on_server_input();
+    void on_stream_error(short events, bool is_server_stream);
+    void on_timeout();
+
+    void send_to_end(bool to_server, uint16_t command, uint32_t flag,
+        const void* data = nullptr, size_t size = 0);
+    void send_to_end(bool to_server, uint16_t command, uint32_t flag,
+        const std::string& data);
+    void send_to_end_with_header(
+        bool to_server, const void* data, size_t size);
+    void send_to_end_with_header(bool to_server, const std::string& data);
+
+    void disconnect();
+
+    bool is_open() const;
+  };
+
+  std::shared_ptr<LinkedSession> get_session();
+  std::shared_ptr<LinkedSession> create_licensed_session(
+    std::shared_ptr<const License> l,
+    uint16_t local_port,
+    GameVersion version,
+    const ClientConfigBB& newserv_client_config);
+  void delete_session(uint64_t id);
 
 private:
-  std::shared_ptr<struct event_base> base;
-  std::map<int, std::unique_ptr<struct evconnlistener, void(*)(struct evconnlistener*)>> listeners;
-  std::unique_ptr<struct bufferevent, void(*)(struct bufferevent*)> client_bev;
-  std::unique_ptr<struct bufferevent, void(*)(struct bufferevent*)> server_bev;
-  struct sockaddr_storage next_destination;
-  int listen_port;
-  GameVersion version;
+  struct ListeningSocket {
+    ProxyServer* server;
 
-  size_t header_size;
-  PSOCommandHeader client_input_header;
-  PSOCommandHeader server_input_header;
-  std::shared_ptr<PSOEncryption> client_input_crypt;
-  std::shared_ptr<PSOEncryption> client_output_crypt;
-  std::shared_ptr<PSOEncryption> server_input_crypt;
-  std::shared_ptr<PSOEncryption> server_output_crypt;
+    PrefixedLogger log;
+    uint16_t port;
+    scoped_fd fd;
+    std::unique_ptr<struct evconnlistener, void(*)(struct evconnlistener*)> listener;
+    GameVersion version;
+    struct sockaddr_storage default_destination;
 
-  struct SavingQuestFile {
-    std::string basename;
-    std::string output_filename;
-    uint32_t remaining_bytes;
-    std::unique_ptr<FILE, std::function<void(FILE*)>> f;
+    ListeningSocket(
+        ProxyServer* server,
+        uint16_t port,
+        GameVersion version,
+        const struct sockaddr_storage* default_destination);
 
-    SavingQuestFile(
-        const std::string& basename,
-        const std::string& output_filename,
-        uint32_t remaining_bytes);
+    static void dispatch_on_listen_accept(struct evconnlistener* listener,
+        evutil_socket_t fd, struct sockaddr *address, int socklen, void* ctx);
+    static void dispatch_on_listen_error(struct evconnlistener* listener, void* ctx);
+    void on_listen_accept(int fd);
+    void on_listen_error();
   };
-  bool save_quests;
-  std::unordered_map<std::string, SavingQuestFile> saving_quest_files;
 
-  void send_to_end(const std::string& data, bool to_server);
+  struct UnlinkedSession {
+    ProxyServer* server;
 
-  static void dispatch_on_listen_accept(struct evconnlistener* listener,
-      evutil_socket_t fd, struct sockaddr *address, int socklen, void* ctx);
-  static void dispatch_on_listen_error(struct evconnlistener* listener, void* ctx);
-  static void dispatch_on_client_input(struct bufferevent* bev, void* ctx);
-  static void dispatch_on_client_error(struct bufferevent* bev, short events,
-      void* ctx);
-  static void dispatch_on_server_input(struct bufferevent* bev, void* ctx);
-  static void dispatch_on_server_error(struct bufferevent* bev, short events,
-      void* ctx);
+    PrefixedLogger log;
+    std::unique_ptr<struct bufferevent, void(*)(struct bufferevent*)> bev;
+    uint16_t local_port;
+    GameVersion version;
+    struct sockaddr_storage next_destination;
 
-  void on_listen_accept(struct evconnlistener* listener, evutil_socket_t fd,
-      struct sockaddr *address, int socklen);
-  void on_listen_error(struct evconnlistener* listener);
-  void on_client_input(struct bufferevent* bev);
-  void on_client_error(struct bufferevent* bev, short events);
-  void on_server_input(struct bufferevent* bev);
-  void on_server_error(struct bufferevent* bev, short events);
+    std::shared_ptr<PSOEncryption> crypt_out;
+    std::shared_ptr<PSOEncryption> crypt_in;
+    std::shared_ptr<PSOBBMultiKeyDetectorEncryption> detector_crypt;
 
-  void on_client_connect(struct bufferevent* bev);
+    UnlinkedSession(ProxyServer* server, struct bufferevent* bev, uint16_t port, GameVersion version);
 
-  size_t get_size_field(const PSOCommandHeader* header);
-  size_t get_command_field(const PSOCommandHeader* header);
+    void receive_and_process_commands();
 
-  void receive_and_process_commands(bool from_server);
+    static void dispatch_on_client_input(struct bufferevent* bev, void* ctx);
+    static void dispatch_on_client_error(struct bufferevent* bev, short events,
+        void* ctx);
+    void on_client_input();
+    void on_client_error(short events);
+  };
+
+  PrefixedLogger log;
+  std::shared_ptr<struct event_base> base;
+  std::shared_ptr<ServerState> state;
+  std::map<int, std::shared_ptr<ListeningSocket>> listeners;
+  std::unordered_map<struct bufferevent*, std::shared_ptr<UnlinkedSession>> bev_to_unlinked_session;
+  std::unordered_map<uint64_t, std::shared_ptr<LinkedSession>> id_to_session;
+  uint64_t next_unlicensed_session_id;
+
+  void on_client_connect(
+      struct bufferevent* bev,
+      uint16_t listen_port,
+      GameVersion version,
+      const struct sockaddr_storage* default_destination);
 };

src/ProxyShell.cc

@@ -1,146 +0,0 @@
-#include "ProxyShell.hh"
-
-#include <event2/event.h>
-#include <stdio.h>
-
-#include <phosg/Strings.hh>
-
-using namespace std;
-
-
-
-ProxyShell::ProxyShell(std::shared_ptr<struct event_base> base,
-    std::shared_ptr<ServerState> state,
-    std::shared_ptr<ProxyServer> proxy_server) : Shell(base, state),
-    proxy_server(proxy_server) { }
-
-void ProxyShell::execute_command(const string& command) {
-  // find the entry in the command table and run the command
-  size_t command_end = skip_non_whitespace(command, 0);
-  size_t args_begin = skip_whitespace(command, command_end);
-  string command_name = command.substr(0, command_end);
-  string command_args = command.substr(args_begin);
-
-  if (command_name == "exit") {
-    throw exit_shell();
-
-  } else if (command_name == "help") {
-    fprintf(stderr, "\
-Commands:\n\
-  help\n\
-    You\'re reading it now.\n\
-  exit (or ctrl+d)\n\
-    Shut down the proxy.\n\
-  sc <data>\n\
-    Send a command to the client.\n\
-  ss <data>\n\
-    Send a command to the server.\n\
-  chat <text>\n\
-    Send a chat message to the server.\n\
-  dchat <data>\n\
-    Send a chat message to the server with arbitrary data in it.\n\
-  info-board <text>\n\
-    Set your info board contents.\n\
-  info-board-data <data>\n\
-    Set your info board contents with arbitrary data.\n\
-  marker <color-id>\n\
-    Send a lobby marker message to the server.\n\
-  event <event-id>\n\
-    Send a lobby event update to yourself.\n\
-  ship\n\
-    Request the ship select menu from the server.\n\
-");
-
-  } else if ((command_name == "sc") || (command_name == "ss")) {
-    bool to_client = (command_name[1] == 'c');
-    string data = parse_data_string(command_args);
-    if (data.size() & 3) {
-      throw invalid_argument("data size is not a multiple of 4");
-    }
-    if (data.size() == 0) {
-      throw invalid_argument("no data given");
-    }
-    uint16_t* size_field = reinterpret_cast<uint16_t*>(data.data() + 2);
-    *size_field = data.size();
-
-    log(INFO, "%s (from proxy):", to_client ? "server" : "client");
-    print_data(stderr, data);
-
-    if (to_client) {
-      this->proxy_server->send_to_client(data);
-    } else {
-      this->proxy_server->send_to_server(data);
-    }
-
-  } else if ((command_name == "chat") || (command_name == "dchat")) {
-    string data(12, '\0');
-    data[0] = 0x06;
-    data.push_back('\x09');
-    data.push_back('E');
-    if (command_name == "dchat") {
-      data += parse_data_string(command_args);
-    } else {
-      data += command_args;
-    }
-    data.push_back('\0');
-    data.resize((data.size() + 3) & (~3));
-    uint16_t* size_field = reinterpret_cast<uint16_t*>(data.data() + 2);
-    *size_field = data.size();
-
-    log(INFO, "Client (from proxy):");
-    print_data(stderr, data);
-    this->proxy_server->send_to_server(data);
-
-  } else if (command_name == "marker") {
-    string data("\x89\x00\x04\x00", 4);
-    data[1] = stod(command_args);
-
-    log(INFO, "Client (from proxy):");
-    print_data(stderr, data);
-    this->proxy_server->send_to_server(data);
-
-  } else if (command_name == "event") {
-    string data("\xDA\x00\x04\x00", 4);
-    data[1] = stod(command_args);
-
-    log(INFO, "Server (from proxy):");
-    print_data(stderr, data);
-    this->proxy_server->send_to_client(data);
-
-  } else if (command_name == "ship") {
-    static const string data("\xA0\x00\x04\x00", 4);
-
-    log(INFO, "Server (from proxy):");
-    print_data(stderr, data);
-    this->proxy_server->send_to_server(data);
-
-  } else if ((command_name == "info-board") || (command_name == "info-board-data")) {
-    string data(4, '\0');
-    data[0] = 0xD9;
-    if (command_name == "info-board-data") {
-      data += parse_data_string(command_args);
-    } else {
-      data += command_args;
-    }
-    data.push_back('\0');
-    data.resize((data.size() + 3) & (~3));
-    uint16_t* size_field = reinterpret_cast<uint16_t*>(data.data() + 2);
-    *size_field = data.size();
-
-    log(INFO, "Client (from proxy):");
-    print_data(stderr, data);
-    this->proxy_server->send_to_server(data);
-
-  } else if (command_name == "set-save-quests") {
-    if (command_args == "on") {
-      this->proxy_server->set_save_quests(true);
-    } else if (command_args == "off") {
-      this->proxy_server->set_save_quests(false);
-    } else {
-      throw invalid_argument("argument must be \"on\" or \"off\"");
-    }
-
-  } else {
-    throw invalid_argument("unknown command; try \'help\'");
-  }
-}

src/ProxyShell.hh

@@ -1,28 +0,0 @@
-#pragma once
-
-#include <memory>
-#include <string>
-
-#include <event2/event.h>
-
-#include "Shell.hh"
-#include "ProxyServer.hh"
-
-
-
-class ProxyShell : public Shell {
-public:
-  ProxyShell(std::shared_ptr<struct event_base> base,
-      std::shared_ptr<ServerState> state,
-      std::shared_ptr<ProxyServer> proxy_server);
-  virtual ~ProxyShell() = default;
-  ProxyShell(const ProxyShell&) = delete;
-  ProxyShell(ProxyShell&&) = delete;
-  ProxyShell& operator=(const ProxyShell&) = delete;
-  ProxyShell& operator=(ProxyShell&&) = delete;
-
-protected:
-  std::shared_ptr<ProxyServer> proxy_server;
-
-  virtual void execute_command(const std::string& command);
-};

src/Quest.cc

@@ -8,6 +8,7 @@
 #include <phosg/Random.hh>
 #include <phosg/Strings.hh>
 
+#include "CommandFormats.hh"
 #include "Compression.hh"
 #include "PSOEncryption.hh"
 #include "Text.hh"
@@ -19,48 +20,48 @@ using namespace std;
 struct PSODownloadQuestHeader {
   // When sending a DLQ to the client, this is the DECOMPRESSED size. When
   // reading it from a GCI file, this is the COMPRESSED size.
-  be_uint32_t size;
+  le_uint32_t size;
   // Note: use PSO PC encryption, even for GC quests.
-  be_uint32_t encryption_seed;
-};
+  le_uint32_t encryption_seed;
+} __attribute__((packed));
 
 
 
 bool category_is_mode(QuestCategory category) {
-  return (category == QuestCategory::Battle) ||
-         (category == QuestCategory::Challenge) ||
-         (category == QuestCategory::Episode3);
+  return (category == QuestCategory::BATTLE) ||
+         (category == QuestCategory::CHALLENGE) ||
+         (category == QuestCategory::EPISODE_3);
 }
 
 const char* name_for_category(QuestCategory category) {
   switch (category) {
-    case QuestCategory::Retrieval:
+    case QuestCategory::RETRIEVAL:
       return "Retrieval";
-    case QuestCategory::Extermination:
+    case QuestCategory::EXTERMINATION:
       return "Extermination";
-    case QuestCategory::Event:
+    case QuestCategory::EVENT:
       return "Event";
-    case QuestCategory::Shop:
+    case QuestCategory::SHOP:
       return "Shop";
     case QuestCategory::VR:
       return "VR";
-    case QuestCategory::Tower:
+    case QuestCategory::TOWER:
       return "Tower";
-    case QuestCategory::GovernmentEpisode1:
+    case QuestCategory::GOVERNMENT_EPISODE_1:
       return "GovernmentEpisode1";
-    case QuestCategory::GovernmentEpisode2:
+    case QuestCategory::GOVERNMENT_EPISODE_2:
       return "GovernmentEpisode2";
-    case QuestCategory::GovernmentEpisode4:
+    case QuestCategory::GOVERNMENT_EPISODE_4:
       return "GovernmentEpisode4";
-    case QuestCategory::Download:
+    case QuestCategory::DOWNLOAD:
       return "Download";
-    case QuestCategory::Battle:
+    case QuestCategory::BATTLE:
       return "Battle";
-    case QuestCategory::Challenge:
+    case QuestCategory::CHALLENGE:
       return "Challenge";
-    case QuestCategory::Solo:
+    case QuestCategory::SOLO:
       return "Solo";
-    case QuestCategory::Episode3:
+    case QuestCategory::EPISODE_3:
       return "Episode3";
     default:
       return "Unknown";
@@ -77,10 +78,10 @@ struct PSOQuestHeaderDC { // same for dc v1 and v2, thankfully
   uint8_t is_download;
   uint8_t unknown1;
   uint16_t quest_number; // 0xFFFF for challenge quests
-  char name[0x20];
-  char short_description[0x80];
-  char long_description[0x120];
-};
+  ptext<char, 0x20> name;
+  ptext<char, 0x80> short_description;
+  ptext<char, 0x120> long_description;
+} __attribute__((packed));
 
 struct PSOQuestHeaderPC {
   uint32_t start_offset;
@@ -90,10 +91,10 @@ struct PSOQuestHeaderPC {
   uint8_t is_download;
   uint8_t unknown1;
   uint16_t quest_number; // 0xFFFF for challenge quests
-  char16_t name[0x20];
-  char16_t short_description[0x80];
-  char16_t long_description[0x120];
-};
+  ptext<char16_t, 0x20> name;
+  ptext<char16_t, 0x80> short_description;
+  ptext<char16_t, 0x120> long_description;
+} __attribute__((packed));
 
 struct PSOQuestHeaderGC {
   uint32_t start_offset;
@@ -104,22 +105,22 @@ struct PSOQuestHeaderGC {
   uint8_t unknown1;
   uint8_t quest_number;
   uint8_t episode; // 1 = ep2. apparently some quests have 0xFF here, which means ep1 (?)
-  char name[0x20];
-  char short_description[0x80];
-  char long_description[0x120];
-};
+  ptext<char, 0x20> name;
+  ptext<char, 0x80> short_description;
+  ptext<char, 0x120> long_description;
+} __attribute__((packed));
 
 struct PSOQuestHeaderGCEpisode3 {
   // there's actually a lot of other important stuff in here but I'm lazy. it
   // looks like map data, cutscene data, and maybe special cards used during
   // the quest
-  uint8_t unused[0x1DF0];
-  char name[0x14];
-  char location[0x14];
-  char location2[0x3C];
-  char description[0x190];
-  uint8_t unused2[0x3A34];
-};
+  parray<uint8_t, 0x1DF0> unknown_a1;
+  ptext<char, 0x14> name;
+  ptext<char, 0x14> location;
+  ptext<char, 0x3C> location2;
+  ptext<char, 0x190> description;
+  parray<uint8_t, 0x3A34> unknown_a2;
+} __attribute__((packed));
 
 struct PSOQuestHeaderBB {
   uint32_t start_offset;
@@ -132,24 +133,31 @@ struct PSOQuestHeaderBB {
   uint8_t max_players;
   uint8_t joinable_in_progress;
   uint8_t unknown;
-  char16_t name[0x20];
-  char16_t short_description[0x80];
-  char16_t long_description[0x120];
-};
+  ptext<char16_t, 0x20> name;
+  ptext<char16_t, 0x80> short_description;
+  ptext<char16_t, 0x120> long_description;
+} __attribute__((packed));
 
 
 
 Quest::Quest(const string& bin_filename)
-  : quest_id(-1),
-    category(QuestCategory::Unknown),
+  : internal_id(-1),
+    menu_item_id(0),
+    category(QuestCategory::UNKNOWN),
     episode(0),
     is_dcv1(false),
     joinable(false),
-    gci_format(false) {
+    file_format(FileFormat::BIN_DAT) {
 
   if (ends_with(bin_filename, ".bin.gci")) {
-    this->gci_format = true;
+    this->file_format = FileFormat::BIN_DAT_GCI;
     this->file_basename = bin_filename.substr(0, bin_filename.size() - 8);
+  } else if (ends_with(bin_filename, ".bin.dlq")) {
+    this->file_format = FileFormat::BIN_DAT_DLQ;
+    this->file_basename = bin_filename.substr(0, bin_filename.size() - 8);
+  } else if (ends_with(bin_filename, ".qst")) {
+    this->file_format = FileFormat::QST;
+    this->file_basename = bin_filename.substr(0, bin_filename.size() - 4);
   } else if (ends_with(bin_filename, ".bin")) {
     this->file_basename = bin_filename.substr(0, bin_filename.size() - 4);
   } else {
@@ -165,7 +173,9 @@ Quest::Quest(const string& bin_filename)
       basename = bin_filename;
     }
   }
-  basename.resize(basename.size() - (this->gci_format ? 8 : 4));
+  bool has_short_extension = (this->file_format == FileFormat::BIN_DAT) ||
+      (this->file_format == FileFormat::QST);
+  basename.resize(basename.size() - (has_short_extension ? 4 : 8));
 
   // quest filenames are like:
   // b###-VV.bin for battle mode
@@ -178,11 +188,11 @@ Quest::Quest(const string& bin_filename)
   }
 
   if (basename[0] == 'b') {
-    this->category = QuestCategory::Battle;
+    this->category = QuestCategory::BATTLE;
   } else if (basename[0] == 'c') {
-    this->category = QuestCategory::Challenge;
+    this->category = QuestCategory::CHALLENGE;
   } else if (basename[0] == 'e') {
-    this->category = QuestCategory::Episode3;
+    this->category = QuestCategory::EPISODE_3;
   } else if (basename[0] != 'q') {
     throw invalid_argument("filename does not indicate mode");
   }
@@ -190,38 +200,29 @@ Quest::Quest(const string& bin_filename)
   // if the quest category is still unknown, expect 3 tokens (one of them will
   // tell us the category)
   vector<string> tokens = split(basename, '-');
-  if (tokens.size() != (2 + (this->category == QuestCategory::Unknown))) {
+  if (tokens.size() != (2 + (this->category == QuestCategory::UNKNOWN))) {
     throw invalid_argument("incorrect filename format");
   }
 
   // parse the number out of the first token
-  this->quest_id = strtoull(tokens[0].c_str() + 1, nullptr, 10);
+  this->internal_id = strtoull(tokens[0].c_str() + 1, nullptr, 10);
 
   // get the category from the second token if needed
-  if (this->category == QuestCategory::Unknown) {
-    if (tokens[1] == "gov") {
-      if (this->episode == 0) {
-        this->category = QuestCategory::GovernmentEpisode1;
-      } else if (this->episode == 1) {
-        this->category = QuestCategory::GovernmentEpisode2;
-      } else if (this->episode == 2) {
-        this->category = QuestCategory::GovernmentEpisode4;
-      } else {
-        throw invalid_argument("government quest has incorrect episode");
-      }
-    } else {
-      static const unordered_map<std::string, QuestCategory> name_to_category({
-        {"ret", QuestCategory::Retrieval},
-        {"ext", QuestCategory::Extermination},
-        {"evt", QuestCategory::Event},
-        {"shp", QuestCategory::Shop},
-        {"vr",  QuestCategory::VR},
-        {"twr", QuestCategory::Tower},
-        {"dl",  QuestCategory::Download},
-        {"1p",  QuestCategory::Solo},
-      });
-      this->category = name_to_category.at(tokens[1]);
-    }
+  if (this->category == QuestCategory::UNKNOWN) {
+    static const unordered_map<std::string, QuestCategory> name_to_category({
+      {"ret", QuestCategory::RETRIEVAL},
+      {"ext", QuestCategory::EXTERMINATION},
+      {"evt", QuestCategory::EVENT},
+      {"shp", QuestCategory::SHOP},
+      {"vr",  QuestCategory::VR},
+      {"twr", QuestCategory::TOWER},
+      // Note: This will be overwritten later for Episode 2 & 4 quests - we
+      // haven't parsed the episode from the quest script yet
+      {"gov", QuestCategory::GOVERNMENT_EPISODE_1},
+      {"dl",  QuestCategory::DOWNLOAD},
+      {"1p",  QuestCategory::SOLO},
+    });
+    this->category = name_to_category.at(tokens[1]);
     tokens.erase(tokens.begin() + 1);
   }
 
@@ -242,7 +243,7 @@ Quest::Quest(const string& bin_filename)
   auto bin_decompressed = prs_decompress(*bin_compressed);
 
   switch (this->version) {
-    case GameVersion::Patch:
+    case GameVersion::PATCH:
       throw invalid_argument("patch server quests are not valid");
       break;
 
@@ -274,7 +275,7 @@ Quest::Quest(const string& bin_filename)
     }
 
     case GameVersion::GC: {
-      if (this->category == QuestCategory::Episode3) {
+      if (this->category == QuestCategory::EPISODE_3) {
         // these all appear to be the same size
         if (bin_decompressed.size() != sizeof(PSOQuestHeaderGCEpisode3)) {
           throw invalid_argument("file is incorrect size");
@@ -309,9 +310,20 @@ Quest::Quest(const string& bin_filename)
       this->name = header->name;
       this->short_description = header->short_description;
       this->long_description = header->long_description;
+      if (this->category == QuestCategory::GOVERNMENT_EPISODE_1) {
+        if (this->episode == 1) {
+          this->category = QuestCategory::GOVERNMENT_EPISODE_2;
+        } else if (this->episode == 2) {
+          this->category = QuestCategory::GOVERNMENT_EPISODE_4;
+        } else if (this->episode != 0) {
+          throw invalid_argument("government quest has invalid episode number");
+        }
+      }
       break;
-
     }
+
+    default:
+      throw logic_error("invalid quest game version");
   }
 }
 
@@ -333,10 +345,24 @@ std::string Quest::dat_filename() const {
 
 shared_ptr<const string> Quest::bin_contents() const {
   if (!this->bin_contents_ptr) {
-    if (this->gci_format) {
-      this->bin_contents_ptr.reset(new string(this->decode_gci(this->file_basename + ".bin.gci")));
-    } else {
-      this->bin_contents_ptr.reset(new string(load_file(this->file_basename + ".bin")));
+    switch (this->file_format) {
+      case FileFormat::BIN_DAT:
+        this->bin_contents_ptr.reset(new string(load_file(this->file_basename + ".bin")));
+        break;
+      case FileFormat::BIN_DAT_GCI:
+        this->bin_contents_ptr.reset(new string(this->decode_gci(this->file_basename + ".bin.gci")));
+        break;
+      case FileFormat::BIN_DAT_DLQ:
+        this->bin_contents_ptr.reset(new string(this->decode_dlq(this->file_basename + ".bin.dlq")));
+        break;
+      case FileFormat::QST: {
+        auto result = this->decode_qst(this->file_basename + ".qst");
+        this->bin_contents_ptr.reset(new string(move(result.first)));
+        this->dat_contents_ptr.reset(new string(move(result.second)));
+        break;
+      }
+      default:
+        throw logic_error("invalid quest file format");
     }
   }
   return this->bin_contents_ptr;
@@ -344,10 +370,24 @@ shared_ptr<const string> Quest::bin_contents() const {
 
 shared_ptr<const string> Quest::dat_contents() const {
   if (!this->dat_contents_ptr) {
-    if (this->gci_format) {
-      this->dat_contents_ptr.reset(new string(this->decode_gci(this->file_basename + ".dat.gci")));
-    } else {
-      this->dat_contents_ptr.reset(new string(load_file(this->file_basename + ".dat")));
+    switch (this->file_format) {
+      case FileFormat::BIN_DAT:
+        this->dat_contents_ptr.reset(new string(load_file(this->file_basename + ".dat")));
+        break;
+      case FileFormat::BIN_DAT_GCI:
+        this->dat_contents_ptr.reset(new string(this->decode_gci(this->file_basename + ".dat.gci")));
+        break;
+      case FileFormat::BIN_DAT_DLQ:
+        this->dat_contents_ptr.reset(new string(this->decode_dlq(this->file_basename + ".dat.dlq")));
+        break;
+      case FileFormat::QST: {
+        auto result = this->decode_qst(this->file_basename + ".qst");
+        this->bin_contents_ptr.reset(new string(move(result.first)));
+        this->dat_contents_ptr.reset(new string(move(result.second)));
+        break;
+      }
+      default:
+        throw logic_error("invalid quest file format");
     }
   }
   return this->dat_contents_ptr;
@@ -375,7 +415,7 @@ string Quest::decode_gci(const string& filename) {
     uint32_t unknown2;
     uint32_t decompressed_size;
     uint32_t unknown4;
-  };
+  } __attribute__((packed));
   if (compressed_data_with_header.size() < sizeof(DecryptedHeader)) {
     throw runtime_error("GCI file compressed data truncated during header");
   }
@@ -400,12 +440,154 @@ string Quest::decode_gci(const string& filename) {
   return data_to_decompress;
 }
 
+string Quest::decode_dlq(const string& filename) {
+  uint32_t decompressed_size;
+  uint32_t key;
+  string data;
+  {
+    auto f = fopen_unique(filename, "rb");
+    decompressed_size = freadx<le_uint32_t>(f.get());
+    key = freadx<le_uint32_t>(f.get());
+    data = read_all(f.get());
+  }
+
+  PSOPCEncryption encr(key);
+
+  // The compressed data size does not need to be a multiple of 4, but the PC
+  // encryption (which is used for all download quests, even in V3) requires the
+  // data size to be a multiple of 4. We'll just temporarily stick a few bytes
+  // on the end, then throw them away later if needed.
+  size_t original_size = data.size();
+  data.resize((data.size() + 3) & (~3));
+  encr.decrypt(data);
+  data.resize(original_size);
+
+  if (prs_decompress_size(data) != decompressed_size) {
+    throw runtime_error("decompressed size does not match size in header");
+  }
+
+  return data;
+}
+
+template <typename HeaderT, typename OpenFileT>
+static pair<string, string> decode_qst_t(FILE* f) {
+  string qst_data = read_all(f);
+  StringReader r(qst_data);
+
+  string bin_contents;
+  string dat_contents;
+  string internal_bin_filename;
+  string internal_dat_filename;
+  uint32_t bin_file_size = 0;
+  uint32_t dat_file_size = 0;
+  while (!r.eof()) {
+    // Handle BB's implicit 8-byte command alignment
+    static constexpr size_t alignment = sizeof(HeaderT);
+    size_t next_command_offset = (r.where() + (alignment - 1)) & ~(alignment - 1);
+    r.go(next_command_offset);
+    if (r.eof()) {
+      break;
+    }
+
+    const auto& header = r.get<HeaderT>();
+    if (header.command == 0x44) {
+      if (header.size != sizeof(HeaderT) + sizeof(OpenFileT)) {
+        throw runtime_error("qst open file command has incorrect size");
+      }
+      const auto& cmd = r.get<OpenFileT>(f);
+      string internal_filename = cmd.filename;
+
+      if (ends_with(internal_filename, ".bin")) {
+        if (internal_bin_filename.empty()) {
+          internal_bin_filename = internal_filename;
+        } else {
+          throw runtime_error("qst contains multiple bin files");
+        }
+        bin_file_size = cmd.file_size;
+
+      } else if (ends_with(internal_filename, ".dat")) {
+        if (internal_dat_filename.empty()) {
+          internal_dat_filename = internal_filename;
+        } else {
+          throw runtime_error("qst contains multiple dat files");
+        }
+        dat_file_size = cmd.file_size;
+
+      } else {
+        throw runtime_error("qst contains non-bin, non-dat file");
+      }
+
+    } else if (header.command == 0x13) {
+      if (header.size != sizeof(HeaderT) + sizeof(S_WriteFile_13_A7)) {
+        throw runtime_error("qst write file command has incorrect size");
+      }
+      const auto& cmd = r.get<S_WriteFile_13_A7>();
+      string filename = cmd.filename;
+
+      string* dest = nullptr;
+      if (filename == internal_bin_filename) {
+        dest = &bin_contents;
+      } else if (filename == internal_dat_filename) {
+        dest = &dat_contents;
+      } else {
+        throw runtime_error("qst contains write commnd for non-open file");
+      }
+
+      if (cmd.data_size > 0x400) {
+        throw runtime_error("qst contains invalid write command");
+      }
+      if (dest->size() & 0x3FF) {
+        throw runtime_error("qst contains uneven chunks out of order");
+      }
+      if (header.flag != dest->size() / 0x400) {
+        throw runtime_error("qst contains chunks out of order");
+      }
+      dest->append(reinterpret_cast<const char*>(cmd.data), cmd.data_size);
+
+    } else {
+      throw runtime_error("invalid command in qst file");
+    }
+  }
+
+  if (bin_contents.size() != bin_file_size) {
+    throw runtime_error("bin file does not match expected size");
+  }
+  if (dat_contents.size() != dat_file_size) {
+    throw runtime_error("dat file does not match expected size");
+  }
+
+  return make_pair(bin_contents, dat_contents);
+}
+
+pair<string, string> Quest::decode_qst(const string& filename) {
+  auto f = fopen_unique(filename, "rb");
+
+  // qst files start with an open file command, but the format differs depending
+  // on the PSO version that the qst file is for. We can detect the format from
+  // the first 4 bytes in the file:
+  // - BB: 58 00 44 00
+  // - PC: 3C ?? 44 00
+  // - DC/GC: 44 ?? 3C 00
+  uint32_t signature = freadx<be_uint32_t>(f.get());
+  fseek(f.get(), 0, SEEK_SET);
+  if (signature == 0x58004400) {
+    return decode_qst_t<PSOCommandHeaderBB, S_OpenFile_BB_44_A6>(f.get());
+  } else if ((signature & 0xFF00FFFF) == 0x3C004400) {
+    return decode_qst_t<PSOCommandHeaderPC, S_OpenFile_PC_GC_44_A6>(f.get());
+  } else if ((signature & 0xFF00FFFF) == 0x44003C00) {
+    return decode_qst_t<PSOCommandHeaderDCGC, S_OpenFile_PC_GC_44_A6>(f.get());
+  } else {
+    throw runtime_error("invalid qst file format");
+  }
+}
 
 
-QuestIndex::QuestIndex(const char* directory) : directory(directory) {
+
+QuestIndex::QuestIndex(const std::string& directory) : directory(directory) {
   auto filename_set = list_directory(this->directory);
   vector<string> filenames(filename_set.begin(), filename_set.end());
   sort(filenames.begin(), filenames.end());
+  uint32_t next_menu_item_id = 1;
   for (const auto& filename : filenames) {
     string full_path = this->directory + "/" + filename;
 
@@ -415,15 +597,27 @@ QuestIndex::QuestIndex(const char* directory) : directory(directory) {
       continue;
     }
 
-    if (ends_with(filename, ".bin") || ends_with(filename, ".bin.gci")) {
+    if (ends_with(filename, ".bin") ||
+        ends_with(filename, ".bin.gci") ||
+        ends_with(filename, ".bin.dlq") ||
+        ends_with(filename, ".qst")) {
       try {
         shared_ptr<Quest> q(new Quest(full_path));
-        this->version_id_to_quest.emplace(make_pair(q->version, q->quest_id), q);
-        this->version_name_to_quest.emplace(make_pair(q->version, q->name), q);
+        q->menu_item_id = next_menu_item_id++;
         string ascii_name = encode_sjis(q->name);
-        log(INFO, "Indexed quest %s (%s-%" PRId64 ", %s, episode=%hhu, joinable=%s, dcv1=%s)",
-            ascii_name.c_str(), name_for_version(q->version), q->quest_id,
-            name_for_category(q->category), q->episode,
+        if (!this->version_menu_item_id_to_quest.emplace(
+            make_pair(q->version, q->menu_item_id), q).second) {
+          throw logic_error("duplicate quest menu item id");
+        }
+        if (!this->version_name_to_quest.emplace(
+            make_pair(q->version, q->name), q).second) {
+          throw runtime_error(string_printf(
+              "duplicate quest name (%s-%" PRId64 "): %s",
+              name_for_version(q->version), q->internal_id, ascii_name.c_str()));
+        }
+        log(INFO, "Indexed quest %s (%s-%" PRId64 " => %" PRIu32 ", %s, episode=%hhu, joinable=%s, dcv1=%s)",
+            ascii_name.c_str(), name_for_version(q->version), q->internal_id,
+            q->menu_item_id, name_for_category(q->category), q->episode,
             q->joinable ? "true" : "false", q->is_dcv1 ? "true" : "false");
       } catch (const exception& e) {
         log(WARNING, "Failed to parse quest file %s (%s)", filename.c_str(), e.what());
@@ -433,8 +627,8 @@ QuestIndex::QuestIndex(const char* directory) : directory(directory) {
 }
 
 shared_ptr<const Quest> QuestIndex::get(GameVersion version,
-    uint32_t id) const {
-  return this->version_id_to_quest.at(make_pair(version, id));
+    uint32_t menu_item_id) const {
+  return this->version_menu_item_id_to_quest.at(make_pair(version, menu_item_id));
 }
 
 shared_ptr<const string> QuestIndex::get_gba(const string& name) const {
@@ -442,9 +636,9 @@ shared_ptr<const string> QuestIndex::get_gba(const string& name) const {
 }
 
 vector<shared_ptr<const Quest>> QuestIndex::filter(GameVersion version,
-    bool is_dcv1, QuestCategory category, int16_t episode) const {
-  auto it = this->version_id_to_quest.lower_bound(make_pair(version, 0));
-  auto end_it = this->version_id_to_quest.upper_bound(make_pair(version, 0xFFFFFFFF));
+    bool is_dcv1, QuestCategory category) const {
+  auto it = this->version_menu_item_id_to_quest.lower_bound(make_pair(version, 0));
+  auto end_it = this->version_menu_item_id_to_quest.upper_bound(make_pair(version, 0xFFFFFFFF));
 
   vector<shared_ptr<const Quest>> ret;
   for (; it != end_it; it++) {
@@ -452,14 +646,6 @@ vector<shared_ptr<const Quest>> QuestIndex::filter(GameVersion version,
     if ((q->is_dcv1 != is_dcv1) || (q->category != category)) {
       continue;
     }
-
-    // Only check episode and solo if the category isn't a mode (that is, ignore
-    // episode if querying for battle/challenge/solo quests). Also, ignore
-    // ignore episode if it's < 0 (e.g. for the download quest menu).
-    if ((episode >= 0) && !category_is_mode(category) && ((q->episode != episode))) {
-      continue;
-    }
-
     ret.emplace_back(q);
   }
 
@@ -469,38 +655,60 @@ vector<shared_ptr<const Quest>> QuestIndex::filter(GameVersion version,
 
 
 static string create_download_quest_file(const string& compressed_data,
-    size_t decompressed_size) {
+    size_t decompressed_size, uint32_t encryption_seed = 0) {
+  // Download quest files are like normal (PRS-compressed) quest files, but they
+  // are encrypted with the PSOPC encryption (even on V3 / PSO GC), and a small
+  // header (PSODownloadQuestHeader) is prepended to the encrypted data.
+
+  if (encryption_seed == 0) {
+    encryption_seed = random_object<uint32_t>();
+  }
+
   string data(8, '\0');
   auto* header = reinterpret_cast<PSODownloadQuestHeader*>(data.data());
-  header->size = decompressed_size + sizeof(PSODownloadQuestHeader);
-  header->encryption_seed = random_object<uint32_t>();
+  header->size = decompressed_size;
+  header->encryption_seed = encryption_seed;
   data += compressed_data;
 
-  // add extra bytes if necessary so encryption won't fail
+  // Add temporary extra bytes if necessary so encryption won't fail - the data
+  // size must be a multiple of 4 for PSO PC encryption.
+  size_t original_size = data.size();
   data.resize((data.size() + 3) & (~3));
 
-  // TODO: for DC quests, do we use DC encryption?
-  PSOPCEncryption encr(header->encryption_seed);
+  PSOPCEncryption encr(encryption_seed);
   encr.encrypt(data.data() + sizeof(PSODownloadQuestHeader),
       data.size() - sizeof(PSODownloadQuestHeader));
+  data.resize(original_size);
+
   return data;
 }
 
 shared_ptr<Quest> Quest::create_download_quest() const {
-  if (this->category == QuestCategory::Download) {
-    throw invalid_argument("quest is already a download quest");
-  }
+  // The download flag needs to be set in the bin header, or else the client
+  // will ignore it when scanning for download quests in an offline game. To set
+  // this flag, we need to decompress the quest's .bin file, set the flag, then
+  // recompress it again.
 
   string decompressed_bin = prs_decompress(*this->bin_contents());
+
   void* data_ptr = decompressed_bin.data();
   switch (this->version) {
     case GameVersion::DC:
+      if (decompressed_bin.size() < sizeof(PSOQuestHeaderDC)) {
+        throw runtime_error("bin file is too small for header");
+      }
       reinterpret_cast<PSOQuestHeaderDC*>(data_ptr)->is_download = 0x01;
       break;
     case GameVersion::PC:
+      if (decompressed_bin.size() < sizeof(PSOQuestHeaderPC)) {
+        throw runtime_error("bin file is too small for header");
+      }
       reinterpret_cast<PSOQuestHeaderPC*>(data_ptr)->is_download = 0x01;
       break;
     case GameVersion::GC:
+      if (decompressed_bin.size() < sizeof(PSOQuestHeaderGC)) {
+        throw runtime_error("bin file is too small for header");
+      }
       reinterpret_cast<PSOQuestHeaderGC*>(data_ptr)->is_download = 0x01;
       break;
     case GameVersion::BB:
@@ -509,15 +717,14 @@ shared_ptr<Quest> Quest::create_download_quest() const {
       throw invalid_argument("unknown game version");
   }
 
+  string compressed_bin = prs_compress(decompressed_bin);
+
+  // We'll create a new Quest object with appropriately-processed .bin and .dat
+  // file contents.
   shared_ptr<Quest> dlq(new Quest(*this));
-  dlq->category = QuestCategory::Download;
-
   dlq->bin_contents_ptr.reset(new string(create_download_quest_file(
-      prs_compress(decompressed_bin), decompressed_bin.size())));
-
-  auto dat_contents = this->dat_contents();
+      compressed_bin, decompressed_bin.size())));
   dlq->dat_contents_ptr.reset(new string(create_download_quest_file(
-      *dat_contents, prs_decompress_size(*dat_contents))));
-
+      *this->dat_contents(), prs_decompress_size(*this->dat_contents()))));
   return dlq;
 }

src/Quest.hh

@@ -12,21 +12,21 @@
 
 
 enum class QuestCategory {
-  Unknown = -1,
-  Retrieval = 0,
-  Extermination,
-  Event,
-  Shop,
+  UNKNOWN = -1,
+  RETRIEVAL = 0,
+  EXTERMINATION,
+  EVENT,
+  SHOP,
   VR,
-  Tower,
-  GovernmentEpisode1,
-  GovernmentEpisode2,
-  GovernmentEpisode4,
-  Download,
-  Battle,
-  Challenge,
-  Solo,
-  Episode3,
+  TOWER,
+  GOVERNMENT_EPISODE_1,
+  GOVERNMENT_EPISODE_2,
+  GOVERNMENT_EPISODE_4,
+  DOWNLOAD,
+  BATTLE,
+  CHALLENGE,
+  SOLO,
+  EPISODE_3,
 };
 
 bool category_is_mode(QuestCategory category);
@@ -37,16 +37,25 @@ const char* name_for_category(QuestCategory category);
 class Quest {
 private:
   static std::string decode_gci(const std::string& filename);
+  static std::string decode_dlq(const std::string& filename);
+  static std::pair<std::string, std::string> decode_qst(const std::string& filename);
 
 public:
-  int64_t quest_id;
+  enum class FileFormat {
+    BIN_DAT = 0,
+    BIN_DAT_GCI,
+    BIN_DAT_DLQ,
+    QST,
+  };
+  int64_t internal_id;
+  uint32_t menu_item_id;
   QuestCategory category;
   uint8_t episode; // 0 = ep1, 1 = ep2, 2 = ep4, 0xFF = ep3
   bool is_dcv1;
   bool joinable;
   GameVersion version;
   std::string file_basename; // we append -<version>.<bin/dat> when reading
-  bool gci_format;
+  FileFormat file_format;
   std::u16string name;
   std::u16string short_description;
   std::u16string long_description;
@@ -73,17 +82,17 @@ public:
 struct QuestIndex {
   std::string directory;
 
-  std::map<std::pair<GameVersion, uint64_t>, std::shared_ptr<Quest>> version_id_to_quest;
+  std::map<std::pair<GameVersion, uint64_t>, std::shared_ptr<Quest>> version_menu_item_id_to_quest;
   std::map<std::pair<GameVersion, std::u16string>, std::shared_ptr<Quest>> version_name_to_quest;
 
   std::map<std::string, std::vector<std::shared_ptr<Quest>>> category_to_quests;
 
   std::map<std::string, std::shared_ptr<std::string>> gba_file_contents;
 
-  QuestIndex(const char* directory);
+  QuestIndex(const std::string& directory);
 
   std::shared_ptr<const Quest> get(GameVersion version, uint32_t id) const;
   std::shared_ptr<const std::string> get_gba(const std::string& name) const;
   std::vector<std::shared_ptr<const Quest>> filter(GameVersion version,
-    bool is_dcv1, QuestCategory category, int16_t episode) const;
+    bool is_dcv1, QuestCategory category) const;
 };

src/RareItemSet.hh

@@ -7,9 +7,10 @@
 struct RareItemDrop {
   uint8_t probability;
   uint8_t item_code[3];
-};
+} __attribute__((packed));
 
 struct RareItemSet {
+  // 0x280 in size; describes one difficulty, section ID, and episode
   RareItemDrop rares[0x65];     // 0000 - 0194 in file
   uint8_t box_areas[0x1E];      // 0194 - 01B2 in file
   RareItemDrop box_rares[0x1E]; // 01B2 - 022A in file
@@ -17,6 +18,6 @@ struct RareItemSet {
 
   RareItemSet(const char* filename, uint8_t episode, uint8_t difficulty,
   	  uint8_t secid);
-}; // 0x280 in size; describes one difficulty, section ID, and episode
+} __attribute__((packed));
 
 bool sample_rare_item(uint8_t pc);

src/ReceiveCommands.hh

@@ -5,8 +5,9 @@
 #include "ServerState.hh"
 
 
+
 void process_connect(std::shared_ptr<ServerState> s, std::shared_ptr<Client> c);
 void process_disconnect(std::shared_ptr<ServerState> s,
     std::shared_ptr<Client> c);
 void process_command(std::shared_ptr<ServerState> s, std::shared_ptr<Client> c,
-    uint16_t command, uint32_t flag, uint16_t size, const void* data);
+    uint16_t command, uint32_t flag, const std::string& data);

src/ReceiveSubcommands.hh

@@ -7,8 +7,9 @@
 #include "ServerState.hh"
 
 
-void check_size(uint16_t size, uint16_t min_size, uint16_t max_size = 0);
 
 void process_subcommand(std::shared_ptr<ServerState> s,
     std::shared_ptr<Lobby> l, std::shared_ptr<Client> c, uint8_t command,
-    uint8_t flag, const PSOSubcommand* sub, size_t count);
+    uint8_t flag, const std::string& data);
+
+bool subcommand_is_implemented(uint8_t which);

src/SendCommands.hh

@@ -13,68 +13,90 @@
 #include "Menu.hh"
 #include "Quest.hh"
 #include "Text.hh"
+#include "CommandFormats.hh"
 
 
 
-#define MAIN_MENU_ID           0x60000000
-#define INFORMATION_MENU_ID    0x60000030
-#define LOBBY_MENU_ID          0x60000060
-#define GAME_MENU_ID           0x60000090
-#define QUEST_MENU_ID          0x600000C0
-#define QUEST_FILTER_MENU_ID   0x600000F0
-
-#define MAIN_MENU_GO_TO_LOBBY      0x00000001
-#define MAIN_MENU_INFORMATION      0x00000002
-#define MAIN_MENU_DOWNLOAD_QUESTS  0x00000003
-#define MAIN_MENU_DISCONNECT       0x00000004
-#define INFORMATION_MENU_GO_BACK   0xFFFFFFFF
-
+// Note: There are so many versions of this function for a few reasons:
+// - There are a lot of different target types (sometimes we want to send a
+//   command to one client, sometimes to everyone in a lobby, etc.)
+// - For the const void* versions, the data and size arguments should not be
+//   independently optional - this can lead to bugs where a non-null data
+//   pointer is given but size is accidentally not given zero (e.g. if the type
+//   of data in the calling function is changed from string to void*).
 
+void send_command(struct bufferevent* bev, GameVersion version,
+    PSOEncryption* crypt, uint16_t command, uint32_t flag, const void* data,
+    size_t size, const char* name_str = nullptr);
 
 void send_command(std::shared_ptr<Client> c, uint16_t command,
-    uint32_t flag = 0, const void* data = nullptr, size_t size = 0);
+    uint32_t flag, const void* data, size_t size);
+
+inline void send_command(std::shared_ptr<Client> c, uint16_t command,
+    uint32_t flag) {
+  send_command(c, command, flag, nullptr, 0);
+}
 
 void send_command_excluding_client(std::shared_ptr<Lobby> l,
-    std::shared_ptr<Client> c, uint16_t command, uint32_t flag = 0,
-    const void* data = nullptr, size_t size = 0);
+    std::shared_ptr<Client> c, uint16_t command, uint32_t flag,
+    const void* data, size_t size);
 
-void send_command(std::shared_ptr<Lobby> l, uint16_t command, uint32_t flag = 0,
-    const void* data = nullptr, size_t size = 0);
+inline void send_command_excluding_client(std::shared_ptr<Lobby> l,
+    std::shared_ptr<Client> c, uint16_t command, uint32_t flag) {
+  send_command_excluding_client(l, c, command, flag, nullptr, 0);
+}
+
+void send_command(std::shared_ptr<Lobby> l, uint16_t command, uint32_t flag,
+    const void* data, size_t size);
+
+inline void send_command(std::shared_ptr<Lobby> l, uint16_t command, uint32_t flag) {
+  send_command(l, command, flag, nullptr, 0);
+}
 
 void send_command(std::shared_ptr<ServerState> s, uint16_t command,
-    uint32_t flag = 0, const void* data = nullptr, size_t size = 0);
+    uint32_t flag, const void* data, size_t size);
 
-template <typename TARGET, typename STRUCT>
-void send_command(std::shared_ptr<TARGET> c, uint16_t command, uint32_t flag,
-    const STRUCT& data) {
+inline void send_command(std::shared_ptr<ServerState> s, uint16_t command,
+    uint32_t flag) {
+  send_command(s, command, flag, nullptr, 0);
+}
+
+template <typename TargetT, typename StructT>
+static void send_command_t(std::shared_ptr<TargetT> c, uint16_t command,
+    uint32_t flag, const StructT& data) {
   send_command(c, command, flag, &data, sizeof(data));
 }
 
-template <typename TARGET>
-void send_command(std::shared_ptr<TARGET> c, uint16_t command, uint32_t flag,
-    const std::string& data) {
+template <typename TargetT>
+static void send_command(std::shared_ptr<TargetT> c, uint16_t command,
+    uint32_t flag, const std::string& data) {
   send_command(c, command, flag, data.data(), data.size());
 }
 
-template <typename TARGET, typename STRUCT>
-void send_command(std::shared_ptr<TARGET> c, uint16_t command, uint32_t flag,
-    const std::vector<STRUCT>& data) {
-  send_command(c, command, flag, data.data(), data.size() * sizeof(STRUCT));
+template <typename TargetT, typename StructT>
+void send_command_vt(std::shared_ptr<TargetT> c, uint16_t command,
+    uint32_t flag, const std::vector<StructT>& data) {
+  send_command(c, command, flag, data.data(), data.size() * sizeof(StructT));
 }
 
-template <typename TARGET, typename STRUCT, typename ENTRY>
-void send_command(std::shared_ptr<TARGET> c, uint16_t command, uint32_t flag,
-    const STRUCT& data, const std::vector<ENTRY>& array_data) {
-  std::string all_data(reinterpret_cast<const char*>(&data), sizeof(STRUCT));
+template <typename TargetT, typename StructT, typename EntryT>
+void send_command_t_vt(std::shared_ptr<TargetT> c, uint16_t command,
+    uint32_t flag, const StructT& data, const std::vector<EntryT>& array_data) {
+  std::string all_data(reinterpret_cast<const char*>(&data), sizeof(StructT));
   all_data.append(reinterpret_cast<const char*>(array_data.data()),
-      array_data.size() * sizeof(ENTRY));
+      array_data.size() * sizeof(EntryT));
   send_command(c, command, flag, all_data.data(), all_data.size());
 }
 
 
 
-
-
+S_ServerInit_DC_PC_GC_02_17 prepare_server_init_contents_dc_pc_gc(
+    bool initial_connection,
+    uint32_t server_key,
+    uint32_t client_key);
+S_ServerInit_BB_03 prepare_server_init_contents_bb(
+    const parray<uint8_t, 0x30>& server_key,
+    const parray<uint8_t, 0x30>& client_key);
 void send_server_init(std::shared_ptr<ServerState> s, std::shared_ptr<Client> c,
     bool initial_connection);
 void send_update_client_config(std::shared_ptr<Client> c);
@@ -90,28 +112,29 @@ void send_player_preview_bb(std::shared_ptr<Client> c, uint8_t player_index,
 void send_accept_client_checksum_bb(std::shared_ptr<Client> c);
 void send_guild_card_header_bb(std::shared_ptr<Client> c);
 void send_guild_card_chunk_bb(std::shared_ptr<Client> c, size_t chunk_index);
-void send_stream_file_bb(std::shared_ptr<Client> c);
+void send_stream_file_index_bb(std::shared_ptr<Client> c);
+void send_stream_file_chunk_bb(std::shared_ptr<Client> c, uint32_t chunk_index);
 void send_approve_player_choice_bb(std::shared_ptr<Client> c);
 void send_complete_player_bb(std::shared_ptr<Client> c);
 
-void send_check_directory_patch(std::shared_ptr<Client> c, const char* dir);
+void send_enter_directory_patch(std::shared_ptr<Client> c, const std::string& dir);
 
-void send_message_box(std::shared_ptr<Client> c, const char16_t* text);
-void send_lobby_name(std::shared_ptr<Client> c, const char16_t* text);
-void send_quest_info(std::shared_ptr<Client> c, const char16_t* text);
-void send_lobby_message_box(std::shared_ptr<Client> c, const char16_t* text);
-void send_ship_info(std::shared_ptr<Client> c, const char16_t* text);
-void send_text_message(std::shared_ptr<Client> c, const char16_t* text);
-void send_text_message(std::shared_ptr<Lobby> l, const char16_t* text);
-void send_text_message(std::shared_ptr<ServerState> l, const char16_t* text);
+void send_message_box(std::shared_ptr<Client> c, const std::u16string& text);
+void send_lobby_name(std::shared_ptr<Client> c, const std::u16string& text);
+void send_quest_info(std::shared_ptr<Client> c, const std::u16string& text);
+void send_lobby_message_box(std::shared_ptr<Client> c, const std::u16string& text);
+void send_ship_info(std::shared_ptr<Client> c, const std::u16string& text);
+void send_text_message(std::shared_ptr<Client> c, const std::u16string& text);
+void send_text_message(std::shared_ptr<Lobby> l, const std::u16string& text);
+void send_text_message(std::shared_ptr<ServerState> l, const std::u16string& text);
 void send_chat_message(std::shared_ptr<Client> c, uint32_t from_serial_number,
-    const char16_t* from_name, const char16_t* text);
+    const std::u16string& from_name, const std::u16string& text);
 void send_simple_mail(std::shared_ptr<Client> c, uint32_t from_serial_number,
-    const char16_t* from_name, const char16_t* text);
+    const std::u16string& from_name, const std::u16string& text);
 
-template <typename TARGET>
+template <typename TargetT>
 __attribute__((format(printf, 2, 3))) void send_text_message_printf(
-    std::shared_ptr<TARGET> t, const char* format, ...) {
+    std::shared_ptr<TargetT> t, const char* format, ...) {
   va_list va;
   va_start(va, format);
   std::string buf = string_vprintf(format, va);
@@ -122,11 +145,14 @@ __attribute__((format(printf, 2, 3))) void send_text_message_printf(
 
 void send_info_board(std::shared_ptr<Client> c, std::shared_ptr<Lobby> l);
 
-void send_card_search_result(std::shared_ptr<ServerState> s, std::shared_ptr<Client> c,
-    std::shared_ptr<Client> result, std::shared_ptr<Lobby> result_lobby);
+void send_card_search_result(
+    std::shared_ptr<ServerState> s,
+    std::shared_ptr<Client> c,
+    std::shared_ptr<Client> result,
+    std::shared_ptr<Lobby> result_lobby);
 
 void send_guild_card(std::shared_ptr<Client> c, std::shared_ptr<Client> source);
-void send_menu(std::shared_ptr<Client> c, const char16_t* menu_name,
+void send_menu(std::shared_ptr<Client> c, const std::u16string& menu_name,
     uint32_t menu_id, const std::vector<MenuItem>& items, bool is_info_menu);
 void send_game_menu(std::shared_ptr<Client> c, std::shared_ptr<ServerState> s);
 void send_quest_menu(std::shared_ptr<Client> c, uint32_t menu_id,
@@ -147,11 +173,11 @@ void send_resume_game(std::shared_ptr<Lobby> l,
     std::shared_ptr<Client> ready_client);
 
 enum PlayerStatsChange {
-  SubtractHP = 0,
-  SubtractTP = 1,
-  SubtractMeseta = 2,
-  AddHP = 3,
-  AddTP = 4,
+  SUBTRACT_HP = 0,
+  SUBTRACT_TP = 1,
+  SUBTRACT_MESETA = 2,
+  ADD_HP = 3,
+  ADD_TP = 4,
 };
 
 void send_player_stats_change(std::shared_ptr<Lobby> l, std::shared_ptr<Client> c,
@@ -164,9 +190,9 @@ void send_set_player_visibility(std::shared_ptr<Lobby> l,
 void send_revive_player(std::shared_ptr<Lobby> l, std::shared_ptr<Client> c);
 
 void send_drop_item(std::shared_ptr<Lobby> l, const ItemData& item,
-    bool from_enemy, uint8_t area, float x, float y, uint16_t request_id);
+    bool from_enemy, uint8_t area, float x, float z, uint16_t request_id);
 void send_drop_stacked_item(std::shared_ptr<Lobby> l, const ItemData& item,
-    uint8_t area, float x, float y);
+    uint8_t area, float x, float z);
 void send_pick_up_item(std::shared_ptr<Lobby> l, std::shared_ptr<Client> c, uint32_t id,
     uint8_t area);
 void send_create_inventory_item(std::shared_ptr<Lobby> l, std::shared_ptr<Client> c,
@@ -183,8 +209,16 @@ void send_ep3_rank_update(std::shared_ptr<Client> c);
 void send_ep3_map_list(std::shared_ptr<Lobby> l);
 void send_ep3_map_data(std::shared_ptr<Lobby> l, uint32_t map_id);
 
-void send_quest_file(std::shared_ptr<Client> c, const std::string& basename,
-    const std::string& contents, bool is_download_quest, bool is_ep3_quest);
+enum class QuestFileType {
+  ONLINE = 0,
+  DOWNLOAD,
+  EPISODE_3,
+  GBA_DEMO,
+};
+
+void send_quest_file(std::shared_ptr<Client> c, const std::string& quest_name,
+    const std::string& basename, const std::string& contents,
+    QuestFileType type);
 
 void send_server_time(std::shared_ptr<Client> c);
 

src/Server.cc

@@ -38,9 +38,9 @@ void Server::disconnect_client(shared_ptr<Client> c) {
 
   int fd = bufferevent_getfd(bev);
   if (fd < 0) {
-    log(INFO, "[Server] Client on virtual connection %p disconnected", bev);
+    this->log(INFO, "Client on virtual connection %p disconnected", bev);
   } else {
-    log(INFO, "[Server] Client on fd %d disconnected", fd);
+    this->log(INFO, "Client on fd %d disconnected", fd);
   }
 
   // if the output buffer is not empty, move the client into the draining pool
@@ -100,13 +100,14 @@ void Server::on_listen_accept(struct evconnlistener* listener,
   try {
     listening_socket = &this->listening_sockets.at(listen_fd);
   } catch (const out_of_range& e) {
-    log(WARNING, "[Server] Can\'t determine version for socket %d; disconnecting client",
+    this->log(WARNING, "Can\'t determine version for socket %d; disconnecting client",
         listen_fd);
     close(fd);
     return;
   }
 
-  log(INFO, "[Server] Client fd %d connected via fd %d", fd, listen_fd);
+  this->log(INFO, "Client fd %d connected via fd %d (%s)",
+      fd, listen_fd, listening_socket->name.c_str());
 
   struct bufferevent *bev = bufferevent_socket_new(this->base.get(), fd,
       BEV_OPT_CLOSE_ON_FREE | BEV_OPT_DEFER_CALLBACKS);
@@ -124,7 +125,7 @@ void Server::on_listen_accept(struct evconnlistener* listener,
 void Server::connect_client(
     struct bufferevent* bev, uint32_t address, uint16_t port,
     GameVersion version, ServerBehavior initial_state) {
-  log(INFO, "[Server] Client connected on virtual connection %p", bev);
+  this->log(INFO, "Client connected on virtual connection %p", bev);
 
   shared_ptr<Client> c(new Client(bev, version, initial_state));
   this->bev_to_client.emplace(make_pair(bev, c));
@@ -145,7 +146,7 @@ void Server::connect_client(
 
 void Server::on_listen_error(struct evconnlistener* listener) {
   int err = EVUTIL_SOCKET_ERROR();
-  log(ERROR, "[Server] Failure on listening socket %d: %d (%s)",
+  this->log(ERROR, "Failure on listening socket %d: %d (%s)",
       evconnlistener_get_fd(listener), err, evutil_socket_error_to_string(err));
   event_base_loopexit(this->base.get(), nullptr);
 }
@@ -155,7 +156,7 @@ void Server::on_client_input(struct bufferevent* bev) {
   try {
     c = this->bev_to_client.at(bev);
   } catch (const out_of_range& e) {
-    log(WARNING, "[Server] Received message from client with no configuration");
+    this->log(WARNING, "Received message from client with no configuration");
 
     // ignore all the data
     // TODO: we probably should disconnect them or something
@@ -186,7 +187,7 @@ void Server::on_disconnecting_client_output(struct bufferevent* bev) {
 void Server::on_client_error(struct bufferevent* bev, short events) {
   if (events & BEV_EVENT_ERROR) {
     int err = EVUTIL_SOCKET_ERROR();
-    log(WARNING, "[Server] Client caused error %d (%s)", err,
+    this->log(WARNING, "Client caused error %d (%s)", err,
         evutil_socket_error_to_string(err));
   }
   if (events & (BEV_EVENT_EOF | BEV_EVENT_ERROR)) {
@@ -198,7 +199,7 @@ void Server::on_disconnecting_client_error(struct bufferevent* bev,
     short events) {
   if (events & BEV_EVENT_ERROR) {
     int err = EVUTIL_SOCKET_ERROR();
-    log(WARNING, "[Server] Disconnecting client caused error %d (%s)", err,
+    this->log(WARNING, "Disconnecting client caused error %d (%s)", err,
         evutil_socket_error_to_string(err));
   }
   if (events & (BEV_EVENT_EOF | BEV_EVENT_ERROR)) {
@@ -208,91 +209,67 @@ void Server::on_disconnecting_client_error(struct bufferevent* bev,
 }
 
 void Server::receive_and_process_commands(shared_ptr<Client> c) {
-  struct evbuffer* buf = bufferevent_get_input(c->bev);
-  size_t header_size = (c->version == GameVersion::BB) ? 8 : 4;
-
-  // read as much data into recv_buffer as we can and decrypt it
-  size_t existing_bytes = c->recv_buffer.size();
-  size_t new_bytes = evbuffer_get_length(buf);
-  new_bytes &= ~(header_size - 1); // only read in multiples of header_size
-  c->recv_buffer.resize(existing_bytes + new_bytes);
-  void* recv_ptr = c->recv_buffer.data() + existing_bytes;
-  if (evbuffer_remove(buf, recv_ptr, new_bytes) != static_cast<ssize_t>(new_bytes)) {
-    throw runtime_error("some bytes could not be read from the receive buffer");
+  try {
+    for_each_received_command(c->bev, c->version, c->crypt_in.get(),
+        [this, c](uint16_t command, uint16_t flag, const std::string& data) {
+          process_command(this->state, c, command, flag, data);
+        });
+  } catch (const exception& e) {
+    this->log(INFO, "Error in client stream: %s", e.what());
+    c->should_disconnect = true;
+    return;
   }
-
-  // decrypt the received data if encryption is enabled
-  if (c->crypt_in.get()) {
-    c->crypt_in->decrypt(recv_ptr, new_bytes);
-  }
-
-  // process as many commands as possible
-  size_t offset = 0;
-  while (offset < c->recv_buffer.size()) {
-    const PSOCommandHeader* header = reinterpret_cast<const PSOCommandHeader*>(
-        c->recv_buffer.data() + offset);
-    size_t size = header->size(c->version);
-    if (offset + size > c->recv_buffer.size()) {
-      break; // don't have a complete command; we're done for now
-    }
-
-    // if we get here, then we have a complete, decrypted command waiting to be
-    // processed. we copy it out and append zeroes on the end so that it's safe
-    // to call string functions on the buffer in command handlers
-    string data = c->recv_buffer.substr(offset + header_size, size - header_size);
-    data.append(4, '\0');
-    try {
-      process_command(this->state, c, header->command(c->version),
-          header->flag(c->version), size - header_size, data.data());
-    } catch (const exception& e) {
-      log(INFO, "[Server] Error in client stream: %s", e.what());
-      c->should_disconnect = true;
-      return;
-    }
-
-    // BB pads commands to 8-byte boundaries, so if we see a shorter command,
-    // skip over the padding
-    offset += (size + header_size - 1) & ~(header_size - 1);
-  }
-
-  // remove the processed commands from the receive buffer
-  c->recv_buffer = c->recv_buffer.substr(offset);
 }
 
-Server::Server(shared_ptr<struct event_base> base,
-    shared_ptr<ServerState> state) : base(base), state(state) { }
+Server::Server(
+    shared_ptr<struct event_base> base,
+    shared_ptr<ServerState> state)
+  : log("[Server] "),
+    base(base),
+    state(state) { }
 
-void Server::listen(const string& socket_path, GameVersion version,
+void Server::listen(
+    const std::string& name,
+    const string& socket_path,
+    GameVersion version,
     ServerBehavior behavior) {
   int fd = ::listen(socket_path, 0, SOMAXCONN);
-  log(INFO, "[Server] Listening on Unix socket %s (%s) on fd %d",
-      socket_path.c_str(), name_for_version(version), fd);
-  this->add_socket(fd, version, behavior);
+  this->log(INFO, "Listening on Unix socket %s (%s) on fd %d (name: %s)",
+      socket_path.c_str(), name_for_version(version), fd, name.c_str());
+  this->add_socket(name, fd, version, behavior);
 }
 
-void Server::listen(const string& addr, int port, GameVersion version,
+void Server::listen(
+    const std::string& name,
+    const string& addr,
+    int port,
+    GameVersion version,
     ServerBehavior behavior) {
   int fd = ::listen(addr, port, SOMAXCONN);
   string netloc_str = render_netloc(addr, port);
-  log(INFO, "[Server] Listening on TCP interface %s (%s) on fd %d",
-      netloc_str.c_str(), name_for_version(version), fd);
-  this->add_socket(fd, version, behavior);
+  this->log(INFO, "Listening on TCP interface %s (%s) on fd %d (name: %s)",
+      netloc_str.c_str(), name_for_version(version), fd, name.c_str());
+  this->add_socket(name, fd, version, behavior);
 }
 
-void Server::listen(int port, GameVersion version, ServerBehavior behavior) {
-  this->listen("", port, version, behavior);
+void Server::listen(const std::string& name, int port, GameVersion version, ServerBehavior behavior) {
+  this->listen(name, "", port, version, behavior);
 }
 
-Server::ListeningSocket::ListeningSocket(Server* s, int fd,
-    GameVersion version, ServerBehavior behavior) :
-    fd(fd), version(version), behavior(behavior), listener(
+Server::ListeningSocket::ListeningSocket(Server* s, const std::string& name,
+    int fd, GameVersion version, ServerBehavior behavior) :
+    name(name), fd(fd), version(version), behavior(behavior), listener(
         evconnlistener_new(s->base.get(), Server::dispatch_on_listen_accept, s,
           LEV_OPT_REUSEABLE, 0, this->fd), evconnlistener_free) {
   evconnlistener_set_error_cb(this->listener.get(),
       Server::dispatch_on_listen_error);
 }
 
-void Server::add_socket(int fd, GameVersion version, ServerBehavior behavior) {
+void Server::add_socket(
+    const std::string& name,
+    int fd,
+    GameVersion version,
+    ServerBehavior behavior) {
   this->listening_sockets.emplace(piecewise_construct, forward_as_tuple(fd),
-      forward_as_tuple(this, fd, version, behavior));
+      forward_as_tuple(this, name, fd, version, behavior));
 }

src/Server.hh

@@ -21,24 +21,30 @@ public:
       std::shared_ptr<ServerState> state);
   virtual ~Server() = default;
 
-  void listen(const std::string& socket_path, GameVersion version, ServerBehavior initial_state);
-  void listen(const std::string& addr, int port, GameVersion version, ServerBehavior initial_state);
-  void listen(int port, GameVersion version, ServerBehavior initial_state);
-  void add_socket(int fd, GameVersion version, ServerBehavior initial_state);
+  void listen(const std::string& name, const std::string& socket_path, GameVersion version, ServerBehavior initial_state);
+  void listen(const std::string& name, const std::string& addr, int port, GameVersion version, ServerBehavior initial_state);
+  void listen(const std::string& name, int port, GameVersion version, ServerBehavior initial_state);
+  void add_socket(const std::string& name, int fd, GameVersion version, ServerBehavior initial_state);
 
   void connect_client(struct bufferevent* bev, uint32_t address, uint16_t port,
       GameVersion version, ServerBehavior initial_state);
 
 private:
+  PrefixedLogger log;
   std::shared_ptr<struct event_base> base;
 
   struct ListeningSocket {
+    std::string name;
     int fd;
     GameVersion version;
     ServerBehavior behavior;
     std::unique_ptr<struct evconnlistener, void(*)(struct evconnlistener*)> listener;
 
-    ListeningSocket(Server* s, int fd, GameVersion version,
+    ListeningSocket(
+        Server* s,
+        const std::string& name,
+        int fd,
+        GameVersion version,
         ServerBehavior behavior);
   };
   std::unordered_map<int, ListeningSocket> listening_sockets;

src/ServerShell.cc

@@ -6,22 +6,41 @@
 
 #include <phosg/Strings.hh>
 
-#include "ChatCommands.hh"
 #include "ServerState.hh"
 #include "SendCommands.hh"
+#include "StaticGameData.hh"
 
 using namespace std;
 
 
 
-ServerShell::ServerShell(std::shared_ptr<struct event_base> base,
-    std::shared_ptr<ServerState> state) : Shell(base, state) { }
+ServerShell::ServerShell(
+    shared_ptr<struct event_base> base,
+    shared_ptr<ServerState> state)
+  : Shell(base, state) { }
 
 void ServerShell::print_prompt() {
   fwrite("newserv> ", 9, 1, stdout);
   fflush(stdout);
 }
 
+shared_ptr<ProxyServer::LinkedSession> ServerShell::get_proxy_session() {
+  if (!this->state->proxy_server.get()) {
+    throw runtime_error("the proxy server is disabled");
+  }
+  return this->state->proxy_server->get_session();
+}
+
+static void set_boolean(bool* target, const string& args) {
+  if (args == "on") {
+    *target = true;
+  } else if (args == "off") {
+    *target = false;
+  } else {
+    throw invalid_argument("argument must be \"on\" or \"off\"");
+  }
+}
+
 void ServerShell::execute_command(const string& command) {
   // find the entry in the command table and run the command
   size_t command_end = skip_non_whitespace(command, 0);
@@ -34,16 +53,20 @@ void ServerShell::execute_command(const string& command) {
 
   } else if (command_name == "help") {
     fprintf(stderr, "\
-Commands:\n\
+General commands:\n\
   help\n\
     You\'re reading it now.\n\
+\n\
+Server commands:\n\
   exit (or ctrl+d)\n\
     Shut down the server.\n\
   reload <item> ...\n\
     Reload data. <item> can be licenses, battle-params, level-table, or quests.\n\
+    Reloading will not affect items that are in use; for example, if a client\'s\n\
+    license is deleted by reloading, they will not be disconnected immediately.\n\
   add-license <parameters>\n\
     Add a license to the server. <parameters> is some subset of the following:\n\
-      username=<username> (BB username)\n\
+      bb-username=<username> (BB username)\n\
       bb-password=<password> (BB password)\n\
       gc-password=<password> (GC password)\n\
       access-key=<access-key> (GC/PC access key)\n\
@@ -66,8 +89,69 @@ Commands:\n\
     Song IDs are 0 through 51; the default song is -1.\n\
   announce <message>\n\
     Send an announcement message to all players.\n\
+\n\
+Proxy commands (these will only work when exactly one client is connected):\n\
+  sc <data>\n\
+    Send a command to the client.\n\
+  ss <data>\n\
+    Send a command to the server.\n\
+  chat <text>\n\
+    Send a chat message to the server.\n\
+  dchat <data>\n\
+    Send a chat message to the server with arbitrary data in it.\n\
+  info-board <text>\n\
+    Set your info board contents. This will affect the current session only,\n\
+    and will not be saved for future sessions.\n\
+  info-board-data <data>\n\
+    Set your info board contents with arbitrary data. Like the above, affects\n\
+    the current session only.\n\
+  marker <color-id>\n\
+    Change your lobby marker color.\n\
+  warp <area-id>\n\
+    Send yourself to a specific area.\n\
+  set-override-section-id [section-id]\n\
+    Override the section ID for games you create or join. This affects the\n\
+    active drop chart if you are the leader of the game and the server doesn't\n\
+    override drops entirely. If no argument is given, clears the override.\n\
+  set-override-event [event]\n\
+    Override the lobby event for all lobbies and games you join. This applies\n\
+    only to you; other players do not see this override.  If no argument is\n\
+    given, clears the override.\n\
+  set-override-lobby-number [number]\n\
+    Override the lobby type for all lobbies you join. This applies only to you;\n\
+    other players do not see this override. If no argument is given, clears the\n\
+    override.\n\
+  set-chat-filter <on|off>\n\
+    Enable or disable chat filtering (enabled by default). Chat filtering\n\
+    applies newserv\'s standard character replacements to chat messages; for\n\
+    example, $ becomes a tab character and # becomes a newline.\n\
+  set-chat-safety <on|off>\n\
+    Enable or disable chat safety (enabled by default). When chat safety is on,\n\
+    all chat messages that begin with a $ are not sent to the remote server.\n\
+    This can prevent embarrassing situations if the remote server isn\'t a\n\
+    newserv instance and you have newserv commands in your chat shortcuts.\n\
+  set-switch-assist <on|off>\n\
+    Enable or disable switch assist. When switch assist is on, the proxy will\n\
+    remember the last \"enable switch\" command that you send, and will send it\n\
+    to you and the server when you step on another switch. Using this, you can\n\
+    unlock any doors that require two players to stand on switches by touching\n\
+    both switches yourself. With this, all online maps can be completed solo.\n\
+  set-save-files <on|off>\n\
+    Enable or disable saving of game files (disabled by default). When this is\n\
+    on, any file that the remote server sends to the client will be saved to\n\
+    the current directory. This includes data like quests, Episode 3 card\n\
+    definitions, and GBA games.\n\
+  set-block-function-calls [return-value]\n\
+    Enable blocking of function calls from the server. When enabled, the proxy\n\
+    responds as if the function was called (with the given return value), but\n\
+    does not send the code to the client. To stop blocking function calls, omit\n\
+    the return value.\n\
 ");
 
+
+
+  // SERVER COMMANDS
+
   } else if (command_name == "reload") {
     auto types = split(command_args, ' ');
     if (types.empty()) {
@@ -95,29 +179,29 @@ Commands:\n\
     shared_ptr<License> l(new License());
 
     for (const string& token : split(command_args, ' ')) {
-      if (starts_with(token, "username=")) {
-        if (token.size() >= 29) {
+      if (starts_with(token, "bb-username=")) {
+        if (token.size() >= 32) {
           throw invalid_argument("username too long");
         }
-        strcpy(l->username, token.c_str() + 9);
+        l->username = token.substr(12);
 
       } else if (starts_with(token, "bb-password=")) {
         if (token.size() >= 32) {
           throw invalid_argument("bb-password too long");
         }
-        strcpy(l->bb_password, token.c_str() + 12);
+        l->bb_password = token.substr(12);
 
       } else if (starts_with(token, "gc-password=")) {
         if (token.size() > 20) {
           throw invalid_argument("gc-password too long");
         }
-        strcpy(l->gc_password, token.c_str() + 12);
+        l->gc_password = token.substr(12);
 
       } else if (starts_with(token, "access-key=")) {
         if (token.size() > 23) {
           throw invalid_argument("access-key is too long");
         }
-        strcpy(l->access_key, token.c_str() + 11);
+        l->access_key = token.substr(11);
 
       } else if (starts_with(token, "serial=")) {
         l->serial_number = stoul(token.substr(7));
@@ -127,11 +211,11 @@ Commands:\n\
         if (mask == "normal") {
           l->privileges = 0;
         } else if (mask == "mod") {
-          l->privileges = Privilege::Moderator;
+          l->privileges = Privilege::MODERATOR;
         } else if (mask == "admin") {
-          l->privileges = Privilege::Administrator;
+          l->privileges = Privilege::ADMINISTRATOR;
         } else if (mask == "root") {
-          l->privileges = Privilege::Root;
+          l->privileges = Privilege::ROOT;
         } else {
           l->privileges = stoul(mask);
         }
@@ -189,6 +273,118 @@ Commands:\n\
     u16string message16 = decode_sjis(command_args);
     send_text_message(this->state, message16.c_str());
 
+
+
+  // PROXY COMMANDS
+
+  } else if ((command_name == "sc") || (command_name == "ss")) {
+    auto session = this->get_proxy_session();
+
+    bool to_server = (command_name[1] == 's');
+    string data = parse_data_string(command_args);
+    if (data.size() & 3) {
+      throw invalid_argument("data size is not a multiple of 4");
+    }
+    if (data.size() == 0) {
+      throw invalid_argument("no data given");
+    }
+
+    session->send_to_end_with_header(to_server, data);
+
+  } else if ((command_name == "chat") || (command_name == "dchat")) {
+    auto session = this->get_proxy_session();
+
+    string data(8, '\0');
+    data.push_back('\x09');
+    data.push_back('E');
+    if (command_name == "dchat") {
+      data += parse_data_string(command_args);
+    } else {
+      data += command_args;
+    }
+    data.push_back('\0');
+    data.resize((data.size() + 3) & (~3));
+
+    session->send_to_end(true, 0x06, 0x00, data);
+
+  } else if (command_name == "marker") {
+    auto session = this->get_proxy_session();
+    session->send_to_end(true, 0x89, stoul(command_args));
+
+  } else if (command_name == "warp") {
+    auto session = this->get_proxy_session();
+
+    PSOSubcommand cmds[2];
+    cmds[0].word[0] = 0x0294;
+    cmds[0].word[1] = session->lobby_client_id;
+    cmds[1].dword = stoul(command_args);
+
+    session->send_to_end(false, 0x60, 0x00, &cmds, sizeof(cmds));
+    session->send_to_end(true, 0x60, 0x00, &cmds, sizeof(cmds));
+
+  } else if ((command_name == "info-board") || (command_name == "info-board-data")) {
+    auto session = this->get_proxy_session();
+
+    string data;
+    if (command_name == "info-board-data") {
+      data += parse_data_string(command_args);
+    } else {
+      data += command_args;
+    }
+    data.push_back('\0');
+    data.resize((data.size() + 3) & (~3));
+
+    session->send_to_end(true, 0xD9, 0x00, data);
+
+  } else if (command_name == "set-override-section-id") {
+    auto session = this->get_proxy_session();
+    if (command_args.empty()) {
+      session->override_section_id = -1;
+    } else {
+      session->override_section_id = section_id_for_name(command_args);
+    }
+
+  } else if (command_name == "set-override-event") {
+    auto session = this->get_proxy_session();
+    if (command_args.empty()) {
+      session->override_lobby_event = -1;
+    } else {
+      session->override_lobby_event = event_for_name(command_args);
+      session->send_to_end(false, 0xDA, session->override_lobby_event);
+    }
+
+  } else if (command_name == "set-override-lobby-number") {
+    auto session = this->get_proxy_session();
+    if (command_args.empty()) {
+      session->override_lobby_number = -1;
+    } else {
+      session->override_lobby_number = lobby_type_for_name(command_args);
+    }
+
+  } else if (command_name == "set-chat-filter") {
+    auto session = this->get_proxy_session();
+    set_boolean(&session->enable_chat_filter, command_args);
+
+  } else if (command_name == "set-chat-safety") {
+    auto session = this->get_proxy_session();
+    set_boolean(&session->suppress_newserv_commands, command_args);
+
+  } else if (command_name == "set-switch-assist") {
+    auto session = this->get_proxy_session();
+    set_boolean(&session->enable_switch_assist, command_args);
+
+  } else if (command_name == "set-save-files") {
+    auto session = this->get_proxy_session();
+    set_boolean(&session->save_files, command_args);
+
+  } else if (command_name == "set-block-function-calls") {
+    auto session = this->get_proxy_session();
+    if (command_args.empty()) {
+      session->function_call_return_value = -1;
+    } else {
+      session->function_call_return_value = stoul(command_args);
+    }
+
   } else {
     throw invalid_argument("unknown command; try \'help\'");
   }

src/ServerShell.hh

@@ -6,12 +6,14 @@
 #include <event2/event.h>
 
 #include "Shell.hh"
+#include "ProxyServer.hh"
 
 
 
 class ServerShell : public Shell {
 public:
-  ServerShell(std::shared_ptr<struct event_base> base,
+  ServerShell(
+      std::shared_ptr<struct event_base> base,
       std::shared_ptr<ServerState> state);
   virtual ~ServerShell() = default;
   ServerShell(const ServerShell&) = delete;
@@ -20,6 +22,8 @@ public:
   ServerShell& operator=(ServerShell&&) = delete;
 
 protected:
+  std::shared_ptr<ProxyServer::LinkedSession> get_proxy_session();
+
   virtual void print_prompt();
   virtual void execute_command(const std::string& command);
 };

src/ServerState.cc

@@ -17,57 +17,65 @@ ServerState::ServerState()
   : dns_server_port(0),
     ip_stack_debug(false),
     allow_unregistered_users(false),
-    run_shell_behavior(RunShellBehavior::Default), next_lobby_id(1),
+    run_shell_behavior(RunShellBehavior::DEFAULT), next_lobby_id(1),
     pre_lobby_event(0),
     ep3_menu_song(-1) {
-  memset(&this->default_key_file, 0, sizeof(this->default_key_file));
-
-  this->main_menu.emplace_back(MAIN_MENU_GO_TO_LOBBY, u"Go to lobby",
-      u"Join the lobby.", 0);
-  this->main_menu.emplace_back(MAIN_MENU_INFORMATION, u"Information",
-      u"View server information.", MenuItemFlag::RequiresMessageBoxes);
-  this->main_menu.emplace_back(MAIN_MENU_DOWNLOAD_QUESTS, u"Download quests",
-      u"Download quests.", 0);
-  this->main_menu.emplace_back(MAIN_MENU_DISCONNECT, u"Disconnect",
-      u"Disconnect.", 0);
+  vector<shared_ptr<Lobby>> ep3_only_lobbies;
 
   for (size_t x = 0; x < 20; x++) {
     auto lobby_name = decode_sjis(string_printf("LOBBY%zu", x + 1));
+    bool is_ep3_only = (x > 14);
+
     shared_ptr<Lobby> l(new Lobby());
-    l->flags |= LobbyFlag::Public | LobbyFlag::Default | LobbyFlag::Persistent |
-        ((x > 14) ? LobbyFlag::Episode3 : 0);
+    l->flags |= Lobby::Flag::PUBLIC | Lobby::Flag::DEFAULT | Lobby::Flag::PERSISTENT |
+        (is_ep3_only ? Lobby::Flag::EPISODE_3_ONLY : 0);
     l->block = x + 1;
     l->type = x;
-    char16cpy(l->name, lobby_name.c_str(), 0x24);
+    l->name = lobby_name;
     l->max_clients = 12;
     this->add_lobby(l);
+
+    if (!is_ep3_only) {
+      this->public_lobby_search_order.emplace_back(l);
+    } else {
+      ep3_only_lobbies.emplace_back(l);
+    }
   }
+
+  this->public_lobby_search_order_ep3 = this->public_lobby_search_order;
+  this->public_lobby_search_order_ep3.insert(
+      this->public_lobby_search_order_ep3.begin(),
+      ep3_only_lobbies.begin(),
+      ep3_only_lobbies.end());
 }
 
 void ServerState::add_client_to_available_lobby(shared_ptr<Client> c) {
-  auto it = this->id_to_lobby.lower_bound(0);
-  for (; it != this->id_to_lobby.end(); it++) {
-    if (!(it->second->flags & LobbyFlag::Public)) {
-      continue;
-    }
+  const auto& search_order = (c->flags & Client::Flag::EPISODE_3)
+      ? this->public_lobby_search_order_ep3
+      : this->public_lobby_search_order;
+
+  shared_ptr<Lobby> added_to_lobby;
+  for (const auto& l : search_order) {
     try {
-      it->second->add_client(c);
+      l->add_client(c);
+      added_to_lobby = l;
       break;
     } catch (const out_of_range&) { }
   }
 
-  if (it == this->id_to_lobby.end()) {
+  if (!added_to_lobby) {
+    // TODO: Add the user to a dynamically-created private lobby instead
     throw out_of_range("all lobbies full");
   }
 
-  // send a join message to the joining player, and notifications to all others
-  this->send_lobby_join_notifications(it->second, c);
+  // Send a join message to the joining player, and notifications to all others
+  this->send_lobby_join_notifications(added_to_lobby, c);
 }
 
 void ServerState::remove_client_from_lobby(shared_ptr<Client> c) {
   auto l = this->id_to_lobby.at(c->lobby_id);
   l->remove_client(c);
-  if (!(l->flags & LobbyFlag::Persistent) && (l->count_clients() == 0)) {
+  if (!(l->flags & Lobby::Flag::PERSISTENT) && (l->count_clients() == 0)) {
     this->remove_lobby(l->lobby_id);
   } else {
     send_player_leave_notification(l, c->lobby_client_id);
@@ -90,7 +98,7 @@ void ServerState::change_client_lobby(shared_ptr<Client> c, shared_ptr<Lobby> ne
   }
 
   if (current_lobby) {
-    if (!(current_lobby->flags & LobbyFlag::Persistent) && (current_lobby->count_clients() == 0)) {
+    if (!(current_lobby->flags & Lobby::Flag::PERSISTENT) && (current_lobby->count_clients() == 0)) {
       this->remove_lobby(current_lobby->lobby_id);
     } else {
       send_player_leave_notification(current_lobby, old_lobby_client_id);
@@ -136,13 +144,12 @@ void ServerState::remove_lobby(uint32_t lobby_id) {
   this->id_to_lobby.erase(lobby_id);
 }
 
-shared_ptr<Client> ServerState::find_client(const char16_t* identifier,
+shared_ptr<Client> ServerState::find_client(const std::u16string* identifier,
     uint64_t serial_number, shared_ptr<Lobby> l) {
 
   if ((serial_number == 0) && identifier) {
     try {
-      string encoded = encode_sjis(identifier);
-      serial_number = stoull(encoded, nullptr, 0);
+      serial_number = stoull(encode_sjis(*identifier), nullptr, 0);
     } catch (const exception&) { }
   }
 
@@ -187,13 +194,46 @@ uint32_t ServerState::connect_address_for_client(std::shared_ptr<Client> c) {
 
 
 
+shared_ptr<const vector<MenuItem>> ServerState::information_menu_for_version(GameVersion version) {
+  if (version == GameVersion::PC) {
+    return this->information_menu_pc;
+  } else if (version == GameVersion::GC) {
+    return this->information_menu_gc;
+  }
+  throw out_of_range("no information menu exists for this version");
+}
+
+const vector<MenuItem>& ServerState::proxy_destinations_menu_for_version(GameVersion version) {
+  if (version == GameVersion::PC) {
+    return this->proxy_destinations_menu_pc;
+  } else if (version == GameVersion::GC) {
+    return this->proxy_destinations_menu_gc;
+  }
+  throw out_of_range("no proxy destinations menu exists for this version");
+}
+
+const vector<pair<string, uint16_t>>& ServerState::proxy_destinations_for_version(GameVersion version) {
+  if (version == GameVersion::PC) {
+    return this->proxy_destinations_pc;
+  } else if (version == GameVersion::GC) {
+    return this->proxy_destinations_gc;
+  }
+  throw out_of_range("no proxy destinations menu exists for this version");
+}
+
+
+
 void ServerState::set_port_configuration(
-    const std::unordered_map<std::string, PortConfiguration>& named_port_configuration) {
-  this->named_port_configuration = named_port_configuration;
-  this->numbered_port_configuration.clear();
-  for (const auto& it : this->named_port_configuration) {
-    if (!this->numbered_port_configuration.emplace(it.second.port, it.second).second) {
-      throw runtime_error("duplicate port in configuration");
+    const vector<PortConfiguration>& port_configs) {
+  this->name_to_port_config.clear();
+  this->number_to_port_config.clear();
+  for (const auto& pc : port_configs) {
+    shared_ptr<PortConfiguration> spc(new PortConfiguration(pc));
+    if (!this->name_to_port_config.emplace(spc->name, spc).second) {
+      throw logic_error("duplicate name in port configuration");
+    }
+    if (!this->number_to_port_config.emplace(spc->port, spc).second) {
+      throw logic_error("duplicate number in port configuration");
     }
   }
 }

src/ServerState.hh

@@ -18,7 +18,11 @@
 
 
 
+// Forwawrd declaration due to reference cycle
+class ProxyServer;
+
 struct PortConfiguration {
+  std::string name;
   uint16_t port;
   GameVersion version;
   ServerBehavior behavior;
@@ -26,21 +30,21 @@ struct PortConfiguration {
 
 struct ServerState {
   enum class RunShellBehavior {
-    Default = 0,
-    Always,
-    Never,
+    DEFAULT = 0,
+    ALWAYS,
+    NEVER,
   };
 
   std::u16string name;
-  std::unordered_map<std::string, PortConfiguration> named_port_configuration;
-  std::unordered_map<uint16_t, PortConfiguration> numbered_port_configuration;
+  std::unordered_map<std::string, std::shared_ptr<PortConfiguration>> name_to_port_config;
+  std::unordered_map<uint16_t, std::shared_ptr<PortConfiguration>> number_to_port_config;
   std::string username;
   uint16_t dns_server_port;
   std::vector<std::string> ip_stack_addresses;
   bool ip_stack_debug;
   bool allow_unregistered_users;
   RunShellBehavior run_shell_behavior;
-  PSOBBEncryption::KeyFile default_key_file;
+  std::vector<std::shared_ptr<const PSOBBEncryption::KeyFile>> bb_private_keys;
   std::shared_ptr<const QuestIndex> quest_index;
   std::shared_ptr<const LevelTable> level_table;
   std::shared_ptr<const BattleParamTable> battle_params;
@@ -49,11 +53,20 @@ struct ServerState {
   std::shared_ptr<LicenseManager> license_manager;
 
   std::vector<MenuItem> main_menu;
-  std::shared_ptr<std::vector<MenuItem>> information_menu;
+  std::shared_ptr<std::vector<MenuItem>> information_menu_pc;
+  std::shared_ptr<std::vector<MenuItem>> information_menu_gc;
   std::shared_ptr<std::vector<std::u16string>> information_contents;
+  std::vector<MenuItem> proxy_destinations_menu_pc;
+  std::vector<MenuItem> proxy_destinations_menu_gc;
+  std::vector<std::pair<std::string, uint16_t>> proxy_destinations_pc;
+  std::vector<std::pair<std::string, uint16_t>> proxy_destinations_gc;
+  std::pair<std::string, uint16_t> proxy_destination_patch;
+  std::pair<std::string, uint16_t> proxy_destination_bb;
   std::u16string welcome_message;
 
   std::map<int64_t, std::shared_ptr<Lobby>> id_to_lobby;
+  std::vector<std::shared_ptr<Lobby>> public_lobby_search_order;
+  std::vector<std::shared_ptr<Lobby>> public_lobby_search_order_ep3;
   std::atomic<int32_t> next_lobby_id;
   uint8_t pre_lobby_event;
   int32_t ep3_menu_song;
@@ -62,6 +75,10 @@ struct ServerState {
   uint32_t local_address;
   uint32_t external_address;
 
+  // TODO: This is only here because the menu selection handler has to call
+  // delete_session on it. Find a cleaner way to do this.
+  std::shared_ptr<ProxyServer> proxy_server;
+
   ServerState();
 
   void add_client_to_available_lobby(std::shared_ptr<Client> c);
@@ -78,11 +95,17 @@ struct ServerState {
   void add_lobby(std::shared_ptr<Lobby> l);
   void remove_lobby(uint32_t lobby_id);
 
-  std::shared_ptr<Client> find_client(const char16_t* identifier = nullptr,
-    uint64_t serial_number = 0, std::shared_ptr<Lobby> l = nullptr);
+  std::shared_ptr<Client> find_client(
+    const std::u16string* identifier = nullptr,
+    uint64_t serial_number = 0,
+    std::shared_ptr<Lobby> l = nullptr);
 
   uint32_t connect_address_for_client(std::shared_ptr<Client> c);
 
+  std::shared_ptr<const std::vector<MenuItem>> information_menu_for_version(GameVersion version);
+  const std::vector<MenuItem>& proxy_destinations_menu_for_version(GameVersion version);
+  const std::vector<std::pair<std::string, uint16_t>>& proxy_destinations_for_version(GameVersion version);
+
   void set_port_configuration(
-      const std::unordered_map<std::string, PortConfiguration>& named_port_configuration);
+    const std::vector<PortConfiguration>& port_configs);
 };

src/StaticGameData.hh

@@ -0,0 +1,41 @@
+#pragma once
+
+#include <stdint.h>
+
+#include <unordered_map>
+
+#include "Player.hh"
+
+
+
+extern const std::unordered_map<uint32_t, uint32_t> combine_item_to_max;
+extern const std::unordered_map<uint8_t, const char*> name_for_weapon_special;
+extern const std::unordered_map<uint8_t, const char*> name_for_s_rank_special;
+extern const std::unordered_map<uint32_t, const char*> name_for_primary_identifier;
+
+const std::string& name_for_technique(uint8_t tech);
+std::u16string u16name_for_technique(uint8_t tech);
+uint8_t technique_for_name(const std::string& name);
+uint8_t technique_for_name(const std::u16string& name);
+
+const std::string& name_for_section_id(uint8_t section_id);
+std::u16string u16name_for_section_id(uint8_t section_id);
+uint8_t section_id_for_name(const std::string& name);
+uint8_t section_id_for_name(const std::u16string& name);
+
+const std::string& name_for_event(uint8_t event);
+std::u16string u16name_for_event(uint8_t event);
+uint8_t event_for_name(const std::string& name);
+uint8_t event_for_name(const std::u16string& name);
+
+const std::string& name_for_lobby_type(uint8_t type);
+std::u16string u16name_for_lobby_type(uint8_t type);
+uint8_t lobby_type_for_name(const std::string& name);
+uint8_t lobby_type_for_name(const std::u16string& name);
+
+const std::string& name_for_npc(uint8_t npc);
+std::u16string u16name_for_npc(uint8_t npc);
+uint8_t npc_for_name(const std::string& name);
+uint8_t npc_for_name(const std::u16string& name);
+
+std::string name_for_item(const ItemData& item, bool include_color_codes);

src/Text.cc

@@ -13,7 +13,7 @@ using namespace std;
 
 
 
-int char16cmp(const char16_t* s1, const char16_t* s2, size_t count) {
+int char16ncmp(const char16_t* s1, const char16_t* s2, size_t count) {
   size_t x;
   for (x = 0; x < count && s1[x] != 0 && s2[x] != 0; x++) {
     if (s1[x] < s2[x]) {
@@ -30,30 +30,14 @@ int char16cmp(const char16_t* s1, const char16_t* s2, size_t count) {
   return 0;
 }
 
-void char16cpy(char16_t* dest, const char16_t* src, size_t count) {
-  size_t x;
-  for (x = 0; x < count && src[x] != 0; x++) {
-    dest[x] = src[x];
-  }
-  if (x < count) {
-    dest[x] = 0;
-  }
-}
-
-size_t char16len(const char16_t* s) {
-  size_t x;
-  for (x = 0; s[x] != 0; x++);
-  return x;
-}
-
 
 
 static vector<char16_t> unicode_to_sjis_table_data;
 static vector<char16_t> sjis_to_unicode_table_data;
 
 static void load_sjis_tables() {
-  unicode_to_sjis_table_data.resize(0x10000);
-  sjis_to_unicode_table_data.resize(0x10000);
+  unicode_to_sjis_table_data.resize(0x10000, 0);
+  sjis_to_unicode_table_data.resize(0x10000, 0);
 
   // TODO: this is inefficient; it makes multiple copies of the string
   auto file_contents = load_file("system/sjis-table.ini");
@@ -85,155 +69,119 @@ static const vector<char16_t>& unicode_to_sjis_table() {
   return unicode_to_sjis_table_data;
 }
 
-void encode_sjis(char* dest, const char16_t* source, size_t max) {
+
+
+std::string encode_sjis(const char16_t* src, size_t src_count) {
   const auto& table = unicode_to_sjis_table();
-  while (*source && (--max)) {
-    *(dest++) = table[*(source++)];
+
+  const char16_t* src_end = src + src_count;
+  string ret;
+  while ((src != src_end) && *src) {
+    uint16_t ch = *(src++);
+    uint16_t translated_c = table[ch];
+    if (translated_c == 0) {
+      throw runtime_error("untranslatable unicode character");
+    } else if (translated_c & 0xFF00) {
+      ret.push_back((translated_c >> 8) & 0xFF);
+      ret.push_back(translated_c & 0xFF);
+    } else {
+      ret.push_back(translated_c & 0xFF);
+    }
   };
-  *dest = 0;
+  return ret;
 }
 
-void decode_sjis(char16_t* dest, const char* source, size_t max) {
+size_t encode_sjis(
+    char* dest,
+    size_t dest_count,
+    const char16_t* src,
+    size_t src_count,
+    bool allow_skip_terminator) {
+  const auto& table = unicode_to_sjis_table();
+
+  if (dest_count == 0) {
+    throw logic_error("cannot encode into zero-length buffer");
+  }
+
+  const char* dest_start = dest;
+  const char16_t* src_end = src + src_count;
+  const char* dest_end = dest + (allow_skip_terminator ? dest_count : (dest_count - 1));
+  while ((dest != dest_end) && (src != src_end) && *src) {
+    uint16_t ch = *(src++);
+    uint16_t translated_c = table[ch];
+    if (translated_c == 0) {
+      throw runtime_error("untranslatable unicode character");
+    } else if (translated_c & 0xFF00) {
+      *(dest++) = (translated_c >> 8) & 0xFF;
+      // If the second byte of this character would cause the null to overrun
+      // the buffer, erase the first byte instead and return early
+      if (dest == dest_end) {
+        *(dest - 1) = 0;
+      } else {
+        *(dest++) = translated_c & 0xFF;
+      }
+    } else {
+      *(dest++) = translated_c & 0xFF;
+    }
+  }
+  if (!allow_skip_terminator || (dest != dest_end)) {
+    *dest = 0;
+    dest++;
+  }
+  return dest - dest_start;
+}
+
+std::u16string decode_sjis(const char* src, size_t src_count) {
   const auto& table = sjis_to_unicode_table();
-  while (*source && (--max)) {
-    char16_t src_char = *(source++);
+
+  const char* src_end = src + src_count;
+  u16string ret;
+  while ((src != src_end) && *src) {
+    uint16_t src_char = *(src++);
     if (src_char & 0x80) {
-      src_char = (src_char << 8) | *(source++);
+      if (src == src_end) {
+        throw runtime_error("incomplete extended character");
+      }
+      src_char = (src_char << 8) | *(src++);
       if ((src_char & 0xFF) == 0) {
-        return;
+        throw runtime_error("incomplete extended character");
+      }
+    }
+    ret.push_back(table[src_char]);
+  };
+  return ret;
+}
+
+size_t decode_sjis(
+    char16_t* dest,
+    size_t dest_count,
+    const char* src,
+    size_t src_count,
+    bool allow_skip_terminator) {
+  const auto& table = sjis_to_unicode_table();
+
+  if (dest_count == 0) {
+    throw logic_error("cannot decode into zero-length buffer");
+  }
+
+  const char16_t* dest_start = dest;
+  const char* src_end = src + src_count;
+  const char16_t* dest_end = dest + (allow_skip_terminator ? dest_count : (dest_count - 1));
+  while ((dest != dest_end) && (src != src_end) && *src) {
+    uint16_t src_char = *(src++);
+    if (src_char & 0x80) {
+      if (src == src_end) {
+        throw runtime_error("incomplete extended character");
+      }
+      src_char = (src_char << 8) | *(src++);
+      if ((src_char & 0xFF) == 0) {
+        throw runtime_error("incomplete extended character");
       }
     }
     *(dest++) = table[src_char];
   };
-  *dest = 0;
-}
-
-std::string encode_sjis(const char16_t* source) {
-  const auto& table = unicode_to_sjis_table();
-  string ret;
-  while (*source) {
-    ret.push_back(table[*(source++)]);
-  };
-  return ret;
-}
-
-std::u16string decode_sjis(const char* source) {
-  const auto& table = sjis_to_unicode_table();
-  u16string ret;
-  while (*source) {
-    char16_t src_char = *(source++);
-    if (src_char & 0x80) {
-      src_char = (src_char << 8) | *(source++);
-      if ((src_char & 0xFF) == 0) {
-        return ret;
-      }
-    }
-    ret.push_back(table[src_char]);
-  };
-  return ret;
-}
-
-std::string encode_sjis(const std::u16string& source) {
-  const auto& table = unicode_to_sjis_table();
-  string ret;
-  for (char16_t ch : source) {
-    ret.push_back(table[ch]);
-  };
-  return ret;
-}
-
-std::u16string decode_sjis(const std::string& source) {
-  const auto& table = sjis_to_unicode_table();
-  u16string ret;
-  for (size_t x = 0; x < source.size(); x++) {
-    char16_t src_char = source[x];
-    if (src_char & 0x80) {
-      src_char = (src_char << 8) | source[++x];
-      if ((src_char & 0xFF) == 0) {
-        return ret;
-      }
-    }
-    ret.push_back(table[src_char]);
-  };
-  return ret;
-}
-
-
-
-void add_language_marker_inplace(char* a, char e, size_t dest_count) {
-  if ((a[0] == '\t') && (a[1] != 'C')) {
-    return;
+  if (!allow_skip_terminator || (dest != dest_end)) {
+    *(dest++) = 0;
   }
-
-  size_t existing_count = strlen(a);
-  if (existing_count > dest_count - 3) {
-    existing_count = dest_count - 3;
-  }
-  memmove(&a[2], a, (existing_count + 1) * sizeof(char));
-  a[0] = '\t';
-  a[1] = e;
-  a[existing_count + 2] = 0;
-}
-
-void add_language_marker_inplace(char16_t* a, char16_t e, size_t dest_count) {
-  if ((a[0] == '\t') && (a[1] != 'C')) {
-    return;
-  }
-
-  size_t existing_count = char16len(a);
-  if (existing_count > dest_count - 3) {
-    existing_count = dest_count - 3;
-  }
-  memmove(&a[2], a, (existing_count + 1) * sizeof(char16_t));
-  a[0] = '\t';
-  a[1] = e;
-  a[existing_count + 2] = 0;
-}
-
-void remove_language_marker_inplace(char* a) {
-  if ((a[0] == '\t') && (a[1] != 'C')) {
-    strcpy(a, &a[2]);
-  }
-}
-
-void remove_language_marker_inplace(char16_t* a) {
-  if ((a[0] == '\t') && (a[1] != 'C')) {
-    char16cpy(a, &a[2], char16len(a) - 2);
-  }
-}
-
-std::string add_language_marker(const std::string& s, char marker) {
-  if ((s.size() >= 2) && (s[0] == '\t') && (s[1] != 'C')) {
-    return s;
-  }
-
-  string ret;
-  ret.push_back('\t');
-  ret.push_back(marker);
-  return ret + s;
-}
-
-std::u16string add_language_marker(const std::u16string& s, char16_t marker) {
-  if ((s.size() >= 2) && (s[0] == L'\t') && (s[1] != L'C')) {
-    return s;
-  }
-
-  u16string ret;
-  ret.push_back(L'\t');
-  ret.push_back(marker);
-  return ret + s;
-}
-
-std::string remove_language_marker(const std::string& s) {
-  if ((s.size() < 2) || (s[0] != '\t') || (s[1] == 'C')) {
-    return s;
-  }
-  return s.substr(2);
-}
-
-std::u16string remove_language_marker(const std::u16string& s) {
-  if ((s.size() < 2) || (s[0] != L'\t') || (s[1] == L'C')) {
-    return s;
-  }
-  return s.substr(2);
+  return dest - dest_start;
 }

src/Text.hh

@@ -2,32 +2,463 @@
 
 #include <inttypes.h>
 #include <stddef.h>
+#include <string.h>
 
 #include <string>
+#include <phosg/Encoding.hh>
+#include <phosg/Strings.hh>
 
 
-int char16cmp(const char16_t* s1, const char16_t* s2, size_t count);
-void char16cpy(char16_t* dest, const char16_t* src, size_t count);
-size_t char16len(const char16_t* s);
+
+// TODO: delete these if not needed
+// int char16ncmp(const char16_t* s1, const char16_t* s2, size_t count);
+// size_t char16len(const char16_t* s);
 
 
-void encode_sjis(char* dest, const char16_t* source, size_t dest_count);
-void decode_sjis(char16_t* dest, const char* source, size_t dest_count);
-std::string encode_sjis(const char16_t* source);
-std::u16string decode_sjis(const char* source);
-std::string encode_sjis(const std::u16string& source);
-std::u16string decode_sjis(const std::string& source);
+
+// (1a) Conversion functions
+
+// These return the number of characters written, including the terminating null
+// character. In the case of encode_sjis, two-byte characters count as two
+// characters, so the returned number is the number of bytes written.
+// allow_skip_terminator means no null byte will be written if dest_count
+// characters are written to the output. If this argument is false, a null
+// terminator is always written, even if the string is truncated.
+size_t encode_sjis(
+    char* dest, size_t dest_count,
+    const char16_t* src, size_t src_count,
+    bool allow_skip_terminator = false);
+size_t decode_sjis(
+    char16_t* dest, size_t dest_count,
+    const char* src, size_t src_count,
+    bool allow_skip_terminator = false);
+
+std::string encode_sjis(const char16_t* source, size_t src_count);
+std::u16string decode_sjis(const char* source, size_t src_count);
+
+inline std::string encode_sjis(const std::u16string& s) {
+  return encode_sjis(s.data(), s.size());
+}
+
+inline std::u16string decode_sjis(const std::string& s) {
+  return decode_sjis(s.data(), s.size());
+}
+
+// (1b) Type-independent utility functions
+
+template <typename T>
+size_t text_strlen_t(const T* s) {
+  size_t ret = 0;
+  for (; s[ret] != 0; ret++) { }
+  return ret;
+}
+
+template <typename T>
+size_t text_strnlen_t(const T* s, size_t count) {
+  size_t ret = 0;
+  for (; s[ret] != 0 && ret < count; ret++) { }
+  return ret;
+}
+
+template <typename T>
+size_t text_streq_t(const T* a, const T* b) {
+  for (;;) {
+    if (*a != *b) {
+      return false;
+    }
+    if (*a == 0) {
+      return true;
+    }
+    a++;
+    b++;
+  }
+}
+
+template <typename T>
+size_t text_strneq_t(const T* a, const T* b, size_t count) {
+  for (; count; count--) {
+    if (*a != *b) {
+      return false;
+    }
+    if (*a == 0) {
+      return true;
+    }
+    a++;
+    b++;
+  }
+  return true;
+}
+
+template <typename T>
+size_t text_strncpy_t(T* dest, const T* src, size_t count) {
+  size_t x;
+  for (x = 0; x < count && src[x] != 0; x++) {
+    dest[x] = src[x];
+  }
+  if (x < count) {
+    dest[x++] = 0;
+  }
+  return x;
+}
+
+// Like strncpy, but *always* null-terminates the string, even if it has to
+// truncate it.
+template <typename T>
+size_t text_strnzcpy_t(T* dest, const T* src, size_t count) {
+  size_t x;
+  for (x = 0; x < count - 1 && src[x] != 0; x++) {
+    dest[x] = src[x];
+  }
+  dest[x++] = 0;
+  return x;
+}
 
 
-void add_language_marker_inplace(char* s, char marker, size_t dest_count);
-void add_language_marker_inplace(char16_t* s, char16_t marker, size_t dest_count);
-void remove_language_marker_inplace(char* s);
-void remove_language_marker_inplace(char16_t* s);
-std::string add_language_marker(const std::string& s, char marker);
-std::u16string add_language_marker(const std::u16string& s, char16_t marker);
-std::string remove_language_marker(const std::string& s);
-std::u16string remove_language_marker(const std::u16string& s);
 
+// (2) Type conversion functions
+
+template <typename DestT, typename SrcT = DestT>
+size_t text_strncpy_t(DestT*, size_t, const SrcT*, size_t) {
+  static_assert(always_false<DestT, SrcT>::v,
+      "unspecialized text_strncpy_t should never be called");
+  return 0;
+}
+
+template <>
+inline size_t text_strncpy_t<char>(
+    char* dest, size_t dest_count, const char* src, size_t src_count) {
+  size_t count = std::min<size_t>(dest_count, src_count);
+  return text_strncpy_t(dest, src, count);
+}
+
+template <>
+inline size_t text_strncpy_t<char, char16_t>(
+    char* dest, size_t dest_count, const char16_t* src, size_t src_count) {
+  return encode_sjis(dest, dest_count, src, src_count, true);
+}
+
+template <>
+inline size_t text_strncpy_t<char16_t, char>(
+    char16_t* dest, size_t dest_count, const char* src, size_t src_count) {
+  return decode_sjis(dest, dest_count, src, src_count, true);
+}
+
+template <>
+inline size_t text_strncpy_t<char16_t>(
+    char16_t* dest, size_t dest_count, const char16_t* src, size_t src_count) {
+  size_t count = std::min<size_t>(dest_count, src_count);
+  return text_strncpy_t(dest, src, count);
+}
+
+template <typename DestT, typename SrcT = DestT>
+size_t text_strnzcpy_t(DestT*, size_t, const SrcT*, size_t) {
+  static_assert(always_false<DestT, SrcT>::v,
+      "unspecialized text_strnzcpy_t should never be called");
+  return 0;
+}
+
+template <>
+inline size_t text_strnzcpy_t<char>(
+    char* dest, size_t dest_count, const char* src, size_t src_count) {
+  size_t count = std::min<size_t>(dest_count, src_count);
+  return text_strnzcpy_t(dest, src, count);
+}
+
+template <>
+inline size_t text_strnzcpy_t<char, char16_t>(
+    char* dest, size_t dest_count, const char16_t* src, size_t src_count) {
+  return encode_sjis(dest, dest_count, src, src_count);
+}
+
+template <>
+inline size_t text_strnzcpy_t<char16_t, char>(
+    char16_t* dest, size_t dest_count, const char* src, size_t src_count) {
+  return decode_sjis(dest, dest_count, src, src_count);
+}
+
+template <>
+inline size_t text_strnzcpy_t<char16_t>(
+    char16_t* dest, size_t dest_count, const char16_t* src, size_t src_count) {
+  size_t count = std::min<size_t>(dest_count, src_count);
+  return text_strnzcpy_t(dest, src, count);
+}
+
+
+
+// (3) Packed text objects for use in protocol structs
+
+template <typename ItemT, size_t Count>
+struct parray {
+  ItemT items[Count];
+
+  parray() {
+    this->clear();
+  }
+  parray(const parray& other) {
+    this->operator=(other);
+  }
+  parray(parray&& s) = delete;
+
+  template <size_t OtherCount>
+  parray(const parray<ItemT, OtherCount>& s) {
+    this->operator=(s);
+  }
+
+  constexpr size_t size() {
+    return Count;
+  }
+  constexpr size_t bytes() {
+    return Count * sizeof(ItemT);
+  }
+  ItemT* data() {
+    return this->items;
+  }
+  const ItemT* data() const {
+    return this->items;
+  }
+
+  // TODO: These can be made faster by only clearing the unused space after the
+  // strncpy_t (if any) instead of clearing all the space every time
+  parray& operator=(const parray& s) {
+    for (size_t x = 0; x < Count; x++) {
+      this->items[x] = s.items[x];
+    }
+    return *this;
+  }
+  parray& operator=(parray&& s) = delete;
+
+  template <size_t OtherCount>
+  parray& operator=(const parray<ItemT, OtherCount>& s) {
+    if (OtherCount <= Count) {
+      size_t x;
+      for (x = 0; x < OtherCount; x++) {
+        this->items[x] = s.items[x];
+      }
+      for (; x < Count; x++) {
+        this->items[x] = 0;
+      }
+    } else {
+      for (size_t x = 0; x < Count; x++) {
+        this->items[x] = s.items[x];
+      }
+    }
+    return *this;
+  }
+
+  parray& operator=(const ItemT* s) {
+    for (size_t x = 0; x < Count; x++) {
+      this->items[x] = s[x];
+    }
+    return *this;
+  }
+
+  bool operator==(const parray& s) const {
+    for (size_t x = 0; x < Count; x++) {
+      if (this->items[x] != s.items[x]) {
+        return false;
+      }
+    }
+    return true;
+  }
+  bool operator!=(const parray& s) const {
+    return !this->operator==(s);
+  }
+
+  void clear(ItemT v = 0) {
+    for (size_t x = 0; x < Count; x++) {
+      this->items[x] = v;
+    }
+  }
+  void clear_after(size_t position, ItemT v = 0) {
+    for (size_t x = position; x < Count; x++) {
+      this->items[x] = v;
+    }
+  }
+} __attribute__((packed));
+
+
+// TODO: It appears that these actually do not have to be null-terminated in PSO
+// commands some of the time. As an example, creating a game with a name with
+// the maximum length results in a C1 command with no null byte between the game
+// name and the password. We should be able to handle this by making ptexts not
+// required to be null-terminated in storage - this will still be safe if we
+// limit all operations by Count.
+template <typename CharT, size_t Count>
+struct ptext : parray<CharT, Count> {
+  ptext() {
+    this->clear();
+  }
+  ptext(const ptext& other) : parray<CharT, Count>(other) { }
+  ptext(ptext&& s) = delete;
+
+  template <typename OtherCharT>
+  ptext(const OtherCharT* s) {
+    this->operator=(s);
+  }
+  template <typename OtherCharT>
+  ptext(const OtherCharT* s, size_t count) {
+    this->assign(s, count);
+  }
+  template <typename OtherCharT>
+  ptext(const std::basic_string<OtherCharT>& s) {
+    this->operator=(s);
+  }
+  template <typename OtherCharT, size_t OtherCount>
+  ptext(const ptext<OtherCharT, OtherCount>& s) {
+    this->operator=(s);
+  }
+
+  size_t len() const {
+    return text_strnlen_t(this->items, Count);
+  }
+
+  // Q: Why is there no c_str() here?
+  // A: Because the contents of a ptext don't have to be null-terminated.
+
+  ptext& operator=(const ptext& s) {
+    memcpy(this->items, s.items, sizeof(CharT) * Count);
+    return *this;
+  }
+  ptext& operator=(ptext&& s) = delete;
+
+  template <typename OtherCharT>
+  ptext& operator=(const OtherCharT* s) {
+    size_t chars_written = text_strncpy_t(this->items, Count, s, Count);
+    this->clear_after(chars_written);
+    return *this;
+  }
+  template <typename OtherCharT>
+  ptext& assign(const OtherCharT* s, size_t s_count) {
+    size_t chars_written = text_strncpy_t(this->items, Count, s, s_count);
+    this->clear_after(chars_written);
+    return *this;
+  }
+  template <typename OtherCharT>
+  ptext& operator=(const std::basic_string<OtherCharT>& s) {
+    size_t chars_written = text_strncpy_t(this->items, Count, s.c_str(), s.size());
+    this->clear_after(chars_written);
+    return *this;
+  }
+  template <typename OtherCharT, size_t OtherCount>
+  ptext& operator=(const ptext<OtherCharT, OtherCount>& s) {
+    size_t chars_written = text_strncpy_t(this->items, Count, s.items, OtherCount);
+    this->clear_after(chars_written);
+    return *this;
+  }
+
+  template <typename OtherCharT>
+  bool operator==(const OtherCharT* s) const {
+    return text_strneq_t(this->items, s, Count);
+  }
+  template <typename OtherCharT>
+  bool operator==(const std::basic_string<OtherCharT>& s) const {
+    return text_strneq_t(this->items, s.c_str(), Count);
+  }
+  template <typename OtherCharT, size_t OtherCount>
+  bool operator==(const ptext<OtherCharT, OtherCount>& s) const {
+    return text_strneq_t(this->items, s.items, std::min<size_t>(Count, OtherCount));
+  }
+  template <typename OtherCharT>
+  bool operator!=(const OtherCharT* s) const {
+    return !this->operator==(s);
+  }
+  template <typename OtherCharT>
+  bool operator!=(const std::basic_string<OtherCharT>& s) const {
+    return !this->operator==(s);
+  }
+  template <typename OtherCharT, size_t OtherCount>
+  bool operator!=(const ptext<OtherCharT, OtherCount>& s) const {
+    return !this->operator==(s);
+  }
+
+  template <typename OtherCharT>
+  bool eq_n(const OtherCharT* s, size_t count) const {
+    return text_strneq_t(this->items, s, count);
+  }
+  template <typename OtherCharT>
+  bool eq_n(const std::basic_string<OtherCharT>& s, size_t count) const {
+    return text_strneq_t(this->items, s.c_str(), count);
+  }
+  template <typename OtherCharT, size_t OtherCount>
+  bool eq_n(const ptext<OtherCharT, OtherCount>& s, size_t count) const {
+    return text_strneq_t(this->items, s.items, count);
+  }
+
+  operator std::basic_string<CharT>() const {
+    std::basic_string<CharT> ret(this->items, Count);
+    strip_trailing_zeroes(ret);
+    return ret;
+  }
+
+  bool empty() const {
+    return (this->items[0] == 0);
+  }
+} __attribute__((packed));
+
+
+
+// (4) Markers and character replacement
+
+template <typename CharT>
+std::basic_string<CharT> add_language_marker(
+    const std::basic_string<CharT>& s, CharT marker) {
+  if ((s.size() >= 2) && (s[0] == '\t') && (s[1] != 'C')) {
+    return s;
+  }
+
+  std::basic_string<CharT> ret;
+  ret.push_back('\t');
+  ret.push_back(marker);
+  ret += s;
+  return ret;
+}
+
+template <typename CharT, size_t Count>
+std::basic_string<CharT> add_language_marker(
+    const ptext<CharT, Count>& s, CharT marker) {
+  if ((s.items[0] == '\t') && (s.items[1] != 'C')) {
+    return s;
+  }
+
+  std::basic_string<CharT> ret;
+  ret.push_back('\t');
+  ret.push_back(marker);
+  ret += s;
+  return ret;
+}
+
+template <typename CharT>
+const CharT* remove_language_marker(const CharT* s) {
+  if ((s[0] != '\t') || (s[1] == 'C')) {
+    return s;
+  }
+  return s + 2;
+}
+
+template <typename CharT, size_t Count>
+std::basic_string<CharT> remove_language_marker(const ptext<CharT, Count>& s) {
+  if ((s.items[0] != '\t') || (s.items[1] == L'C')) {
+    return s;
+  }
+  return &s.items[2];
+}
+
+template <typename CharT>
+std::basic_string<CharT> remove_language_marker(
+    const std::basic_string<CharT>& s) {
+  if ((s.size() < 2) || (s[0] != L'\t') || (s[1] == L'C')) {
+    return s;
+  }
+  return s.substr(2);
+}
+
+template <typename CharT, size_t Count>
+void remove_language_marker_inplace(ptext<CharT, Count>& a) {
+  if ((a.items[0] == '\t') && (a.items[1] != 'C')) {
+    text_strnzcpy_t(a.items, Count, &a.items[2], Count);
+    a.items[text_strlen_t(a.items) + 1] = 0;
+  }
+}
 
 template <typename T>
 void replace_char_inplace(T* a, T f, T r) {
@@ -40,23 +471,26 @@ void replace_char_inplace(T* a, T f, T r) {
 }
 
 template <typename T>
-size_t add_color_inplace(T* a) {
+size_t add_color_inplace(T* a, size_t max_chars) {
   T* d = a;
   T* orig_d = d;
 
-  while (*a) {
+  for (size_t x = 0; (x < max_chars) && *a; x++) {
     if (*a == '$') {
       *(d++) = '\t';
     } else if (*a == '#') {
       *(d++) = '\n';
     } else if (*a == '%') {
       a++;
+      x++;
       if (*a == 's') {
         *(d++) = '$';
       } else if (*a == '%') {
         *(d++) = '%';
       } else if (*a == 'n') {
         *(d++) = '#';
+      } else if (*a == '\0') {
+        break;
       } else {
         *(d++) = *a;
       }
@@ -66,12 +500,69 @@ size_t add_color_inplace(T* a) {
     a++;
   }
   *d = 0;
+  // TODO: we should clear the chars after the null if the new string is shorter
+  // than the original
 
   return d - orig_d;
 }
 
 template <typename T>
-void add_color_inplace(std::basic_string<T>& a, size_t header_bytes) {
-  size_t count = add_color_inplace(a.data() + header_bytes);
-  a.resize(count + header_bytes);
+void add_color(StringWriter& w, const T* src, size_t max_input_chars) {
+  for (size_t x = 0; (x < max_input_chars) && *src; x++) {
+    if (*src == '$') {
+      w.put<T>('\t');
+    } else if (*src == '#') {
+      w.put<T>('\n');
+    } else if (*src == '%') {
+      src++;
+      x++;
+      if (*src == 's') {
+        w.put<T>('$');
+      } else if (*src == '%') {
+        w.put<T>('%');
+      } else if (*src == 'n') {
+        w.put<T>('#');
+      } else if (*src == '\0') {
+        break;
+      } else {
+        w.put<T>(*src);
+      }
+    } else {
+      w.put<T>(*src);
+    }
+    src++;
+  }
+  w.put<T>(0);
+}
+
+template <typename CharT, size_t Count>
+void add_color_inplace(ptext<CharT, Count>& t) {
+  size_t sx = 0;
+  size_t dx = 0;
+  for (; (sx < Count - 1) && t.items[sx]; sx++) {
+    if (t.items[sx] == '$') {
+      t.items[dx] = '\t';
+    } else if (t.items[sx] == '#') {
+      t.items[dx] = '\n';
+    } else if (t.items[sx] == '%') {
+      sx++;
+      if ((sx == Count - 1) || (t.items[sx] == '\0')) {
+        break;
+      } else if (t.items[sx] == 's') {
+        t.items[dx] = '$';
+      } else if (t.items[sx] == '%') {
+        t.items[dx] = '%';
+      } else if (t.items[sx] == 'n') {
+        t.items[dx] = '#';
+      } else {
+        t.items[dx] = t.items[sx];
+      }
+    } else {
+      t.items[dx] = t.items[sx];
+    }
+    dx++;
+  }
+  for (; dx < Count; dx++) {
+    t.items[dx] = 0;
+  }
 }

src/Version.cc

@@ -1,8 +1,10 @@
 #include "Version.hh"
 
+#include <strings.h>
+
 #include <stdexcept>
 
-#include <strings.h>
+#include "Client.hh"
 
 using namespace std;
 
@@ -13,34 +15,34 @@ uint16_t flags_for_version(GameVersion version, uint8_t sub_version) {
     case 0x00: // initial check (before 9E recognition)
       switch (version) {
         case GameVersion::DC:
-          return ClientFlag::DefaultV2DC;
+          return Client::Flag::DEFAULT_V2_DC;
         case GameVersion::GC:
-          return ClientFlag::DefaultV3GC;
+          return Client::Flag::DEFAULT_V3_GC;
         case GameVersion::PC:
-          return ClientFlag::DefaultV2PC;
-        case GameVersion::Patch:
-          return ClientFlag::DefaultV2PC;
+          return Client::Flag::DEFAULT_V2_PC;
+        case GameVersion::PATCH:
+          return Client::Flag::DEFAULT_V2_PC;
         case GameVersion::BB:
-          return ClientFlag::DefaultV3BB;
+          return Client::Flag::DEFAULT_V4_BB;
       }
       break;
     case 0x29: // PSO PC
-      return ClientFlag::DefaultV2PC;
+      return Client::Flag::DEFAULT_V2_PC;
     case 0x30: // ???
     case 0x31: // PSO Ep1&2 US10, US11, EU10, JP10
     case 0x33: // PSO Ep1&2 EU50HZ
     case 0x34: // PSO Ep1&2 JP11
-      return ClientFlag::DefaultV3GC;
+      return Client::Flag::DEFAULT_V3_GC;
     case 0x32: // PSO Ep1&2 US12, JP12
     case 0x35: // PSO Ep1&2 US12, JP12
     case 0x36: // PSO Ep1&2 US12, JP12
     case 0x39: // PSO Ep1&2 US12, JP12
-      return ClientFlag::DefaultV3GCPlus;
+      return Client::Flag::DEFAULT_V3_GC_PLUS;
     case 0x40: // PSO Ep3 trial
     case 0x41: // PSO Ep3 US
     case 0x42: // PSO Ep3 JP
     case 0x43: // PSO Ep3 UK
-      return ClientFlag::DefaultV4;
+      return Client::Flag::DEFAULT_V3_GC_EP3;
   }
   return 0;
 }
@@ -55,7 +57,7 @@ const char* name_for_version(GameVersion version) {
       return "BB";
     case GameVersion::DC:
       return "DC";
-    case GameVersion::Patch:
+    case GameVersion::PATCH:
       return "Patch";
     default:
       return "Unknown";
@@ -65,16 +67,53 @@ const char* name_for_version(GameVersion version) {
 GameVersion version_for_name(const char* name) {
   if (!strcasecmp(name, "DC") || !strcasecmp(name, "DreamCast")) {
     return GameVersion::DC;
-  }
-  if (!strcasecmp(name, "PC")) {
+  } else if (!strcasecmp(name, "PC")) {
     return GameVersion::PC;
-  }
-  if (!strcasecmp(name, "GC") || !strcasecmp(name, "GameCube")) {
+  } else if (!strcasecmp(name, "GC") || !strcasecmp(name, "GameCube")) {
     return GameVersion::GC;
-  }
-  if (!strcasecmp(name, "BB") || !strcasecmp(name, "BlueBurst") ||
+  } else if (!strcasecmp(name, "BB") || !strcasecmp(name, "BlueBurst") ||
       !strcasecmp(name, "Blue Burst")) {
     return GameVersion::BB;
+  } else if (!strcasecmp(name, "Patch")) {
+    return GameVersion::PATCH;
+  } else {
+    throw invalid_argument("incorrect version name");
   }
-  throw invalid_argument("incorrect version name");
 }
+
+const char* name_for_server_behavior(ServerBehavior behavior) {
+  switch (behavior) {
+    case ServerBehavior::SPLIT_RECONNECT:
+      return "split_reconnect";
+    case ServerBehavior::LOGIN_SERVER:
+      return "login_server";
+    case ServerBehavior::LOBBY_SERVER:
+      return "lobby_server";
+    case ServerBehavior::DATA_SERVER_BB:
+      return "data_server_bb";
+    case ServerBehavior::PATCH_SERVER:
+      return "patch_server";
+    case ServerBehavior::PROXY_SERVER:
+      return "proxy_server";
+    default:
+      throw logic_error("invalid server behavior");
+  }
+}
+
+ServerBehavior server_behavior_for_name(const char* name) {
+  if (!strcasecmp(name, "split_reconnect")) {
+    return ServerBehavior::SPLIT_RECONNECT;
+  } else if (!strcasecmp(name, "login_server") || !strcasecmp(name, "login")) {
+    return ServerBehavior::LOGIN_SERVER;
+  } else if (!strcasecmp(name, "lobby_server") || !strcasecmp(name, "lobby")) {
+    return ServerBehavior::LOBBY_SERVER;
+  } else if (!strcasecmp(name, "data_server_bb") || !strcasecmp(name, "data_server") || !strcasecmp(name, "data")) {
+    return ServerBehavior::DATA_SERVER_BB;
+  } else if (!strcasecmp(name, "patch_server") || !strcasecmp(name, "patch")) {
+    return ServerBehavior::PATCH_SERVER;
+  } else if (!strcasecmp(name, "proxy_server") || !strcasecmp(name, "proxy")) {
+    return ServerBehavior::PROXY_SERVER;
+  } else {
+    throw invalid_argument("incorrect server behavior name");
+  }
+}

src/Version.hh

@@ -7,41 +7,23 @@
 enum class GameVersion {
   DC = 0,
   PC,
-  Patch,
+  PATCH,
   GC,
   BB,
 };
 
-enum ClientFlag {
-  // After joining a lobby, client will no longer send D6 commands when they close message boxes
-  NoMessageBoxCloseConfirmationAfterLobbyJoin = 0x0004,
-  // Client has the above flag and has already joined a lobby
-  NoMessageBoxCloseConfirmation = 0x0008,
-  // Client can see Ep3 lobbies
-  CanSeeExtraLobbies = 0x0010,
-  // Client is episode 3 and should use its game mechanic
-  Episode3Games = 0x0020,
-  // Client is DC v1 (disables some features)
-  IsDCv1 = 0x0040,
-  // Client is loading into a game
-  Loading = 0x0080,
-
-  // Client is in the information menu (login server only)
-  InInformationMenu = 0x0100,
-  // Client is at the welcome message (login server only)
-  AtWelcomeMessage = 0x0200,
-
-  // Note: There isn't a good way to detect Episode 3 until the player data is
-  // sent (via a 61 command), so the Episode3Games flag is set in that handler
-  DefaultV1 = IsDCv1,
-  DefaultV2DC = 0x0000,
-  DefaultV2PC = 0x0000,
-  DefaultV3GC = 0x0000,
-  DefaultV3GCPlus = NoMessageBoxCloseConfirmationAfterLobbyJoin,
-  DefaultV3BB = NoMessageBoxCloseConfirmationAfterLobbyJoin | NoMessageBoxCloseConfirmation,
-  DefaultV4 = NoMessageBoxCloseConfirmationAfterLobbyJoin | CanSeeExtraLobbies | Episode3Games,
+enum class ServerBehavior {
+  SPLIT_RECONNECT = 0,
+  LOGIN_SERVER,
+  LOBBY_SERVER,
+  DATA_SERVER_BB,
+  PATCH_SERVER,
+  PROXY_SERVER,
 };
 
 uint16_t flags_for_version(GameVersion version, uint8_t sub_version);
 const char* name_for_version(GameVersion version);
 GameVersion version_for_name(const char* name);
+
+const char* name_for_server_behavior(ServerBehavior behavior);
+ServerBehavior server_behavior_for_name(const char* name);

system/config.example.json

@@ -2,7 +2,7 @@
   // Configuration file for newserv. This file is standard JSON with comments.
 
   // Server's name (maximum 16 characters)
-  "ServerName": "Alexandria",
+  "ServerName": "newserv",
 
   // Address to connect local clients to (IP address or interface name)
   "LocalAddress": "en0",
@@ -11,24 +11,75 @@
   // address.
   "ExternalAddress": "10.0.1.5",
 
-  // Port to listen for DNS queries on. The DNS server is disabled by default;
-  // to enable it, uncomment this line.
-  // "DNSServerPort": 53,
+  // Port to listen for DNS queries on. To disable the DNS server, comment this
+  // out or set it to zero.
+  "DNSServerPort": 53,
+
+  // Ports to listen for game connections on.
+  "PortConfiguration": {
+    // name: [port, version, behavior]
+
+    // Various versions of PSO hardcode these ports in the clients. Don't change
+    // these unless you don't want to support certain versions of PSO.
+    "gc-jp10":   [9000,  "gc",    "login_server"],
+    "gc-jp11":   [9001,  "gc",    "login_server"],
+    "gc-jp3":    [9003,  "gc",    "login_server"],
+    "gc-us10":   [9100,  "pc",    "split_reconnect"],
+    "gc-us3":    [9103,  "gc",    "login_server"],
+    "gc-eu10":   [9200,  "gc",    "login_server"],
+    "gc-eu11":   [9201,  "gc",    "login_server"],
+    "gc-eu3":    [9203,  "gc",    "login_server"],
+    "pc-login":  [9300,  "pc",    "login_server"],
+    "pc-patch":  [10000, "patch", "patch_server"],
+    "bb-patch":  [11000, "patch", "patch_server"],
+    "bb-init":   [12000, "bb",    "data_server_bb"],
+
+    // Schthack PSOBB uses these ports.
+    // "bb-patch2": [10500, "patch", "patch_server"],
+    // "bb-init2":  [13000, "bb",    "data_server_bb"],
+
+    // Ephinea PSOBB uses these ports. Note that 13000 is also used by Schthack
+    // PSOBB, but not for the patch server; this means you unfortunately can't
+    // support both Schthack and Ephinea PSOBB clients at the same time.
+    // "bb-patch3": [13000, "patch", "patch_server"],
+    // "bb-init3":  [14000, "bb",    "data_server_bb"],
+
+    // newserv uses these ports, but there is no external reason that these
+    // numbers were chosen. You can change the port numbers here without any
+    // issues. Note that the bb-data1 and bb-data2 ports must be sequential;
+    // that is, the bb-data2 port must be the bb-data1 port + 1.
+    "pc-lobby":  [9420,  "pc",    "lobby_server"],
+    "gc-lobby":  [9421,  "gc",    "lobby_server"],
+    "bb-lobby":  [9422,  "bb",    "lobby_server"],
+    "pc-proxy":  [9520,  "pc",    "proxy_server"],
+    "gc-proxy":  [9521,  "gc",    "proxy_server"],
+    "bb-proxy":  [9522,  "bb",    "proxy_server"],
+    "bb-data1":  [12004, "bb",    "data_server_bb"],
+    "bb-data2":  [12005, "bb",    "data_server_bb"],
+  },
 
   // Where to listen for IP stack clients. This exists to interface with PSO GC
-  // clients running in a local Dolphin emulator; the default value given here
-  // matches the socket path that Dolphin uses for BBA connections. (To disable
-  // this, comment out this line.) To enable local Dolphin clients to connect,
-  // set this to ["/tmp/dolphin-tap"] and configure Dolphin to use the tapserver
-  // type of broadband adapter. You do not need to install or run tapserver. See
-  // README.md for details on how to get PSO to connect via this interface.
+  // clients running in a local Dolphin emulator. To enable local Dolphin
+  // clients to connect, set this to ["/tmp/dolphin-tap"] and configure Dolphin
+  // to use the tapserver type of broadband adapter. You do not need to install
+  // or run tapserver. See README.md for details on how to get PSO to connect
+  // via this interface.
   // If you're doing unusual things, you can also add numbers or "address:port"
   // strings to this list to listen for tapserver connections on a TCP port.
-  "IPStackListen": ["/tmp/dolphin-tap"],
+  "IPStackListen": [],
 
-  // Set this to true to show a lot of debugging information from the IP stack
-  // simulator. This is generally only useful for finding bugs in the interface.
-  // "IPStackDebug": true,
+  // Other servers to support proxying to. If either of these is empty, the
+  // proxy server is disabled for that game version. Entries are like
+  // "name": "address:port"; the names are used in the proxy server menu.
+  "ProxyDestinations-GC": {},
+  "ProxyDestinations-PC": {},
+  // Proxy destination for patch server clients. If this is given, the internal
+  // patch server (for PC and BB) is bypassed, and any client that connects to
+  // the patch server is instead proxied to this destination.
+  // "ProxyDestination-Patch": "",
+  // Proxy destination for BB clients. If this is given, all BB clients that
+  // connect to newserv will be proxied to this destination.
+  // "ProxyDestination-BB": "",
 
   // By default, the interactive shell runs if stdin is a terminal, and doesn't
   // run if it's not. This option, if present, overrides that behavior.
@@ -41,15 +92,11 @@
   // users cannot be banned!
   "AllowUnregisteredUsers": false,
 
-  // If you want to use Blue Burst clients with a different private key, put a
-  // .nsk file in system/blueburst/keys and put its name here.
-  "BlueBurstKeyFile": "default",
-
   // User to run the server as. If present, newserv will attempt to switch to
   // this user's permissions after loading its configuration and opening
   // listening sockets. The special value $SUDO_USER causes newserv to look up
   // the desired username in the $SUDO_USER variable instead.
-  "User": "$SUDO_USER",
+  // "User": "$SUDO_USER",
 
   // Information menu contents. Each entry is a 3-list of
   // [title, short description, full contents]. In the short description and
@@ -73,9 +120,11 @@
     ["Ep3 lobby types", "$C7Display lobby type\nlist for Episode\nIII", "These values can be used with the %sln command.\n$C6*$C7 indicates lobbies where players can't move.\n$C8Pink$C7 indicates Episode 3 only lobbies.\n\nnormal - standard lobby\n$C8planet$C7 - Blank Ragol Lobby\n$C8clouds$C7 - Blank Sky Lobby\n$C8cave$C7 - Unguis Lapis\n$C8jungle$C7 - Episode 2 Jungle\n$C8forest2-1$C7 - Episode 1 Forest 2 (ground)\n$C8forest2-2$C7 - Episode 1 Forest 2 (near Dome)\n$C8windpower$C7\n$C8overview$C7\n$C8seaside$C7 - Episode 2 Seaside\n$C8some?$C7\n$C8dmorgue$C7 - Destroyed Morgue\n$C8caelum$C7 - Caelum\n$C8digital$C7\n$C8boss1$C7\n$C8boss2$C7\n$C8boss3$C7\n$C8knight$C7 - Leukon Knight stage\n$C8sky$C7 - Via Tubus\n$C8morgue$C7 - Morgue"],
     ["Area list",       "$C7Display stage code\nlist", "These values can be used with the $C6%swarp$C7 command.\n\n$C2Green$C7 areas will be empty unless you are in a quest.\n$C6Yellow$C7 areas will not allow you to move.\n\n   $C8Episode 1  / Episode 2        / Episode 4$C7\n0: Pioneer 2  / Pioneer 2        / Pioneer 2\n1: Forest 1   / Temple Alpha    / Crater East\n2: Forest 2   / Temple Beta     / Crater West\n3: Caves 1    / Spaceship Alpha / Crater South\n4: Caves 2    / Spaceship Beta  / Crater North\n5: Caves 3    / CCA              / Crater Interior\n6: Mines 1    / Jungle North     / Desert 1\n7: Mines 2    / Jungle South     / Desert 2\n8: Ruins 1     / Mountain         / Desert 3\n9: Ruins 2     / Seaside          / Saint Million\n10: Ruins 3   / Seabed Upper    / $C6Purgatory$C7\n11: Dragon    / Seabed Lower\n12: De Rol Le / Gal Gryphon\n13: Vol Opt   / Olga Flow\n14: Dark Falz / Barba Ray\n15: $C2Lobby$C7     / Gol Dragon\n16: $C6Battle 1$C7  / $C6Seaside Night$C7\n17: $C6Battle 2$C7  / $C2Tower$C7"],
   ],
-  // Welcome message. If not blank, this message will be shown to console users
-  // upon first connecting.
-  "WelcomeMessage": "Welcome to $C6Alexandria$C7, a private PSO server\npowered by newserv.",
+  // Welcome message. If not blank, this message will be shown to PSO GC users
+  // upon first connecting. (If this is blank, they will be taken directly to
+  // the main menu instead.) The welcome message is never shown to PSO PC or PSO
+  // BB users.
+  "WelcomeMessage": "",
 
   // Item drop rates for non-rare items in BB games. For each type (boxes or
   // enemies), all the categories must add up to a number less than 0x100000000.