hide other players' EXP values in ServerEXPDisplay

update

.github/workflows/cmake.yml

@@ -27,7 +27,7 @@ jobs:
       - name: Install libraries (macOS)
         if: ${{ matrix.os == 'macos-latest' }}
         run: |
-          brew install cmake asio libiconv
+          brew install asio libiconv
 
           cat << EOF > nproc
           #!/bin/sh

.gitignore

@@ -14,9 +14,11 @@ CTestTestfile.cmake
 install_manifest.txt
 Makefile
 Testing
+build
 
 # Files modified by the user and/or server that don't have defaults
 system/config.json
+system/ep3/battle-records/*.mzr
 system/ep3/battle-records/*.mzrd
 system/ep3/tournament-state.json
 system/licenses.nsi
@@ -35,6 +37,7 @@ system/patch-bb/.metadata-cache.json
 # repository
 files
 make_release.py
+notes-private
 old-khyller
 old-newserv
 release

CMakeLists.txt

@@ -101,8 +101,9 @@ set(SOURCES
     src/Map.cc
     src/Menu.cc
     src/NetworkAddresses.cc
+    src/PatchDownloadSession.cc
     src/PatchFileIndex.cc
-    src/PlayerFilesManager.cc
+    src/PlayerInventory.cc
     src/PlayerSubordinates.cc
     src/PPKArchive.cc
     src/ProxyCommands.cc
@@ -111,6 +112,7 @@ set(SOURCES
     src/PSOGCObjectGraph.cc
     src/PSOProtocol.cc
     src/Quest.cc
+    src/QuestMetadata.cc
     src/QuestScript.cc
     src/RareItemSet.cc
     src/ReceiveCommands.cc
@@ -136,6 +138,7 @@ target_link_libraries(newserv phosg::phosg ${Iconv_LIBRARIES} pthread resource_f
 if (WIN32)
     target_compile_definitions(newserv PUBLIC -DWINVER=0x0A00 -D_WIN32_WINNT=0x0A00)
     target_link_libraries(newserv ws2_32 mswsock bcrypt iphlpapi -static -static-libgcc -static-libstdc++)
+    target_compile_options(newserv PRIVATE -Wa,-mbig-obj)
 endif()
 add_dependencies(newserv newserv-Revision-cc)
 

README.md

@@ -26,7 +26,7 @@ See TODO.md for a list of known issues and future work I've curated, or go to th
     * [Cross-version play](#cross-version-play)
     * [Server-side saves](#server-side-saves)
     * [Episode 3 features](#episode-3-features)
-    * [Memory patches, client functions, and DOL files](#memory-patches-client-functions-and-dol-files)
+    * [Memory patches, client functions, and DOL files](#memory-patches-and-client-functions)
     * [Using newserv as a proxy](#using-newserv-as-a-proxy)
     * [Chat commands](#chat-commands)
     * [REST API](#rest-api)
@@ -54,16 +54,17 @@ At the time of its inception, Aeon was also called newserv, and you may find som
 
 Independently of this project, there are many other PSO servers out there. Those that I know of that are (or were) public are listed here in approximate chronological order:
 
-* (Early 2000s) **[Schtserv](https://schtserv.com/)**: The first public-access PSO server; written in Delphi by Schthack. Still active and popular as of early 2025. Schtserv is also the only other unofficial server to support all versions of PSO, including Episode 3. (Their implementation of Episode 3 is based on newserv's, which is itself based on Sega's.)
+* (Early 2000s) **[Schtserv](https://schtserv.com/)**: The first public-access PSO server, written in Delphi by Schthack. Schtserv is the only other unofficial server to support Episode 3, their implementation of which is based on newserv's (which is based on Sega's).
 * (2005) **Khyller**: An early attempt of mine to support PSO PC, GC, and BB. See above for more details.
 * (2006) **Aeon**: My second attempt. Better than Khyller, but still unreliable.
-* (2008) **Tethealla**: A fairly extensive implementation of PSOBB, written in C by Sodaboy. The public version of Tethealla has been [officially disowned](https://www.pioneer2.net/community/threads/tethealla-server-forums-removal.26365/) (as it is now more than 15 years old), but closed-source development continues. [Ephinea](https://ephinea.pioneer2.net/), currently the most popular PSOBB server, is the continuation of this project. Several other modern PSOBB servers are forks of the initial public version of Tethealla as well.
-* (2008) **[Sylverant](https://sylverant.net/)** [(source)](https://sourceforge.net/projects/sylverant/): The second public-access PSO server; written in C by BlueCrab. Still active and popular as of early 2025.
+* (2008) **Tethealla**: A fairly extensive implementation of PSOBB, written in C by Sodaboy. The public version of Tethealla has been [officially disowned](https://www.pioneer2.net/community/threads/tethealla-server-forums-removal.26365/) as it is now more than 15 years old, but closed-source development continues. [Ephinea](https://ephinea.pioneer2.net/) is the continuation of this project. Several other modern PSOBB servers are forks of the initial public version of Tethealla as well.
+* (2008) **[Sylverant](https://sylverant.net/)** [(source)](https://sourceforge.net/projects/sylverant/): The second public-access PSO server, written in C by BlueCrab.
 * (2015) **[Archon](https://github.com/dcrodman/archon)**: A PSOBB server written in Go by Drew Rodman.
 * (2015) **[Idola](https://github.com/HybridEidolon/idolapsoserv)**: A PSOBB server written in Rust by HybridEidolon. Functionality status unknown; the project has been archived.
-* (2017) **[Aselia](https://github.com/Solybum/Aselia)**: A PSOBB server written written in C# by Soly. It seems this was planned to be open-source at some point, but that has not (yet) happened.
+* (2017) **[Aselia](https://github.com/Solybum/Aselia)**: A PSOBB server written in C# by Soly. It seems this was planned to be open-source at some point, but that has not (yet) happened.
 * (2018) **newserv**: This project right here.
 * (2019) **[Mechonis](https://gitlab.com/sora3087/mechonis)**: A PSOBB server with a microservice architecture written in TypeScript by TrueVision.
+* (2020) **[Booma.Server](https://github.com/HelloKitty/Booma.Server)**: A PSOBB server written in C# by Glader, with Soly's help.
 * (2021) **[Phantasmal World](https://github.com/DaanVandenBosch/phantasmal-world)**: A set of PSO tools, including a web-based model viewer and quest builder, and a PSO server, written by Daan Vanden Bosch.
 * (2021) **[Elseware](http://git.sharnoth.com/jake/elseware)**: A PSOBB server written in Rust by Jake.
 
@@ -117,14 +118,15 @@ newserv supports all known versions of PSO, including various development protot
 | GC Ep1&2 Plus   | Yes      | Yes      | Yes      |
 | GC Ep3 NTE      | Yes      | Yes (2)  | Yes      |
 | GC Ep3          | Yes      | Yes      | Yes      |
-| Xbox Ep1&2 Beta | Yes      | Yes      | Yes      |
-| Xbox Ep1&2      | Yes      | Yes      | Yes      |
+| Xbox Ep1&2 Beta | Yes (3)  | Yes (3)  | Yes (3)  |
+| Xbox Ep1&2      | Yes (3)  | Yes (3)  | Yes (3)  |
 | BB (vanilla)    | Yes      | Yes      | Yes      |
 | BB (Tethealla)  | Yes      | Yes      | Yes      |
 
 *Notes:*
 1. *This is the only version of PSO that doesn't have any way to identify the player's account - there is no serial number or username. For this reason, AllowUnregisteredUsers must be enabled in config.json to support PC NTE, and PC NTE players receive a random Guild Card number every time they connect. To prevent abuse, PC NTE support can be disabled in config.json.*
 2. *Episode 3 NTE battles are not well-tested; some things may not work. See notes/ep3-nte-differences.txt for a list of known differences between NTE and the final version. NTE and non-NTE players cannot battle each other.*
+3. *PSO Xbox connects through Xbox Live, so you can't easily host a private server for this version of the game. See the [How to connect](#pso-xbox) section.*
 
 # Setup
 
@@ -136,12 +138,10 @@ Currently newserv works on macOS, Windows, and Ubuntu Linux. It will likely work
 
 1. Download the latest release.zip file from the [releases page](https://github.com/fuzziqersoftware/newserv/releases).
 2. Extract the contents of the archive to some location on your computer.
-3. (Optional) If you want to change any config options, go into the system/ folder, open config.json in a text editor, and edit it to your liking. There are comments in the file that describe what all the options do.
+3. Go into the system/ folder, open config.json in a text editor, and edit it to your liking. There are comments in the file that describe what all the options do. Most of the options can be left alone if you want default behavior, but on Windows, you must change LocalAddress and ExternalAddress.
 4. (Optional) If you plan to play Blue Burst on newserv, set up the patch directory. See [client patch directories](#client-patch-directories) for details.
 5. Run the newserv executable.
 
-If you're on an older version of Windows (before Windows 10), the Cygwin libraries included with the release may be incompatible. See [this issue](https://github.com/fuzziqersoftware/newserv/issues/621) for a possible workaround.
-
 ### Linux
 
 There are currently no precompiled releases for Linux. To run newserv on Linux, you'll have to build it from source - see the section below.
@@ -168,6 +168,12 @@ To use newserv in other ways (e.g. for translating data), see the end of this do
 
 The current version of newserv is cross-compiled using mingw-w64 on a macOS build machine, with the necessary libraries manually installed. Setting up such a build environment is tedious and not recommended; it's recommended to just use a release version instead.
 
+Here is a rough outline of the Windows build process. You should only attempt this yourself if you're familiar with setting up build environments and can deal with issues you may encounter along the way.
+1. Install recent versions of MinGW and CMake.
+2. Build and install zlib, libiconv, asio, phosg, and resource_dasm into your MinGW environment.
+3. Clone the newserv repository with symlinks enabled: `git clone -c core.symlinks=true https://github.com/fuzziqersoftware/newserv.git`
+4. Build newserv via CMake.
+
 ## Client patch directories
 
 newserv implements a patch server for PSO PC and PSO BB game data. Any file or directory you put in the system/patch-bb or system/patch-pc directories will be synced to clients when they connect to the patch server.
@@ -257,6 +263,10 @@ If you're using the tapserver BBA or modem type, you can make it connect to a ne
 3. In PSO's network settings, enable DHCP ("Automatically obtain an IP address"), set DNS server address to "Automatic", and leave DHCP Hostname as "Not set". Leave the proxy server settings blank.
 4. Start an online game.
 
+### PSO Xbox
+
+Unfortunately, you can't easily host a private server for PSO Xbox because the Xbox version of the game tunnels its connections through Xbox Live. There is a modern replacement for Xbox Live named [Insignia](https://insignia.live/), which supports the three main PSO Xbox servers, but as of now does not support other private PSO servers.
+
 ### PSO BB
 
 The PSO BB client has been modified and distributed in many different forms. newserv supports most, but not all, of the common distributions. Unlike other versions, it's common for various BB clients to have different map files. It's important that the client and server have the same map files, so make sure to set up the patch directory based on the client you'll be using with newserv. (See the [client patch directories](#client-patch-directories) section for instructions on setting this up.)
@@ -308,7 +318,10 @@ For .dat files, the `LANGUAGE` token may be omitted. If it's present, then that
 
 For example, the GameCube version of Lost HEAT SWORD is in two files named `q058-gc-e.bin` and `q058-gc.dat`. newserv knows these files are quests because they're in the system/quests/ directory, it knows they're for PSO GC because the filenames contain `-gc`, it knows this is the English version of the quest because the .bin filename ends with `-e` (even though the .dat filename does not), and it puts them in the Retrieval category because the files are within the retrieval/ directory within system/quests/.
 
-Some quests (mostly battle and challenge mode quests) have additional JSON metadata files that describe how the server should handle them. These files include flags that can be used to hide the quest unless a preceding quest has been cleared, or to hide the quest unless purchased as a BB team reward. These metadata files are generally named similarly to their .bin and .dat counterparts, except the `VERSION` token may also be omitted if the metadata applies to all languages of the quest on all PSO versions. See system/quests/battle/b88001.json for documentation on the exact format of the JSON file.
+Some quests have additional JSON metadata files that describe how the server should handle them. These metadata files are generally named similarly to their .bin and .dat counterparts, except the `VERSION` token may also be omitted if the metadata applies to all languages of the quest on all PSO versions. See the comments in system/quests/retrieval/q058.json for all of the available options and how to use them. Some of the options are:
+- Disable or hide the quest if certain preceding quests aren't cleared or other conditions aren't met
+- Enable the quest to be joined while in progress
+- Override the common and/or rare item tables and set the allowed drop modes
 
 Some quests may also include a .pvr file, which contains an image used in the quest. These files are named similarly to their .bin and .dat counterparts.
 
@@ -342,7 +355,7 @@ There are multiple PSO quest formats out there; newserv supports all of them. It
 4. *Episode 3 quests don't go in the system/quests directory. See the [Episode 3 section](#episode-3-features) section below.*
 5. *Quest source can be assembled into a .bin or .bind file with `newserv assemble-quest-script FILENAME.txt`. See system/quests/retrieval/q058-gc-e.bin.txt for an annotated example; this is the English GameCube version of Lost HEAT SWORD.*
 
-Episode 3 download quests consist only of a .bin file - there is no corresponding .dat file. Episode 3 download quest files may be named with the .mnm extension instead of .bin, since the format is the same as the standard map files (in system/ep3/). These files can be encoded in any of the formats described above, except .qst.
+Episode 3 download quests consist only of a .bin file - there is no corresponding .dat file. Episode 3 download quest files may be named with the .mnm extension instead of .bin, since the format is the same as the standard map files (in system/ep3/maps/). These files can be encoded in any of the formats described above, except .qst.
 
 When newserv indexes the quests during startup, it will warn (but not fail) if any quests are corrupt or in unrecognized formats.
 
@@ -367,7 +380,7 @@ In the server drop modes, the item tables used to generate common items are in t
 
 ## Cross-version play
 
-All versions of PSO can see and interact with each other in the lobby. By default, newserv allows V1 and V2 players to play together, and allows GC and Xbox players to play together. You can change these rules to allow all versions to play together, or to prevent versions from playing together, with the CompatibilityGroups setting in config.json.
+All versions of PSO can see and interact with each other in the lobby. By default, newserv allows V1 and V2 players to play in games together, and allows GC and Xbox players to play in games together. You can change these rules to allow all versions to play in games together, or to prevent versions from playing in games together, with the CompatibilityGroups setting in config.json.
 
 There are several cross-version restrictions that always apply regardless of the compatibility groups setting:
 * DC V1 players cannot join DC V2 games if the game creator didn't choose to allow them.
@@ -439,16 +452,14 @@ Episode 3 state and game data is stored in the system/ep3 directory. The files i
 * card-text.mnr: Compressed card text archive. Generally only used for debugging.
 * card-text.mnrd: Decompressed card text archive; same format as TextCardE.bin. Generally only used for debugging.
 * com-decks.json: COM decks used in tournaments. The default decks in this file come from logs from Sega's servers, so the file doesn't include every COM deck Sega ever made - the rest are probably lost to time.
-* maps/: Online free battle and quest maps (.mnm/.bin/.mnmd/.bind files). newserv comes with the default online maps, as well as some fan-made variations and quests to help new players get up to speed.
-* maps-download/: Download maps and quests (.mnm/.bin/.mnmd/.bind files). There are two subcategories by default (download maps and Trial Edition download maps), but you can add more by editing QuestCategories in config.json. Categories that have flag 0x40 (Ep3 download) set are indexed from this directory; all others are indexed from system/quests/. Files in maps-download/ subdirectories have the same format as those in the maps/ directory, but should be named like `e###-gc3-LANGUAGE.EXT` (similar to how non-Episode 3 quests are named in the system/quests/ directory). If you want a map to be available for online play and for downloading, the file must exist in both maps/ and in a maps-download/ subdirectory (a symbolic link is acceptable).
-* maps-offline/: Offline map files. These are all the offline quests and free battle maps from the client, including some debugging/test maps that were inaccessible during normal play. To make them playable online, put the files in the maps/ directory.
+* maps/: Online free battle and quest maps (.mnm/.bin/.mnmd/.bind files). newserv comes with the default online maps, as well as some fan-made variations and quests to help new players get up to speed. Within the maps/ directory, each subdirectory is treated as a separate category and may be optionally downloadable or available at the battle setup counter. The category.json file in each subdirectory specifies the category's behavior; see system/ep3/maps/online/category.json for a documented example.
 * tournament-state.json: State of all active tournaments. This file is automatically written when any tournament changes state for any reason (e.g. a tournament is created/started/deleted or a match is resolved).
 
 There is no public editor for Episode 3 maps and quests, but the format is described fairly thoroughly in src/Episode3/DataIndexes.hh (see the MapDefinition structure). You'll need to use `newserv decompress-prs ...` to decompress a .bin or .mnm file before editing it, but you don't need to compress it again to use it - just put the .bind or .mnmd file in the maps directory and newserv will make it available.
 
 Like quests, Episode 3 card definitions, maps, and quests are cached in memory. If you've changed any of these files, you can run `reload ep3-cards` or `reload ep3-maps` in the interactive shell to make the changes take effect without restarting the server.
 
-## Memory patches, client functions, and DOL files
+## Memory patches and client functions
 
 You can put assembly files in the system/client-functions directory with filenames like PatchName.VERS.patch.s and they will appear in the Patches menu for clients that support client functions. Client functions are written in SH-4, PowerPC, or x86 assembly and are compiled when newserv is started. The assembly system's features are documented in the comments in system/client-functions/System/WriteMemoryGC.ppc.s.
 
@@ -456,50 +467,54 @@ The VERS token in client function filenames refers to the specific version of th
 
 The specific versions are:
 
-| Game                         | VERS | Architecture  |
-|------------------------------|------|---------------|
-| PSO DC Network Trial Edition | 1OJ1 | Not supported |
-| PSO DC 11/2000 prototype     | 1OJ2 | Not supported |
-| PSO DC 12/2000 prototype     | 1OJ3 | Not supported |
-| PSO DC 01/2001 prototype     | 1OJ4 | Not supported |
-| PSO DC v1 JP                 | 1OJF | Not supported |
-| PSO DC v1 US                 | 1OEF | Not supported |
-| PSO DC v1 EU                 | 1OPF | Not supported |
-| PSO DC 08/2001 prototype     | 2OJ5 | SH-4          |
-| PSO DC v2 JP                 | 2OJF | SH-4          |
-| PSO DC v2 US                 | 2OEF | SH-4          |
-| PSO DC v2 EU                 | 2OPF | SH-4          |
-| PSO PC (v2)                  | 2OJW | Not supported |
-| PSO GC Trial Edition         | 3OJT | PowerPC       |
-| PSO GC v1.2 JP               | 3OJ2 | PowerPC       |
-| PSO GC v1.3 JP               | 3OJ3 | PowerPC       |
-| PSO GC v1.4 (Plus) JP        | 3OJ4 | PowerPC       |
-| PSO GC v1.5 (Plus) JP        | 3OJ5 | PowerPC (1)   |
-| PSO GC v1.0 US               | 3OE0 | PowerPC       |
-| PSO GC v1.1 US               | 3OE1 | PowerPC       |
-| PSO GC v1.2 (Plus) US        | 3OE2 | PowerPC (1)   |
-| PSO GC v1.0 EU               | 3OP0 | PowerPC       |
-| PSO GC Ep3 Trial Edition     | 3SJT | PowerPC       |
-| PSO GC Ep3 JP                | 3SJ0 | PowerPC       |
-| PSO GC Ep3 US                | 3SE0 | PowerPC (1)   |
-| PSO GC Ep3 EU                | 3SP0 | PowerPC (1)   |
-| PSO Xbox Beta                | 4OJB | x86           |
-| PSO Xbox JP Disc             | 4OJD | x86           |
-| PSO Xbox JP TU               | 4OJU | x86           |
-| PSO Xbox US Disc             | 4OED | x86           |
-| PSO Xbox US TU               | 4OEU | x86           |
-| PSO Xbox EU Disc             | 4OPD | x86           |
-| PSO Xbox EU TU               | 4OPU | x86           |
-| PSO BB JP 1.25.11            | 59NJ | x86           |
-| PSO BB JP 1.25.13            | 59NL | x86           |
-| PSO BB Tethealla             | 59NL | x86           |
+| Game                         | VERS | CPU architecture               |
+|------------------------------|------|--------------------------------|
+| PSO DC Network Trial Edition | 1OJ1 | Client functions not supported |
+| PSO DC 11/2000 prototype     | 1OJ2 | Client functions not supported |
+| PSO DC 12/2000 prototype     | 1OJ3 | Client functions not supported |
+| PSO DC 01/2001 prototype     | 1OJ4 | Client functions not supported |
+| PSO DC v1 JP                 | 1OJF | Client functions not supported |
+| PSO DC v1 US                 | 1OEF | Client functions not supported |
+| PSO DC v1 EU                 | 1OPF | Client functions not supported |
+| PSO DC 08/2001 prototype     | 2OJ5 | SH-4                           |
+| PSO DC v2 JP                 | 2OJF | SH-4                           |
+| PSO DC v2 US                 | 2OEF | SH-4                           |
+| PSO DC v2 EU                 | 2OPF | SH-4                           |
+| PSO PC (v2) Trial Edition    | 2OJT | Client functions not supported |
+| PSO PC (v2) 04/2002          | 2OJW | Client functions not supported |
+| PSO PC (v2) 02/2003          | 2OJZ | Client functions not supported |
+| PSO GC Trial Edition         | 3OJT | PowerPC                        |
+| PSO GC v1.2 JP               | 3OJ2 | PowerPC                        |
+| PSO GC v1.3 JP               | 3OJ3 | PowerPC                        |
+| PSO GC v1.4 (Plus) JP        | 3OJ4 | PowerPC                        |
+| PSO GC v1.5 (Plus) JP        | 3OJ5 | PowerPC (1)                    |
+| PSO GC v1.0 US               | 3OE0 | PowerPC                        |
+| PSO GC v1.1 US               | 3OE1 | PowerPC                        |
+| PSO GC v1.2 (Plus) US        | 3OE2 | PowerPC (1)                    |
+| PSO GC v1.0 EU               | 3OP0 | PowerPC                        |
+| PSO GC Ep3 Trial Edition     | 3SJT | PowerPC                        |
+| PSO GC Ep3 JP                | 3SJ0 | PowerPC                        |
+| PSO GC Ep3 US                | 3SE0 | PowerPC (1)                    |
+| PSO GC Ep3 EU                | 3SP0 | PowerPC (1)                    |
+| PSO Xbox Beta                | 4OJB | x86                            |
+| PSO Xbox JP Disc             | 4OJD | x86                            |
+| PSO Xbox JP TU               | 4OJU | x86                            |
+| PSO Xbox US Disc             | 4OED | x86                            |
+| PSO Xbox US TU               | 4OEU | x86                            |
+| PSO Xbox EU Disc             | 4OPD | x86                            |
+| PSO Xbox EU TU               | 4OPU | x86                            |
+| PSO BB JP 1.25.11            | 59NJ | x86                            |
+| PSO BB JP 1.25.13            | 59NL | x86                            |
+| PSO BB Tethealla             | 59NL | x86                            |
 
 *Notes:*
 1. *Client functions are only supported on these versions if EnableSendFunctionCallQuestNumbers is set in config.json. See the comments there for more information.*
 
-newserv comes with a set of patches for many of the above versions, based on AR codes originally made by Ralf at GC-Forever and Aleron Ives. Many of them were originally posted in [this thread](https://www.gc-forever.com/forums/viewtopic.php?f=38&t=2050).
+newserv comes with a set of patches for many of the above versions. These are organized in subdirectories within system/client-functions/.
 
-You can also put DOL files in the system/dol directory, and they will appear in the Programs menu for GC clients. Selecting a DOL file there will load the file into the GameCube's memory and run it, just like the old homebrew loaders (PSUL and PSOload) did. For this to work, ReadMemoryWordGC.ppc.s, WriteMemoryGC.ppc.s, and RunDOL.ppc.s must be present in the system/client-functions/System directory. This has been tested on Dolphin but not on a real GameCube, so results may vary.
+### DOL loader
+
+You can put DOL files in the system/dol directory, and they will appear in the Programs menu for GC clients. Selecting a DOL file there will load the file into the GameCube's memory and run it, just like the old homebrew loaders (PSUL and PSOload) did. For this to work, ReadMemoryWordGC.ppc.s, WriteMemoryGC.ppc.s, and RunDOL.ppc.s must be present in the system/client-functions/System directory. This has been tested on Dolphin but not on a real GameCube, so results may vary.
 
 Like other kinds of data, functions and DOL files are cached in memory. If you've changed any of these files, you can run `reload functions` or `reload dol-files` in the interactive shell to make the changes take effect without restarting the server.
 
@@ -562,22 +577,25 @@ Some commands only work for clients not in proxy sessions. The chat commands are
     * `$where`: Show your current floor number and coordinates. Mainly useful for debugging.
     * `$qfread <field-name>` (non-proxy only): Show the value of a quest counter in your player data. The field names are defined in config.json.
 
-* Debugging commands
-    * `$debug`: Enable or disable debug. You need the DEBUG flag in your user account to use this command. Enabling debug does several things:
+* Basic debugging commands (special permissions not required)
+    * `$whatobj` and `$whatene` (non-proxy only): Tells you what the closest object or enemy spawn point is to your position, along with its coordinates and object or enemy ID. The full definition is also printed to the server's log.
+    * `$qcheck <flag-num>` (non-proxy only): Show the value of a quest flag. If you're in a game, show the value of the flag in that game; if you're in the lobby, show the saved value of that quest flag for your character (BB only).
+    * `$qgread <flag-num>` (non-proxy only): Show the value of a quest counter ("global flag").
+    * `$sound <sound-id>`: Play the given sound (GC only).
+
+* Restricted debugging commands (`$debug` permission required)
+    * `$debug`: Enable debug mode. You need the DEBUG flag in your user account to use this command. Enabling debug does several things:
+        * You'll be able to use the rest of the commands in this section.
         * You'll see in-game messages from the server when you take some actions, like killing enemies, opening boxes, or flipping switches.
         * You'll see the rare seed value and floor variations when you join a game.
         * You'll be placed into the last available slot in lobbies and games instead of the first, unless you're joining a BB solo-mode game.
         * You'll be able to join games with any PSO version, not only those for which cross-version play is normally enabled. See the "Cross-version play" section above for details on this.
-        * Most of the commands in this section are enabled. (A few of them are always enabled and don't require `$debug`.)
-    * `$whatobj` and `$whatene` (non-proxy only): Tells you what the closest object or enemy spawn point is to your position, along with its coordinates and object or enemy ID. The full definition is also printed to the server's log. These commands can be used without `$debug` enabled.
     * `$readmem <address>`: Read 4 bytes from the given address and show you the values.
     * `$writemem <address> <data>`: Write data to the given address. Data is not required to be any specific size.
     * `$nativecall <address> [arg1 ...]` (GC only): Call a native function on your client. Only arguments passed in registers are supported; calling functions that take many arguments is not supported.
     * `$quest <number>` (non-proxy only): Load a quest by quest number. Can be used to load battle or challenge quests with only one player present. `$debug` is not required for this command if the specified quest has the AllowStartFromChatCommand field set in its metadata file.
     * `$qcall <function-id>`: Call a quest function on your client.
-    * `$qcheck <flag-num>` (non-proxy only): Show the value of a quest flag. This command can be used without `$debug` enabled. If you're in a game, show the value of the flag in that game; if you're in the lobby, show the saved value of that quest flag for your character (BB only).
     * `$qset <flag-num>` or `$qclear <flag-num>`: Set or clear a quest flag for everyone in the game. If you're in the lobby and on BB, set or clear the saved value of a quest flag in your character file.
-    * `$qgread <flag-num>` (non-proxy only): Show the value of a quest counter ("global flag"). This command can be used without `$debug` enabled.
     * `$qgwrite <flag-num> <value>` (non-proxy only): Set the value of a quest counter ("global flag") for yourself.
     * `$qsync <reg-num> <value>`: Set a quest register's value for yourself only. `<reg-num>` should be either rXX (e.g. r60) or fXX (e.g. f60); if the latter, `<value>` is parsed as a floating-point value instead of as an integer.
     * `$qsyncall <reg-num> <value>`: Set a quest register's value for everyone in the game. `<reg-num>` should be either rXX (e.g. r60) or fXX (e.g. f60); if the latter, `<value>` is parsed as a floating-point value instead of as an integer.
@@ -587,7 +605,8 @@ Some commands only work for clients not in proxy sessions. The chat commands are
     * `$sc <data>`: Send a command to yourself.
     * `$ss <data>`: Send a command to the remote server (if in a proxy session) or to the game server.
     * `$sb <data>`: Send a command to yourself, and to the remote server or game server.
-    * `$auction` (Episode 3 only): Bring up the CARD Auction menu, regardless of how many players are in the game or if you have a VIP card.
+    * `$auction` (Episode 3 only): Bring up the CARD Auction menu, even if there are fewer than 4 players are in the game or you don't have a VIP card.
+    * `$makeobj <type> [coords...] [angles...] [params...]`: Create a map object. This is only implemented for a few specific client versions. The type is an integer like `273` or `0x0107`. Coordinates are specified as e.g. `x:30 y:0 z:-25.5`; if coordinates are not specified, the object is created at the player's coordinates. Angles are specified as e.g. `r:0 p:0x1000 w:-0x400` (for roll, pitch, and yaw, respectively). Parameters are specified as e.g. `1:2.0 2:0.0 5:0x4000`; any unspecified parameters are set to zero. The object is only created for the calling player and is not added to the server's map state; if the object ever sends update commands (e.g. 6x0B), it will likely result in a disconnection.
 
 * Personal state commands
     * `$arrow <color-id>`: Change your lobby arrow color. The color may be specified by number (0-12) or by name (red, blue, green, yellow, purple, cyan, orange, pink, white, white2, white3, or black).
@@ -599,8 +618,9 @@ Some commands only work for clients not in proxy sessions. The chat commands are
     * `$patch <name>`: Run a patch on your client. `<name>` must exactly match the name of a patch on the server.
 
 * Character data commands (non-proxy only)
+    * `$switchchar <slot>` (BB only): Switch to a different character from your account without logging out.
     * `$savechar <slot>`: Save your current character data on the server in the specified slot. See the [server-side saves section](#server-side-saves) for more details.
-    * `$loadchar <slot>`: Save your current character data on the server in the specified slot. See the [server-side saves section](#server-side-saves) for more details.
+    * `$loadchar <slot>`: Load character data from the specified slot on the server, and replace your current character with it. See the [server-side saves section](#server-side-saves) for more details.
     * `$bbchar <username> <password> <slot>`: Save your current character data on the server in a different account's BB character slots. See the [server-side saves section](#server-side-saves) for more details.
     * `$checkchar [slot]`: Tells you basic information about a server-side character previously saved using `$savechar`. If `slot` is not given, tells you which slots are used and which are free.
     * `$deletechar <slot>`: Deletes a server-side character previously saved using `$savechar`.
@@ -624,17 +644,31 @@ Some commands only work for clients not in proxy sessions. The chat commands are
     * `$stat <what>`: Show a statistic about your player or team in the current battle. `<what>` can be `duration`, `fcs-destroyed`, `cards-destroyed`, `damage-given`, `damage-taken`, `opp-cards-destroyed`, `own-cards-destroyed`, `move-distance`, `cards-set`, `fcs-set`, `attack-actions-set`, `techs-set`, `assists-set`, `defenses-self`, `defenses-ally`, `cards-drawn`, `max-attack-damage`, `max-combo`, `attacks-given`, `attacks-taken`, `sc-damage`, `damage-defended`, or `rank`.
     * `$surrender`: Cause your team to immediately lose the current battle. If your story character is already defeated, you can't surrender - only your teammate can.
     * `$saverec <name>`: Save the recording of the last battle.
-    * `$playrec <name>`: Play a battle recording. This command creates a spectator team immediately but the replay does not start automatically, to give other players a chance to join. To start the battle replay within the spectator team, run `$playrec` again (with no name). There is a bug in Dolphin that makes this command unstable in emulation (see the "Battle records" section above).
+    * `$playrec <name>`: Play a battle recording. This command creates a spectator team and plays the specified recording as if it were happening in real time. By default, playback will start immediately when the spectator team is ready; you can delay this to allow others to join by prepending a `!` to the recording name. In that case, using `$playrec` again (with no argument) within the spectator team will start playback.
 
 * Cheat mode commands
     * `$cheat` (non-proxy only): Enable or disable cheat mode for the current game. All other cheat mode commands do nothing if cheat mode is disabled. By default, cheat mode is off in new games but can be enabled; there is an option in config.json that allows you to disable cheat mode entirely, or set it to on by default in new games. Cheat mode is always enabled on the proxy, unless cheat mode is disabled on the entire server.
-    * `$infhp`: Enable or disable infinite HP mode. Applies to only you; does not affect other players. When enabled, one-hit KO attacks will still kill you, but on most versions of the game (not DCv1, GC US 1.2, or GC JP 1.5), the server will automatically revive you if you die. On all versions except GC US 1.2 and GC JP 1.5, infinite HP also automatically cures status ailments.
-    * `$inftp`: Enable or disable infinite TP mode. Applies to only you; does not affect other players.
+    * `$infhp`: Enable or disable infinite HP mode. Applies to only you; does not affect other players. When enabled, one-hit KO attacks will still kill you, but on most versions of the game, the server will automatically revive you if you die. Infinite HP also automatically cures status ailments.
+    * `$inftp`: Enable or disable infinite TP mode. Applies to only you; does not affect other players. Does not work on DCv1 or earlier versions.
     * `$warpme <floor-id>` (or `$warp <floor-id>`): Warp yourself to the given floor.
     * `$warpall <floor-id>`: Warp everyone in the game to the given floor. You must be the leader to use this command, unless you're on the proxy.
     * `$next`: Warp yourself to the next floor.
-    * `$item <desc>` (or `$i <desc>`): Create an item. `desc` may be a description of the item (e.g. "Hell Saber +5 0/10/25/0/10") or a string of hex data specifying the item code. Item codes are 16 hex bytes; at least 2 bytes must be specified, and all unspecified bytes are zeroes. If you are on the proxy, you must not be using Blue Burst for this command to work. On the game server, this command works for all versions.
-    * `$unset <index>` (non-proxy only): In an Episode 3 battle, removes one of your set cards from the field. `<index>` is the index of the set card as it appears on your screen - 1 is the card next to your SC's icon, 2 is the card to the right of 1, etc. This does not cause a Hunters-side SC to lose HP, as they normally do when their items are destroyed.
+    * `$item <desc>` (or `$i <desc>`): Create an item. `desc` may be a description of the item or a string of hex data specifying the item code. Item codes are 16 hex bytes; at least 2 bytes must be specified, and all unspecified bytes are zeroes. If you are on the proxy, you must not be using Blue Burst for this command to work. On the game server, this command works for all versions. Here are some examples to illustrate the syntax (nothing is case-sensitive, and everything except the item name itself is optional):
+        * `$item Saber +5 0/10/25/0/10` (weapon with special, grind and attributes)
+        * `$item ???? Draw Autogun` (untekked weapon with special; can have grind/attributes too, as above)
+        * `$item SEALED J-SWORD K:2000` (weapon with kill count)
+        * `$item ES APHEX ZALURE TWIN +200` (ES weapon must be prefixed with "ES"; name comes before special)
+        * `$item DF FIELD +10DEF +20EVP +4` (armor with DFP bonus, EVP bonus, and slot count)
+        * `$item RED MERGE +10DFP +20EVP` (shield; same as armor except without slot count)
+        * `$item Knight/Power +9` (unit with specific modifier)
+        * `$item Knight/Power++` (unit with normal modifier; ++/-- are +4/-4 and +/- are +2/-2)
+        * `$item LIMITER K:1000` (sealed unit with kill count)
+        * `$item Tapas PB:F,G,M&Y 120% 200IQ 5/195/0/0 green` (mag with PBs, synchro, IO, stats, and color)
+        * `$item Trimate x10` (tool with stack size)
+        * `$item Disk:Reverser` (technique disk without level)
+        * `$item Disk:Razonde Lv.30` (technique disk with level)
+        * `$item 1000 Meseta`
+    * `$unset <index>` (non-proxy only): In an Episode 3 battle, removes one of your set cards from the field. `<index>` is the index of the set card as it appears on your screen - 1 is the card next to your SC's icon, 2 is the card to the right of 1, etc. This does not cause a Hunters-side SC to lose HP, as they normally do when their items are destroyed. You can also destroy the assist card set on yourself with `$unset 0`.
     * `$dropmode [mode]` (proxy only): Change the way item drops behave in the current game, if you are not on BB. Unlike the game server version of this command, using this on the proxy requires cheats to be enabled. This works by intercepting the drop requests sent to and from the leader. (So, if you are the leader and not using server drop mode on the remote server, it affects the entire game; otherwise, it affects only items generated by your actions.) `mode` can be `none` (no drops), `default` (normal drops), or `proxy` (use newserv's drop tables instead of the remote server's). If `mode` is not given, tells you the current drop mode without changing it.
 
 * Aesthetic commands
@@ -753,6 +787,7 @@ The data formats that newserv can convert to/from are:
 | PSO DC save file (.vms)             | `encrypt-vms-save`        | `decrypt-vms-save`           |
 | PSO PC save file                    | `encrypt-pc-save`         | `decrypt-pc-save`            |
 | PSO GC save file (.gci)             | `encrypt-gci-save`        | `decrypt-gci-save`           |
+| PSO Xbox save file                  | None                      | `decrypt-xbox-save`          |
 | PSO GC snapshot file                | None                      | `decode-gci-snapshot`        |
 | Quest script (.bin)                 | `assemble-quest-script`   | `disassemble-quest-script`   |
 | Quest map (.dat)                    | None                      | `disassemble-quest-map`      |

notes/ar-codes.txt

@@ -17,7 +17,9 @@ Version codes (from README.md):
   2OJF: PSO DC v2 JP
   2OEF: PSO DC v2 US
   2OPF: PSO DC v2 EU
-  2OJW: PSO PC (v2)
+  2OJT: PSO PC Trial Edition
+  2OJW: PSO PC (v2) 04/2002
+  2OJZ: PSO PC (v2) 02/2003
   3OJT: PSO GC Trial Edition
   3OJ2: PSO GC v1.2 JP
   3OJ3: PSO GC v1.3 JP
@@ -69,14 +71,65 @@ Disable serial number validation (untested)
         8C2670B6 01E0
 
 Disable item equip restrictions ("God of equip")
+3OE0 => 0410521C 38000005
 3OE1 => 0410521C 38000005
+3OE2 => 041050E4 38000005
+3OJ2 => 04104F78 38000005
+3OJ3 => 04105154 38000005
+3OJ4 => 04105240 38000005
+3OJ5 => 041050D4 38000005
+3OJT => 0415BF50 38000005
+3OP0 => 041052D4 38000005
 59NL => 005C9F31 E9A7000000
 
+All items visible in Pioneer 2
+3OE1 => 04102D88 38600000
+
+Mags visible in Pioneer 2
+59NL => 005D8F4B EB04
+
+Disable pause menu background + offset
+3OE1 => 0424BD5C 48000370
+        0428735C 4800000C
+3OE2 => 0424CED8 48000370
+        042887D8 4800000C
+59NL => 00719B54 9090
+        00733BA7 9090
+        00733A0E 90E9
+
 All rareable enemies are rare
+3OE0 => 040AC944 60000000 // Hildeblue
+        040C1B70 60000000 // Rappies
+        040C3FC8 60000000 // Nar Lily
+        040EB050 48000010 // Pouilly Slime
 3OE1 => 040AC944 60000000 // Hildeblue
         040C1B70 60000000 // Rappies
         040C3FC8 60000000 // Nar Lily
         040EB050 48000010 // Pouilly Slime
+3OE2 => 040ACAFC 60000000 // Hildeblue
+        040C1D08 60000000 // Rappies
+        040C4160 60000000 // Nar Lily
+        040EB1E8 48000010 // Pouilly Slime
+3OJ2 => 040AC6B8 60000000 // Hildeblue
+        040C18CC 60000000 // Rappies
+        040C3D24 60000000 // Nar Lily
+        040EADAC 48000010 // Pouilly Slime
+3OJ3 => 040AC9C4 60000000 // Hildeblue
+        040C1BD0 60000000 // Rappies
+        040C4028 60000000 // Nar Lily
+        040EB0B0 48000010 // Pouilly Slime
+3OJ4 => 040ACB3C 60000000 // Hildeblue
+        040C1E04 60000000 // Rappies
+        040C41A0 60000000 // Nar Lily
+        040EB374 48000010 // Pouilly Slime
+3OJ5 => 040ACAEC 60000000 // Hildeblue
+        040C1CF8 60000000 // Rappies
+        040C4150 60000000 // Nar Lily
+        040EB1D8 48000010 // Pouilly Slime
+3OP0 => 040ACAC4 60000000 // Hildeblue
+        040C1CD0 60000000 // Rappies
+        040C4128 60000000 // Nar Lily
+        040EB1B0 48000010 // Pouilly Slime
 
 Unlock all songs in BGM test
 Note: sadly, there are no secret/unused ones
@@ -198,6 +251,16 @@ Unlock all COM decks
 3SP0 => 042CB414 38600001
 3SE0 => 042CA908 38600001
 
+Enable marker color menu in all lobbies
+3OJ2 => 04138200 3800000E
+3OJ3 => 04138508 3800000E
+3OJ4 => 041390AC 3800000E
+3OJ5 => 041385B0 3800000E
+3OE0 => 041384BC 3800000E
+3OE1 => 041384BC 3800000E
+3OE2 => 041385C0 3800000E
+3OP0 => 04138840 3800000E
+
 Enable all lobby counter options in non-CARD lobbies
 3SE0 => 04096A8C 480000C0
         04096B4C 38800007
@@ -218,7 +281,27 @@ Change HUD color mask
         0438CA90 6000BBAA
 
 Disable lobby event music (but keep the visuals)
+3OJT => 040B2394 38000000
 3SE0 => 040B705C 38000000
+3SJ0 => 040B7078 38000000
+3SP0 => 040B74A0 38000000
+
+Disable rate limit for lobby chair movement
+3OJ2 => 041C73B0 60000000
+3OJ3 => 041C786C 60000000
+3OJ4 => 041C7DA8 60000000
+3OJ5 => 041C7938 60000000
+3OE0 => 041C77CC 60000000
+3OE1 => 041C77CC 60000000
+3OE2 => 041C799C 60000000
+3OP0 => 041C7E58 60000000
+3SJT => 040E290C 60000000
+3SJ0 => 040DE6C4 60000000
+3SE0 => 040DE6A8 60000000
+3SP0 => 040DEAEC 60000000
+
+Make lobby chairs fast (client-side only)
+3SE0 => 0457E618 40000000
 
 Enable Pinz's Shop Super Card Capsule Machine as a fourth option
 3SE0 => 043101C0 38800004
@@ -266,8 +349,14 @@ Unlock all offline free battle maps
 This unlocks ALL maps, including a bunch of maps with garbage names that crash if you try to play them
 3SJT => 042BE538 38600001
 3SJ0 => 042C9C2C 38600001
-3SP0 => 042CB50C 38600001
 3SE0 => 042CAA00 38600001
+3SP0 => 042CB50C 38600001
+
+Card auctions accessible with fewer than 4 players
+3SJT => 042DD618 38600004
+3SJ0 => 042F4F20 38600004
+3SE0 => 042F5D88 38600004
+3SP0 => 042F698C 38600004
 
 Talk to auction counter offline to get all cards
 3SE0 => 042F5D18 4BD160E8
@@ -433,6 +522,11 @@ Note: Without a TextEnglish.pr2/pr3 patch, the menu items for these sounds will
         0442B6E0 802C0000
 
 Use English language files
+3OJT => 04189FE8 38000001
+        0418A010 38000001
+        0418A0A0 38000001
+        0418A0C8 38000001
+        04189EC4 3BC00001
 3SJT => 0408E414 38600001
         0408E448 38000001
         0408E44C 900DA62C
@@ -475,6 +569,68 @@ Heaven Punisher's special always works
 3OE2 => 0412AD84 38800001
 3OP0 => 0412AF5C 38800001
 
+Fast tekker (skips wind-up jingle)
+1OJ1 => 8C15B0CA mov    r1, 1
+        8C15B0E6 nop
+1OJ2 => 8C162302 mov    r1, 1
+        8C16231E nop
+1OJ3 => 8C175E66 mov    r1, 1
+        8C175E82 nop
+1OJ4 => 8C1780AE mov    r1, 1
+        8C1780CA nop
+1OJF => 8C17600E mov    r1, 1
+        8C17602A nop
+1OEF => 8C17863E mov    r1, 1
+        8C17865A nop
+1OPF => 8C1783FA mov    r1, 1
+        8C178416 nop
+2OJ5 => 8C19BD4A mov    r1, 1
+        8C19BD66 nop
+2OJF => 8C19ADB6 mov    r1, 1
+        8C19ADD2 nop
+2OEF => 8C19BD4A mov    r1, 1
+        8C19BD66 nop
+2OPF => 8C19B7E2 mov    r1, 1
+        8C19B7FE nop
+2OJW => 005B14A3 mov    dword [ebx + 0x150], 1
+        005B14BF jmp    +0x0D
+2OJZ => 005B0193 mov    dword [ebx + 0x150], 1
+        005B01AF jmp    +0x0D
+3OJT => 0426FAE8 38000001
+        0426FB10 60000000
+3OJ2 => 0421F8CC 38000001
+        0421F8F4 60000000
+3OJ3 => 04220250 38000001
+        04220278 60000000
+3OJ4 => 04221154 38000001
+        0422117C 60000000
+3OJ5 => 04220EF0 38000001
+        04220F18 60000000
+3OE0 => 04220170 38000001
+        04220198 60000000
+3OE1 => 04220170 38000001
+        04220198 60000000
+3OE2 => 04221224 38000001
+        0422124C 60000000
+3OP0 => 04220ABC 38000001
+        04220AE4 60000000
+4OED => 0023EF3C mov    dword [ebp + 0x14C], 1
+        0023EF57 jmp    +0x0A
+4OEU => 0023F0BC mov    dword [ebp + 0x14C], 1
+        0023F0D7 jmp    +0x0A
+4OJB => 0023EC5C mov    dword [ebp + 0x14C], 1
+        0023EC77 jmp    +0x0A
+4OJD => 0023EEAC mov    dword [ebp + 0x14C], 1
+        0023EEC7 jmp    +0x0A
+4OJU => 0023F21C mov    dword [ebp + 0x14C], 1
+        0023F237 jmp    +0x0A
+4OPD => 0023EF5C mov    dword [ebp + 0x14C], 1
+        0023EF77 jmp    +0x0A
+4OPU => 0023F14C mov    dword [ebp + 0x14C], 1
+        0023F167 jmp    +0x0A
+59NL => 006DA113 mov    dword [edi + 0x14C], 1
+        006DA130 jmp    +0x0B
+
 Allow loading corrupted save files
 3OJ2 => 041FC784 38600007
         041FC788 4E800020
@@ -721,4 +877,47 @@ Show extended item info when targeting a dropped item
         04005190 4E800020
 
 All weapons can do 3-hit combos
+3OE0 => 041D3248 38000001
 3OE1 => 041D3248 38000001
+3OE2 => 041D3448 38000001
+3OJ2 => 041D2DEC 38000001
+3OJ3 => 041D3318 38000001
+3OJ4 => 041D3144 38000001
+3OJ5 => 041D33E4 38000001
+3OP0 => 041D3904 38000001
+
+Disable save file signature validation (for moving Xbox saves across consoles)
+4OJB => 002F01CB 9090
+4OJD => 002F0CDB 9090
+4OJU => 002F22DB 9090
+4OED => 002F212B 9090
+4OEU => 002F22DB 9090
+4OPD => 002F215B 9090
+4OPU => 002F234B 9090
+
+Enable UDP test mode online
+3OE1 => 041A3D60 38600001
+
+Main warp door opens in Challenge mode
+3OE1 => 041820A4 38600001
+        041820A8 4E800020
+
+Allow arbitrary tech disk levels
+3OE1 => 0410EBE8 60000000
+        04100D18 60000000
+        041D6C0C 60000000
+        041D6C5C 60000000
+        0422CB50 60000000
+        042CD74C 4E800020
+
+Change particle colors in quest loading screen
+3OE1 => 04472C20 AARRGGBB // Default color
+        04472C24 AARRGGBB // Color after 1 A press
+        04472C28 AARRGGBB // Color after 2 A presses
+        04472C2C AARRGGBB // Color after 3 A presses
+        04472C30 AARRGGBB // Color after 4 A presses
+        04472C34 AARRGGBB // Color after 5 A presses
+
+Floor warp loading screen speed modifier
+// XXXX = speed; default is 01B4; 0800 = very fast/wobbly; 0020 = very slow
+3OE1 => 0434A350 3863XXXX

notes/patch-ports.txt

@@ -557,7 +557,7 @@ BugFixes
 8000C6E8 807F0024 8000C6E8 807F0024 8000C6E8 807F0024 8000C6E8 807F0024 8000C6E8 807F0024 8000C6E8 807F0024 8000C6E8 807F0024 8000C6E8 807F0024 lwz       r3, [r31 + 0x0024]
 8000C6EC 48165AA0 8000C6EC 482147D4 8000C6EC 482156C0 8000C6EC 48215474 8000C6EC 482146F4 8000C6EC 482146F4 8000C6EC 482157A8 8000C6EC 48215040 b         +0x002146F4 /* 80220DE0 */
 8021D098 4BDEF638 8021D9FC 4BDEECD4 8021E8E8 4BDEDDE8 8021E69C 4BDEE034 8021D91C 4BDEEDB4 8021D91C 4BDEEDB4 8021E9D0 4BDEDD00 8021E268 4BDEE468 b         -0x0021124C /* 8000C6D0 */
-80172188 4BE9A558 80220EBC 4BDEB824 80221DA8 4BDEA938 80221B5C 4BDEAB84 80220DDC 4BDEB904 80220DDC 4BDEB904 80221E90 4BDEA850 80221728 4BDEAFB8 b         -0x002146FC /* 8000C6E0 */
+80220528 4BE9A558 80220EBC 4BDEB824 80221DA8 4BDEA938 80221B5C 4BDEAB84 80220DDC 4BDEB904 80220DDC 4BDEB904 80221E90 4BDEA850 80221728 4BDEAFB8 b         -0x002146FC /* 8000C6E0 */
 
 Dropped Mag Colour Bug Fix
 BugFixes

notes/quest-flags.txt

@@ -17,7 +17,7 @@
 0019 = P2 Scientist after defeating dragon
 001E = Entered Caves 1 (Gov 2-1)
 001F = Entered De Rol Le in 2-4
-0020 = De Ro lee defeated
+0020 = De Rol Le defeated
 0021 = Mines unlocked (P2 Tyrell after defeating De Rol Le)
 0028 = Entered Mines 1
 0029 = Entered Vol Opt Area

src/AFSArchive.hh

@@ -23,6 +23,10 @@ public:
     return this->entries;
   }
 
+  inline size_t num_entries() const {
+    return this->entries.size();
+  }
+
   std::pair<const void*, size_t> get(size_t index) const;
   std::string get_copy(size_t index) const;
   phosg::StringReader get_reader(size_t index) const;

src/Account.cc

@@ -420,26 +420,6 @@ void Account::delete_file() const {
   remove(filename.c_str());
 }
 
-uint64_t Login::proxy_session_id() const {
-  uint64_t low_part = 0;
-  if (this->dc_nte_license) {
-    low_part = this->dc_nte_license->proxy_session_id_part();
-  } else if (this->dc_license) {
-    low_part = this->dc_license->proxy_session_id_part();
-  } else if (this->pc_license) {
-    low_part = this->pc_license->proxy_session_id_part();
-  } else if (this->gc_license) {
-    low_part = this->gc_license->proxy_session_id_part();
-  } else if (this->xb_license) {
-    low_part = this->xb_license->proxy_session_id_part();
-  } else if (this->bb_license) {
-    low_part = this->bb_license->proxy_session_id_part();
-  } else {
-    throw logic_error("none of the licenses in a Login were present");
-  }
-  return (static_cast<uint64_t>(this->account->account_id) << 32) | low_part;
-}
-
 string Login::str() const {
   string ret = std::format("Account:{:08X}", this->account->account_id);
   if (this->account_was_created) {

src/Account.hh

@@ -17,10 +17,6 @@ struct DCNTELicense {
   std::string serial_number;
   std::string access_key;
 
-  inline uint64_t proxy_session_id_part() const {
-    return phosg::fnv1a32(this->serial_number);
-  }
-
   static std::shared_ptr<DCNTELicense> from_json(const phosg::JSON& json);
   phosg::JSON json() const;
 };
@@ -29,10 +25,6 @@ struct V1V2License {
   uint32_t serial_number = 0;
   std::string access_key;
 
-  inline uint64_t proxy_session_id_part() const {
-    return this->serial_number;
-  }
-
   static std::shared_ptr<V1V2License> from_json(const phosg::JSON& json);
   phosg::JSON json() const;
 };
@@ -42,10 +34,6 @@ struct GCLicense {
   std::string access_key;
   std::string password;
 
-  inline uint64_t proxy_session_id_part() const {
-    return this->serial_number;
-  }
-
   static std::shared_ptr<GCLicense> from_json(const phosg::JSON& json);
   phosg::JSON json() const;
 };
@@ -55,10 +43,6 @@ struct XBLicense {
   uint64_t user_id = 0;
   uint64_t account_id = 0;
 
-  inline uint64_t proxy_session_id_part() const {
-    return phosg::fnv1a32(this->gamertag);
-  }
-
   static std::shared_ptr<XBLicense> from_json(const phosg::JSON& json);
   phosg::JSON json() const;
 };
@@ -67,10 +51,6 @@ struct BBLicense {
   std::string username;
   std::string password;
 
-  inline uint64_t proxy_session_id_part() const {
-    return phosg::fnv1a32(this->username);
-  }
-
   static std::shared_ptr<BBLicense> from_json(const phosg::JSON& json);
   phosg::JSON json() const;
 };
@@ -178,8 +158,6 @@ struct Login {
   std::shared_ptr<XBLicense> xb_license;
   std::shared_ptr<BBLicense> bb_license;
 
-  uint64_t proxy_session_id() const;
-
   std::string str() const;
 };
 

src/AddressTranslator.cc

@@ -247,9 +247,9 @@ public:
     static constexpr bool IsBE = BE;
 
     U16T<BE> type;
-    U16T<BE> unknown_a1;
+    U16T<BE> unused;
     U32T<BE> constructor_addr;
-    F32T<BE> unknown_a2;
+    F32T<BE> max_dist2; // Only applies for objects
     U32T<BE> default_num_children;
   } __attribute__((packed));
 
@@ -259,9 +259,9 @@ public:
 
     pstring<TextEncoding::ASCII, 0x10> debug_name;
     U16T<BE> type;
-    U16T<BE> unknown_a1;
+    U16T<BE> unused;
     U32T<BE> constructor_addr;
-    F32T<BE> unknown_a2;
+    F32T<BE> max_dist2; // Only applies for objects
     U32T<BE> default_num_children;
   } __attribute__((packed));
 
@@ -635,7 +635,13 @@ public:
     throw runtime_error("scan field too long; too many matches");
   }
 
-  void find_all_matches(uint32_t src_addr, uint32_t src_size) const {
+  enum class MatchType {
+    ANY = 0,
+    TEXT,
+    DATA,
+  };
+
+  void find_all_matches(uint32_t src_addr, uint32_t src_size, MatchType type) const {
     if (!this->src_mem) {
       throw runtime_error("no source file selected");
     }
@@ -660,19 +666,48 @@ public:
             ExpandMethod::PPC_DATA_BACKWARD,
             ExpandMethod::PPC_DATA_BOTH,
         };
+        static const vector<ExpandMethod> ppc_text_methods = {
+            ExpandMethod::PPC_TEXT_FORWARD,
+            ExpandMethod::PPC_TEXT_FORWARD_WITH_BARRIER,
+            ExpandMethod::PPC_TEXT_BACKWARD,
+            ExpandMethod::PPC_TEXT_BACKWARD_WITH_BARRIER,
+            ExpandMethod::PPC_TEXT_BOTH,
+            ExpandMethod::PPC_TEXT_BOTH_WITH_BARRIER,
+            ExpandMethod::PPC_TEXT_BOTH_IGNORE_ORIGIN,
+        };
+        static const vector<ExpandMethod> ppc_data_methods = {
+            ExpandMethod::PPC_DATA_FORWARD,
+            ExpandMethod::PPC_DATA_BACKWARD,
+            ExpandMethod::PPC_DATA_BOTH,
+        };
         static const vector<ExpandMethod> raw_methods = {
             ExpandMethod::RAW_FORWARD,
             ExpandMethod::RAW_BACKWARD,
             ExpandMethod::RAW_BOTH,
         };
-        const auto& methods = this->ppc_mems.count(it.second) ? ppc_methods : raw_methods;
-        for (size_t z = 0; z < methods.size(); z++) {
-          futures.emplace_back(async(&AddressTranslator::find_match, this, it.second, src_addr, src_size, methods[z]));
+
+        const vector<ExpandMethod>* methods;
+        if (this->ppc_mems.count(it.second)) {
+          if (type == MatchType::ANY) {
+            methods = &ppc_methods;
+          } else if (type == MatchType::TEXT) {
+            methods = &ppc_text_methods;
+          } else if (type == MatchType::DATA) {
+            methods = &ppc_data_methods;
+          } else {
+            throw logic_error("invalid match type");
+          }
+        } else {
+          methods = &raw_methods;
+        }
+
+        for (size_t z = 0; z < methods->size(); z++) {
+          futures.emplace_back(async(&AddressTranslator::find_match, this, it.second, src_addr, src_size, methods->at(z)));
         }
 
         unordered_set<uint32_t> match_addrs;
         for (size_t z = 0; z < futures.size(); z++) {
-          const char* method_name = this->name_for_expand_method(methods[z]);
+          const char* method_name = this->name_for_expand_method(methods->at(z));
           try {
             uint32_t ret = futures[z].get();
             log.info_f("({}) ({}) {:08X}", it.first, method_name, ret);
@@ -831,7 +866,18 @@ public:
     } else if (tokens[0] == "match") {
       this->find_all_matches(
           stoul(tokens.at(1), nullptr, 16),
-          tokens.size() >= 3 ? stoul(tokens[2], nullptr, 16) : 0);
+          tokens.size() >= 3 ? stoul(tokens[2], nullptr, 16) : 0,
+          MatchType::ANY);
+    } else if (tokens[0] == "match-text") {
+      this->find_all_matches(
+          stoul(tokens.at(1), nullptr, 16),
+          tokens.size() >= 3 ? stoul(tokens[2], nullptr, 16) : 0,
+          MatchType::TEXT);
+    } else if (tokens[0] == "match-data") {
+      this->find_all_matches(
+          stoul(tokens.at(1), nullptr, 16),
+          tokens.size() >= 3 ? stoul(tokens[2], nullptr, 16) : 0,
+          MatchType::DATA);
     } else if (tokens[0] == "match-be-le") {
       this->find_all_be_to_le_data_matches(
           stoul(tokens.at(1), nullptr, 16),

src/AsyncHTTPServer.cc

@@ -163,7 +163,7 @@ asio::awaitable<HTTPRequest> HTTPClient::recv_http_request(size_t max_line_size,
   } else if (method_token == "TRACE") {
     req.method = HTTPRequest::Method::TRACE;
   } else {
-    throw HTTPError(400, "unknown request method");
+    throw HTTPError(400, "Unknown request method");
   }
 
   req.http_version = std::move(line_tokens[2]);
@@ -237,26 +237,26 @@ asio::awaitable<HTTPRequest> HTTPClient::recv_http_request(size_t max_line_size,
       size_t parse_offset = 0;
       size_t chunk_size = stoull(line, &parse_offset, 16);
       if (parse_offset != line.size()) {
-        throw HTTPError(400, "invalid chunk header during chunked encoding");
+        throw HTTPError(400, "Invalid chunk header during chunked encoding");
       }
       if (chunk_size == 0) {
         break;
       }
       total_data_bytes += chunk_size;
       if (total_data_bytes > max_body_size) {
-        throw HTTPError(400, "request data size too large");
+        throw HTTPError(400, "Request data size too large");
       }
       chunks.emplace_back(co_await this->r.read_data(chunk_size));
       auto after_chunk_data = co_await this->r.read_line("\r\n", 0x20);
       if (!after_chunk_data.empty()) {
-        throw HTTPError(400, "incorrect trailing sequence after chunk data");
+        throw HTTPError(400, "Incorrect trailing sequence after chunk data");
       }
     }
   } else {
     auto content_length_header = req.get_header("content-length");
     size_t content_length = content_length_header ? stoull(*content_length_header) : 0;
     if (content_length > max_body_size) {
-      throw HTTPError(400, "request data size too large");
+      throw HTTPError(400, "Request data size too large");
     } else if (content_length > 0) {
       req.data = co_await this->r.read_data(content_length);
     }
@@ -289,8 +289,7 @@ asio::awaitable<WebSocketMessage> HTTPClient::recv_websocket_message(size_t max_
   while (this->r.get_socket().is_open()) {
     WebSocketMessage msg;
 
-    // We need at most 10 bytes to determine if there's a valid frame, or as
-    // little as 2
+    // We need at most 10 bytes to determine if there's a valid frame, or as little as 2
     co_await this->r.read_data_into(msg.header, 2);
 
     // Get the payload size
@@ -335,7 +334,7 @@ asio::awaitable<WebSocketMessage> HTTPClient::recv_websocket_message(size_t max_
       } else if (opcode == 0x08) {
         // Close message
         co_await this->send_websocket_message(msg.data, msg.opcode);
-        this->r.get_socket().close();
+        this->r.close();
 
       } else if (opcode == 0x09) {
         // Ping message
@@ -343,7 +342,7 @@ asio::awaitable<WebSocketMessage> HTTPClient::recv_websocket_message(size_t max_
 
       } else {
         // Unknown control message type
-        this->r.get_socket().close();
+        this->r.close();
       }
       continue;
     }
@@ -351,7 +350,7 @@ asio::awaitable<WebSocketMessage> HTTPClient::recv_websocket_message(size_t max_
     // If there's an existing fragment, the current message's opcode should be
     // zero; if there's no pending message, it must not be zero
     if (prev_msg_present == (opcode != 0)) {
-      this->r.get_socket().close();
+      this->r.close();
       continue;
     }
 

src/AsyncHTTPServer.hh

@@ -9,6 +9,7 @@
 #include <functional>
 #include <memory>
 #include <optional>
+#include <phosg/Encoding.hh>
 #include <phosg/Hash.hh>
 #include <phosg/Time.hh>
 #include <string>
@@ -82,12 +83,112 @@ struct HTTPClient {
   asio::awaitable<void> send_websocket_message(const std::string& data, uint8_t opcode = 0x01);
 };
 
+template <typename RetT>
+class HTTPRouter {
+public:
+  struct Args {
+    std::shared_ptr<HTTPClient> client;
+    const HTTPRequest& req;
+    std::unordered_map<std::string, std::string> params;
+    phosg::JSON post_data;
+
+    template <typename T>
+      requires(std::is_integral_v<T>)
+    T get_param(const char* name, bool hex = false) const {
+      const auto& value_str = this->params.at(name);
+      size_t conversion_end;
+      int64_t v = std::stoull(value_str, &conversion_end, hex ? 16 : 0);
+      if (conversion_end != value_str.size()) {
+        throw HTTPError(400, "Invalid integer value");
+      }
+
+      uint64_t uv = static_cast<uint64_t>(v);
+      if constexpr (std::is_unsigned_v<T>) {
+        if (uv & (~phosg::mask_for_type<T>)) {
+          throw HTTPError(400, "Unsigned value out of range");
+        }
+        return uv;
+      } else {
+        if (((uv & (~(phosg::mask_for_type<T> >> 1))) != 0) && ((uv & (~(phosg::mask_for_type<T> >> 1))) != (~(phosg::mask_for_type<T> >> 1)))) {
+          throw HTTPError(400, "Signed value out of range");
+        }
+        return v;
+      }
+    }
+  };
+
+  using Handler = std::function<asio::awaitable<RetT>(Args&&)>;
+
+  static std::vector<std::string> split_and_normalize_path(const std::string& path) {
+    auto path_tokens = phosg::split(path, '/');
+    while (!path_tokens.empty() && path_tokens.back().empty()) {
+      path_tokens.pop_back();
+    }
+    return path_tokens;
+  }
+
+  void add(HTTPRequest::Method method, const std::string& path_pattern, Handler handler) {
+    this->routes.emplace_back(Route{
+        .method = method, .path_tokens = this->split_and_normalize_path(path_pattern), .handler = handler});
+  }
+
+  asio::awaitable<RetT> call_handler(std::shared_ptr<HTTPClient> c, const HTTPRequest& req) {
+    Args args = {.client = c, .req = req, .params = {}, .post_data = phosg::JSON()};
+
+    auto tokens = this->split_and_normalize_path(req.path);
+    for (const auto& route : this->routes) {
+      if (route.path_tokens.size() != tokens.size()) {
+        continue;
+      }
+
+      bool matched = true;
+      args.params.clear();
+      for (size_t z = 0; z < tokens.size(); z++) {
+        if (route.path_tokens[z].starts_with(':')) {
+          args.params.emplace(route.path_tokens[z].substr(1), tokens[z]);
+        } else if (route.path_tokens[z] != tokens[z]) {
+          matched = false;
+          break;
+        }
+      }
+
+      if (matched) {
+        if (req.method != route.method) {
+          throw HTTPError(405, "Incorrect HTTP method");
+        }
+        if (req.method == HTTPRequest::Method::POST) {
+          auto* content_type = req.get_header("content-type");
+          if (!content_type || (*content_type != "application/json")) {
+            throw HTTPError(400, "POST requests must use the application/json content type");
+          }
+          try {
+            args.post_data = phosg::JSON::parse(req.data);
+          } catch (const std::exception& e) {
+            throw HTTPError(400, std::format("Invalid JSON: {}", e.what()));
+          }
+        }
+        co_return co_await route.handler(std::move(args));
+      }
+    }
+
+    throw HTTPError(404, "Request path did not match any route");
+  }
+
+private:
+  struct Route {
+    HTTPRequest::Method method;
+    std::vector<std::string> path_tokens;
+    Handler handler;
+  };
+  std::vector<Route> routes;
+};
+
 struct HTTPServerLimits {
   size_t max_http_request_line_size = 0x1000; // 4KB
   size_t max_http_data_size = 0x200000; // 2MB
   size_t max_http_keepalive_idle_usecs = 300 * 1000 * 1000; // 5 minutes (0 = no limit)
   size_t max_websocket_message_size = 0x200000; // 2MB
-  size_t max_websocket_idle_usecs = 300 * 1000 * 1000; // 5 minutes (0 = no limit)
+  size_t max_websocket_idle_usecs = 0; // No limit by default
 };
 
 extern const HTTPServerLimits DEFAULT_HTTP_LIMITS;
@@ -120,6 +221,29 @@ public:
 protected:
   HTTPServerLimits limits;
 
+  void require_GET(const HTTPRequest& req) {
+    if (req.method != HTTPRequest::Method::GET) {
+      throw HTTPError(405, "GET method required for this endpoint");
+    }
+  }
+
+  phosg::JSON require_JSON_POST(const HTTPRequest& req) {
+    if (req.method != HTTPRequest::Method::POST) {
+      throw HTTPError(405, "POST method required for this endpoint");
+    }
+
+    auto* content_type = req.get_header("content-type");
+    if (!content_type || (*content_type != "application/json")) {
+      throw HTTPError(400, "POST requests must use the application/json content type");
+    }
+
+    try {
+      return phosg::JSON::parse(req.data);
+    } catch (const std::exception& e) {
+      throw HTTPError(400, std::format("Invalid JSON: {}", e.what()));
+    }
+  }
+
   // Attempts to switch the client to WebSockets. Returns true if this is done
   // successfully (and the caller should then receive/send WebSocket messages),
   // or false if this failed (and the caller should send an HTTP response).
@@ -205,9 +329,11 @@ protected:
         if (resp) {
           co_await c->send_http_response(*resp);
         }
-        auto* conn_header = req.get_header("connection");
-        if (!conn_header || (*conn_header != "keep-alive")) {
-          c->r.close();
+        if (!c->is_websocket) {
+          auto* conn_header = req.get_header("connection");
+          if (!conn_header || (*conn_header != "keep-alive")) {
+            c->r.close();
+          }
         }
       }
 

src/AsyncUtils.cc

@@ -13,15 +13,18 @@ AsyncEvent::AsyncEvent(asio::any_io_executor ex)
     : executor(ex), is_set(false) {}
 
 void AsyncEvent::set() {
-  lock_guard g(this->lock);
-  this->is_set = true;
-  for (auto& waiter : this->waiters) {
+  std::vector<std::unique_ptr<asio::detail::awaitable_handler<asio::any_io_executor>>> waiters_to_resume;
+  {
+    lock_guard g(this->lock);
+    this->is_set = true;
+    this->waiters.swap(waiters_to_resume);
+  }
+  for (auto& waiter : waiters_to_resume) {
     asio::post(this->executor,
         [handler = std::move(waiter)]() mutable {
           (*handler)();
         });
   }
-  this->waiters.clear();
 }
 
 void AsyncEvent::clear() {

src/AsyncUtils.hh

@@ -34,7 +34,7 @@ public:
   }
 
   void set_value(T&& result) {
-    if (this->exc || this->val.has_value()) {
+    if (this->done()) {
       throw std::logic_error("attempted to set value on completed promise");
     }
     this->val = result;
@@ -42,7 +42,7 @@ public:
   }
 
   void set_exception(std::exception_ptr ex) {
-    if (this->exc || this->val.has_value()) {
+    if (this->done()) {
       throw std::logic_error("attempted to set value on completed promise");
     }
     this->exc = ex;
@@ -67,12 +67,13 @@ private:
   std::optional<ResolverRef> resolver_ref;
 
   void resolve() {
-    if (this->resolver_ref.has_value()) {
+    if (this->resolver_ref) {
       auto* executor = this->resolver_ref->executor;
-      asio::post(*executor, [ref = std::move(this->resolver_ref)]() mutable -> void {
-        ref->resolve(std::error_code{});
-      });
+      ResolverRef ref = std::move(*this->resolver_ref);
       this->resolver_ref.reset();
+      asio::post(*executor, [ref = std::move(ref)]() mutable -> void {
+        ref.resolve(std::error_code{});
+      });
     }
   }
 };
@@ -102,7 +103,7 @@ public:
   }
 
   void set_value() {
-    if (this->exc || this->returned) {
+    if (this->done()) {
       throw std::logic_error("attempted to set value on completed promise");
     }
     this->returned = true;
@@ -110,7 +111,7 @@ public:
   }
 
   void set_exception(std::exception_ptr ex) {
-    if (this->exc || this->returned) {
+    if (this->done()) {
       throw std::logic_error("attempted to set value on completed promise");
     }
     this->exc = ex;
@@ -130,17 +131,18 @@ private:
     asio::detail::awaitable_handler<asio::any_io_executor, std::error_code> resolve;
     asio::any_io_executor* executor;
   };
-  bool returned;
+  bool returned = false;
   std::exception_ptr exc;
   std::optional<ResolverRef> resolver_ref;
 
   void resolve() {
-    if (this->resolver_ref.has_value()) {
+    if (this->resolver_ref) {
       auto* executor = this->resolver_ref->executor;
-      asio::post(*executor, [ref = std::move(this->resolver_ref)]() mutable -> void {
-        ref->resolve(std::error_code{});
-      });
+      ResolverRef ref = std::move(*this->resolver_ref);
       this->resolver_ref.reset();
+      asio::post(*executor, [ref = std::move(ref)]() mutable -> void {
+        ref.resolve(std::error_code{});
+      });
     }
   }
 };
@@ -187,8 +189,14 @@ public:
     return this->sock;
   }
 
+  inline bool is_open() const {
+    return this->sock.is_open();
+  }
+
   inline void close() {
-    this->sock.close();
+    if (this->sock.is_open()) {
+      this->sock.close();
+    }
   }
 
 private:
@@ -224,6 +232,14 @@ inline asio::ip::tcp::endpoint make_endpoint_ipv4(uint32_t addr, uint16_t port)
   return asio::ip::tcp::endpoint(asio::ip::address_v4(addr), port);
 }
 
+inline asio::ip::tcp::endpoint make_endpoint_ipv6(const void* addr, uint16_t port) {
+  std::array<uint8_t, 0x10> bytes;
+  for (size_t z = 0; z < 0x10; z++) {
+    bytes[z] = reinterpret_cast<const uint8_t*>(addr)[z];
+  }
+  return asio::ip::tcp::endpoint(asio::ip::address_v6(bytes), port);
+}
+
 inline std::string str_for_endpoint(const asio::ip::tcp::endpoint& ep) {
   return ep.address().to_string() + std::format(":{}", ep.port());
 }
@@ -232,7 +248,7 @@ inline uint32_t ipv4_addr_for_asio_addr(const asio::ip::address& addr) {
   if (!addr.is_v4()) {
     throw std::runtime_error("Address is not IPv4");
   }
-  return ntohl(addr.to_v4().to_uint());
+  return addr.to_v4().to_uint();
 }
 
 asio::awaitable<asio::ip::tcp::socket> async_connect_tcp(uint32_t ipv4_addr, uint16_t port);
@@ -243,10 +259,17 @@ template <typename FnT, typename... ArgTs>
 asio::awaitable<std::invoke_result_t<FnT, ArgTs...>> call_on_thread_pool(asio::thread_pool& pool, FnT&& f, ArgTs&&... args) {
   using ReturnT = std::invoke_result_t<FnT, ArgTs...>;
   auto bound = std::bind(std::forward<FnT>(f), std::forward<ArgTs>(args)...);
-  AsyncPromise<ReturnT> promise;
 
-  asio::post(pool, [&promise, &bound]() -> void {
-    promise.set_value(bound());
+  // We have to use a shared_ptr here in case call_on_thread_pool is canceled
+  // (in that case, the posted callback will try to use promise after the
+  // call_on_thread_pool coroutine has been destroyed)
+  auto promise = std::make_shared<AsyncPromise<ReturnT>>();
+  asio::post(pool, [bound = std::move(bound), promise]() mutable {
+    try {
+      promise->set_value(bound());
+    } catch (...) {
+      promise->set_exception(std::current_exception());
+    }
   });
-  co_return co_await promise.get();
+  co_return co_await promise->get();
 }

src/BattleParamsIndex.cc

@@ -9,10 +9,17 @@
 
 using namespace std;
 
-void BattleParamsIndex::Table::print(FILE* stream) const {
-  auto print_entry = +[](FILE* stream, const PlayerStats& e) {
+void BattleParamsIndex::Table::print(FILE* stream, Episode episode) const {
+  auto print_entry = [stream, episode](const PlayerStats& e, size_t z) {
+    string names_str;
+    for (auto type : enemy_types_for_battle_param_index(episode, z)) {
+      if (!names_str.empty()) {
+        names_str += ", ";
+      }
+      names_str += phosg::name_for_enum(type);
+    }
     phosg::fwrite_fmt(stream,
-        "{:5} {:5} {:5} {:5} {:5} {:5} {:5} {:5} {:5} {:5}",
+        "{:5} {:5} {:5} {:5} {:5} {:5} {:5} {:5} {:5} {:5}  {}",
         e.char_stats.atp,
         e.char_stats.mst,
         e.char_stats.evp,
@@ -22,15 +29,16 @@ void BattleParamsIndex::Table::print(FILE* stream) const {
         e.char_stats.lck,
         e.esp,
         e.experience,
-        e.meseta);
+        e.meseta,
+        names_str);
   };
 
-  for (size_t diff = 0; diff < 4; diff++) {
-    phosg::fwrite_fmt(stream, "{} ZZ   ATP   PSV   EVP    HP   DFP   ATA   LCK   ESP   EXP  DIFF\n",
-        abbreviation_for_difficulty(diff));
+  for (Difficulty difficulty : ALL_DIFFICULTIES_V234) {
+    phosg::fwrite_fmt(stream, "{} ZZ   ATP   PSV   EVP    HP   DFP   ATA   LCK   ESP   EXP  DIFF  NAMES\n",
+        abbreviation_for_difficulty(difficulty));
     for (size_t z = 0; z < 0x60; z++) {
       phosg::fwrite_fmt(stream, "  {:02X} ", z);
-      print_entry(stream, this->stats[diff][z]);
+      print_entry(this->stats[static_cast<size_t>(difficulty)][z], z);
       fputc('\n', stream);
     }
   }

src/BattleParamsIndex.hh

@@ -70,13 +70,26 @@ public:
   } __packed_ws__(MovementData, 0x30);
 
   struct Table {
-    /* 0000 */ parray<parray<PlayerStats, 0x60>, 4> stats;
-    /* 3600 */ parray<parray<AttackData, 0x60>, 4> attack_data;
-    /* 7E00 */ parray<parray<ResistData, 0x60>, 4> resist_data;
-    /* AE00 */ parray<parray<MovementData, 0x60>, 4> movement_data;
+    /* 0000 */ parray<parray<PlayerStats, 0x60>, 4> stats; // [difficulty][bp_index]
+    /* 3600 */ parray<parray<AttackData, 0x60>, 4> attack_data; // [difficulty][bp_index]
+    /* 7E00 */ parray<parray<ResistData, 0x60>, 4> resist_data; // [difficulty][bp_index]
+    /* AE00 */ parray<parray<MovementData, 0x60>, 4> movement_data; // [difficulty][bp_index]
     /* F600 */
 
-    void print(FILE* stream) const;
+    const PlayerStats& stats_for_index(Difficulty difficulty, uint8_t index) const {
+      return this->stats.at(static_cast<size_t>(difficulty)).at(index);
+    }
+    const AttackData& attack_data_for_index(Difficulty difficulty, uint8_t index) const {
+      return this->attack_data.at(static_cast<size_t>(difficulty)).at(index);
+    }
+    const ResistData& resist_data_for_index(Difficulty difficulty, uint8_t index) const {
+      return this->resist_data.at(static_cast<size_t>(difficulty)).at(index);
+    }
+    const MovementData& movement_data_for_index(Difficulty difficulty, uint8_t index) const {
+      return this->movement_data.at(static_cast<size_t>(difficulty)).at(index);
+    }
+
+    void print(FILE* stream, Episode episode) const;
   } __packed_ws__(Table, 0xF600);
 
   BattleParamsIndex(

src/Channel.cc

@@ -8,6 +8,7 @@
 #include <phosg/Time.hh>
 
 #include "Loggers.hh"
+#include "StaticGameData.hh"
 #include "Version.hh"
 
 using namespace std;
@@ -16,7 +17,7 @@ extern bool use_terminal_colors;
 
 Channel::Channel(
     Version version,
-    uint8_t language,
+    Language language,
     const string& name,
     phosg::TerminalFormat terminal_send_color,
     phosg::TerminalFormat terminal_recv_color)
@@ -249,7 +250,7 @@ shared_ptr<SocketChannel> SocketChannel::create(
     std::shared_ptr<asio::io_context> io_context,
     std::unique_ptr<asio::ip::tcp::socket>&& sock,
     Version version,
-    uint8_t language,
+    Language language,
     const string& name,
     phosg::TerminalFormat terminal_send_color,
     phosg::TerminalFormat terminal_recv_color) {
@@ -263,7 +264,7 @@ SocketChannel::SocketChannel(
     std::shared_ptr<asio::io_context> io_context,
     std::unique_ptr<asio::ip::tcp::socket>&& sock,
     Version version,
-    uint8_t language,
+    Language language,
     const string& name,
     phosg::TerminalFormat terminal_send_color,
     phosg::TerminalFormat terminal_recv_color)
@@ -331,7 +332,7 @@ asio::awaitable<void> SocketChannel::send_task() {
 PeerChannel::PeerChannel(
     std::shared_ptr<asio::io_context> io_context,
     Version version,
-    uint8_t language,
+    Language language,
     const std::string& name,
     phosg::TerminalFormat terminal_send_color,
     phosg::TerminalFormat terminal_recv_color)

src/Channel.hh

@@ -12,7 +12,7 @@
 class Channel {
 public:
   Version version;
-  uint8_t language;
+  Language language;
   std::shared_ptr<PSOEncryption> crypt_in;
   std::shared_ptr<PSOEncryption> crypt_out;
 
@@ -87,7 +87,7 @@ public:
 protected:
   Channel(
       Version version,
-      uint8_t language,
+      Language language,
       const std::string& name,
       phosg::TerminalFormat terminal_send_color = phosg::TerminalFormat::END,
       phosg::TerminalFormat terminal_recv_color = phosg::TerminalFormat::END);
@@ -120,7 +120,7 @@ public:
   static std::shared_ptr<SocketChannel> create(std::shared_ptr<asio::io_context> io_context,
       std::unique_ptr<asio::ip::tcp::socket>&& sock,
       Version version,
-      uint8_t language,
+      Language language,
       const std::string& name = "",
       phosg::TerminalFormat terminal_send_color = phosg::TerminalFormat::END,
       phosg::TerminalFormat terminal_recv_color = phosg::TerminalFormat::END);
@@ -138,7 +138,7 @@ private:
       std::shared_ptr<asio::io_context> io_context,
       std::unique_ptr<asio::ip::tcp::socket>&& sock,
       Version version,
-      uint8_t language,
+      Language language,
       const std::string& name,
       phosg::TerminalFormat terminal_send_color,
       phosg::TerminalFormat terminal_recv_color);
@@ -158,7 +158,7 @@ public:
   PeerChannel(
       std::shared_ptr<asio::io_context> io_context,
       Version version,
-      uint8_t language,
+      Language language,
       const std::string& name = "",
       phosg::TerminalFormat terminal_send_color = phosg::TerminalFormat::END,
       phosg::TerminalFormat terminal_recv_color = phosg::TerminalFormat::END);

src/ChatCommands.cc

@@ -1,5 +1,6 @@
 #include "ChatCommands.hh"
 
+#include <ctype.h>
 #include <string.h>
 
 #include <filesystem>
@@ -225,7 +226,7 @@ static asio::awaitable<void> server_command_announce_inner(const Args& a, bool m
       send_text_or_scrolling_message(s, a.text, a.text);
     }
   } else {
-    auto from_name = a.c->character()->disp.name.decode(a.c->language());
+    auto from_name = a.c->character_file()->disp.name.decode(a.c->language());
     if (mail) {
       send_simple_mail(s, 0, from_name, a.text);
     } else {
@@ -332,7 +333,7 @@ ChatCommandDefinition cc_auction(
     });
 
 static string name_for_client(shared_ptr<Client> c) {
-  auto player = c->character(false);
+  auto player = c->character_file(false);
   if (player.get()) {
     return escape_player_name(player->disp.name.decode(player->inventory.language));
   }
@@ -417,28 +418,24 @@ ChatCommandDefinition cc_bank(
 
       ssize_t new_char_index = a.text.empty() ? (a.c->bb_character_index + 1) : stol(a.text, nullptr, 0);
 
-      if (new_char_index == 0) {
-        if (a.c->use_shared_bank()) {
-          send_text_message(a.c, "$C6Using shared bank (0)");
-        } else {
-          send_text_message(a.c, "$C6Created shared bank (0)");
-        }
-      } else if (new_char_index <= 4) {
-        a.c->use_character_bank(new_char_index - 1);
-        auto bp = a.c->current_bank_character();
+      if (new_char_index <= 0) {
+        a.c->change_bank(-1);
+        send_text_message(a.c, "$C6Using shared bank");
+
+      } else if (new_char_index <= 127) {
+        a.c->change_bank(new_char_index - 1);
+        send_text_message_fmt(a.c, "$C6Using character {}'s bank", new_char_index);
 
-        auto name = escape_player_name(bp->disp.name.decode(a.c->language()));
-        send_text_message_fmt(a.c, "$C6Using {}\'s bank ({})", name, new_char_index);
       } else {
         throw precondition_failed("$C6Invalid bank number");
       }
 
-      auto& bank = a.c->current_bank();
-      bank.assign_ids(0x99000000 + (a.c->lobby_client_id << 20));
+      auto bank = a.c->bank_file();
+      bank->assign_ids(0x99000000 + (a.c->lobby_client_id << 20));
       a.c->log.info_f("Assigned bank item IDs");
       a.c->print_bank();
 
-      send_text_message_fmt(a.c, "{} items\n{} Meseta", bank.num_items, bank.meseta);
+      send_text_message_fmt(a.c, "{} items\n{} Meseta", bank->items.size(), bank->meseta);
       co_return;
     });
 
@@ -487,17 +484,18 @@ static asio::awaitable<void> server_command_bbchar_savechar(const Args& a, bool
 
   // If the client isn't BB, request the player info. (If they are BB, the
   // server already has it)
-  auto ch = co_await send_get_player_info(a.c, true);
+  GetPlayerInfoResult ch;
+  if (a.c->version() == Version::BB_V4) {
+    ch.character = a.c->character_file();
+    ch.is_full_info = true;
+  } else {
+    ch = co_await send_get_player_info(a.c, true);
+  }
 
   string filename = dest_bb_license
       ? Client::character_filename(dest_bb_license->username, dest_character_index)
       : Client::backup_character_filename(dest_account->account_id, dest_character_index, is_ep3(a.c->version()));
 
-  if (s->player_files_manager->get_character(filename)) {
-    send_text_message(a.c, "$C6The target player\nis currently loaded.\nSign off in Blue\nBurst and try again.");
-    co_return;
-  }
-
   if (ch.is_full_info) {
     // Client sent 30; ch contains the verbatim save file from the client
     if (ch.ep3_character) {
@@ -777,41 +775,40 @@ ChatCommandDefinition cc_dropmode(
 
       if (a.c->proxy_session) {
 
-        using DropMode = ProxySession::DropMode;
         if (a.text.empty()) {
           switch (a.c->proxy_session->drop_mode) {
-            case DropMode::DISABLED:
+            case ProxyDropMode::DISABLED:
               send_text_message(a.c, "Drop mode: disabled");
               break;
-            case DropMode::PASSTHROUGH:
+            case ProxyDropMode::PASSTHROUGH:
               send_text_message(a.c, "Drop mode: default");
               break;
-            case DropMode::INTERCEPT:
+            case ProxyDropMode::INTERCEPT:
               send_text_message(a.c, "Drop mode: proxy");
               break;
           }
 
         } else {
-          DropMode new_mode;
+          ProxyDropMode new_mode;
           if ((a.text == "none") || (a.text == "disabled")) {
-            new_mode = DropMode::DISABLED;
+            new_mode = ProxyDropMode::DISABLED;
           } else if ((a.text == "default") || (a.text == "passthrough")) {
-            new_mode = DropMode::PASSTHROUGH;
+            new_mode = ProxyDropMode::PASSTHROUGH;
           } else if ((a.text == "proxy") || (a.text == "intercept")) {
-            new_mode = DropMode::INTERCEPT;
+            new_mode = ProxyDropMode::INTERCEPT;
           } else {
             throw precondition_failed("Invalid drop mode");
           }
 
           a.c->proxy_session->set_drop_mode(s, a.c->version(), a.c->override_random_seed, new_mode);
           switch (a.c->proxy_session->drop_mode) {
-            case DropMode::DISABLED:
+            case ProxyDropMode::DISABLED:
               send_text_message(a.c->channel, "Item drops disabled");
               break;
-            case DropMode::PASSTHROUGH:
+            case ProxyDropMode::PASSTHROUGH:
               send_text_message(a.c->channel, "Item drops changed\nto default mode");
               break;
-            case DropMode::INTERCEPT:
+            case ProxyDropMode::INTERCEPT:
               send_text_message(a.c->channel, "Item drops changed\nto proxy mode");
               break;
           }
@@ -821,36 +818,36 @@ ChatCommandDefinition cc_dropmode(
         auto l = a.c->require_lobby();
         if (a.text.empty()) {
           switch (l->drop_mode) {
-            case Lobby::DropMode::DISABLED:
+            case ServerDropMode::DISABLED:
               send_text_message(a.c, "Drop mode: disabled");
               break;
-            case Lobby::DropMode::CLIENT:
+            case ServerDropMode::CLIENT:
               send_text_message(a.c, "Drop mode: client");
               break;
-            case Lobby::DropMode::SERVER_SHARED:
+            case ServerDropMode::SERVER_SHARED:
               send_text_message(a.c, "Drop mode: server\nshared");
               break;
-            case Lobby::DropMode::SERVER_PRIVATE:
+            case ServerDropMode::SERVER_PRIVATE:
               send_text_message(a.c, "Drop mode: server\nprivate");
               break;
-            case Lobby::DropMode::SERVER_DUPLICATE:
+            case ServerDropMode::SERVER_DUPLICATE:
               send_text_message(a.c, "Drop mode: server\nduplicate");
               break;
           }
 
         } else {
           a.check_is_leader();
-          Lobby::DropMode new_mode;
+          ServerDropMode new_mode;
           if ((a.text == "none") || (a.text == "disabled")) {
-            new_mode = Lobby::DropMode::DISABLED;
+            new_mode = ServerDropMode::DISABLED;
           } else if (a.text == "client") {
-            new_mode = Lobby::DropMode::CLIENT;
+            new_mode = ServerDropMode::CLIENT;
           } else if ((a.text == "shared") || (a.text == "server")) {
-            new_mode = Lobby::DropMode::SERVER_SHARED;
+            new_mode = ServerDropMode::SERVER_SHARED;
           } else if ((a.text == "private") || (a.text == "priv")) {
-            new_mode = Lobby::DropMode::SERVER_PRIVATE;
+            new_mode = ServerDropMode::SERVER_PRIVATE;
           } else if ((a.text == "duplicate") || (a.text == "dup")) {
-            new_mode = Lobby::DropMode::SERVER_DUPLICATE;
+            new_mode = ServerDropMode::SERVER_DUPLICATE;
           } else {
             throw precondition_failed("Invalid drop mode");
           }
@@ -861,19 +858,19 @@ ChatCommandDefinition cc_dropmode(
 
           l->drop_mode = new_mode;
           switch (l->drop_mode) {
-            case Lobby::DropMode::DISABLED:
+            case ServerDropMode::DISABLED:
               send_text_message(l, "Item drops disabled");
               break;
-            case Lobby::DropMode::CLIENT:
+            case ServerDropMode::CLIENT:
               send_text_message(l, "Item drops changed\nto client mode");
               break;
-            case Lobby::DropMode::SERVER_SHARED:
+            case ServerDropMode::SERVER_SHARED:
               send_text_message(l, "Item drops changed\nto server shared\nmode");
               break;
-            case Lobby::DropMode::SERVER_PRIVATE:
+            case ServerDropMode::SERVER_PRIVATE:
               send_text_message(l, "Item drops changed\nto server private\nmode");
               break;
-            case Lobby::DropMode::SERVER_DUPLICATE:
+            case ServerDropMode::SERVER_DUPLICATE:
               send_text_message(l, "Item drops changed\nto server duplicate\nmode");
               break;
           }
@@ -904,7 +901,7 @@ ChatCommandDefinition cc_edit(
       using MatType = PSOBBCharacterFile::MaterialType;
 
       try {
-        auto p = a.c->character();
+        auto p = a.c->character_file();
         if (tokens.at(0) == "atp" && (cheats_allowed || !s->cheat_flags.edit_stats)) {
           p->disp.stats.char_stats.atp = stoul(tokens.at(1));
         } else if (tokens.at(0) == "mst" && (cheats_allowed || !s->cheat_flags.edit_stats)) {
@@ -925,7 +922,7 @@ ChatCommandDefinition cc_edit(
           p->disp.stats.experience = stoul(tokens.at(1));
         } else if (tokens.at(0) == "level" && (cheats_allowed || !s->cheat_flags.edit_stats)) {
           p->disp.stats.level = stoul(tokens.at(1)) - 1;
-          p->recompute_stats(s->level_table(a.c->version()));
+          p->recompute_stats(s->level_table(a.c->version()), true);
         } else if (((tokens.at(0) == "material") || (tokens.at(0) == "mat")) && !is_v1_or_v2(a.c->version()) && (cheats_allowed || !s->cheat_flags.reset_materials)) {
           if (tokens.at(1) == "reset") {
             const auto& which = tokens.at(2);
@@ -963,14 +960,14 @@ ChatCommandDefinition cc_edit(
           } else {
             throw precondition_failed("$C6Invalid subcommand");
           }
-          p->recompute_stats(s->level_table(a.c->version()));
+          p->recompute_stats(s->level_table(a.c->version()), false);
         } else if (tokens.at(0) == "namecolor") {
           p->disp.visual.name_color = stoul(tokens.at(1), nullptr, 16);
         } else if (tokens.at(0) == "language" || tokens.at(0) == "lang") {
           if (tokens.at(1).size() != 1) {
             throw runtime_error("invalid language");
           }
-          uint8_t new_language = language_code_for_char(tokens.at(1).at(0));
+          Language new_language = language_for_char(tokens.at(1).at(0));
           a.c->channel->language = new_language;
           p->inventory.language = new_language;
           p->guild_card.language = new_language;
@@ -989,52 +986,19 @@ ChatCommandDefinition cc_edit(
             p->disp.visual.section_id = secid;
           }
         } else if (tokens.at(0) == "name") {
-          vector<string> orig_tokens = phosg::split(a.text, ' ');
+          vector<string> orig_tokens = phosg::split(a.text, ' ', 1);
           p->disp.name.encode(orig_tokens.at(1), p->inventory.language);
         } else if (tokens.at(0) == "npc") {
           if (tokens.at(1) == "none") {
             p->disp.visual.extra_model = 0;
             p->disp.visual.validation_flags &= 0xFD;
-            // Restore saved fields, if any
-            if (p->disp.visual.unused[0] == 0x8D) {
-              p->disp.visual.char_class = p->disp.visual.unused[1];
-              p->disp.visual.head = p->disp.visual.unused[2];
-              p->disp.visual.hair = p->disp.visual.unused[3];
-              p->disp.visual.unused.clear(0);
-            }
+            p->disp.visual.restore_npc_saved_fields();
           } else {
             uint8_t npc = npc_for_name(tokens.at(1), a.c->version());
             if (npc == 0xFF) {
               throw precondition_failed("$C6No such NPC");
             }
-
-            // Some NPCs can crash the client if the character's class is
-            // incorrect. To handle this, we save the affected fields in the unused
-            // bytes after extra_model.
-            int8_t replacement_class = -1;
-            switch (npc) {
-              case 1: // Rico (replace with HUnewearl)
-              case 6: // Elly (replace with HUnewearl)
-                replacement_class = 0x01;
-                break;
-              case 0: // Ninja (replace with HUmar)
-              case 2: // Sonic (replace with HUmar)
-              case 5: // Flowen (replace with HUmar)
-                replacement_class = 0x00;
-                break;
-            }
-            if (replacement_class >= 0) {
-              if (p->disp.visual.unused[0] != 0x8D) {
-                p->disp.visual.unused[0] = 0x8D;
-                p->disp.visual.unused[1] = p->disp.visual.char_class;
-                p->disp.visual.unused[2] = p->disp.visual.head;
-                p->disp.visual.unused[3] = p->disp.visual.hair;
-              }
-              p->disp.visual.char_class = replacement_class;
-              p->disp.visual.head = 0x00;
-              p->disp.visual.hair = 0x00;
-            }
-
+            p->disp.visual.backup_npc_saved_fields();
             p->disp.visual.extra_model = npc;
             p->disp.visual.validation_flags |= 0x02;
           }
@@ -1140,14 +1104,12 @@ ChatCommandDefinition cc_exit(
         is_in_quest = a.c->proxy_session->is_in_quest;
       } else {
         auto l = a.c->require_lobby();
-        is_in_quest = (l->check_flag(Lobby::Flag::QUEST_IN_PROGRESS) ||
-            l->check_flag(Lobby::Flag::JOINABLE_QUEST_IN_PROGRESS));
+        is_in_quest = (l->check_flag(Lobby::Flag::QUEST_IN_PROGRESS) || l->check_flag(Lobby::Flag::JOINABLE_QUEST_IN_PROGRESS));
       }
       if (is_in_quest) {
         // Client is in a quest; command 6x73 triggers game exit
         G_UnusedHeader cmd = {0x73, 0x01, 0x0000};
         a.c->channel->send(0x60, 0x00, cmd);
-        a.c->floor = 0;
         co_return;
       }
 
@@ -1287,7 +1249,7 @@ ChatCommandDefinition cc_item(
         item = s->parse_item_description(a.c->version(), a.text);
         item.id = l->generate_item_id(a.c->lobby_client_id);
 
-        if ((l->drop_mode == Lobby::DropMode::SERVER_PRIVATE) || (l->drop_mode == Lobby::DropMode::SERVER_DUPLICATE)) {
+        if ((l->drop_mode == ServerDropMode::SERVER_PRIVATE) || (l->drop_mode == ServerDropMode::SERVER_DUPLICATE)) {
           l->add_item(a.c->floor, item, a.c->pos, nullptr, nullptr, (1 << a.c->lobby_client_id));
           send_drop_stacked_item_to_channel(s, a.c->channel, item, a.c->floor, a.c->pos);
         } else {
@@ -1296,7 +1258,7 @@ ChatCommandDefinition cc_item(
         }
       }
 
-      string name = s->describe_item(a.c->version(), item, true);
+      string name = s->describe_item(a.c->version(), item, ItemNameIndex::Flag::INCLUDE_PSO_COLOR_ESCAPES);
       send_text_message(a.c, "$C7Item created:\n" + name);
       co_return;
     });
@@ -1357,29 +1319,39 @@ ChatCommandDefinition cc_killcount(
     +[](const Args& a) -> asio::awaitable<void> {
       a.check_is_proxy(false);
 
-      auto p = a.c->character();
-      size_t item_index;
-      try {
-        item_index = p->inventory.find_equipped_item(EquipSlot::WEAPON);
-      } catch (const out_of_range&) {
-        throw precondition_failed("No weapon equipped");
+      auto p = a.c->character_file();
+      vector<size_t> item_indexes;
+      for (size_t z = 0; z < p->inventory.num_items; z++) {
+        const auto& item = p->inventory.items[z];
+        if (item.is_equipped() && item.data.has_kill_count()) {
+          item_indexes.emplace_back(z);
+        }
       }
 
-      const auto& item = p->inventory.items.at(item_index);
-      if (!item.data.has_kill_count()) {
-        throw precondition_failed("Weapon does not\nhave a kill count");
-      }
+      if (item_indexes.empty()) {
+        throw precondition_failed("No equipped items\nhave kill counts");
 
-      // Kill counts are only accurate on the server side at all times on BB. On
-      // other versions, we update the server's view of the client's inventory
-      // during games, but we can't track kills because the client doesn't inform
-      // the server whether it counted a kill for any individual enemy. So, on
-      // non-BB versions, the kill count is accurate at all times in the lobby
-      // (since kills can't occur there), or at the beginning of a game.
-      if ((a.c->version() == Version::BB_V4) || !a.c->require_lobby()->is_game()) {
-        send_text_message_fmt(a.c, "{} kills", item.data.get_kill_count());
       } else {
-        send_text_message_fmt(a.c, "{} kills as of\ngame join", item.data.get_kill_count());
+        // Kill counts are only accurate on the server side at all times on BB.
+        // On other versions, we update the server's view of the client's
+        // inventory during games, but we can't track kills because the client
+        // doesn't inform the server whether it counted a kill for any
+        // individual enemy. So, on non-BB versions, the kill count is accurate
+        // at all times in the lobby (since kills can't occur there), or at the
+        // beginning of a game.
+        if ((a.c->version() == Version::BB_V4) || !a.c->require_lobby()->is_game()) {
+          send_text_message(a.c, "As of now:");
+        } else {
+          send_text_message(a.c, "As of game join:");
+        }
+
+        auto s = a.c->require_server_state();
+        for (size_t z : item_indexes) {
+          const auto& item = p->inventory.items[z];
+          string name = s->describe_item(
+              a.c->version(), item.data, ItemNameIndex::Flag::INCLUDE_PSO_COLOR_ESCAPES | ItemNameIndex::Flag::NAME_ONLY);
+          send_text_message_fmt(a.c, "{}$C7: {} kills", name, item.data.get_kill_count());
+        }
       }
       co_return;
     });
@@ -1460,26 +1432,27 @@ ChatCommandDefinition cc_lobby_info(
               if (l->max_level == 0xFFFFFFFF) {
                 lines.emplace_back(std::format("$C6{:08X}$C7 L$C6{}+$C7", l->lobby_id, l->min_level + 1));
               } else {
-                lines.emplace_back(std::format(
-                    "$C6{:08X}$C7 L$C6{}-{}$C7", l->lobby_id, l->min_level + 1, l->max_level + 1));
+                lines.emplace_back(std::format("$C6{:08X}$C7 L$C6{}-{}$C7", l->lobby_id, l->min_level + 1, l->max_level + 1));
+              }
+              uint8_t effective_section_id = l->effective_section_id();
+              if (effective_section_id < 10) {
+                lines.emplace_back(std::format("$C7Section ID: $C6{}$C7", name_for_section_id(effective_section_id)));
               }
-              lines.emplace_back(std::format(
-                  "$C7Section ID: $C6{}$C7", name_for_section_id(l->effective_section_id())));
 
               switch (l->drop_mode) {
-                case Lobby::DropMode::DISABLED:
+                case ServerDropMode::DISABLED:
                   lines.emplace_back("Drops disabled");
                   break;
-                case Lobby::DropMode::CLIENT:
+                case ServerDropMode::CLIENT:
                   lines.emplace_back("Client item table");
                   break;
-                case Lobby::DropMode::SERVER_SHARED:
+                case ServerDropMode::SERVER_SHARED:
                   lines.emplace_back("Server item table");
                   break;
-                case Lobby::DropMode::SERVER_PRIVATE:
+                case ServerDropMode::SERVER_PRIVATE:
                   lines.emplace_back("Server indiv items");
                   break;
-                case Lobby::DropMode::SERVER_DUPLICATE:
+                case ServerDropMode::SERVER_DUPLICATE:
                   lines.emplace_back("Server dup items");
                   break;
                 default:
@@ -1613,13 +1586,13 @@ ChatCommandDefinition cc_loadchar(
         };
 
         if (a.c->version() == Version::DC_V2) {
-          PSODCV2CharacterFile::Character dc_char = *a.c->character();
+          PSODCV2CharacterFile::Character dc_char = *a.c->character_file();
           co_await send_set_extended_player_info(dc_char);
         } else if (a.c->version() == Version::GC_NTE) {
-          PSOGCNTECharacterFileCharacter gc_char = *a.c->character();
+          PSOGCNTECharacterFileCharacter gc_char = *a.c->character_file();
           co_await send_set_extended_player_info(gc_char);
         } else if (a.c->version() == Version::GC_V3) {
-          PSOGCCharacterFile::Character gc_char = *a.c->character();
+          PSOGCCharacterFile::Character gc_char = *a.c->character_file();
           co_await send_set_extended_player_info(gc_char);
         } else if (a.c->version() == Version::GC_EP3_NTE) {
           PSOGCEp3NTECharacter nte_char = *ep3_char;
@@ -1630,7 +1603,7 @@ ChatCommandDefinition cc_loadchar(
           if (!a.c->login || !a.c->login->xb_license) {
             throw runtime_error("XB client is not logged in");
           }
-          PSOXBCharacterFileCharacter xb_char = *a.c->character();
+          PSOXBCharacterFile::Character xb_char = *a.c->character_file();
           xb_char.guild_card.xb_user_id_high = (a.c->login->xb_license->user_id >> 32) & 0xFFFFFFFF;
           xb_char.guild_card.xb_user_id_low = a.c->login->xb_license->user_id & 0xFFFFFFFF;
           co_await send_set_extended_player_info(xb_char);
@@ -1648,12 +1621,103 @@ ChatCommandDefinition cc_loadchar(
       co_return;
     });
 
+ChatCommandDefinition cc_makeobj(
+    {"$makeobj"},
+    +[](const Args& a) -> asio::awaitable<void> {
+      a.check_debug_enabled();
+
+      auto tokens = phosg::split(a.text, ' ');
+      if (tokens.size() < 1) {
+        throw runtime_error("not enough arguments");
+      }
+
+      uint32_t base_type_high = stoul(tokens[0], nullptr, 0) << 16;
+      VectorXYZF pos = a.c->pos;
+      VectorXYZI angle{0, 0, 0};
+      VectorXYZF param123{0, 0, 0};
+      VectorXYZI param456{0, 0, 0};
+      for (size_t z = 1; z < tokens.size(); z++) {
+        auto subtokens = phosg::split(tokens[z], ':');
+        if (subtokens.size() != 2 || subtokens[0].size() != 1) {
+          throw runtime_error("invalid argument: " + tokens[z]);
+        }
+        switch (tolower(subtokens[0].front())) {
+          case 'X':
+          case 'x':
+            pos.x = stof(subtokens[1]);
+            break;
+          case 'Y':
+          case 'y':
+            pos.y = stof(subtokens[1]);
+            break;
+          case 'Z':
+          case 'z':
+            pos.z = stof(subtokens[1]);
+            break;
+          case 'R':
+          case 'r':
+            angle.x = stol(subtokens[1], nullptr, 0);
+            break;
+          case 'P':
+          case 'p':
+            angle.y = stol(subtokens[1], nullptr, 0);
+            break;
+          case 'W':
+          case 'w':
+            angle.z = stol(subtokens[1], nullptr, 0);
+            break;
+          case '1':
+            param123.x = stof(subtokens[1]);
+            break;
+          case '2':
+            param123.y = stof(subtokens[1]);
+            break;
+          case '3':
+            param123.z = stof(subtokens[1]);
+            break;
+          case '4':
+            param456.x = stol(subtokens[1], nullptr, 0);
+            break;
+          case '5':
+            param456.y = stol(subtokens[1], nullptr, 0);
+            break;
+          case '6':
+            param456.z = stol(subtokens[1], nullptr, 0);
+            break;
+          default:
+            throw runtime_error("invalid argument: " + tokens[z]);
+        }
+      }
+
+      unordered_map<string, uint32_t> label_writes{
+          {"base_type_high", base_type_high},
+          {"floor_low", a.c->floor},
+          {"pos_x", std::bit_cast<uint32_t>(pos.x.load())},
+          {"pos_y", std::bit_cast<uint32_t>(pos.y.load())},
+          {"pos_z", std::bit_cast<uint32_t>(pos.z.load())},
+          {"angle_x", angle.x},
+          {"angle_y", angle.y},
+          {"angle_z", angle.z},
+          {"param1", std::bit_cast<uint32_t>(param123.x.load())},
+          {"param2", std::bit_cast<uint32_t>(param123.y.load())},
+          {"param3", std::bit_cast<uint32_t>(param123.z.load())},
+          {"param4", param456.x},
+          {"param5", param456.y},
+          {"param6", param456.z},
+      };
+
+      co_await prepare_client_for_patches(a.c);
+      auto s = a.c->require_server_state();
+      auto fn = s->function_code_index->get_patch("CreateObject", a.c->specific_version);
+      co_await send_function_call(a.c, fn, label_writes);
+    });
+
 ChatCommandDefinition cc_matcount(
     {"$matcount"},
     +[](const Args& a) -> asio::awaitable<void> {
       a.check_is_proxy(false);
 
-      auto p = a.c->character();
+      auto p = a.c->character_file();
       if (is_v1_or_v2(a.c->version())) {
         send_text_message_fmt(a.c, "{} HP, {} TP",
             p->get_material_usage(PSOBBCharacterFile::MaterialType::HP),
@@ -1818,13 +1882,13 @@ ChatCommandDefinition cc_ping(
       co_return;
     });
 
-static string file_path_for_recording(const std::string& args, uint32_t account_id) {
+static string file_path_for_recording(const std::string& args, uint32_t account_id, bool compressed) {
   for (char ch : args) {
     if (ch <= 0x20 || ch > 0x7E || ch == '/') {
       throw runtime_error("invalid recording name");
     }
   }
-  return std::format("system/ep3/battle-records/{:010}_{}.mzrd", account_id, args);
+  return std::format("system/ep3/battle-records/{:010}_{}.mzr{}", account_id, args, compressed ? "" : "d");
 }
 
 ChatCommandDefinition cc_playrec(
@@ -1837,25 +1901,28 @@ ChatCommandDefinition cc_playrec(
       if (l->is_game() && l->battle_player) {
         l->battle_player->start();
       } else if (!l->is_game()) {
-        string file_path = file_path_for_recording(a.text, a.c->login->account->account_id);
 
         auto s = a.c->require_server_state();
         string filename = a.text;
-        bool start_battle_player_immediately = (filename[0] == '!');
-        if (start_battle_player_immediately) {
+        bool start_battle_player_immediately = (filename.at(0) != '!');
+        if (!start_battle_player_immediately) {
           filename = filename.substr(1);
         }
 
         string data;
         try {
-          data = phosg::load_file(file_path);
+          data = phosg::load_file(file_path_for_recording(filename, a.c->login->account->account_id, false));
         } catch (const phosg::cannot_open_file&) {
-          throw precondition_failed("$C4The recording does\nnot exist");
+          try {
+            data = prs_decompress(phosg::load_file(file_path_for_recording(filename, a.c->login->account->account_id, true)));
+          } catch (const phosg::cannot_open_file&) {
+            throw precondition_failed("$C4The recording does\nnot exist");
+          }
         }
         auto record = make_shared<Episode3::BattleRecord>(data);
         auto battle_player = make_shared<Episode3::BattleRecordPlayer>(s->io_context, record);
         auto game = create_game_generic(
-            s, a.c, a.text, "", Episode::EP3, GameMode::NORMAL, 0, false, nullptr, battle_player);
+            s, a.c, filename, "", Episode::EP3, GameMode::NORMAL, Difficulty::NORMAL, false, nullptr, battle_player);
         if (game) {
           if (start_battle_player_immediately) {
             game->set_flag(Lobby::Flag::START_BATTLE_PLAYER_IMMEDIATELY);
@@ -1895,7 +1962,7 @@ ChatCommandDefinition cc_qcheck(
         if (!l->quest_flags_known || l->quest_flags_known->get(l->difficulty, flag_num)) {
           send_text_message_fmt(a.c, "$C7Game: flag 0x{:X} ({})\nis {} on {}",
               flag_num, flag_num,
-              a.c->character()->quest_flags.get(l->difficulty, flag_num) ? "set" : "not set",
+              a.c->character_file()->quest_flags.get(l->difficulty, flag_num) ? "set" : "not set",
               name_for_difficulty(l->difficulty));
         } else {
           send_text_message_fmt(a.c, "$C7Game: flag 0x{:X} ({})\nis unknown on {}",
@@ -1904,7 +1971,7 @@ ChatCommandDefinition cc_qcheck(
       } else if (a.c->version() == Version::BB_V4) {
         send_text_message_fmt(a.c, "$C7Player: flag 0x{:X} ({})\nis {} on {}",
             flag_num, flag_num,
-            a.c->character()->quest_flags.get(l->difficulty, flag_num) ? "set" : "not set",
+            a.c->character_file()->quest_flags.get(l->difficulty, flag_num) ? "set" : "not set",
             name_for_difficulty(l->difficulty));
       }
       co_return;
@@ -1929,7 +1996,7 @@ static void command_qset_qclear(const Args& a, bool should_set) {
       }
     }
 
-    auto p = a.c->character(false);
+    auto p = a.c->character_file(false);
     if (p) {
       if (should_set) {
         p->quest_flags.set(l->difficulty, flag_num);
@@ -1946,8 +2013,8 @@ static void command_qset_qclear(const Args& a, bool should_set) {
       a.c->proxy_session->server_channel->send(0x60, 0x00, &cmd, sizeof(cmd));
     }
   } else {
-    uint8_t difficulty = a.c->proxy_session ? a.c->proxy_session->lobby_difficulty : a.c->require_lobby()->difficulty;
-    G_UpdateQuestFlag_V3_BB_6x75 cmd = {{{0x75, 0x03, 0x0000}, flag_num, should_set ? 0 : 1}, difficulty, 0x0000};
+    Difficulty difficulty = a.c->proxy_session ? a.c->proxy_session->lobby_difficulty : a.c->require_lobby()->difficulty;
+    G_UpdateQuestFlag_V3_BB_6x75 cmd = {{{0x75, 0x03, 0x0000}, flag_num, should_set ? 0 : 1}, static_cast<uint16_t>(difficulty), 0x0000};
     a.c->channel->send(0x60, 0x00, &cmd, sizeof(cmd));
     if (a.c->proxy_session) {
       a.c->proxy_session->server_channel->send(0x60, 0x00, &cmd, sizeof(cmd));
@@ -1982,7 +2049,7 @@ ChatCommandDefinition cc_qfread(
         throw runtime_error("invalid quest counter definition");
       }
 
-      uint32_t counter_value = a.c->character()->quest_counters.at(counter_index) & mask;
+      uint32_t counter_value = a.c->character_file()->quest_counters.at(counter_index) & mask;
 
       while (!(mask & 1)) {
         mask >>= 1;
@@ -2002,7 +2069,7 @@ ChatCommandDefinition cc_qgread(
     +[](const Args& a) -> asio::awaitable<void> {
       a.check_is_proxy(false);
       uint8_t counter_num = stoul(a.text, nullptr, 0);
-      const auto& counters = a.c->character()->quest_counters;
+      const auto& counters = a.c->character_file()->quest_counters;
       if (counter_num >= counters.size()) {
         throw precondition_failed("$C7Counter ID must be\nless than {}", counters.size());
       } else {
@@ -2031,11 +2098,11 @@ ChatCommandDefinition cc_qgwrite(
 
       uint8_t counter_num = stoul(tokens[0], nullptr, 0);
       uint32_t value = stoul(tokens[1], nullptr, 0);
-      auto& counters = a.c->character()->quest_counters;
+      auto& counters = a.c->character_file()->quest_counters;
       if (counter_num >= counters.size()) {
         throw precondition_failed("$C7Counter ID must be\nless than {}", counters.size());
       } else {
-        a.c->character()->quest_counters[counter_num] = value;
+        a.c->character_file()->quest_counters[counter_num] = value;
         G_SetQuestCounter_BB_6xD2 cmd = {{0xD2, sizeof(G_SetQuestCounter_BB_6xD2) / 4, a.c->lobby_client_id}, counter_num, value};
         send_command_t(a.c, 0x60, 0x00, cmd);
         send_text_message_fmt(a.c, "$C7Quest counter {}\nset to {}", counter_num, value);
@@ -2101,8 +2168,7 @@ ChatCommandDefinition cc_quest(
       a.check_is_game(true);
 
       auto s = a.c->require_server_state();
-      Version effective_version = is_ep3(a.c->version()) ? Version::GC_V3 : a.c->version();
-      auto q = s->quest_index(effective_version)->get(stoul(a.text));
+      auto q = s->quest_index->get(stoul(a.text));
       if (!q) {
         throw precondition_failed("$C6Quest not found");
       }
@@ -2112,11 +2178,20 @@ ChatCommandDefinition cc_quest(
         if (l->count_clients() > 1) {
           throw precondition_failed("$C6This command can only\nbe used with no\nother players present");
         }
-        if (!q->allow_start_from_chat_command) {
+        if (!q->meta.allow_start_from_chat_command) {
           throw precondition_failed("$C6This quest cannot\nbe started with the\n%squest command");
         }
       }
 
+      for (size_t client_id = 0; client_id < l->max_clients; client_id++) {
+        auto lc = l->clients[client_id];
+        if (lc) {
+          if (!q->version(lc->version(), lc->language())) {
+            throw precondition_failed("$C6Quest does not exist\nfor all players\' game\nversions");
+          }
+        }
+      }
+
       set_lobby_quest(a.c->require_lobby(), q, true);
       co_return;
     });
@@ -2216,7 +2291,7 @@ ChatCommandDefinition cc_saverec(
       if (!a.c->ep3_prev_battle_record) {
         throw precondition_failed("$C4No finished\nrecording is\npresent");
       }
-      string file_path = file_path_for_recording(a.text, a.c->login->account->account_id);
+      string file_path = file_path_for_recording(a.text, a.c->login->account->account_id, false);
       string data = a.c->ep3_prev_battle_record->serialize();
       phosg::save_file(file_path, data);
       send_text_message(a.c, "$C7Recording saved");
@@ -2386,6 +2461,27 @@ ChatCommandDefinition cc_song(
       co_return;
     });
 
+ChatCommandDefinition cc_sound(
+    {"$sound"},
+    +[](const Args& a) -> asio::awaitable<void> {
+      bool echo_to_all = (!a.text.empty() && a.text[0] == '!');
+      uint32_t sound_id = stoul(echo_to_all ? a.text.substr(1) : a.text, nullptr, 16);
+
+      auto l = a.c->require_lobby();
+      uint8_t area = l->is_game() ? l->area_for_floor(a.c->version(), a.c->floor) : 0x0F;
+      G_PlaySoundFromPlayer_6xB2 cmd = {{0xB2, 0x03, 0x0000}, area, 0x00, a.c->lobby_client_id, sound_id};
+      if (!echo_to_all) {
+        send_command_t(a.c, 0x60, 0x00, cmd);
+      } else if (a.c->proxy_session) {
+        send_command_t(a.c, 0x60, 0x00, cmd);
+        send_command_t(a.c->proxy_session->server_channel, 0x60, 0x00, cmd);
+      } else {
+        a.check_debug_enabled();
+        send_command_t(a.c->require_lobby(), 0x60, 0x00, cmd);
+      }
+      co_return;
+    });
+
 ChatCommandDefinition cc_spec(
     {"$spec"},
     +[](const Args& a) -> asio::awaitable<void> {
@@ -2530,7 +2626,7 @@ ChatCommandDefinition cc_surrender(
       if (!ps || !ps->is_alive()) {
         throw precondition_failed("$C6Defeated players\ncannot surrender");
       }
-      string name = remove_color(a.c->character()->disp.name.decode(a.c->language()));
+      string name = remove_color(a.c->character_file()->disp.name.decode(a.c->language()));
       send_text_message_fmt(l, "$C6{} has\nsurrendered", name);
       for (const auto& watcher_l : l->watcher_lobbies) {
         send_text_message_fmt(watcher_l, "$C6{} has\nsurrendered", name);
@@ -2635,6 +2731,47 @@ ChatCommandDefinition cc_swsetall(
       co_return;
     });
 
+ChatCommandDefinition cc_switchchar(
+    {"$switchchar"},
+    +[](const Args& a) -> asio::awaitable<void> {
+      auto l = a.c->require_lobby();
+      auto s = a.c->require_server_state();
+
+      a.check_is_proxy(false);
+      a.check_is_game(false);
+      if (a.c->version() != Version::BB_V4) {
+        throw precondition_failed("This command can only\nbe used on BB");
+      }
+
+      int32_t index = stol(a.text, nullptr, 0) - 1;
+      if (index < 0) {
+        throw precondition_failed("Invalid slot number");
+      }
+      auto filename = Client::character_filename(a.c->login->bb_license->username, index);
+      if (!std::filesystem::is_regular_file(filename)) {
+        throw precondition_failed("No character exists\nin that slot");
+      }
+
+      a.c->unload_character(true);
+      a.c->bb_character_index = index;
+      a.c->bb_bank_character_index = index;
+
+      // TODO: This can trigger a client bug where the previous character's
+      // name label object isn't deleted if the leave and join notifications
+      // are received on the same frame. This results in the receiving player
+      // seeing both labels over the new character, with the latest one
+      // appearing on top. We could fix this by requiring each recipient to
+      // reply to a ping between the two commands, similar to how the 64 and
+      // 6x6D commands are split during game joining, but implementing that
+      // here seems not worth the effort given the low likelihood and impact of
+      // this bug.
+      send_complete_player_bb(a.c);
+      send_player_leave_notification(l, a.c->lobby_client_id);
+      s->send_lobby_join_notifications(l, a.c);
+
+      co_return;
+    });
+
 ChatCommandDefinition cc_unset(
     {"$unset"},
     +[](const Args& a) -> asio::awaitable<void> {
@@ -2654,8 +2791,12 @@ ChatCommandDefinition cc_unset(
         throw precondition_failed("$C6Battle has not\nyet begun");
       }
 
-      size_t index = stoull(a.text) - 1;
-      l->ep3_server->force_destroy_field_character(a.c->lobby_client_id, index);
+      size_t index = stoull(a.text);
+      if (index == 0) {
+        l->ep3_server->force_replace_assist_card(a.c->lobby_client_id, 0xFFFF);
+      } else {
+        l->ep3_server->force_destroy_field_character(a.c->lobby_client_id, index - 1);
+      }
       co_return;
     });
 
@@ -2762,7 +2903,7 @@ ChatCommandDefinition cc_what(
         throw precondition_failed("$C4No items are near you");
       } else {
         auto s = a.c->require_server_state();
-        string name = s->describe_item(a.c->version(), nearest_fi->data, true);
+        string name = s->describe_item(a.c->version(), nearest_fi->data, ItemNameIndex::Flag::INCLUDE_PSO_COLOR_ESCAPES);
         send_text_message(a.c, name);
       }
       co_return;
@@ -2778,13 +2919,10 @@ static void whatobj_whatene_fn(const Args& a, bool include_objs, bool include_en
     throw precondition_failed("$C4No map loaded");
   }
 
-  // TODO: We should use the actual area if a loaded quest has reassigned
-  // them; it's likely that the variations will be wrong if we don't
   uint8_t area, layout_var;
   auto s = a.c->require_server_state();
   if (l->episode != Episode::EP3) {
-    auto sdt = s->set_data_table(a.c->version(), l->episode, l->mode, l->difficulty);
-    area = sdt->default_area_for_floor(l->episode, a.c->floor);
+    area = l->area_for_floor(a.c->version(), a.c->floor);
     layout_var = (a.c->floor < 0x10) ? l->variations.entries[a.c->floor].layout.load() : 0x00;
   } else {
     area = a.c->floor;
@@ -2847,7 +2985,7 @@ static void whatobj_whatene_fn(const Args& a, bool include_objs, bool include_en
     const auto* set_entry = nearest_ene->super_ene->version(a.c->version()).set_entry;
     string type_name = MapFile::name_for_enemy_type(set_entry->base_type, a.c->version(), area);
     send_text_message_fmt(a.c, "$C5E-{:03X}\n$C6{}\n$C2{}\n$C7X:{:.2f} Z:{:.2f}",
-        nearest_ene->e_id, phosg::name_for_enum(nearest_ene->type(a.c->version(), l->episode, l->event)),
+        nearest_ene->e_id, phosg::name_for_enum(nearest_ene->type(a.c->version(), l->episode, l->difficulty, l->event)),
         type_name, nearest_worldspace_pos.x, nearest_worldspace_pos.z);
     auto set_str = set_entry->str(a.c->version(), area);
     a.c->log.info_f("Enemy found via $whatobj: E-{:03X} {} at x={:g} y={:g} z={:g}",
@@ -2908,7 +3046,7 @@ ChatCommandDefinition cc_where(
       if (!a.c->proxy_session && l && l->is_game()) {
         for (auto lc : l->clients) {
           if (lc && (lc != a.c)) {
-            string name = lc->character()->disp.name.decode(lc->language());
+            string name = lc->character_file()->disp.name.decode(lc->language());
             send_text_message_fmt(a.c, "$C6{}$C7 {:X}:{}",
                 name, lc->floor, FloorDefinition::get(l->episode, lc->floor).short_name);
           }

src/ChoiceSearch.cc

@@ -28,10 +28,10 @@ const vector<ChoiceSearchCategory> CHOICE_SEARCH_CATEGORIES({
           if (choice_id == 0x0000) {
             return true;
           }
-          uint32_t target_level = target_c->character()->disp.stats.level + 1;
+          uint32_t target_level = target_c->character_file()->disp.stats.level + 1;
           switch (choice_id) {
             case 0x0001:
-              return (labs(static_cast<int32_t>(target_level - searcher_c->character()->disp.stats.level)) <= 5);
+              return (labs(static_cast<int32_t>(target_level - searcher_c->character_file()->disp.stats.level)) <= 5);
             case 0x0002:
               return (target_level <= 10);
             case 0x0003:
@@ -80,13 +80,13 @@ const vector<ChoiceSearchCategory> CHOICE_SEARCH_CATEGORIES({
             case 0x0000:
               return true;
             case 0x0010:
-              return target_c->character()->disp.visual.class_flags & 0x20;
+              return target_c->character_file()->disp.visual.class_flags & 0x20;
             case 0x0011:
-              return target_c->character()->disp.visual.class_flags & 0x40;
+              return target_c->character_file()->disp.visual.class_flags & 0x40;
             case 0x0012:
-              return target_c->character()->disp.visual.class_flags & 0x80;
+              return target_c->character_file()->disp.visual.class_flags & 0x80;
             default:
-              return ((choice_id - 1) == target_c->character()->disp.visual.char_class);
+              return ((choice_id - 1) == target_c->character_file()->disp.visual.char_class);
           }
         },
     },
@@ -143,7 +143,7 @@ const vector<ChoiceSearchCategory> CHOICE_SEARCH_CATEGORIES({
             {0x0006, "Challenge"},
         },
         .client_matches = +[](shared_ptr<Client>, shared_ptr<Client> target_c, uint16_t choice_id) -> bool {
-          uint16_t target_choice_id = target_c->character()->choice_search_config.get_setting(0x0204);
+          uint16_t target_choice_id = target_c->character_file()->choice_search_config.get_setting(0x0204);
           return (choice_id == 0) || (target_choice_id == 0) || (choice_id == target_choice_id);
         },
     },

src/Client.cc

@@ -244,12 +244,11 @@ Client::~Client() {
 void Client::update_channel_name() {
   string default_name = this->channel->default_name();
 
-  auto player = this->character(false, false);
+  auto player = this->character_file(false, false);
   if (player) {
     string name_str = player->disp.name.decode(this->language());
     size_t level = player->disp.stats.level + 1;
-    this->channel->name = std::format("C-{:X} ({} Lv.{}) @ {}",
-        this->id, name_str, level, default_name);
+    this->channel->name = std::format("C-{:X} ({} Lv.{}) @ {}", this->id, name_str, level, default_name);
   } else {
     this->channel->name = std::format("C-{:X} @ {}", this->id, default_name);
   }
@@ -261,11 +260,12 @@ void Client::reschedule_save_game_data_timer() {
     return;
   }
   this->save_game_data_timer.expires_after(std::chrono::seconds(60));
-  this->idle_timeout_timer.async_wait([this](std::error_code ec) {
+  this->save_game_data_timer.async_wait([this](std::error_code ec) {
     if (!ec) {
-      if (this->character(false)) {
+      if (this->character_file(false)) {
         this->save_all();
       }
+      this->reschedule_save_game_data_timer();
     }
   });
 }
@@ -335,7 +335,7 @@ shared_ptr<const TeamIndex::Team> Client::team() const {
     return nullptr;
   }
 
-  auto p = this->character(false);
+  auto p = this->character_file(false);
   auto s = this->require_server_state();
   auto team = s->team_index->get_by_id(this->login->account->bb_team_id);
   if (!team) {
@@ -371,7 +371,7 @@ bool Client::evaluate_quest_availability_expression(
     shared_ptr<const IntegralExpression> expr,
     shared_ptr<const Lobby> game,
     uint8_t event,
-    uint8_t difficulty,
+    Difficulty difficulty,
     size_t num_players,
     bool v1_present) const {
   if (this->login && this->login->account->check_flag(Account::Flag::DISABLE_QUEST_REQUIREMENTS)) {
@@ -383,9 +383,9 @@ bool Client::evaluate_quest_availability_expression(
   if (game && !game->quest_flag_values) {
     throw logic_error("quest flags are missing from game");
   }
-  auto p = this->character();
+  auto p = this->character_file();
   IntegralExpression::Env env = {
-      .flags = &p->quest_flags.data.at(difficulty),
+      .flags = &p->quest_flags.data.at(static_cast<size_t>(difficulty)),
       .challenge_records = &p->challenge_records,
       .team = this->team(),
       .num_players = num_players,
@@ -404,29 +404,31 @@ bool Client::can_see_quest(
     shared_ptr<const Quest> q,
     shared_ptr<const Lobby> game,
     uint8_t event,
-    uint8_t difficulty,
+    Difficulty difficulty,
     size_t num_players,
     bool v1_present) const {
   if (!q->has_version_any_language(this->version())) {
     return false;
   }
-  return this->evaluate_quest_availability_expression(q->available_expression, game, event, difficulty, num_players, v1_present);
+  return this->evaluate_quest_availability_expression(
+      q->meta.available_expression, game, event, difficulty, num_players, v1_present);
 }
 
 bool Client::can_play_quest(
     shared_ptr<const Quest> q,
     shared_ptr<const Lobby> game,
     uint8_t event,
-    uint8_t difficulty,
+    Difficulty difficulty,
     size_t num_players,
     bool v1_present) const {
   if (!q->has_version_any_language(this->version())) {
     return false;
   }
-  if (num_players > q->max_players) {
+  if (num_players > q->meta.max_players) {
     return false;
   }
-  return this->evaluate_quest_availability_expression(q->enabled_expression, game, event, difficulty, num_players, v1_present);
+  return this->evaluate_quest_availability_expression(
+      q->meta.enabled_expression, game, event, difficulty, num_players, v1_present);
 }
 
 bool Client::can_use_chat_commands() const {
@@ -447,8 +449,187 @@ void Client::set_login(shared_ptr<Login> login) {
   }
 }
 
+// System file
+
+string Client::system_filename(const string& bb_username) {
+  return std::format("system/players/system_{}.psosys", bb_username);
+}
+
+string Client::system_filename() const {
+  if (this->version() != Version::BB_V4) {
+    throw logic_error("non-BB players do not have system data");
+  }
+  if (!this->login || !this->login->bb_license) {
+    throw logic_error("client is not logged in");
+  }
+  return this->system_filename(this->login->bb_license->username);
+}
+
+shared_ptr<PSOBBBaseSystemFile> Client::system_file(bool allow_load) {
+  if (!this->system_data && allow_load) {
+    this->load_all_files();
+  }
+  return this->system_data;
+}
+
+shared_ptr<const PSOBBBaseSystemFile> Client::system_file(bool throw_if_missing) const {
+  if (!this->system_data.get() && throw_if_missing) {
+    throw runtime_error("system file is not loaded");
+  }
+  return this->system_data;
+}
+
+void Client::save_system_file() const {
+  if (!this->system_data) {
+    throw logic_error("no system file loaded");
+  }
+  string filename = this->system_filename();
+  phosg::save_object_file(filename, *this->system_data);
+  this->log.info_f("Saved system file {}", filename);
+}
+
+// Guild Card file
+
+string Client::guild_card_filename(const string& bb_username) {
+  return std::format("system/players/guild_cards_{}.psocard", bb_username);
+}
+
+string Client::guild_card_filename() const {
+  if (this->version() != Version::BB_V4) {
+    throw logic_error("non-BB players do not have saved Guild Card files");
+  }
+  if (!this->login || !this->login->bb_license) {
+    throw logic_error("client is not logged in");
+  }
+  return this->guild_card_filename(this->login->bb_license->username);
+}
+
+shared_ptr<PSOBBGuildCardFile> Client::guild_card_file(bool allow_load) {
+  if (!this->guild_card_data && allow_load) {
+    this->load_all_files();
+  }
+  return this->guild_card_data;
+}
+
+shared_ptr<const PSOBBGuildCardFile> Client::guild_card_file(bool allow_load) const {
+  if (!this->guild_card_data && allow_load) {
+    throw runtime_error("account data is not loaded");
+  }
+  return this->guild_card_data;
+}
+
+void Client::save_guild_card_file() const {
+  if (!this->guild_card_data.get()) {
+    throw logic_error("no Guild Card file loaded");
+  }
+  string filename = this->guild_card_filename();
+  phosg::save_object_file(filename, *this->guild_card_data);
+  this->log.info_f("Saved Guild Card file {}", filename);
+}
+
+// Character file
+
+string Client::character_filename(const std::string& bb_username, ssize_t index) {
+  if (bb_username.empty()) {
+    throw logic_error("non-BB players do not have saved character filenames");
+  }
+  if (index < 0) {
+    throw logic_error("character index is not set");
+  }
+  return std::format("system/players/player_{}_{}.psochar", bb_username, index);
+}
+
+string Client::backup_character_filename(uint32_t account_id, size_t index, bool is_ep3) {
+  return std::format("system/players/backup_player_{}_{}.{}",
+      account_id, index, is_ep3 ? "pso3char" : "psochar");
+}
+
+string Client::character_filename() const {
+  if (this->version() != Version::BB_V4) {
+    throw logic_error("non-BB players do not have saved character filenames");
+  }
+  if (!this->login || !this->login->bb_license) {
+    throw logic_error("client is not logged in");
+  }
+  return this->character_filename(this->login->bb_license->username, this->bb_character_index);
+}
+
+shared_ptr<PSOBBCharacterFile> Client::character_file(bool allow_load, bool allow_overlay) {
+  if (this->overlay_character_data && allow_overlay) {
+    return this->overlay_character_data;
+  }
+  if (!this->character_data && allow_load) {
+    if ((this->version() == Version::BB_V4) && (this->bb_character_index < 0)) {
+      throw runtime_error("character index not specified");
+    }
+    this->load_all_files();
+    if (!this->character_data) {
+      throw std::runtime_error("none of the corresponding character files exist");
+    }
+  }
+  return this->character_data;
+}
+
+shared_ptr<const PSOBBCharacterFile> Client::character_file(bool throw_if_missing, bool allow_overlay) const {
+  if (allow_overlay && this->overlay_character_data) {
+    return this->overlay_character_data;
+  }
+  if (!this->character_data && throw_if_missing) {
+    throw runtime_error("character data is not loaded");
+  }
+  return this->character_data;
+}
+
+void Client::save_character_file(
+    const string& filename,
+    shared_ptr<const PSOBBBaseSystemFile> system,
+    shared_ptr<const PSOBBCharacterFile> character) {
+  PSOCHARFile::save(filename, system, character);
+}
+
+void Client::save_ep3_character_file(
+    const string& filename,
+    const PSOGCEp3CharacterFile::Character& character) {
+  phosg::save_file(filename, &character, sizeof(character));
+}
+
+void Client::save_character_file() {
+  if (!this->system_data.get()) {
+    throw logic_error("no system file loaded");
+  }
+  if (!this->character_data.get()) {
+    throw logic_error("no character file loaded");
+  }
+  if (this->should_update_play_time) {
+    // This is slightly inaccurate, since fractions of a second are truncated
+    // off each time we save. I'm lazy, so insert shrug emoji here.
+    uint64_t t = phosg::now();
+    uint64_t seconds = (t - this->last_play_time_update) / 1000000;
+    this->character_data->play_time_seconds += seconds;
+    this->log.info_f("Added {} seconds to play time", seconds);
+    this->last_play_time_update = t;
+    if (this->bank_data && (this->bb_bank_character_index == this->bb_character_index)) {
+      this->character_data->bank = *this->bank_data;
+      this->log.info_f("Committed bank data back to character file");
+    }
+  }
+
+  auto filename = this->character_filename();
+  this->save_character_file(filename, this->system_data, this->character_data);
+  this->log.info_f("Saved character file {}", filename);
+}
+
+void Client::create_character_file(
+    uint32_t guild_card_number,
+    Language language,
+    const PlayerDispDataBBPreview& preview,
+    shared_ptr<const LevelTable> level_table) {
+  this->character_data = PSOBBCharacterFile::create_from_preview(guild_card_number, language, preview, level_table);
+  this->save_character_file();
+}
+
 void Client::create_battle_overlay(shared_ptr<const BattleRules> rules, shared_ptr<const LevelTable> level_table) {
-  this->overlay_character_data = make_shared<PSOBBCharacterFile>(*this->character(true, false));
+  this->overlay_character_data = make_shared<PSOBBCharacterFile>(*this->character_file(true, false));
 
   if (rules->weapon_and_armor_mode != BattleRules::WeaponAndArmorMode::ALLOW) {
     this->overlay_character_data->inventory.remove_all_items_of_type(0);
@@ -498,7 +679,7 @@ void Client::create_battle_overlay(shared_ptr<const BattleRules> rules, shared_p
 }
 
 void Client::create_challenge_overlay(Version version, size_t template_index, shared_ptr<const LevelTable> level_table) {
-  auto p = this->character(true, false);
+  auto p = this->character_file(true, false);
   const auto& tpl = get_challenge_template_definition(version, p->disp.visual.class_flags, template_index);
 
   this->overlay_character_data = make_shared<PSOBBCharacterFile>(*p);
@@ -542,124 +723,113 @@ void Client::create_challenge_overlay(Version version, size_t template_index, sh
   }
 }
 
-void Client::import_blocked_senders(const parray<le_uint32_t, 30>& blocked_senders) {
-  this->blocked_senders.clear();
-  for (size_t z = 0; z < blocked_senders.size(); z++) {
-    if (blocked_senders[z]) {
-      this->blocked_senders.emplace(blocked_senders[z]);
-    }
-  }
-}
+// Bank file
 
-shared_ptr<PSOBBBaseSystemFile> Client::system_file(bool allow_load) {
-  if (!this->system_data && allow_load) {
-    this->load_all_files();
-  }
-  return this->system_data;
-}
-
-shared_ptr<const PSOBBBaseSystemFile> Client::system_file(bool allow_load) const {
-  if (!this->system_data.get() && allow_load) {
-    throw runtime_error("system data is not loaded");
-  }
-  return this->system_data;
-}
-
-shared_ptr<PSOBBCharacterFile> Client::character(bool allow_load, bool allow_overlay) {
-  if (this->overlay_character_data && allow_overlay) {
-    return this->overlay_character_data;
-  }
-  if (!this->character_data && allow_load) {
-    if ((this->version() == Version::BB_V4) && (this->bb_character_index < 0)) {
-      throw runtime_error("character index not specified");
-    }
-    this->load_all_files();
-  }
-  return this->character_data;
-}
-
-shared_ptr<const PSOBBCharacterFile> Client::character(bool allow_load, bool allow_overlay) const {
-  if (allow_overlay && this->overlay_character_data) {
-    return this->overlay_character_data;
-  }
-  if (!this->character_data && allow_load) {
-    throw runtime_error("character data is not loaded");
-  }
-  return this->character_data;
-}
-
-shared_ptr<PSOBBGuildCardFile> Client::guild_card_file(bool allow_load) {
-  if (!this->guild_card_data && allow_load) {
-    this->load_all_files();
-  }
-  return this->guild_card_data;
-}
-
-shared_ptr<const PSOBBGuildCardFile> Client::guild_card_file(bool allow_load) const {
-  if (!this->guild_card_data && allow_load) {
-    throw runtime_error("account data is not loaded");
-  }
-  return this->guild_card_data;
-}
-
-string Client::system_filename() const {
-  if (this->version() != Version::BB_V4) {
-    throw logic_error("non-BB players do not have system data");
-  }
-  if (!this->login || !this->login->bb_license) {
-    throw logic_error("client is not logged in");
-  }
-  return std::format("system/players/system_{}.psosys", this->login->bb_license->username);
-}
-
-string Client::character_filename(const std::string& bb_username, ssize_t index) {
+string Client::bank_filename(const std::string& bb_username, ssize_t index) {
   if (bb_username.empty()) {
-    throw logic_error("non-BB players do not have character data");
+    throw logic_error("non-BB players do not have saved bank files");
   }
   if (index < 0) {
-    throw logic_error("character index is not set");
+    return std::format("system/players/shared_bank_{}.psobank", bb_username);
+  } else {
+    return std::format("system/players/player_{}_{}.psobank", bb_username, index);
   }
-  return std::format("system/players/player_{}_{}.psochar", bb_username, index);
 }
 
-string Client::backup_character_filename(uint32_t account_id, size_t index, bool is_ep3) {
-  return std::format("system/players/backup_player_{}_{}.{}",
-      account_id, index, is_ep3 ? "pso3char" : "psochar");
-}
-
-string Client::character_filename(ssize_t index) const {
+string Client::bank_filename() const {
   if (this->version() != Version::BB_V4) {
-    throw logic_error("non-BB players do not have character data");
+    throw logic_error("non-BB players do not have saved bank filenames");
   }
   if (!this->login || !this->login->bb_license) {
     throw logic_error("client is not logged in");
   }
-  return this->character_filename(this->login->bb_license->username, (index < 0) ? this->bb_character_index : index);
+  return this->bank_filename(this->login->bb_license->username, this->bb_bank_character_index);
 }
 
-string Client::guild_card_filename() const {
+std::shared_ptr<PlayerBank> Client::bank_file(bool allow_load) {
   if (this->version() != Version::BB_V4) {
-    throw logic_error("non-BB players do not have character data");
+    throw logic_error("non-BB players do not have saved bank files");
   }
-  if (!this->login || !this->login->bb_license) {
-    throw logic_error("client is not logged in");
+  if (this->has_overlay()) {
+    throw std::runtime_error("bank is inaccessible when overlay is present");
   }
-  return std::format("system/players/guild_cards_{}.psocard", this->login->bb_license->username);
+  if (!this->bank_data && allow_load) {
+    try {
+      // If there's a psobank file, load it and ignore the character file bank
+      auto filename = this->bank_filename();
+      auto f = phosg::fopen_unique(filename, "rb");
+      this->bank_data = make_shared<PlayerBank>();
+      this->bank_data->load(f.get());
+      this->log.info_f("Loaded bank data from {}", filename);
+    } catch (const phosg::cannot_open_file&) {
+      // If there isn't a psobank file, use the loaded character data if the
+      // bank character index matches the current character index (that is, we
+      // should use the current character's bank); otherwise, load the
+      // corresponding character and parse the bank from that character file
+      if (this->bb_bank_character_index == this->bb_character_index) {
+        this->bank_data = std::make_shared<PlayerBank>(this->character_file(true, false)->bank);
+        this->log.info_f("Using bank data from loaded character");
+      } else if (this->bb_bank_character_index >= 0) {
+        if (!this->login || !this->login->bb_license) {
+          throw logic_error("client is not logged in");
+        }
+        string filename = this->character_filename(this->login->bb_license->username, this->bb_bank_character_index);
+        auto character = PSOCHARFile::load_shared(filename, false).character_file;
+        this->bank_data = std::make_shared<PlayerBank>(character->bank);
+        this->log.info_f("Using bank data from {}", filename);
+      } else {
+        // The shared bank doesn't exist; make a new one
+        this->bank_data = make_shared<PlayerBank>();
+        this->log.info_f("Created new shared bank");
+      }
+    }
+
+    auto s = this->require_server_state();
+    this->bank_data->max_items = s->bb_max_bank_items;
+    this->bank_data->max_meseta = s->bb_max_bank_meseta;
+  }
+  return this->bank_data;
 }
 
-string Client::shared_bank_filename() const {
-  if (this->version() != Version::BB_V4) {
-    throw logic_error("non-BB players do not have character data");
+std::shared_ptr<const PlayerBank> Client::bank_file(bool throw_if_missing) const {
+  if (!this->bank_data && throw_if_missing) {
+    throw std::runtime_error("bank is not loaded");
   }
-  if (!this->login || !this->login->bb_license) {
-    throw logic_error("client is not logged in");
-  }
-  return std::format("system/players/shared_bank_{}.psobank", this->login->bb_license->username);
+  return this->bank_data;
 }
 
+void Client::save_bank_file(const string& filename, const PlayerBank& bank) {
+  auto f = phosg::fopen_unique(filename, "wb");
+  bank.save(f.get());
+}
+
+void Client::save_bank_file() const {
+  if (!this->bank_data) {
+    throw logic_error("no bank file loaded");
+  }
+  auto filename = this->bank_filename();
+  this->save_bank_file(filename, *this->bank_data);
+  this->log.info_f("Saved bank file {}", filename);
+}
+
+void Client::change_bank(ssize_t index) {
+  if (this->bank_data) {
+    this->save_bank_file();
+    this->bank_data.reset();
+    if (this->bb_bank_character_index < 0) {
+      this->log.info_f("Unloaded shared bank");
+    } else {
+      this->log.info_f("Unloaded bank from character {}", this->bb_bank_character_index);
+    }
+  }
+  this->bb_bank_character_index = index;
+}
+
+// Legacy files
+
 string Client::legacy_account_filename() const {
   if (this->version() != Version::BB_V4) {
-    throw logic_error("non-BB players do not have character data");
+    throw logic_error("non-BB players do not have saved account data");
   }
   if (!this->login || !this->login->bb_license) {
     throw logic_error("client is not logged in");
@@ -669,7 +839,7 @@ string Client::legacy_account_filename() const {
 
 string Client::legacy_player_filename() const {
   if (this->version() != Version::BB_V4) {
-    throw logic_error("non-BB players do not have character data");
+    throw logic_error("non-BB players do not have saved player files");
   }
   if (!this->login || !this->login->bb_license) {
     throw logic_error("client is not logged in");
@@ -683,13 +853,13 @@ string Client::legacy_player_filename() const {
       static_cast<ssize_t>(this->bb_character_index + 1));
 }
 
-void Client::create_character_file(
-    uint32_t guild_card_number,
-    uint8_t language,
-    const PlayerDispDataBBPreview& preview,
-    shared_ptr<const LevelTable> level_table) {
-  this->character_data = PSOBBCharacterFile::create_from_preview(guild_card_number, language, preview, level_table);
-  this->save_character_file();
+void Client::import_blocked_senders(const parray<le_uint32_t, 30>& blocked_senders) {
+  this->blocked_senders.clear();
+  for (size_t z = 0; z < blocked_senders.size(); z++) {
+    if (blocked_senders[z]) {
+      this->blocked_senders.emplace(blocked_senders[z]);
+    }
+  }
 }
 
 void Client::load_all_files() {
@@ -697,40 +867,29 @@ void Client::load_all_files() {
     this->system_data = make_shared<PSOBBBaseSystemFile>();
     this->character_data = make_shared<PSOBBCharacterFile>();
     this->guild_card_data = make_shared<PSOBBGuildCardFile>();
+    this->bank_data = make_shared<PlayerBank>();
     return;
   }
   if (!this->login || !this->login->bb_license) {
     throw logic_error("cannot load BB player data until client is logged in");
   }
 
-  this->system_data.reset();
-  this->character_data.reset();
-  this->guild_card_data.reset();
-
-  auto files_manager = this->require_server_state()->player_files_manager;
-
-  string sys_filename = this->system_filename();
-  this->system_data = files_manager->get_system(sys_filename);
-  if (this->system_data) {
-    player_data_log.info_f("Using loaded system file {}", sys_filename);
-  } else if (std::filesystem::is_regular_file(sys_filename)) {
-    this->system_data = make_shared<PSOBBBaseSystemFile>(phosg::load_object_file<PSOBBBaseSystemFile>(sys_filename, true));
-    files_manager->set_system(sys_filename, this->system_data);
-    player_data_log.info_f("Loaded system data from {}", sys_filename);
-  } else {
-    player_data_log.info_f("System file is missing: {}", sys_filename);
+  if (!this->system_data) {
+    string sys_filename = this->system_filename();
+    if (std::filesystem::is_regular_file(sys_filename)) {
+      this->system_data = make_shared<PSOBBBaseSystemFile>(phosg::load_object_file<PSOBBBaseSystemFile>(sys_filename, true));
+      this->log.info_f("Loaded system data from {}", sys_filename);
+    } else {
+      this->log.info_f("System file is missing: {}", sys_filename);
+    }
   }
 
-  if (this->bb_character_index >= 0) {
+  if (!this->character_data && (this->bb_character_index >= 0)) {
     string char_filename = this->character_filename();
-    this->character_data = files_manager->get_character(char_filename);
-    if (this->character_data) {
-      player_data_log.info_f("Using loaded character file {}", char_filename);
-    } else if (std::filesystem::is_regular_file(char_filename)) {
+    if (std::filesystem::is_regular_file(char_filename)) {
       auto psochar = PSOCHARFile::load_shared(char_filename, !this->system_data);
       this->character_data = psochar.character_file;
-      files_manager->set_character(char_filename, this->character_data);
-      player_data_log.info_f("Loaded character data from {}", char_filename);
+      this->log.info_f("Loaded character data from {}", char_filename);
 
       // If there was no .psosys file, use the system file from the .psochar
       // file instead
@@ -739,31 +898,29 @@ void Client::load_all_files() {
           throw logic_error("account system data not present, and also not loaded from psochar file");
         }
         this->system_data = psochar.system_file;
-        files_manager->set_system(sys_filename, this->system_data);
-        player_data_log.info_f("Loaded system data from {}", char_filename);
+        this->log.info_f("Loaded system data from {}", char_filename);
       }
 
       this->update_character_data_after_load(this->character_data);
       this->system_data->language = this->language();
 
     } else {
-      player_data_log.info_f("Character file is missing: {}", char_filename);
+      this->log.info_f("Character file is missing: {}", char_filename);
     }
   }
 
-  string card_filename = this->guild_card_filename();
-  this->guild_card_data = files_manager->get_guild_card(card_filename);
-  if (this->guild_card_data) {
-    player_data_log.info_f("Using loaded Guild Card file {}", card_filename);
-  } else if (std::filesystem::is_regular_file(card_filename)) {
-    this->guild_card_data = make_shared<PSOBBGuildCardFile>(phosg::load_object_file<PSOBBGuildCardFile>(card_filename));
-    files_manager->set_guild_card(card_filename, this->guild_card_data);
-    player_data_log.info_f("Loaded Guild Card data from {}", card_filename);
-  } else {
-    player_data_log.info_f("Guild Card file is missing: {}", card_filename);
+  if (!this->guild_card_data) {
+    string card_filename = this->guild_card_filename();
+    if (std::filesystem::is_regular_file(card_filename)) {
+      this->guild_card_data = make_shared<PSOBBGuildCardFile>(phosg::load_object_file<PSOBBGuildCardFile>(card_filename));
+      this->log.info_f("Loaded Guild Card data from {}", card_filename);
+    } else {
+      this->log.info_f("Guild Card file is missing: {}", card_filename);
+    }
   }
 
-  // If any of the above files were missing, try to load from .nsa/.nsc files instead
+  // If any of the above files are still missing, try to load from .nsa/.nsc
+  // files instead
   if (!this->system_data || (!this->character_data && (this->bb_character_index >= 0)) || !this->guild_card_data) {
     string nsa_filename = this->legacy_account_filename();
     shared_ptr<LegacySavedAccountDataBB> nsa_data;
@@ -774,80 +931,83 @@ void Client::load_all_files() {
       }
       if (!this->system_data) {
         this->system_data = make_shared<PSOBBBaseSystemFile>(nsa_data->system_file);
-        files_manager->set_system(sys_filename, this->system_data);
-        player_data_log.info_f("Loaded legacy system data from {}", nsa_filename);
+        this->log.info_f("Loaded legacy system data from {}", nsa_filename);
       }
       if (!this->guild_card_data) {
         this->guild_card_data = make_shared<PSOBBGuildCardFile>(nsa_data->guild_card_file);
-        files_manager->set_guild_card(card_filename, this->guild_card_data);
-        player_data_log.info_f("Loaded legacy Guild Card data from {}", nsa_filename);
+        this->log.info_f("Loaded legacy Guild Card data from {}", nsa_filename);
       }
     }
 
-    if (!this->system_data) {
-      this->system_data = make_shared<PSOBBBaseSystemFile>();
-      auto s = this->require_server_state();
-      if (s->bb_default_keyboard_config) {
-        this->system_data->key_config = *s->bb_default_keyboard_config;
-      }
-      if (s->bb_default_joystick_config) {
-        this->system_data->joystick_config = *s->bb_default_joystick_config;
-      }
-      files_manager->set_system(sys_filename, this->system_data);
-      player_data_log.info_f("Created new system data");
-    }
-    if (!this->guild_card_data) {
-      this->guild_card_data = make_shared<PSOBBGuildCardFile>();
-      files_manager->set_guild_card(card_filename, this->guild_card_data);
-      player_data_log.info_f("Created new Guild Card data");
-    }
-
     if (!this->character_data && (this->bb_character_index >= 0)) {
       string nsc_filename = this->legacy_player_filename();
-      auto nsc_data = phosg::load_object_file<LegacySavedPlayerDataBB>(nsc_filename);
-      if (nsc_data.signature == LegacySavedPlayerDataBB::SIGNATURE_V0) {
-        nsc_data.signature = LegacySavedPlayerDataBB::SIGNATURE_V0;
-        nsc_data.unused.clear();
-        nsc_data.battle_records.place_counts.clear(0);
-        nsc_data.battle_records.disconnect_count = 0;
-        nsc_data.battle_records.unknown_a1.clear(0);
-      } else if (nsc_data.signature != LegacySavedPlayerDataBB::SIGNATURE_V1) {
-        throw runtime_error("legacy player data has incorrect signature");
-      }
+      if (std::filesystem::is_regular_file(nsc_filename)) {
+        auto nsc_data = phosg::load_object_file<LegacySavedPlayerDataBB>(nsc_filename);
+        if (nsc_data.signature == LegacySavedPlayerDataBB::SIGNATURE_V0) {
+          nsc_data.signature = LegacySavedPlayerDataBB::SIGNATURE_V0;
+          nsc_data.unused.clear();
+          nsc_data.battle_records.place_counts.clear(0);
+          nsc_data.battle_records.disconnect_count = 0;
+          nsc_data.battle_records.unknown_a1.clear(0);
+        } else if (nsc_data.signature != LegacySavedPlayerDataBB::SIGNATURE_V1) {
+          throw runtime_error("legacy player data has incorrect signature");
+        }
 
-      this->character_data = make_shared<PSOBBCharacterFile>();
-      files_manager->set_character(this->character_filename(), this->character_data);
-      this->character_data->inventory = nsc_data.inventory;
-      this->character_data->disp = nsc_data.disp;
-      this->character_data->play_time_seconds = 0;
-      this->character_data->quest_flags = nsc_data.quest_flags;
-      this->character_data->death_count = nsc_data.death_count;
-      this->character_data->bank = nsc_data.bank;
-      this->character_data->guild_card.guild_card_number = this->login->account->account_id;
-      this->character_data->guild_card.name = nsc_data.disp.name;
-      this->character_data->guild_card.description = nsc_data.guild_card_description;
-      this->character_data->guild_card.present = 1;
-      this->character_data->guild_card.language = nsc_data.inventory.language;
-      this->character_data->guild_card.section_id = nsc_data.disp.visual.section_id;
-      this->character_data->guild_card.char_class = nsc_data.disp.visual.char_class;
-      this->character_data->auto_reply = nsc_data.auto_reply;
-      this->character_data->info_board = nsc_data.info_board;
-      this->character_data->battle_records = nsc_data.battle_records;
-      this->character_data->challenge_records = nsc_data.challenge_records;
-      this->character_data->tech_menu_shortcut_entries = nsc_data.tech_menu_shortcut_entries;
-      this->character_data->quest_counters = nsc_data.quest_counters;
-      if (nsa_data) {
-        this->character_data->option_flags = nsa_data->option_flags;
-        this->character_data->symbol_chats = nsa_data->symbol_chats;
-        this->character_data->shortcuts = nsa_data->shortcuts;
-        player_data_log.info_f("Loaded legacy player data from {} and {}", nsa_filename, nsc_filename);
-      } else {
-        player_data_log.info_f("Loaded legacy player data from {}", nsc_filename);
+        this->character_data = make_shared<PSOBBCharacterFile>();
+        this->character_data->inventory = nsc_data.inventory;
+        this->character_data->disp = nsc_data.disp;
+        this->character_data->play_time_seconds = 0;
+        this->character_data->quest_flags = nsc_data.quest_flags;
+        this->character_data->death_count = nsc_data.death_count;
+        this->character_data->bank = nsc_data.bank;
+        this->character_data->guild_card.guild_card_number = this->login->account->account_id;
+        this->character_data->guild_card.name = nsc_data.disp.name;
+        this->character_data->guild_card.description = nsc_data.guild_card_description;
+        this->character_data->guild_card.present = 1;
+        this->character_data->guild_card.language = nsc_data.inventory.language;
+        this->character_data->guild_card.section_id = nsc_data.disp.visual.section_id;
+        this->character_data->guild_card.char_class = nsc_data.disp.visual.char_class;
+        this->character_data->auto_reply = nsc_data.auto_reply;
+        this->character_data->info_board = nsc_data.info_board;
+        this->character_data->battle_records = nsc_data.battle_records;
+        this->character_data->challenge_records = nsc_data.challenge_records;
+        this->character_data->tech_menu_shortcut_entries = nsc_data.tech_menu_shortcut_entries;
+        this->character_data->quest_counters = nsc_data.quest_counters;
+        if (nsa_data) {
+          this->character_data->option_flags = nsa_data->option_flags;
+          this->character_data->symbol_chats = nsa_data->symbol_chats;
+          this->character_data->shortcuts = nsa_data->shortcuts;
+          this->log.info_f("Loaded legacy player data from {} and {}", nsa_filename, nsc_filename);
+        } else {
+          this->log.info_f("Loaded legacy player data from {}", nsc_filename);
+        }
+        this->update_character_data_after_load(this->character_data);
       }
-      this->update_character_data_after_load(this->character_data);
     }
   }
 
+  // The system and Guild Card files can be auto-created if they can't be
+  // loaded. After this, system_data and guild_card_data are always non-null,
+  // but character_data may still be null
+  if (!this->system_data) {
+    this->system_data = make_shared<PSOBBBaseSystemFile>();
+    auto s = this->require_server_state();
+    if (s->bb_default_keyboard_config) {
+      this->system_data->key_config = *s->bb_default_keyboard_config;
+    }
+    if (s->bb_default_joystick_config) {
+      this->system_data->joystick_config = *s->bb_default_joystick_config;
+    }
+    this->log.info_f("Created new system data");
+  }
+  if (!this->guild_card_data) {
+    this->guild_card_data = make_shared<PSOBBGuildCardFile>();
+    this->log.info_f("Created new Guild Card data");
+  }
+
+  auto s = this->require_server_state();
+  auto stack_limits = s->item_stack_limits(this->version());
+
   this->blocked_senders.clear();
   for (size_t z = 0; z < this->guild_card_data->blocked.size(); z++) {
     if (this->guild_card_data->blocked[z].present) {
@@ -858,17 +1018,23 @@ void Client::load_all_files() {
   if (this->character_data) {
     // Clear legacy play_time field
     this->character_data->disp.name.clear_after_bytes(0x18);
+    this->character_data->inventory.enforce_stack_limits(stack_limits);
     this->login->account->auto_reply_message = this->character_data->auto_reply.decode();
     this->login->account->save();
     this->last_play_time_update = phosg::now();
+    if (this->bb_character_index >= 0) {
+      // Note that bank_file() can't recur infinitely here because
+      // character_file is already set; it will not call load_all_files() again
+      this->bank_file()->enforce_stack_limits(stack_limits);
+    }
   }
 }
 
 void Client::update_character_data_after_load(shared_ptr<PSOBBCharacterFile> charfile) {
   charfile->import_tethealla_material_usage(this->require_server_state()->level_table(this->version()));
 
-  uint8_t lang = this->language();
-  player_data_log.info_f("Overriding language fields in save files with {:02X} ({})", lang, char_for_language_code(lang));
+  Language lang = this->language();
+  this->log.info_f("Overriding language fields in save files with {}", name_for_language(lang));
   charfile->inventory.language = lang;
   charfile->guild_card.language = lang;
 }
@@ -883,70 +1049,9 @@ void Client::save_all() {
   if (this->guild_card_data) {
     this->save_guild_card_file();
   }
-  if (this->external_bank) {
-    string filename = this->shared_bank_filename();
-    phosg::save_object_file<PlayerBank200>(filename, *this->external_bank);
-    player_data_log.info_f("Saved shared bank file {}", filename);
+  if (this->bank_data) {
+    this->save_bank_file();
   }
-  if (this->external_bank_character) {
-    this->save_character_file(
-        this->character_filename(this->external_bank_character_index),
-        this->system_data,
-        this->external_bank_character);
-  }
-}
-
-void Client::save_system_file() const {
-  if (!this->system_data) {
-    throw logic_error("no system file loaded");
-  }
-  string filename = this->system_filename();
-  phosg::save_object_file(filename, *this->system_data);
-  player_data_log.info_f("Saved system file {}", filename);
-}
-
-void Client::save_character_file(
-    const string& filename,
-    shared_ptr<const PSOBBBaseSystemFile> system,
-    shared_ptr<const PSOBBCharacterFile> character) {
-  PSOCHARFile::save(filename, system, character);
-  player_data_log.info_f("Saved character file {}", filename);
-}
-
-void Client::save_ep3_character_file(
-    const string& filename,
-    const PSOGCEp3CharacterFile::Character& character) {
-  phosg::save_file(filename, &character, sizeof(character));
-  player_data_log.info_f("Saved Episode 3 character file {}", filename);
-}
-
-void Client::save_character_file() {
-  if (!this->system_data.get()) {
-    throw logic_error("no system file loaded");
-  }
-  if (!this->character_data.get()) {
-    throw logic_error("no character file loaded");
-  }
-  if (this->should_update_play_time) {
-    // This is slightly inaccurate, since fractions of a second are truncated
-    // off each time we save. I'm lazy, so insert shrug emoji here.
-    uint64_t t = phosg::now();
-    uint64_t seconds = (t - this->last_play_time_update) / 1000000;
-    this->character_data->play_time_seconds += seconds;
-    player_data_log.info_f("Added {} seconds to play time", seconds);
-    this->last_play_time_update = t;
-  }
-
-  this->save_character_file(this->character_filename(), this->system_data, this->character_data);
-}
-
-void Client::save_guild_card_file() const {
-  if (!this->guild_card_data.get()) {
-    throw logic_error("no Guild Card file loaded");
-  }
-  string filename = this->guild_card_filename();
-  phosg::save_object_file(filename, *this->guild_card_data);
-  player_data_log.info_f("Saved Guild Card file {}", filename);
 }
 
 void Client::load_backup_character(uint32_t account_id, size_t index) {
@@ -966,114 +1071,50 @@ shared_ptr<PSOGCEp3CharacterFile::Character> Client::load_ep3_backup_character(u
   return ch;
 }
 
-void Client::save_and_unload_character() {
+void Client::unload_character(bool save) {
   if (this->character_data) {
-    this->save_character_file();
+    if (save) {
+      this->save_character_file();
+    }
     this->character_data.reset();
     this->log.info_f("Unloaded character");
-  }
-}
-
-PlayerBank200& Client::current_bank() {
-  if (this->external_bank) {
-    return *this->external_bank;
-  } else if (this->external_bank_character) {
-    return this->external_bank_character->bank;
-  }
-  return this->character()->bank;
-}
-
-const PlayerBank200& Client::current_bank() const {
-  return const_cast<Client*>(this)->current_bank();
-}
-
-std::shared_ptr<PSOBBCharacterFile> Client::current_bank_character() {
-  return this->external_bank_character ? this->external_bank_character : this->character();
-}
-
-void Client::use_default_bank() {
-  if (this->external_bank) {
-    string filename = this->shared_bank_filename();
-    phosg::save_object_file<PlayerBank200>(filename, *this->external_bank);
-    this->external_bank.reset();
-    player_data_log.info_f("Detached shared bank {}", filename);
-  }
-  if (this->external_bank_character) {
-    string filename = this->character_filename(this->external_bank_character_index);
-    this->save_character_file(filename, this->system_data, this->external_bank_character);
-    this->external_bank_character.reset();
-    player_data_log.info_f("Detached character {} from bank", filename);
-  }
-}
-
-bool Client::use_shared_bank() {
-  this->use_default_bank();
-
-  string filename = this->shared_bank_filename();
-  auto files_manager = this->require_server_state()->player_files_manager;
-  this->external_bank = files_manager->get_bank(filename);
-  if (this->external_bank) {
-    player_data_log.info_f("Using loaded shared bank {}", filename);
-    return true;
-  } else if (std::filesystem::is_regular_file(filename)) {
-    this->external_bank = make_shared<PlayerBank200>(phosg::load_object_file<PlayerBank200>(filename));
-    files_manager->set_bank(filename, this->external_bank);
-    player_data_log.info_f("Loaded shared bank {}", filename);
-    return true;
-  } else {
-    this->external_bank = make_shared<PlayerBank200>();
-    files_manager->set_bank(filename, this->external_bank);
-    player_data_log.info_f("Created shared bank for {}", filename);
-    return false;
-  }
-}
-
-void Client::use_character_bank(ssize_t index) {
-  this->use_default_bank();
-  if (index != this->bb_character_index) {
-    auto files_manager = this->require_server_state()->player_files_manager;
-
-    string filename = this->character_filename(index);
-    this->external_bank_character = files_manager->get_character(filename);
-    if (this->external_bank_character) {
-      this->external_bank_character_index = index;
-      player_data_log.info_f("Using loaded character file {} for external bank", filename);
-    } else if (std::filesystem::is_regular_file(filename)) {
-      this->external_bank_character = PSOCHARFile::load_shared(filename, false).character_file;
-      this->update_character_data_after_load(this->external_bank_character);
-      this->external_bank_character_index = index;
-      files_manager->set_character(filename, this->external_bank_character);
-      player_data_log.info_f("Loaded character data from {} for external bank", filename);
-    } else {
-      throw runtime_error("character does not exist");
+    if (this->bank_data) {
+      if (save) {
+        this->save_bank_file();
+      }
+      this->bank_data.reset();
+      this->log.info_f("Unloaded bank");
     }
   }
 }
 
 void Client::print_inventory() const {
   auto s = this->require_server_state();
-  auto p = this->character();
+  auto p = this->character_file();
   this->log.info_f("[PlayerInventory] Meseta: {}", p->disp.stats.meseta);
   this->log.info_f("[PlayerInventory] {} items", p->inventory.num_items);
   for (size_t x = 0; x < p->inventory.num_items; x++) {
     const auto& item = p->inventory.items[x];
     auto hex = item.data.hex();
-    auto name = s->describe_item(this->version(), item.data, false);
+    auto name = s->describe_item(this->version(), item.data);
     this->log.info_f("[PlayerInventory]   {:2}: [+{:08X}] {} ({})", x, item.flags, hex, name);
   }
 }
 
 void Client::print_bank() const {
-  auto s = this->require_server_state();
-  auto bank = this->current_bank();
-  this->log.info_f("[PlayerBank] Meseta: {}", bank.meseta);
-  this->log.info_f("[PlayerBank] {} items", bank.num_items);
-  for (size_t x = 0; x < bank.num_items; x++) {
-    const auto& item = bank.items[x];
-    const char* present_token = item.present ? "" : " (missing present flag)";
-    auto hex = item.data.hex();
-    auto name = s->describe_item(this->version(), item.data, false);
-    this->log.info_f("[PlayerBank]   {:3}: {} ({}) (x{}){}", x, hex, name, item.amount, present_token);
+  if (this->bank_data) {
+    auto s = this->require_server_state();
+    this->log.info_f("[PlayerBank] Meseta: {}", this->bank_data->meseta);
+    this->log.info_f("[PlayerBank] {} items", this->bank_data->items.size());
+    for (size_t x = 0; x < this->bank_data->items.size(); x++) {
+      const auto& item = this->bank_data->items[x];
+      const char* present_token = item.present ? "" : " (missing present flag)";
+      auto hex = item.data.hex();
+      auto name = s->describe_item(this->version(), item.data);
+      this->log.info_f("[PlayerBank]   {:3}: {} ({}) (x{}){}", x, hex, name, item.amount, present_token);
+    }
+  } else {
+    this->log.info_f("[PlayerBank] Bank data not loaded");
   }
 }
 

src/Client.hh

@@ -50,29 +50,29 @@ public:
     SEND_FUNCTION_CALL_ACTUALLY_RUNS_CODE      = 0x0000000000004000,
     SEND_FUNCTION_CALL_NO_CACHE_PATCH          = 0x0000000000008000,
     CAN_RECEIVE_ENABLE_B2_QUEST                = 0x0000000000020000,
-    AWAITING_ENABLE_B2_QUEST                   = 0x0000000000040000, // Server-side only
+    AWAITING_ENABLE_B2_QUEST                   = 0x0000000000040000,
 
     // State flags
-    LOADING                                    = 0x0000000000100000, // Server-side only
-    LOADING_QUEST                              = 0x0000000000200000, // Server-side only
-    LOADING_RUNNING_JOINABLE_QUEST             = 0x0000000000400000, // Server-side only
-    LOADING_TOURNAMENT                         = 0x0000000000800000, // Server-side only
-    IN_INFORMATION_MENU                        = 0x0000000001000000, // Server-side only
-    AT_WELCOME_MESSAGE                         = 0x0000000002000000, // Server-side only
+    LOADING                                    = 0x0000000000100000,
+    LOADING_QUEST                              = 0x0000000000200000,
+    LOADING_RUNNING_JOINABLE_QUEST             = 0x0000000000400000,
+    LOADING_TOURNAMENT                         = 0x0000000000800000,
+    IN_INFORMATION_MENU                        = 0x0000000001000000,
+    AT_WELCOME_MESSAGE                         = 0x0000000002000000,
     SAVE_ENABLED                               = 0x0000000004000000,
     HAS_EP3_CARD_DEFS                          = 0x0000000008000000,
     HAS_EP3_MEDIA_UPDATES                      = 0x0000000010000000,
     HAS_AUTO_PATCHES                           = 0x0000004000000000,
-    AT_BANK_COUNTER                            = 0x0000000080000000, // Server-side only
-    SHOULD_SEND_ARTIFICIAL_ITEM_STATE          = 0x0001000000000000, // Server-side only
-    SHOULD_SEND_ARTIFICIAL_ENEMY_AND_SET_STATE = 0x0040000000000000, // Server-side only
-    SHOULD_SEND_ARTIFICIAL_OBJECT_STATE        = 0x0080000000000000, // Server-side only
-    SHOULD_SEND_ARTIFICIAL_FLAG_STATE          = 0x0002000000000000, // Server-side only
-    SHOULD_SEND_ARTIFICIAL_PLAYER_STATES       = 0x0200000000000000, // Server-side only
+    AT_BANK_COUNTER                            = 0x0000000080000000,
+    SHOULD_SEND_ARTIFICIAL_ITEM_STATE          = 0x0001000000000000,
+    SHOULD_SEND_ARTIFICIAL_ENEMY_AND_SET_STATE = 0x0040000000000000,
+    SHOULD_SEND_ARTIFICIAL_OBJECT_STATE        = 0x0080000000000000,
+    SHOULD_SEND_ARTIFICIAL_FLAG_STATE          = 0x0002000000000000,
+    SHOULD_SEND_ARTIFICIAL_PLAYER_STATES       = 0x0200000000000000,
     SHOULD_SEND_ENABLE_SAVE                    = 0x0004000000000000,
     SWITCH_ASSIST_ENABLED                      = 0x0000000100000000,
     IS_CLIENT_CUSTOMIZATION                    = 0x0100000000000000,
-    EP3_ALLOW_6xBC                             = 0x1000000000000000, // Server-side only
+    EP3_ALLOW_6xBC                             = 0x1000000000000000,
 
     // Cheat mode and option flags
     INFINITE_HP_ENABLED                        = 0x0000000200000000,
@@ -80,6 +80,7 @@ public:
     DEBUG_ENABLED                              = 0x0000000800000000,
     ITEM_DROP_NOTIFICATIONS_1                  = 0x0010000000000000,
     ITEM_DROP_NOTIFICATIONS_2                  = 0x0020000000000000,
+    HAS_ENEMY_DAMAGE_SYNC_PATCH                = 0x2000000000000000, // Must be same as in EnemyDamageSync*.s
 
     // Proxy option flags
     PROXY_SAVE_FILES                           = 0x0000001000000000,
@@ -113,6 +114,7 @@ public:
   uint8_t bb_client_code = 0;
   uint8_t bb_connection_phase = 0xFF;
   ssize_t bb_character_index = -1; // -1 = not set
+  ssize_t bb_bank_character_index = -1; // -1 = shared bank
   uint32_t bb_security_token = 0;
   parray<uint8_t, 0x28> bb_client_config;
   std::string login_character_name;
@@ -147,7 +149,7 @@ public:
   uint8_t override_lobby_number = 0x80; // 80 = no override
   int64_t override_random_seed = -1;
   std::unique_ptr<Variations> override_variations;
-  VectorXZF pos;
+  VectorXYZF pos;
   uint32_t floor = 0x0F;
   std::weak_ptr<Lobby> lobby;
   uint8_t lobby_client_id = 0;
@@ -226,7 +228,7 @@ public:
   inline Version version() const {
     return this->channel->version;
   }
-  inline uint8_t language() const {
+  inline Language language() const {
     return this->channel->language;
   }
 
@@ -265,21 +267,21 @@ public:
       std::shared_ptr<const IntegralExpression> expr,
       std::shared_ptr<const Lobby> game,
       uint8_t event,
-      uint8_t difficulty,
+      Difficulty difficulty,
       size_t num_players,
       bool v1_present) const;
   bool can_see_quest(
       std::shared_ptr<const Quest> q,
       std::shared_ptr<const Lobby> game,
       uint8_t event,
-      uint8_t difficulty,
+      Difficulty difficulty,
       size_t num_players,
       bool v1_present) const;
   bool can_play_quest(
       std::shared_ptr<const Quest> q,
       std::shared_ptr<const Lobby> game,
       uint8_t event,
-      uint8_t difficulty,
+      Difficulty difficulty,
       size_t num_players,
       bool v1_present) const;
 
@@ -287,6 +289,36 @@ public:
 
   void set_login(std::shared_ptr<Login> login);
 
+  void import_blocked_senders(const parray<le_uint32_t, 30>& blocked_senders);
+
+  static std::string system_filename(const std::string& bb_username);
+  std::string system_filename() const;
+  std::shared_ptr<PSOBBBaseSystemFile> system_file(bool allow_load = true);
+  std::shared_ptr<const PSOBBBaseSystemFile> system_file(bool throw_if_missing = true) const;
+  void save_system_file() const;
+
+  static std::string guild_card_filename(const std::string& bb_username);
+  std::string guild_card_filename() const;
+  std::shared_ptr<PSOBBGuildCardFile> guild_card_file(bool allow_load = true);
+  std::shared_ptr<const PSOBBGuildCardFile> guild_card_file(bool allow_load = true) const;
+  void save_guild_card_file() const;
+
+  static std::string character_filename(const std::string& bb_username, ssize_t index);
+  static std::string backup_character_filename(uint32_t account_id, size_t index, bool is_ep3);
+  std::string character_filename() const;
+  std::shared_ptr<PSOBBCharacterFile> character_file(bool allow_load = true, bool allow_overlay = true);
+  std::shared_ptr<const PSOBBCharacterFile> character_file(bool throw_if_missing = true, bool allow_overlay = true) const;
+  static void save_character_file(
+      const std::string& filename,
+      std::shared_ptr<const PSOBBBaseSystemFile> sys,
+      std::shared_ptr<const PSOBBCharacterFile> character);
+  static void save_ep3_character_file(const std::string& filename, const PSOGCEp3CharacterFile::Character& character);
+  void save_character_file();
+  void create_character_file(
+      uint32_t guild_card_number,
+      Language language,
+      const PlayerDispDataBBPreview& preview,
+      std::shared_ptr<const LevelTable> level_table);
   void create_battle_overlay(std::shared_ptr<const BattleRules> rules, std::shared_ptr<const LevelTable> level_table);
   void create_challenge_overlay(Version version, size_t template_index, std::shared_ptr<const LevelTable> level_table);
   inline void delete_overlay() {
@@ -296,54 +328,22 @@ public:
     return this->overlay_character_data.get() != nullptr;
   }
 
-  void import_blocked_senders(const parray<le_uint32_t, 30>& blocked_senders);
+  static std::string bank_filename(const std::string& bb_username, ssize_t index);
+  std::string bank_filename() const;
+  std::shared_ptr<PlayerBank> bank_file(bool allow_load = true);
+  std::shared_ptr<const PlayerBank> bank_file(bool throw_if_missing = true) const;
+  static void save_bank_file(const std::string& filename, const PlayerBank& bank);
+  void save_bank_file() const;
+  void change_bank(ssize_t bb_character_index); // -1 = use shared bank
 
-  std::shared_ptr<PSOBBBaseSystemFile> system_file(bool allow_load = true);
-  std::shared_ptr<PSOBBCharacterFile> character(bool allow_load = true, bool allow_overlay = true);
-  std::shared_ptr<PSOBBGuildCardFile> guild_card_file(bool allow_load = true);
-  std::shared_ptr<const PSOBBBaseSystemFile> system_file(bool allow_load = true) const;
-  std::shared_ptr<const PSOBBCharacterFile> character(bool allow_load = true, bool allow_overlay = true) const;
-  std::shared_ptr<const PSOBBGuildCardFile> guild_card_file(bool allow_load = true) const;
-
-  void create_character_file(
-      uint32_t guild_card_number,
-      uint8_t language,
-      const PlayerDispDataBBPreview& preview,
-      std::shared_ptr<const LevelTable> level_table);
-
-  std::string system_filename() const;
-  static std::string character_filename(const std::string& bb_username, ssize_t index);
-  static std::string backup_character_filename(uint32_t account_id, size_t index, bool is_ep3);
-  std::string character_filename(ssize_t index = -1) const;
-  std::string guild_card_filename() const;
-  std::string shared_bank_filename() const;
-
-  std::string legacy_player_filename() const;
   std::string legacy_account_filename() const;
+  std::string legacy_player_filename() const;
 
   void save_all();
-  void save_system_file() const;
-  static void save_character_file(
-      const std::string& filename,
-      std::shared_ptr<const PSOBBBaseSystemFile> sys,
-      std::shared_ptr<const PSOBBCharacterFile> character);
-  static void save_ep3_character_file(
-      const std::string& filename,
-      const PSOGCEp3CharacterFile::Character& character);
-  // Note: This function is not const because it updates the player's play time.
-  void save_character_file();
-  void save_guild_card_file() const;
 
   void load_backup_character(uint32_t account_id, size_t index);
   std::shared_ptr<PSOGCEp3CharacterFile::Character> load_ep3_backup_character(uint32_t account_id, size_t index);
-  void save_and_unload_character();
-
-  PlayerBank200& current_bank();
-  const PlayerBank200& current_bank() const;
-  std::shared_ptr<PSOBBCharacterFile> current_bank_character();
-  bool use_shared_bank(); // Returns true if the bank exists; false if it was created
-  void use_character_bank(ssize_t bb_character_index);
-  void use_default_bank();
+  void unload_character(bool save);
 
   void print_inventory() const;
   void print_bank() const;
@@ -358,9 +358,7 @@ private:
   std::shared_ptr<PSOBBCharacterFile> overlay_character_data;
   std::shared_ptr<PSOBBCharacterFile> character_data;
   std::shared_ptr<PSOBBGuildCardFile> guild_card_data;
-  std::shared_ptr<PlayerBank200> external_bank;
-  std::shared_ptr<PSOBBCharacterFile> external_bank_character;
-  ssize_t external_bank_character_index = -1;
+  std::shared_ptr<PlayerBank> bank_data;
   uint64_t last_play_time_update = 0;
 
   void load_all_files();

src/CommonItemSet.cc

@@ -48,11 +48,7 @@ void from_json_into(const phosg::JSON& json, parray<CommonItemSet::Table::Range<
 
 template <typename IntT>
 phosg::JSON to_json(const CommonItemSet::Table::Range<IntT>& v) {
-  if (v.min == v.max) {
-    return phosg::JSON(v.min);
-  } else {
-    return phosg::JSON::list({v.min, v.max});
-  }
+  return (v.min == v.max) ? phosg::JSON(v.min) : phosg::JSON::list({v.min, v.max});
 }
 
 template <typename IntT>
@@ -126,15 +122,20 @@ CommonItemSet::Table::Table(const phosg::JSON& json, Episode episode)
   const auto& enemy_type_drop_probs_json = json.at("EnemyTypeDropProbs").as_dict();
   const auto& enemy_item_classes_json = json.at("EnemyItemClasses").as_dict();
   for (size_t z = 0; z < 0x64; z++) {
-    static const array<Episode, 3> episodes = {Episode::EP1, Episode::EP2, Episode::EP4};
-    for (Episode episode : episodes) {
-      for (auto type : enemy_types_for_rare_table_index(episode, z)) {
-        string name = std::format("{}:{}", abbreviation_for_episode(episode), phosg::name_for_enum(type));
-        from_json_into(*enemy_meseta_ranges_json.at(name), this->enemy_meseta_ranges[z]);
-        this->enemy_type_drop_probs[z] = enemy_type_drop_probs_json.at(name)->as_int();
-        this->enemy_item_classes[z] = enemy_item_classes_json.at(name)->as_int();
+    auto types = enemy_types_for_rare_table_index(episode, z);
+    vector<string> names;
+    if (types.empty()) {
+      names.emplace_back(std::format("{}:!{:02X}", abbreviation_for_episode(episode), z));
+    } else {
+      for (auto type : types) {
+        names.emplace_back(std::format("{}:{}", abbreviation_for_episode(episode), phosg::name_for_enum(type)));
       }
     }
+    for (const auto& name : names) {
+      from_json_into(*enemy_meseta_ranges_json.at(name), this->enemy_meseta_ranges[z]);
+      this->enemy_type_drop_probs[z] = enemy_type_drop_probs_json.at(name)->as_int();
+      this->enemy_item_classes[z] = enemy_item_classes_json.at(name)->as_int();
+    }
   }
 }
 
@@ -336,15 +337,138 @@ void CommonItemSet::Table::print(FILE* stream) const {
   }
 }
 
+void CommonItemSet::Table::print_diff(FILE* stream, const Table& other) const {
+  if (this->episode != other.episode) {
+    phosg::fwrite_fmt(stream, ">   Episode: {} -> {}\n", name_for_episode(this->episode), name_for_episode(other.episode));
+  }
+  if (this->base_weapon_type_prob_table != other.base_weapon_type_prob_table) {
+    phosg::fwrite_fmt(stream, ">   base_weapon_type_prob_table: {} -> {}\n",
+        phosg::format_data_string(&this->base_weapon_type_prob_table, sizeof(this->base_weapon_type_prob_table)),
+        phosg::format_data_string(&other.base_weapon_type_prob_table, sizeof(other.base_weapon_type_prob_table)));
+  }
+  if (this->subtype_base_table != other.subtype_base_table) {
+    phosg::fwrite_fmt(stream, ">   subtype_base_table: {} -> {}\n",
+        phosg::format_data_string(&this->subtype_base_table, sizeof(this->subtype_base_table)),
+        phosg::format_data_string(&other.subtype_base_table, sizeof(other.subtype_base_table)));
+  }
+  if (this->subtype_area_length_table != other.subtype_area_length_table) {
+    phosg::fwrite_fmt(stream, ">   subtype_area_length_table: {} -> {}\n",
+        phosg::format_data_string(&this->subtype_area_length_table, sizeof(this->subtype_area_length_table)),
+        phosg::format_data_string(&other.subtype_area_length_table, sizeof(other.subtype_area_length_table)));
+  }
+  if (this->grind_prob_table != other.grind_prob_table) {
+    phosg::fwrite_fmt(stream, ">   grind_prob_table: {} -> {}\n",
+        phosg::format_data_string(&this->grind_prob_table, sizeof(this->grind_prob_table)),
+        phosg::format_data_string(&other.grind_prob_table, sizeof(other.grind_prob_table)));
+  }
+  if (this->armor_shield_type_index_prob_table != other.armor_shield_type_index_prob_table) {
+    phosg::fwrite_fmt(stream, ">   armor_shield_type_index_prob_table: {} -> {}\n",
+        phosg::format_data_string(&this->armor_shield_type_index_prob_table, sizeof(this->armor_shield_type_index_prob_table)),
+        phosg::format_data_string(&other.armor_shield_type_index_prob_table, sizeof(other.armor_shield_type_index_prob_table)));
+  }
+  if (this->armor_slot_count_prob_table != other.armor_slot_count_prob_table) {
+    phosg::fwrite_fmt(stream, ">   armor_slot_count_prob_table: {} -> {}\n",
+        phosg::format_data_string(&this->armor_slot_count_prob_table, sizeof(this->armor_slot_count_prob_table)),
+        phosg::format_data_string(&other.armor_slot_count_prob_table, sizeof(other.armor_slot_count_prob_table)));
+  }
+  if (this->enemy_meseta_ranges != other.enemy_meseta_ranges) {
+    phosg::fwrite_fmt(stream, ">   enemy_meseta_ranges: {} -> {}\n",
+        phosg::format_data_string(&this->enemy_meseta_ranges, sizeof(this->enemy_meseta_ranges)),
+        phosg::format_data_string(&other.enemy_meseta_ranges, sizeof(other.enemy_meseta_ranges)));
+  }
+  if (this->enemy_type_drop_probs != other.enemy_type_drop_probs) {
+    phosg::fwrite_fmt(stream, ">   enemy_type_drop_probs: {} -> {}\n",
+        phosg::format_data_string(&this->enemy_type_drop_probs, sizeof(this->enemy_type_drop_probs)),
+        phosg::format_data_string(&other.enemy_type_drop_probs, sizeof(other.enemy_type_drop_probs)));
+  }
+  if (this->enemy_item_classes != other.enemy_item_classes) {
+    phosg::fwrite_fmt(stream, ">   enemy_item_classes: {} -> {}\n",
+        phosg::format_data_string(&this->enemy_item_classes, sizeof(this->enemy_item_classes)),
+        phosg::format_data_string(&other.enemy_item_classes, sizeof(other.enemy_item_classes)));
+  }
+  if (this->box_meseta_ranges != other.box_meseta_ranges) {
+    phosg::fwrite_fmt(stream, ">   box_meseta_ranges: {} -> {}\n",
+        phosg::format_data_string(&this->box_meseta_ranges, sizeof(this->box_meseta_ranges)),
+        phosg::format_data_string(&other.box_meseta_ranges, sizeof(other.box_meseta_ranges)));
+  }
+  if (this->has_rare_bonus_value_prob_table != other.has_rare_bonus_value_prob_table) {
+    phosg::fwrite_fmt(stream, ">   Has rare bonus value prob table: {} -> {}\n",
+        this->has_rare_bonus_value_prob_table ? "true" : "false",
+        other.has_rare_bonus_value_prob_table ? "true" : "false");
+  }
+  if (this->bonus_value_prob_table != other.bonus_value_prob_table) {
+    phosg::fwrite_fmt(stream, ">   bonus_value_prob_table: {} -> {}\n",
+        phosg::format_data_string(&this->bonus_value_prob_table, sizeof(this->bonus_value_prob_table)),
+        phosg::format_data_string(&other.bonus_value_prob_table, sizeof(other.bonus_value_prob_table)));
+  }
+  if (this->nonrare_bonus_prob_spec != other.nonrare_bonus_prob_spec) {
+    phosg::fwrite_fmt(stream, ">   nonrare_bonus_prob_spec: {} -> {}\n",
+        phosg::format_data_string(&this->nonrare_bonus_prob_spec, sizeof(this->nonrare_bonus_prob_spec)),
+        phosg::format_data_string(&other.nonrare_bonus_prob_spec, sizeof(other.nonrare_bonus_prob_spec)));
+  }
+  if (this->bonus_type_prob_table != other.bonus_type_prob_table) {
+    phosg::fwrite_fmt(stream, ">   bonus_type_prob_table: {} -> {}\n",
+        phosg::format_data_string(&this->bonus_type_prob_table, sizeof(this->bonus_type_prob_table)),
+        phosg::format_data_string(&other.bonus_type_prob_table, sizeof(other.bonus_type_prob_table)));
+  }
+  if (this->special_mult != other.special_mult) {
+    phosg::fwrite_fmt(stream, ">   special_mult: {} -> {}\n",
+        phosg::format_data_string(&this->special_mult, sizeof(this->special_mult)),
+        phosg::format_data_string(&other.special_mult, sizeof(other.special_mult)));
+  }
+  if (this->special_percent != other.special_percent) {
+    phosg::fwrite_fmt(stream, ">   special_percent: {} -> {}\n",
+        phosg::format_data_string(&this->special_percent, sizeof(this->special_percent)),
+        phosg::format_data_string(&other.special_percent, sizeof(other.special_percent)));
+  }
+  if (this->tool_class_prob_table != other.tool_class_prob_table) {
+    phosg::fwrite_fmt(stream, ">   tool_class_prob_table: {} -> {}\n",
+        phosg::format_data_string(&this->tool_class_prob_table, sizeof(this->tool_class_prob_table)),
+        phosg::format_data_string(&other.tool_class_prob_table, sizeof(other.tool_class_prob_table)));
+  }
+  if (this->technique_index_prob_table != other.technique_index_prob_table) {
+    phosg::fwrite_fmt(stream, ">   technique_index_prob_table: {} -> {}\n",
+        phosg::format_data_string(&this->technique_index_prob_table, sizeof(this->technique_index_prob_table)),
+        phosg::format_data_string(&other.technique_index_prob_table, sizeof(other.technique_index_prob_table)));
+  }
+  if (this->technique_level_ranges != other.technique_level_ranges) {
+    phosg::fwrite_fmt(stream, ">   technique_level_ranges: {} -> {}\n",
+        phosg::format_data_string(&this->technique_level_ranges, sizeof(this->technique_level_ranges)),
+        phosg::format_data_string(&other.technique_level_ranges, sizeof(other.technique_level_ranges)));
+  }
+  if (this->armor_or_shield_type_bias != other.armor_or_shield_type_bias) {
+    phosg::fwrite_fmt(stream, ">   Armor/shield type bias: {} -> {}\n",
+        this->armor_or_shield_type_bias ? "true" : "false",
+        other.armor_or_shield_type_bias ? "true" : "false");
+  }
+  if (this->unit_max_stars_table != other.unit_max_stars_table) {
+    phosg::fwrite_fmt(stream, ">   unit_max_stars_table: {} -> {}\n",
+        phosg::format_data_string(&this->unit_max_stars_table, sizeof(this->unit_max_stars_table)),
+        phosg::format_data_string(&other.unit_max_stars_table, sizeof(other.unit_max_stars_table)));
+  }
+  if (this->box_item_class_prob_table != other.box_item_class_prob_table) {
+    phosg::fwrite_fmt(stream, ">   box_item_class_prob_table: {} -> {}\n",
+        phosg::format_data_string(&this->box_item_class_prob_table, sizeof(this->box_item_class_prob_table)),
+        phosg::format_data_string(&other.box_item_class_prob_table, sizeof(other.box_item_class_prob_table)));
+  }
+}
+
 phosg::JSON CommonItemSet::Table::json() const {
   phosg::JSON enemy_meseta_ranges_json = phosg::JSON::dict();
   phosg::JSON enemy_type_drop_probs_json = phosg::JSON::dict();
   phosg::JSON enemy_item_classes_json = phosg::JSON::dict();
   for (size_t z = 0; z < 0x64; z++) {
-    static const array<Episode, 3> episodes = {Episode::EP1, Episode::EP2, Episode::EP4};
-    for (Episode episode : episodes) {
-      for (auto type : enemy_types_for_rare_table_index(episode, z)) {
-        string name = std::format("{}:{}", abbreviation_for_episode(episode), phosg::name_for_enum(type));
+    for (Episode episode : ALL_EPISODES_V4) {
+      auto types = enemy_types_for_rare_table_index(episode, z);
+      vector<string> names;
+      if (types.empty()) {
+        names.emplace_back(std::format("{}:!{:02X}", abbreviation_for_episode(episode), z));
+      } else {
+        for (auto type : types) {
+          names.emplace_back(std::format("{}:{}", abbreviation_for_episode(episode), phosg::name_for_enum(type)));
+        }
+      }
+      for (const auto& name : names) {
         enemy_meseta_ranges_json.emplace(name, to_json(this->enemy_meseta_ranges[z]));
         enemy_type_drop_probs_json.emplace(name, this->enemy_type_drop_probs[z]);
         enemy_item_classes_json.emplace(name, this->enemy_item_classes[z]);
@@ -379,13 +503,11 @@ phosg::JSON CommonItemSet::Table::json() const {
 
 phosg::JSON CommonItemSet::json() const {
   auto modes_dict = phosg::JSON::dict();
-  static const array<GameMode, 4> modes = {GameMode::NORMAL, GameMode::BATTLE, GameMode::CHALLENGE, GameMode::SOLO};
-  for (const auto& mode : modes) {
+  for (const auto& mode : ALL_GAME_MODES_V4) {
     auto episodes_dict = phosg::JSON::dict();
-    static const array<Episode, 3> episodes = {Episode::EP1, Episode::EP2, Episode::EP4};
-    for (const auto& episode : episodes) {
+    for (const auto& episode : ALL_EPISODES_V4) {
       auto difficulty_dict = phosg::JSON::dict();
-      for (uint8_t difficulty = 0; difficulty < 4; difficulty++) {
+      for (const auto& difficulty : ALL_DIFFICULTIES_V234) {
         auto section_id_dict = phosg::JSON::dict();
         for (uint8_t section_id = 0; section_id < 10; section_id++) {
           try {
@@ -405,11 +527,9 @@ phosg::JSON CommonItemSet::json() const {
 }
 
 void CommonItemSet::print(FILE* stream) const {
-  static const array<GameMode, 4> modes = {GameMode::NORMAL, GameMode::BATTLE, GameMode::CHALLENGE, GameMode::SOLO};
-  for (const auto& mode : modes) {
-    static const array<Episode, 3> episodes = {Episode::EP1, Episode::EP2, Episode::EP4};
-    for (const auto& episode : episodes) {
-      for (uint8_t difficulty = 0; difficulty < 4; difficulty++) {
+  for (const auto& mode : ALL_GAME_MODES_V4) {
+    for (const auto& episode : ALL_EPISODES_V4) {
+      for (Difficulty difficulty : ALL_DIFFICULTIES_V234) {
         for (uint8_t section_id = 0; section_id < 10; section_id++) {
           try {
             auto table = this->get_table(episode, mode, difficulty, section_id);
@@ -424,6 +544,41 @@ void CommonItemSet::print(FILE* stream) const {
   }
 }
 
+void CommonItemSet::print_diff(FILE* stream, const CommonItemSet& other) const {
+  for (const auto& mode : ALL_GAME_MODES_V4) {
+    for (const auto& episode : ALL_EPISODES_V4) {
+      for (const auto& difficulty : ALL_DIFFICULTIES_V234) {
+        for (uint8_t section_id = 0; section_id < 10; section_id++) {
+          shared_ptr<const Table> this_table;
+          shared_ptr<const Table> other_table;
+          try {
+            this_table = this->get_table(episode, mode, difficulty, section_id);
+          } catch (const runtime_error&) {
+          }
+          try {
+            other_table = other.get_table(episode, mode, difficulty, section_id);
+          } catch (const runtime_error&) {
+          }
+
+          if (!this_table && !other_table) {
+            continue;
+          } else if (!this_table) {
+            phosg::fwrite_fmt(stream, "> Table present in other but not this: {} {} {} {}\n",
+                name_for_mode(mode), name_for_episode(episode), name_for_difficulty(difficulty), name_for_section_id(section_id));
+          } else if (!other_table) {
+            phosg::fwrite_fmt(stream, "> Table present in this but not other: {} {} {} {}\n",
+                name_for_mode(mode), name_for_episode(episode), name_for_difficulty(difficulty), name_for_section_id(section_id));
+          } else if (*this_table != *other_table) {
+            phosg::fwrite_fmt(stream, "> Tables do not match: {} {} {} {}\n",
+                name_for_mode(mode), name_for_episode(episode), name_for_difficulty(difficulty), name_for_section_id(section_id));
+            this_table->print_diff(stream, *other_table);
+          }
+        }
+      }
+    }
+  }
+}
+
 CommonItemSet::Table::Table(const phosg::StringReader& r, bool is_big_endian, bool is_v3, Episode episode)
     : episode(episode) {
   if (is_big_endian) {
@@ -490,7 +645,7 @@ void CommonItemSet::Table::parse_itempt_t(const phosg::StringReader& r, bool is_
   this->box_item_class_prob_table = r.pget<parray<parray<uint8_t, 10>, 7>>(offsets.box_item_class_prob_table_offset);
 }
 
-uint16_t CommonItemSet::key_for_table(Episode episode, GameMode mode, uint8_t difficulty, uint8_t secid) {
+uint16_t CommonItemSet::key_for_table(Episode episode, GameMode mode, Difficulty difficulty, uint8_t secid) {
   // Bits: -----EEEMMDDSSSS
   return (((static_cast<uint16_t>(episode) << 8) & 0x0700) |
       ((static_cast<uint16_t>(mode) << 6) & 0x00C0) |
@@ -499,39 +654,65 @@ uint16_t CommonItemSet::key_for_table(Episode episode, GameMode mode, uint8_t di
 }
 
 shared_ptr<const CommonItemSet::Table> CommonItemSet::get_table(
-    Episode episode, GameMode mode, uint8_t difficulty, uint8_t secid) const {
+    Episode episode, GameMode mode, Difficulty difficulty, uint8_t secid) const {
   try {
     return this->tables.at(this->key_for_table(episode, mode, difficulty, secid));
   } catch (const out_of_range&) {
     throw runtime_error(std::format("common item table not available for episode={}, mode={}, difficulty={}, secid={}",
-        name_for_episode(episode), name_for_mode(mode), difficulty, secid));
+        name_for_episode(episode), name_for_mode(mode), name_for_difficulty(difficulty), secid));
   }
 }
 
 AFSV2CommonItemSet::AFSV2CommonItemSet(
-    std::shared_ptr<const std::string> pt_afs_data,
-    std::shared_ptr<const std::string> ct_afs_data) {
-  // ItemPT.afs has 40 entries; the first 10 are for Normal, then Hard, etc.
-  AFSArchive pt_afs(pt_afs_data);
-  for (size_t difficulty = 0; difficulty < 4; difficulty++) {
-    for (size_t section_id = 0; section_id < 10; section_id++) {
-      auto entry = pt_afs.get(difficulty * 10 + section_id);
-      phosg::StringReader r(entry.first, entry.second);
-      auto table = make_shared<Table>(r, false, false, Episode::EP1);
-      this->tables.emplace(this->key_for_table(Episode::EP1, GameMode::NORMAL, difficulty, section_id), table);
-      this->tables.emplace(this->key_for_table(Episode::EP1, GameMode::BATTLE, difficulty, section_id), table);
-      this->tables.emplace(this->key_for_table(Episode::EP1, GameMode::SOLO, difficulty, section_id), table);
+    std::shared_ptr<const std::string> pt_afs_data, std::shared_ptr<const std::string> ct_afs_data) {
+  // Each AFS file has 40 entries (30 on v1); the first 10 are for Normal, then
+  // Hard, etc.
+  {
+    AFSArchive pt_afs(pt_afs_data);
+    bool include_ultimate;
+    if (pt_afs.num_entries() >= 40) {
+      include_ultimate = true;
+    } else if (pt_afs.num_entries() >= 30) {
+      include_ultimate = false;
+    } else {
+      throw std::runtime_error(std::format("PT AFS file has unexpected entry count ({})", pt_afs.num_entries()));
+    }
+    for (Difficulty difficulty : ALL_DIFFICULTIES_V234) {
+      if ((difficulty == Difficulty::ULTIMATE) && !include_ultimate) {
+        continue;
+      }
+      for (size_t section_id = 0; section_id < 10; section_id++) {
+        auto entry = pt_afs.get(static_cast<size_t>(difficulty) * 10 + section_id);
+        phosg::StringReader r(entry.first, entry.second);
+        auto table = make_shared<Table>(r, false, false, Episode::EP1);
+        this->tables.emplace(this->key_for_table(Episode::EP1, GameMode::NORMAL, difficulty, section_id), table);
+        this->tables.emplace(this->key_for_table(Episode::EP1, GameMode::BATTLE, difficulty, section_id), table);
+        this->tables.emplace(this->key_for_table(Episode::EP1, GameMode::SOLO, difficulty, section_id), table);
+      }
     }
   }
 
-  // ItemCT.afs also has 40 entries, but only the 0th, 10th, 20th, and 30th are
-  // used (section_id is ignored)
-  AFSArchive ct_afs(ct_afs_data);
-  for (size_t difficulty = 0; difficulty < 4; difficulty++) {
-    auto r = ct_afs.get_reader(difficulty * 10);
-    auto table = make_shared<Table>(r, false, false, Episode::EP1);
-    for (size_t section_id = 0; section_id < 10; section_id++) {
-      this->tables.emplace(this->key_for_table(Episode::EP1, GameMode::CHALLENGE, difficulty, section_id), table);
+  // ItemCT AFS files also have 40 entries, but only the 0th, 10th, 20th, and
+  // 30th are used (section_id is ignored)
+  if (ct_afs_data) {
+    AFSArchive ct_afs(ct_afs_data);
+    bool include_ultimate;
+    if (ct_afs.num_entries() >= 40) {
+      include_ultimate = true;
+    } else if (ct_afs.num_entries() >= 30) {
+      include_ultimate = false;
+    } else {
+      throw std::runtime_error(std::format("CT AFS file has unexpected entry count ({})", ct_afs.num_entries()));
+    }
+    for (Difficulty difficulty : ALL_DIFFICULTIES_V234) {
+      if ((difficulty == Difficulty::ULTIMATE) && !include_ultimate) {
+        continue;
+      }
+      auto r = ct_afs.get_reader(static_cast<size_t>(difficulty) * 10);
+      auto table = make_shared<Table>(r, false, false, Episode::EP1);
+      for (size_t section_id = 0; section_id < 10; section_id++) {
+        this->tables.emplace(this->key_for_table(Episode::EP1, GameMode::CHALLENGE, difficulty, section_id), table);
+      }
     }
   }
 }
@@ -539,7 +720,7 @@ AFSV2CommonItemSet::AFSV2CommonItemSet(
 GSLV3V4CommonItemSet::GSLV3V4CommonItemSet(std::shared_ptr<const std::string> gsl_data, bool is_big_endian) {
   GSLArchive gsl(gsl_data, is_big_endian);
 
-  auto filename_for_table = +[](Episode episode, uint8_t difficulty, uint8_t section_id, bool is_challenge) -> string {
+  auto filename_for_table = +[](Episode episode, Difficulty difficulty, uint8_t section_id, bool is_challenge) -> string {
     const char* episode_token = "";
     switch (episode) {
       case Episode::EP1:
@@ -562,9 +743,8 @@ GSLV3V4CommonItemSet::GSLV3V4CommonItemSet(std::shared_ptr<const std::string> gs
         section_id);
   };
 
-  vector<Episode> episodes = {Episode::EP1, Episode::EP2, Episode::EP4};
-  for (Episode episode : episodes) {
-    for (size_t difficulty = 0; difficulty < 4; difficulty++) {
+  for (Episode episode : ALL_EPISODES_V4) {
+    for (Difficulty difficulty : ALL_DIFFICULTIES_V234) {
       for (size_t section_id = 0; section_id < 10; section_id++) {
         phosg::StringReader r;
         try {
@@ -589,11 +769,15 @@ GSLV3V4CommonItemSet::GSLV3V4CommonItemSet(std::shared_ptr<const std::string> gs
     }
 
     if (episode != Episode::EP4) {
-      for (size_t difficulty = 0; difficulty < 4; difficulty++) {
-        auto r = gsl.get_reader(filename_for_table(episode, difficulty, 0, true));
-        auto table = make_shared<Table>(r, is_big_endian, true, episode);
-        for (size_t section_id = 0; section_id < 10; section_id++) {
-          this->tables.emplace(this->key_for_table(episode, GameMode::CHALLENGE, difficulty, section_id), table);
+      for (Difficulty difficulty : ALL_DIFFICULTIES_V234) {
+        try {
+          auto r = gsl.get_reader(filename_for_table(episode, difficulty, 0, true));
+          auto table = make_shared<Table>(r, is_big_endian, true, episode);
+          for (size_t section_id = 0; section_id < 10; section_id++) {
+            this->tables.emplace(this->key_for_table(episode, GameMode::CHALLENGE, difficulty, section_id), table);
+          }
+        } catch (const out_of_range&) {
+          // GC NTE doesn't have Ep2 challenge; just skip adding the table
         }
       }
     }
@@ -612,9 +796,9 @@ JSONCommonItemSet::JSONCommonItemSet(const phosg::JSON& json) {
       Episode episode = episode_keys.at(episode_it.first);
 
       for (const auto& difficulty_it : episode_it.second->as_dict()) {
-        static const unordered_map<string, uint8_t> difficulty_keys(
-            {{"Normal", 0}, {"Hard", 1}, {"VeryHard", 2}, {"Ultimate", 3}});
-        uint8_t difficulty = difficulty_keys.at(difficulty_it.first);
+        static const unordered_map<string, Difficulty> difficulty_keys(
+            {{"Normal", Difficulty::NORMAL}, {"Hard", Difficulty::HARD}, {"VeryHard", Difficulty::VERY_HARD}, {"Ultimate", Difficulty::ULTIMATE}});
+        Difficulty difficulty = difficulty_keys.at(difficulty_it.first);
 
         for (const auto& section_id_it : difficulty_it.second->as_dict()) {
           uint8_t section_id = section_id_for_name(section_id_it.first);
@@ -628,8 +812,7 @@ JSONCommonItemSet::JSONCommonItemSet(const phosg::JSON& json) {
 }
 
 RELFileSet::RELFileSet(std::shared_ptr<const std::string> data)
-    : data(data),
-      r(*this->data) {}
+    : data(data), r(*this->data) {}
 
 ArmorRandomSet::ArmorRandomSet(std::shared_ptr<const std::string> data)
     : RELFileSet(data) {
@@ -657,18 +840,13 @@ ArmorRandomSet::get_unit_table(size_t index) const {
 ToolRandomSet::ToolRandomSet(std::shared_ptr<const std::string> data)
     : RELFileSet(data) {
   uint32_t specs_offset = r.pget_u32b(data->size() - 0x10);
-  this->common_recovery_table_spec = &r.pget<TableSpec>(r.pget_u32b(
-      specs_offset));
-  this->rare_recovery_table_spec = &r.pget<TableSpec>(
-      r.pget_u32b(specs_offset + sizeof(uint32_t)),
-      2 * sizeof(TableSpec));
+  this->common_recovery_table_spec = &r.pget<TableSpec>(r.pget_u32b(specs_offset));
+  this->rare_recovery_table_spec = &r.pget<TableSpec>(r.pget_u32b(specs_offset + sizeof(uint32_t)), 2 * sizeof(TableSpec));
   this->tech_disk_table_spec = this->rare_recovery_table_spec + 1;
-  this->tech_disk_level_table_spec = &r.pget<TableSpec>(r.pget_u32b(
-      specs_offset + 2 * sizeof(uint32_t)));
+  this->tech_disk_level_table_spec = &r.pget<TableSpec>(r.pget_u32b(specs_offset + 2 * sizeof(uint32_t)));
 }
 
-pair<const uint8_t*, size_t> ToolRandomSet::get_common_recovery_table(
-    size_t index) const {
+pair<const uint8_t*, size_t> ToolRandomSet::get_common_recovery_table(size_t index) const {
   return this->get_table<uint8_t>(*this->common_recovery_table_spec, index);
 }
 
@@ -695,25 +873,21 @@ WeaponRandomSet::WeaponRandomSet(std::shared_ptr<const std::string> data)
 
 std::pair<const WeaponRandomSet::WeightTableEntry8*, size_t>
 WeaponRandomSet::get_weapon_type_table(size_t index) const {
-  const auto& spec = this->r.pget<TableSpec>(
-      this->offsets->weapon_type_table + index * sizeof(TableSpec));
-  const auto* data = &this->r.pget<WeightTableEntry8>(
-      spec.offset, spec.entries_per_table * sizeof(WeightTableEntry8));
+  const auto& spec = this->r.pget<TableSpec>(this->offsets->weapon_type_table + index * sizeof(TableSpec));
+  const auto* data = &this->r.pget<WeightTableEntry8>(spec.offset, spec.entries_per_table * sizeof(WeightTableEntry8));
   return make_pair(data, spec.entries_per_table);
 }
 
 const parray<WeaponRandomSet::WeightTableEntry32, 6>*
 WeaponRandomSet::get_bonus_type_table(size_t which, size_t index) const {
   uint32_t base_offset = which ? this->offsets->bonus_type_table2 : this->offsets->bonus_type_table1;
-  return &this->r.pget<parray<WeightTableEntry32, 6>>(
-      base_offset + sizeof(parray<WeightTableEntry32, 6>) * index);
+  return &this->r.pget<parray<WeightTableEntry32, 6>>(base_offset + sizeof(parray<WeightTableEntry32, 6>) * index);
 }
 
 const WeaponRandomSet::RangeTableEntry*
 WeaponRandomSet::get_bonus_range(size_t which, size_t index) const {
   uint32_t base_offset = which ? this->offsets->bonus_range_table2 : this->offsets->bonus_range_table1;
-  return &this->r.pget<RangeTableEntry>(
-      base_offset + sizeof(RangeTableEntry) * index);
+  return &this->r.pget<RangeTableEntry>(base_offset + sizeof(RangeTableEntry) * index);
 }
 
 const parray<WeaponRandomSet::WeightTableEntry32, 3>*
@@ -724,19 +898,16 @@ WeaponRandomSet::get_special_mode_table(size_t index) const {
 
 const WeaponRandomSet::RangeTableEntry*
 WeaponRandomSet::get_standard_grind_range(size_t index) const {
-  return &this->r.pget<RangeTableEntry>(
-      this->offsets->standard_grind_range_table + sizeof(RangeTableEntry) * index);
+  return &this->r.pget<RangeTableEntry>(this->offsets->standard_grind_range_table + sizeof(RangeTableEntry) * index);
 }
 
 const WeaponRandomSet::RangeTableEntry*
 WeaponRandomSet::get_favored_grind_range(size_t index) const {
-  return &this->r.pget<RangeTableEntry>(
-      this->offsets->favored_grind_range_table + sizeof(RangeTableEntry) * index);
+  return &this->r.pget<RangeTableEntry>(this->offsets->favored_grind_range_table + sizeof(RangeTableEntry) * index);
 }
 
 TekkerAdjustmentSet::TekkerAdjustmentSet(std::shared_ptr<const std::string> data)
-    : data(data),
-      r(*data) {
+    : data(data), r(*data) {
   this->offsets = &this->r.pget<Offsets>(this->r.pget_u32b(this->r.size() - 0x10));
 }
 

src/CommonItemSet.hh

@@ -18,10 +18,16 @@ public:
     Table(const phosg::JSON& json, Episode episode);
     Table(const phosg::StringReader& r, bool big_endian, bool is_v3, Episode episode);
 
+    bool operator==(const Table& other) const = default;
+    bool operator!=(const Table& other) const = default;
+
     template <typename IntT>
     struct Range {
       IntT min;
       IntT max;
+
+      bool operator==(const Range& other) const = default;
+      bool operator!=(const Range& other) const = default;
     } __attribute__((packed));
 
     Episode episode;
@@ -50,6 +56,7 @@ public:
 
     phosg::JSON json() const;
     void print(FILE* stream) const;
+    void print_diff(FILE* stream, const Table& other) const;
 
   private:
     template <bool BE>
@@ -261,14 +268,18 @@ public:
     check_struct_size(OffsetsBE, 0x54);
   };
 
-  std::shared_ptr<const Table> get_table(Episode episode, GameMode mode, uint8_t difficulty, uint8_t secid) const;
+  bool operator==(const CommonItemSet& other) const = default;
+  bool operator!=(const CommonItemSet& other) const = default;
+
+  std::shared_ptr<const Table> get_table(Episode episode, GameMode mode, Difficulty difficulty, uint8_t secid) const;
   phosg::JSON json() const;
   void print(FILE* stream) const;
+  void print_diff(FILE* stream, const CommonItemSet& other) const;
 
 protected:
   CommonItemSet() = default;
 
-  static uint16_t key_for_table(Episode episode, GameMode mode, uint8_t difficulty, uint8_t secid);
+  static uint16_t key_for_table(Episode episode, GameMode mode, Difficulty difficulty, uint8_t secid);
 
   std::unordered_map<uint16_t, std::shared_ptr<Table>> tables;
 };

src/DownloadSession.cc

@@ -40,7 +40,7 @@ DownloadSession::DownloadSession(
     uint16_t remote_port,
     const std::string& output_dir,
     Version version,
-    uint8_t language,
+    Language language,
     std::shared_ptr<const PSOBBEncryption::KeyFile> bb_key_file,
     uint32_t serial_number2,
     uint32_t serial_number,
@@ -184,7 +184,7 @@ void DownloadSession::send_93_9D_9E(bool extended) {
     ret.access_key2 = ret.access_key;
     ret.login_character_name.encode(this->character->disp.name.decode());
     ret.client_config = this->client_config;
-    this->channel->send(0x9E, 0x01, &ret, extended ? sizeof(ret) : sizeof(C_Login_GC_9E));
+    this->channel->send(0x9E, 0x01, &ret, extended ? sizeof(ret) : sizeof(C_Login_PC_GC_9E));
 
   } else if (this->version == Version::XB_V3) {
     C_LoginExtended_XB_9E ret;
@@ -222,13 +222,13 @@ void DownloadSession::send_61_98(bool is_98) {
   if (is_v1(this->version)) {
     C_CharacterData_DCv1_61_98 ret;
     ret.inventory = this->character->inventory;
-    ret.disp = convert_player_disp_data<PlayerDispDataDCPCV3, PlayerDispDataBB>(this->character->disp, 1, 1);
+    ret.disp = convert_player_disp_data<PlayerDispDataDCPCV3, PlayerDispDataBB>(this->character->disp, Language::ENGLISH, Language::ENGLISH);
     this->channel->send(command, 0x01, ret);
 
   } else if (this->version == Version::DC_V2) {
     C_CharacterData_DCv2_61_98 ret;
     ret.inventory = this->character->inventory;
-    ret.disp = convert_player_disp_data<PlayerDispDataDCPCV3, PlayerDispDataBB>(this->character->disp, 1, 1);
+    ret.disp = convert_player_disp_data<PlayerDispDataDCPCV3, PlayerDispDataBB>(this->character->disp, Language::ENGLISH, Language::ENGLISH);
     ret.records.challenge = this->character->challenge_records;
     ret.records.battle = this->character->battle_records;
     ret.choice_search_config = this->character->choice_search_config;
@@ -237,7 +237,7 @@ void DownloadSession::send_61_98(bool is_98) {
   } else if (this->version == Version::PC_V2) {
     C_CharacterData_PC_61_98 ret;
     ret.inventory = this->character->inventory;
-    ret.disp = convert_player_disp_data<PlayerDispDataDCPCV3, PlayerDispDataBB>(this->character->disp, 1, 1);
+    ret.disp = convert_player_disp_data<PlayerDispDataDCPCV3, PlayerDispDataBB>(this->character->disp, Language::ENGLISH, Language::ENGLISH);
     ret.records.challenge = this->character->challenge_records;
     ret.records.battle = this->character->battle_records;
     ret.choice_search_config = this->character->choice_search_config;
@@ -246,7 +246,7 @@ void DownloadSession::send_61_98(bool is_98) {
   } else if (is_v3(this->version)) {
     C_CharacterData_V3_61_98 ret;
     ret.inventory = this->character->inventory;
-    ret.disp = convert_player_disp_data<PlayerDispDataDCPCV3, PlayerDispDataBB>(this->character->disp, 1, 1);
+    ret.disp = convert_player_disp_data<PlayerDispDataDCPCV3, PlayerDispDataBB>(this->character->disp, Language::ENGLISH, Language::ENGLISH);
     ret.records.challenge = this->character->challenge_records;
     ret.records.battle = this->character->battle_records;
     ret.choice_search_config = this->character->choice_search_config;
@@ -457,7 +457,7 @@ asio::awaitable<void> DownloadSession::on_message(Channel::Message& msg) {
     case 0x1F:
     case 0xA0:
     case 0xA1: {
-      C_MenuSelection_10_Flag00 ret;
+      C_MenuSelectionBase_10 ret;
 
       auto handle_command = [&]<typename CmdT>() {
         const auto* items = check_size_vec_t<CmdT>(msg.data, msg.flag + 1);
@@ -552,7 +552,7 @@ asio::awaitable<void> DownloadSession::on_message(Channel::Message& msg) {
         C_CreateGame_PC_C1 ret;
         ret.name.encode(random_name());
         ret.password.encode(random_name());
-        ret.difficulty = 0;
+        ret.difficulty = Difficulty::NORMAL;
         ret.battle_mode = (game_config.mode == GameMode::BATTLE);
         ret.challenge_mode = (game_config.mode == GameMode::CHALLENGE);
         ret.episode = 1;
@@ -562,7 +562,7 @@ asio::awaitable<void> DownloadSession::on_message(Channel::Message& msg) {
         C_CreateGame_DC_V3_0C_C1_Ep3_EC ret;
         ret.name.encode(random_name());
         ret.password.encode(random_name());
-        ret.difficulty = 0;
+        ret.difficulty = Difficulty::NORMAL;
         ret.battle_mode = (game_config.mode == GameMode::BATTLE);
         ret.challenge_mode = (game_config.mode == GameMode::CHALLENGE);
         if (is_v1(this->version)) {
@@ -582,7 +582,7 @@ asio::awaitable<void> DownloadSession::on_message(Channel::Message& msg) {
         C_CreateGame_BB_C1 ret;
         ret.name.encode(random_name());
         ret.password.encode(random_name());
-        ret.difficulty = 0;
+        ret.difficulty = Difficulty::NORMAL;
         ret.battle_mode = (game_config.mode == GameMode::BATTLE);
         ret.challenge_mode = (game_config.mode == GameMode::CHALLENGE);
         if (game_config.episode == Episode::EP1) {
@@ -651,12 +651,12 @@ asio::awaitable<void> DownloadSession::on_message(Channel::Message& msg) {
           filtered_name.push_back((isalnum(ch) || (ch == '-') || (ch == '.') || (ch == '_')) ? ch : '_');
         }
         string local_filename = std::format(
-            "{}/{:016X}_{}_{}_{}_{}",
+            "{}/{:016X}_{}_{}_{:c}_{}",
             this->output_dir,
             this->current_request,
             phosg::now(),
             phosg::name_for_enum(this->version),
-            char_for_language_code(this->language),
+            char_for_language(this->language),
             filtered_name);
         this->open_files.emplace(internal_name, OpenFile{.request = this->current_request, .filename = local_filename, .total_size = cmd.file_size, .data = ""});
       };
@@ -764,7 +764,7 @@ void DownloadSession::send_next_request() {
       this->log.info_f("Sending request {:016X}", this->current_request);
     }
 
-    C_MenuSelection_10_Flag00 cmd;
+    C_MenuSelectionBase_10 cmd;
     cmd.menu_id = (this->current_request >> 32) & 0xFFFFFFFF;
     cmd.item_id = this->current_request & 0xFFFFFFFF;
     this->channel->send(0x10, 0x00, cmd);

src/DownloadSession.hh

@@ -21,7 +21,7 @@ public:
       uint16_t remote_port,
       const std::string& output_dir,
       Version version,
-      uint8_t language,
+      Language language,
       std::shared_ptr<const PSOBBEncryption::KeyFile> bb_key_file,
       uint32_t serial_number2,
       uint32_t serial_number,
@@ -50,7 +50,7 @@ protected:
   uint16_t remote_port;
   std::string output_dir;
   Version version;
-  uint8_t language;
+  Language language;
   bool show_command_data;
   std::shared_ptr<const PSOBBEncryption::KeyFile> bb_key_file;
   uint32_t serial_number;

src/EnemyType.cc

@@ -198,6 +198,33 @@ const vector<EnemyType>& enemy_types_for_rare_table_index(Episode episode, uint8
   }
 }
 
+const vector<EnemyType>& enemy_types_for_battle_param_index(Episode episode, uint8_t bp_index) {
+  const auto& generate_table = +[](Episode episode) -> vector<vector<EnemyType>> {
+    vector<vector<EnemyType>> ret;
+    for (const auto& def : type_defs) {
+      if (def.valid_in_episode(episode) && (def.bp_index != 0xFF)) {
+        if (def.bp_index >= ret.size()) {
+          ret.resize(def.bp_index + 1);
+        }
+        ret[def.bp_index].emplace_back(def.type);
+      }
+    }
+    return ret;
+  };
+
+  static array<vector<vector<EnemyType>>, 5> data;
+  auto& ret = data.at(static_cast<size_t>(episode));
+  if (ret.empty()) {
+    ret = generate_table(episode);
+  }
+  try {
+    return ret.at(bp_index);
+  } catch (const out_of_range&) {
+    static const vector<EnemyType> empty_vec;
+    return empty_vec;
+  }
+}
+
 EnemyType EnemyTypeDefinition::rare_type(Episode episode, uint8_t event, uint8_t floor) const {
   switch (this->type) {
     case EnemyType::HILDEBEAR:

src/EnemyType.hh

@@ -181,3 +181,4 @@ template <>
 EnemyType phosg::enum_for_name<EnemyType>(const char* name);
 
 const std::vector<EnemyType>& enemy_types_for_rare_table_index(Episode episode, uint8_t rt_index);
+const std::vector<EnemyType>& enemy_types_for_battle_param_index(Episode episode, uint8_t bp_index);

src/Episode3/BattleRecord.cc

@@ -174,21 +174,6 @@ string BattleRecord::serialize() const {
   return std::move(w.str());
 }
 
-bool BattleRecord::writable() const {
-  return this->is_writable;
-}
-
-bool BattleRecord::battle_in_progress() const {
-  return (this->battle_start_timestamp != 0);
-}
-
-const BattleRecord::Event* BattleRecord::get_first_event() const {
-  if (this->events.empty()) {
-    return nullptr;
-  }
-  return &this->events.front();
-}
-
 void BattleRecord::add_player(
     const PlayerLobbyDataDCGC& lobby_data,
     const PlayerInventory& inventory,
@@ -256,16 +241,6 @@ void BattleRecord::add_random_data(const void* data, size_t size) {
   this->random_stream.append(reinterpret_cast<const char*>(data), size);
 }
 
-vector<string> BattleRecord::get_all_server_data_commands() const {
-  vector<string> ret;
-  for (const auto& event : this->events) {
-    if (event.type == Event::Type::SERVER_DATA_COMMAND) {
-      ret.emplace_back(event.data);
-    }
-  }
-  return ret;
-}
-
 const string& BattleRecord::get_random_stream() const {
   return this->random_stream;
 }

src/Episode3/BattleRecord.hh

@@ -62,10 +62,25 @@ public:
   explicit BattleRecord(const std::string& data);
   std::string serialize() const;
 
-  bool writable() const;
-  bool battle_in_progress() const;
+  inline bool writable() const {
+    return this->is_writable;
+  }
 
-  const Event* get_first_event() const;
+  inline uint32_t get_behavior_flags() const {
+    return this->behavior_flags;
+  }
+
+  inline bool battle_in_progress() const {
+    return (this->battle_start_timestamp != 0);
+  }
+
+  inline const Event* get_first_event() const {
+    return this->events.empty() ? nullptr : &this->events.front();
+  }
+
+  inline std::deque<Event> get_all_events() const {
+    return this->events;
+  }
 
   void add_player(
       const PlayerLobbyDataDCGC& lobby_data,
@@ -86,7 +101,6 @@ public:
 
   void print(FILE* stream) const;
 
-  std::vector<std::string> get_all_server_data_commands() const;
   const std::string& get_random_stream() const;
 
 private:

src/Episode3/DataIndexes.cc

@@ -1177,6 +1177,19 @@ void PlayerConfig::encrypt(uint8_t basis) {
   this->basis = basis;
 }
 
+bool PlayerConfig::card_count_checksums_correct() const {
+  for (size_t z = 0; z < this->card_count_checksums.size(); z++) {
+    uint16_t checksum_value = 0;
+    for (size_t w = 0; w < 20; w++) {
+      checksum_value += reinterpret_cast<const uint8_t*>(&this->card_counts)[z * 50 + w];
+    }
+    if (this->card_count_checksums[z] != checksum_value) {
+      return false;
+    }
+  }
+  return true;
+}
+
 PlayerConfigNTE::PlayerConfigNTE(const PlayerConfig& config)
     : rank_text(config.rank_text),
       unknown_a1(config.unknown_a1),
@@ -1707,7 +1720,7 @@ phosg::JSON MapDefinition::CameraSpec::json() const {
   });
 }
 
-phosg::JSON MapDefinition::NPCDeck::json(uint8_t language) const {
+phosg::JSON MapDefinition::NPCDeck::json(Language language) const {
   phosg::JSON card_ids_json = phosg::JSON::list();
   for (size_t z = 0; z < this->card_ids.size(); z++) {
     if (this->card_ids[z] != 0xFFFF) {
@@ -1720,7 +1733,7 @@ phosg::JSON MapDefinition::NPCDeck::json(uint8_t language) const {
   });
 }
 
-phosg::JSON MapDefinition::AIParams::json(uint8_t language) const {
+phosg::JSON MapDefinition::AIParams::json(Language language) const {
   phosg::JSON params_json = phosg::JSON::list();
   for (size_t z = 0; z < this->params.size(); z++) {
     params_json.emplace_back(this->params[z].load());
@@ -1732,7 +1745,7 @@ phosg::JSON MapDefinition::AIParams::json(uint8_t language) const {
   });
 }
 
-phosg::JSON MapDefinition::DialogueSet::json(uint8_t language) const {
+phosg::JSON MapDefinition::DialogueSet::json(Language language) const {
   phosg::JSON strings_json = phosg::JSON::list();
   for (size_t z = 0; z < this->strings.size(); z++) {
     strings_json.emplace_back(this->strings[z].decode(language));
@@ -1805,7 +1818,7 @@ string MapDefinition::CameraSpec::str() const {
       this->unknown_a2[1], this->unknown_a2[2]);
 }
 
-string MapDefinition::str(const CardIndex* card_index, uint8_t language) const {
+string MapDefinition::str(const CardIndex* card_index, Language language) const {
   deque<string> lines;
   auto add_map = [&](const parray<parray<uint8_t, 0x10>, 0x10>& tiles) {
     for (size_t y = 0; y < this->height; y++) {
@@ -2490,7 +2503,7 @@ CardIndex::CardIndex(
 
       // Some cards intentionally have the same name, so we just leave them
       // unindexed (they can still be looked up by ID, of course)
-      string name = entry->def.en_name.decode(1);
+      string name = entry->def.en_name.decode(Language::ENGLISH);
       this->card_definitions_by_name.emplace(name, entry);
       this->card_definitions_by_name_normalized.emplace(this->normalize_card_name(name), entry);
 
@@ -2607,14 +2620,14 @@ string CardIndex::normalize_card_name(const string& name) {
   return ret;
 }
 
-MapIndex::VersionedMap::VersionedMap(shared_ptr<const MapDefinition> map, uint8_t language)
+MapIndex::VersionedMap::VersionedMap(shared_ptr<const MapDefinition> map, Language language)
     : map(map),
       language(language) {}
 
-MapIndex::VersionedMap::VersionedMap(std::string&& compressed_data, uint8_t language)
+MapIndex::VersionedMap::VersionedMap(std::string&& compressed_data, Language language)
     : language(language),
-      compressed_data(std::move(compressed_data)) {
-  string decompressed = prs_decompress(this->compressed_data);
+      compressed_data(make_shared<string>(std::move(compressed_data))) {
+  string decompressed = prs_decompress(*this->compressed_data);
   if (decompressed.size() == sizeof(MapDefinitionTrial)) {
     this->map = make_shared<MapDefinition>(*reinterpret_cast<const MapDefinitionTrial*>(decompressed.data()));
   } else if (decompressed.size() == sizeof(MapDefinition)) {
@@ -2633,51 +2646,67 @@ shared_ptr<const MapDefinitionTrial> MapIndex::VersionedMap::trial() const {
   return this->trial_map;
 }
 
-const std::string& MapIndex::VersionedMap::compressed(bool is_nte) const {
-  if (is_nte) {
-    if (this->compressed_trial_data.empty()) {
+std::shared_ptr<const std::string> MapIndex::VersionedMap::compressed(bool trial) const {
+  if (trial) {
+    if (!this->compressed_data_trial) {
       auto md = this->trial();
-      this->compressed_trial_data = prs_compress(md.get(), sizeof(*md));
+      this->compressed_data_trial = make_shared<string>(prs_compress(md.get(), sizeof(*md)));
     }
-    return this->compressed_trial_data;
+    return this->compressed_data_trial;
   } else {
-    if (this->compressed_data.empty()) {
-      this->compressed_data = prs_compress(this->map.get(), sizeof(*this->map));
+    if (!this->compressed_data) {
+      this->compressed_data = make_shared<string>(prs_compress(this->map.get(), sizeof(*this->map)));
     }
     return this->compressed_data;
   }
 }
 
-MapIndex::Map::Map(shared_ptr<const VersionedMap> initial_version)
+std::shared_ptr<const std::string> MapIndex::VersionedMap::trial_download() const {
+  if (!this->download_data_trial) {
+    MapDefinitionTrial trial_map = *this->map;
+    trial_map.tag = 0x96;
+    this->download_data_trial = make_shared<string>(prs_compress(&trial_map, sizeof(trial_map)));
+  }
+  return this->download_data_trial;
+}
+
+MapIndex::Map::Map(shared_ptr<const VersionedMap> initial_version, uint8_t visibility_flags)
     : map_number(initial_version->map->map_number),
+      visibility_flags(visibility_flags),
       initial_version(initial_version) {
-  this->versions.resize(this->initial_version->language + 1);
-  this->versions[this->initial_version->language] = initial_version;
+  size_t lang_index = static_cast<size_t>(this->initial_version->language);
+  this->versions.resize(lang_index + 1);
+  this->versions[lang_index] = initial_version;
 }
 
 void MapIndex::Map::add_version(std::shared_ptr<const VersionedMap> vm) {
-  if (this->versions.size() <= vm->language) {
-    this->versions.resize(vm->language + 1);
+  size_t lang_index = static_cast<size_t>(vm->language);
+  if (this->versions.size() <= lang_index) {
+    this->versions.resize(lang_index + 1);
   }
-  if (this->versions[vm->language]) {
+  if (this->versions[lang_index]) {
     throw runtime_error("map version already exists");
   }
   this->initial_version->map->assert_semantically_equivalent(*vm->map);
-  this->versions[vm->language] = vm;
+  this->versions[lang_index] = vm;
 }
 
-bool MapIndex::Map::has_version(uint8_t language) const {
-  return (this->versions.size() > language) && !!this->versions[language];
+bool MapIndex::Map::has_version(Language language) const {
+  size_t lang_index = static_cast<size_t>(language);
+  return (this->versions.size() > lang_index) && !!this->versions[lang_index];
 }
 
-shared_ptr<const MapIndex::VersionedMap> MapIndex::Map::version(uint8_t language) const {
+shared_ptr<const MapIndex::VersionedMap> MapIndex::Map::version(Language language) const {
+  size_t lang_index = static_cast<size_t>(language);
+
   // If the requested language exists, return it
-  if ((language < this->versions.size()) && this->versions[language]) {
-    return this->versions[language];
+  if ((lang_index < this->versions.size()) && this->versions[lang_index]) {
+    return this->versions[lang_index];
   }
   // If English exists, return it
-  if ((1 < this->versions.size()) && this->versions[1]) {
-    return this->versions[1];
+  constexpr size_t english_lang_index = static_cast<size_t>(Language::ENGLISH);
+  if ((english_lang_index < this->versions.size()) && this->versions[english_lang_index]) {
+    return this->versions[english_lang_index];
   }
   // Return the first version that exists
   for (const auto& vm : this->versions) {
@@ -2690,39 +2719,47 @@ shared_ptr<const MapIndex::VersionedMap> MapIndex::Map::version(uint8_t language
   throw logic_error("no map versions exist");
 }
 
-MapIndex::MapIndex(const string& directory) {
-  for (const auto& item : std::filesystem::directory_iterator(directory)) {
-    string filename = item.path().filename().string();
+MapIndex::Category::Category(uint32_t category_id, const phosg::JSON& json)
+    : category_id(category_id),
+      visibility_flags(json.get_int("VisibilityFlags")),
+      name(json.get_string("Name", "")),
+      description(json.get_string("Description", "")) {}
+
+MapIndex::MapIndex(const string& directory, bool raise_on_any_failure) {
+  map<uint32_t, shared_ptr<Map>> mutable_maps;
+
+  auto try_add_map_file = [&](std::shared_ptr<Category> category, const std::string& file_path) -> void {
     try {
+      string filename = phosg::basename(file_path);
       string base_filename;
       string compressed_data;
       shared_ptr<MapDefinition> decompressed_data;
       if (filename.ends_with(".mnmd") || filename.ends_with(".bind")) {
-        decompressed_data = make_shared<MapDefinition>(phosg::load_object_file<MapDefinition>(directory + "/" + filename));
+        decompressed_data = make_shared<MapDefinition>(phosg::load_object_file<MapDefinition>(file_path));
         base_filename = filename.substr(0, filename.size() - 5);
       } else if (filename.ends_with(".mnm") || filename.ends_with(".bin")) {
-        compressed_data = phosg::load_file(directory + "/" + filename);
+        compressed_data = phosg::load_file(file_path);
         base_filename = filename.substr(0, filename.size() - 4);
       } else if (filename.ends_with(".bin.gci") || filename.ends_with(".mnm.gci")) {
-        compressed_data = decode_gci_data(phosg::load_file(directory + "/" + filename));
+        compressed_data = decode_gci_data(phosg::load_file(file_path));
         base_filename = filename.substr(0, filename.size() - 8);
       } else if (filename.ends_with(".gci")) {
-        compressed_data = decode_gci_data(phosg::load_file(directory + "/" + filename));
+        compressed_data = decode_gci_data(phosg::load_file(file_path));
         base_filename = filename.substr(0, filename.size() - 4);
       } else if (filename.ends_with(".bin.vms") || filename.ends_with(".mnm.vms")) {
-        compressed_data = decode_vms_data(phosg::load_file(directory + "/" + filename));
+        compressed_data = decode_vms_data(phosg::load_file(file_path));
         base_filename = filename.substr(0, filename.size() - 8);
       } else if (filename.ends_with(".vms")) {
-        compressed_data = decode_vms_data(phosg::load_file(directory + "/" + filename));
+        compressed_data = decode_vms_data(phosg::load_file(file_path));
         base_filename = filename.substr(0, filename.size() - 4);
       } else if (filename.ends_with(".bin.dlq") || filename.ends_with(".mnm.dlq")) {
-        compressed_data = decode_dlq_data(phosg::load_file(directory + "/" + filename));
+        compressed_data = decode_dlq_data(phosg::load_file(file_path));
         base_filename = filename.substr(0, filename.size() - 8);
       } else if (filename.ends_with(".dlq")) {
-        compressed_data = decode_dlq_data(phosg::load_file(directory + "/" + filename));
+        compressed_data = decode_dlq_data(phosg::load_file(file_path));
         base_filename = filename.substr(0, filename.size() - 4);
       } else {
-        continue; // Silently skip file
+        return; // Silently skip file
       }
 
       if (base_filename.size() < 2) {
@@ -2731,7 +2768,7 @@ MapIndex::MapIndex(const string& directory) {
       if (base_filename[base_filename.size() - 2] != '-') {
         throw runtime_error("language code not present");
       }
-      uint8_t language = language_code_for_char(base_filename[base_filename.size() - 1]);
+      Language language = language_for_char(base_filename[base_filename.size() - 1]);
 
       shared_ptr<VersionedMap> vm;
       if (decompressed_data) {
@@ -2742,35 +2779,90 @@ MapIndex::MapIndex(const string& directory) {
         throw runtime_error("unknown map file format");
       }
 
+      uint8_t visibility_flags = category ? category->visibility_flags : 0x00;
+
       string name = vm->map->name.decode(vm->language);
-      auto map_it = this->maps.find(vm->map->map_number);
-      if (map_it == this->maps.end()) {
-        map_it = this->maps.emplace(vm->map->map_number, make_shared<Map>(vm)).first;
-        static_game_data_log.debug_f("({}) Created Episode 3 map {:08X} {} ({}; {})",
+      auto map_it = mutable_maps.find(vm->map->map_number);
+      if (map_it == mutable_maps.end()) {
+        map_it = mutable_maps.emplace(vm->map->map_number, make_shared<Map>(vm, visibility_flags)).first;
+        this->maps.emplace(vm->map->map_number, map_it->second);
+
+        string in_category_str;
+        if (category) {
+          in_category_str = std::format(" in category {}", category->name);
+          category->add_map(map_it->second);
+        }
+        static_game_data_log.debug_f("({}) Created Episode 3 map {:08X} {}{} ({}; {})",
             filename,
             vm->map->map_number,
-            char_for_language_code(vm->language),
+            char_for_language(vm->language),
+            in_category_str,
             vm->map->is_quest() ? "quest" : "free",
             name);
       } else {
+        if (map_it->second->visibility_flags != visibility_flags) {
+          throw std::runtime_error(std::format("visibility flags {:02X} for added map {} do not match existing flags {}",
+              map_it->second->visibility_flags, file_path, visibility_flags));
+        }
         map_it->second->add_version(vm);
         static_game_data_log.debug_f("({}) Added Episode 3 map version {:08X} {} ({}; {})",
             filename,
             vm->map->map_number,
-            char_for_language_code(vm->language),
+            char_for_language(vm->language),
             vm->map->is_quest() ? "quest" : "free",
             name);
       }
       this->maps_by_name.emplace(vm->map->name.decode(vm->language), map_it->second);
 
     } catch (const exception& e) {
-      static_game_data_log.warning_f("Failed to index Episode 3 map {}: {}",
-          filename, e.what());
+      if (raise_on_any_failure) {
+        throw;
+      }
+      static_game_data_log.warning_f("Failed to index Episode 3 map {}: {}", file_path, e.what());
+    }
+  };
+
+  std::vector<std::filesystem::directory_entry> cat_items;
+  for (const auto& cat_item : std::filesystem::directory_iterator(directory)) {
+    cat_items.emplace_back(cat_item);
+  }
+  auto sort_fn = +[](const std::filesystem::directory_entry& a, const std::filesystem::directory_entry& b) -> bool {
+    return a.path().filename().string() < b.path().filename().string();
+  };
+  sort(cat_items.begin(), cat_items.end(), sort_fn);
+
+  for (const auto& cat_item : cat_items) {
+    string cat_dir_path = cat_item.path().string();
+
+    if (cat_item.is_directory()) {
+      shared_ptr<Category> category;
+      try {
+        string json_filename = std::format("{}/{}", cat_item.path().string(), "category.json");
+        auto category_json = phosg::JSON::parse(phosg::load_file(json_filename));
+        uint32_t category_id = this->categories.size() + 1;
+        auto category = make_shared<Category>(category_id, category_json);
+        this->categories.emplace(category_id, category);
+        static_game_data_log.debug_f("({}) Created Episode 3 map category {:08X} ({})",
+            cat_item.path().filename().string(), category_id, category->name);
+
+        for (const auto& map_item : std::filesystem::directory_iterator(cat_item)) {
+          try_add_map_file(category, map_item.path().string());
+        }
+
+      } catch (const exception& e) {
+        if (raise_on_any_failure) {
+          throw;
+        }
+        static_game_data_log.warning_f("Failed to index Episode 3 map category {}: {}", cat_item.path().string(), e.what());
+      }
+
+    } else {
+      try_add_map_file(nullptr, cat_dir_path);
     }
   }
 }
 
-const string& MapIndex::get_compressed_list(size_t num_players, uint8_t language) const {
+const string& MapIndex::get_compressed_list(size_t num_players, Language language, bool is_trial) const {
   if (num_players == 0) {
     throw runtime_error("cannot generate map list for no players");
   }
@@ -2778,16 +2870,27 @@ const string& MapIndex::get_compressed_list(size_t num_players, uint8_t language
     throw logic_error("player count is too high in map list generation");
   }
 
-  if (language >= this->compressed_map_lists.size()) {
-    this->compressed_map_lists.resize(language + 1);
+  auto& compressed_lists = is_trial ? this->compressed_map_lists_trial : this->compressed_map_lists_final;
+
+  size_t lang_index = static_cast<size_t>(language);
+  if (lang_index >= compressed_lists.size()) {
+    compressed_lists.resize(lang_index + 1);
   }
-  string& compressed_map_list = this->compressed_map_lists[language].at(num_players - 1);
+  string& compressed_map_list = compressed_lists[lang_index].at(num_players - 1);
   if (compressed_map_list.empty()) {
     phosg::StringWriter entries_w;
     phosg::StringWriter strings_w;
 
+    auto vis_flag = is_trial
+        ? Episode3::MapIndex::VisibilityFlag::ONLINE_TRIAL
+        : Episode3::MapIndex::VisibilityFlag::ONLINE_FINAL;
+
     size_t num_maps = 0;
     for (const auto& map_it : this->maps) {
+      if (!map_it.second->check_visibility_flag(vis_flag)) {
+        continue;
+      }
+
       auto vm = map_it.second->version(language);
       size_t map_num_players = 0;
       for (size_t z = 0; z < 4; z++) {
@@ -2844,31 +2947,16 @@ const string& MapIndex::get_compressed_list(size_t num_players, uint8_t language
     compressed_w.write(prs.close());
     compressed_map_list = std::move(compressed_w.str());
     if (compressed_map_list.size() > 0x7BEC) {
-      throw runtime_error(std::format("compressed map list for {} players is too large (0x{:X} bytes)", num_players, compressed_map_list.size()));
+      throw runtime_error(std::format("compressed {} map list for {} players is too large (0x{:X} bytes)",
+          is_trial ? "trial" : "final", num_players, compressed_map_list.size()));
     }
     size_t decompressed_size = sizeof(header) + entries_w.size() + strings_w.size();
-    static_game_data_log.info_f("Generated Episode 3 compressed map list for {} player(s) ({} maps; 0x{:X} -> 0x{:X} bytes)",
-        num_players, num_maps, decompressed_size, compressed_map_list.size());
+    static_game_data_log.info_f("Generated Episode 3 compressed {} map list for {} player(s) ({} maps; 0x{:X} -> 0x{:X} bytes)",
+        is_trial ? "trial" : "final", num_players, num_maps, decompressed_size, compressed_map_list.size());
   }
   return compressed_map_list;
 }
 
-shared_ptr<const MapIndex::Map> MapIndex::for_number(uint32_t id) const {
-  return this->maps.at(id);
-}
-
-shared_ptr<const MapIndex::Map> MapIndex::for_name(const string& name) const {
-  return this->maps_by_name.at(name);
-}
-
-set<uint32_t> MapIndex::all_numbers() const {
-  set<uint32_t> ret;
-  for (const auto& it : this->maps) {
-    ret.emplace(it.first);
-  }
-  return ret;
-}
-
 COMDeckIndex::COMDeckIndex(const string& filename) {
   try {
     auto json = phosg::JSON::parse(phosg::load_file(filename));

src/Episode3/DataIndexes.hh

@@ -605,9 +605,10 @@ struct CardDefinition {
   // actions, and assist cards have 1 here.
   /* 008F */ uint8_t cannot_attack;
   /* 0090 */ uint8_t unused3;
-  // If cannot_drop is 0, this card can't appear in post-battle rewards. A
-  // value of 0 here also prevents the card from being used as a God Whim
-  // random assist.
+  // If cannot_drop is 1, this card can't appear in post-battle rewards and is
+  // considered unobtainable by players, so the game will remove it from the
+  // player's collection if they have any copies of it. A value of 1 here also
+  // prevents the card from being used as a God Whim random assist.
   /* 0091 */ uint8_t cannot_drop;
   // This criterion code specifies who can use the card, and when it can be
   // used. This specifies which Hero-side SCs can use which items, for example,
@@ -864,7 +865,8 @@ struct PlayerConfig {
   /* 0138:---- */ PlayerRecordsBattleBE unused_offline_records;
   /* 0150:---- */ parray<uint8_t, 4> unknown_a4;
   // The PlayerDataSegment structure begins here. In newserv, we combine this
-  // structure into PlayerConfig since the two are always used together.
+  // structure into PlayerConfig since the two are always used together on the
+  // server side.
   /* 0154:0000 */ uint8_t is_encrypted;
   /* 0155:0001 */ uint8_t basis;
   /* 0156:0002 */ parray<uint8_t, 2> unused;
@@ -937,6 +939,8 @@ struct PlayerConfig {
 
   void decrypt();
   void encrypt(uint8_t basis);
+
+  bool card_count_checksums_correct() const;
 } __packed_ws__(PlayerConfig, 0x2350);
 
 struct PlayerConfigNTE {
@@ -1175,7 +1179,8 @@ struct OverlayState {
 struct MapDefinition { // .mnmd format; also the format of (decompressed) quests
   // If tag is not 0x00000100, the game considers the map to be corrupt in
   // offline mode and will delete it (if it's a download quest). The tag field
-  // doesn't seem to have any other use.
+  // doesn't seem to have any other use. In Trial Edition, download quests are
+  // expected to have 0x96 here instead.
   /* 0000 */ be_uint32_t tag;
 
   /* 0004 */ be_uint32_t map_number; // Must be unique across all maps
@@ -1310,7 +1315,7 @@ struct MapDefinition { // .mnmd format; also the format of (decompressed) quests
     /* 00 */ pstring<TextEncoding::MARKED, 0x18> deck_name;
     /* 18 */ parray<be_uint16_t, 0x20> card_ids; // Last one appears to always be FFFF
     /* 58 */
-    phosg::JSON json(uint8_t language) const;
+    phosg::JSON json(Language language) const;
   } __packed_ws__(NPCDeck, 0x58);
   /* 1FE8 */ parray<NPCDeck, 3> npc_decks; // Unused if name[0] == 0
 
@@ -1326,7 +1331,7 @@ struct MapDefinition { // .mnmd format; also the format of (decompressed) quests
     // TODO: Figure out exactly how these are used and document here.
     /* 0018 */ parray<be_uint16_t, 0x7E> params;
     /* 0114 */
-    phosg::JSON json(uint8_t language) const;
+    phosg::JSON json(Language language) const;
   } __packed_ws__(AIParams, 0x114);
   /* 20F0 */ parray<AIParams, 3> npc_ai_params; // Unused if name[0] == 0
 
@@ -1379,7 +1384,7 @@ struct MapDefinition { // .mnmd format; also the format of (decompressed) quests
     // strings, excluding any that are empty or begin with the character '^'.
     /* 0004 */ parray<pstring<TextEncoding::MARKED, 0x40>, 4> strings;
     /* 0104 */
-    phosg::JSON json(uint8_t language) const;
+    phosg::JSON json(Language language) const;
   } __packed_ws__(DialogueSet, 0x104);
 
   // There are up to 0x10 of these per valid NPC, but only the first 13 of them
@@ -1485,8 +1490,8 @@ struct MapDefinition { // .mnmd format; also the format of (decompressed) quests
   // text may differ.
   void assert_semantically_equivalent(const MapDefinition& other) const;
 
-  std::string str(const CardIndex* card_index, uint8_t language) const;
-  phosg::JSON json(uint8_t language) const;
+  std::string str(const CardIndex* card_index, Language language) const;
+  phosg::JSON json(Language language) const;
 } __packed_ws__(MapDefinition, 0x5A18);
 
 struct MapDefinitionTrial {
@@ -1582,35 +1587,47 @@ private:
 
 class MapIndex {
 public:
-  MapIndex(const std::string& directory);
+  enum class VisibilityFlag : uint8_t {
+    ONLINE_TRIAL = 0x01,
+    ONLINE_FINAL = 0x02,
+    DOWNLOAD_TRIAL = 0x04,
+    DOWNLOAD_FINAL = 0x08,
+  };
 
   class VersionedMap {
   public:
     std::shared_ptr<const MapDefinition> map;
-    uint8_t language;
+    Language language;
 
-    VersionedMap(std::shared_ptr<const MapDefinition> map, uint8_t language);
-    VersionedMap(std::string&& compressed_data, uint8_t language);
+    VersionedMap(std::shared_ptr<const MapDefinition> map, Language language);
+    VersionedMap(std::string&& compressed_data, Language language);
 
     std::shared_ptr<const MapDefinitionTrial> trial() const;
-    const std::string& compressed(bool is_nte) const;
+    std::shared_ptr<const std::string> compressed(bool trial) const;
+    std::shared_ptr<const std::string> trial_download() const;
 
   private:
     mutable std::shared_ptr<const MapDefinitionTrial> trial_map;
-    mutable std::string compressed_data;
-    mutable std::string compressed_trial_data;
+    mutable std::shared_ptr<std::string> compressed_data;
+    mutable std::shared_ptr<std::string> compressed_data_trial;
+    mutable std::shared_ptr<std::string> download_data_trial;
   };
 
   class Map {
   public:
     uint32_t map_number;
+    uint8_t visibility_flags;
     std::shared_ptr<const VersionedMap> initial_version;
 
-    explicit Map(std::shared_ptr<const VersionedMap> initial_version);
+    Map(std::shared_ptr<const VersionedMap> initial_version, uint8_t visibility_flags);
+
+    inline bool check_visibility_flag(VisibilityFlag flag) const {
+      return (this->visibility_flags & static_cast<uint8_t>(flag));
+    }
 
     void add_version(std::shared_ptr<const VersionedMap> vm);
-    bool has_version(uint8_t language) const;
-    std::shared_ptr<const VersionedMap> version(uint8_t language) const;
+    bool has_version(Language language) const;
+    std::shared_ptr<const VersionedMap> version(Language language) const;
     inline const std::vector<std::shared_ptr<const VersionedMap>>& all_versions() const {
       return this->versions;
     }
@@ -1619,18 +1636,76 @@ public:
     std::vector<std::shared_ptr<const VersionedMap>> versions;
   };
 
-  const std::string& get_compressed_list(size_t num_players, uint8_t language) const;
-  std::shared_ptr<const Map> for_number(uint32_t id) const;
-  std::shared_ptr<const Map> for_name(const std::string& name) const;
-  std::set<uint32_t> all_numbers() const;
+  class Category {
+  public:
+    uint32_t category_id;
+    uint8_t visibility_flags;
+    std::string name;
+    std::string description;
+
+    Category(uint32_t category_id, const phosg::JSON& json);
+
+    inline bool check_visibility_flag(VisibilityFlag flag) const {
+      return (this->visibility_flags & static_cast<uint8_t>(flag));
+    }
+
+    inline void add_map(std::shared_ptr<const Map> map) {
+      this->maps.emplace(map->map_number, map);
+    }
+    inline const std::map<uint32_t, std::shared_ptr<const Map>>& all_maps() const {
+      return this->maps;
+    }
+
+  private:
+    std::map<uint32_t, std::shared_ptr<const Map>> maps;
+  };
+
+  explicit MapIndex(const std::string& directory, bool raise_on_any_failure = false);
+
+  const std::string& get_compressed_list(size_t num_players, Language language, bool is_trial) const;
+  inline std::shared_ptr<const Map> map_for_id(uint32_t id) const {
+    return this->maps.at(id);
+  }
+  inline std::shared_ptr<const Map> map_for_name(const std::string& name) const {
+    return this->maps_by_name.at(name);
+  }
+  inline const std::map<uint32_t, std::shared_ptr<const Map>>& all_maps() const {
+    return this->maps;
+  }
+
+  inline std::shared_ptr<const Category> category_for_id(uint32_t id) const {
+    return this->categories.at(id);
+  }
+  inline const std::map<uint32_t, std::shared_ptr<const Category>>& all_categories() const {
+    return this->categories;
+  }
 
 private:
-  // The compressed map lists are generated on demand from the maps map below
-  mutable std::vector<std::array<std::string, 4>> compressed_map_lists;
-  std::map<uint32_t, std::shared_ptr<Map>> maps;
+  // The compressed map lists are generated on demand from the maps map below.
+  // THey are indexed as [language][num_players]
+  mutable std::vector<std::array<std::string, 4>> compressed_map_lists_trial;
+  mutable std::vector<std::array<std::string, 4>> compressed_map_lists_final;
+
+  std::map<uint32_t, std::shared_ptr<const Category>> categories;
+  std::map<uint32_t, std::shared_ptr<const Map>> maps;
   std::unordered_map<std::string, std::shared_ptr<Map>> maps_by_name;
 };
 
+class MapCategoryIndex {
+public:
+  explicit MapCategoryIndex(const std::string& directory);
+
+  inline std::shared_ptr<const MapIndex> get(uint32_t id) const {
+    return this->indexes.at(id);
+  }
+  inline const std::map<uint32_t, std::shared_ptr<const MapIndex>>& all() const {
+    return this->indexes;
+  }
+
+private:
+  std::map<uint32_t, std::shared_ptr<const MapIndex>> indexes;
+};
+
 class COMDeckIndex {
 public:
   COMDeckIndex(const std::string& filename);

src/Episode3/DeckState.cc

@@ -320,7 +320,7 @@ void DeckState::print(FILE* stream, std::shared_ptr<const CardIndex> card_index)
       }
     }
     if (ce) {
-      string name = ce->def.en_name.decode(1);
+      string name = ce->def.en_name.decode(Language::ENGLISH);
       phosg::fwrite_fmt(stream, "  ({:02}) index={:02X} ref=@{:04X} card_id=#{:04X} \"{}\" {}\n",
           z, e.deck_index, this->card_refs[z], e.card_id, name, name_for_card_state(e.state));
     } else {

src/Episode3/Server.cc

@@ -231,6 +231,11 @@ int8_t Server::get_winner_team_id() const {
 
 void Server::send(const void* data, size_t size, uint8_t command, bool enable_masking) const {
   // Note: This function is (obviously) not part of the original implementation.
+
+  if (this->options.output_queue) {
+    this->options.output_queue->emplace_back(reinterpret_cast<const char*>(data), size);
+  }
+
   if (this->has_lobby) {
     auto l = this->lobby.lock();
     if (!l) {
@@ -274,22 +279,22 @@ void Server::send_6xB4x46() const {
   // NTE doesn't have the date_str2 field, but we send it anyway to make
   // debugging easier.
   G_ServerVersionStrings_Ep3_6xB4x46 cmd;
-  cmd.version_signature.encode(this->options.is_nte() ? VERSION_SIGNATURE_NTE : VERSION_SIGNATURE, 1);
-  cmd.date_str1.encode(std::format("Card definitions: {:016X}", this->options.card_index->definitions_hash()), 1);
+  cmd.version_signature.encode(this->options.is_nte() ? VERSION_SIGNATURE_NTE : VERSION_SIGNATURE, Language::ENGLISH);
+  cmd.date_str1.encode(std::format("Card definitions: {:016X}", this->options.card_index->definitions_hash()), Language::ENGLISH);
   string build_date = phosg::format_time(BUILD_TIMESTAMP);
-  cmd.date_str2.encode(std::format("newserv {} compiled at {}", GIT_REVISION_HASH, build_date), 1);
+  cmd.date_str2.encode(std::format("newserv {} compiled at {}", GIT_REVISION_HASH, build_date), Language::ENGLISH);
   this->send(cmd);
 }
 
-string Server::prepare_6xB6x41_map_definition(shared_ptr<const MapIndex::Map> map, uint8_t language, bool is_nte) {
+string Server::prepare_6xB6x41_map_definition(shared_ptr<const MapIndex::Map> map, Language language, bool is_nte) {
   auto vm = map->version(language);
 
   const auto& compressed = vm->compressed(is_nte);
 
   phosg::StringWriter w;
-  uint32_t subcommand_size = (compressed.size() + sizeof(G_MapData_Ep3_6xB6x41) + 3) & (~3);
-  w.put<G_MapData_Ep3_6xB6x41>({{{{0xB6, 0, 0}, subcommand_size}, 0x41, {}}, vm->map->map_number.load(), compressed.size(), 0});
-  w.write(compressed);
+  uint32_t subcommand_size = (compressed->size() + sizeof(G_MapData_Ep3_6xB6x41) + 3) & (~3);
+  w.put<G_MapData_Ep3_6xB6x41>({{{{0xB6, 0, 0}, subcommand_size}, 0x41, {}}, vm->map->map_number.load(), compressed->size(), 0});
+  w.write(*compressed);
   return std::move(w.str());
 }
 
@@ -306,7 +311,7 @@ void Server::send_commands_for_joining_spectator(std::shared_ptr<Channel> ch) co
 
   if (this->last_chosen_map) {
     string data = this->prepare_6xB6x41_map_definition(this->last_chosen_map, ch->language, this->options.is_nte());
-    this->log().info_f("Sending {} version of map {:08X}", char_for_language_code(ch->language), this->last_chosen_map->map_number);
+    this->log().info_f("Sending {} version of map {:08X}", name_for_language(ch->language), this->last_chosen_map->map_number);
     ch->send(0x6C, 0x00, data);
   }
 
@@ -592,7 +597,14 @@ void Server::force_replace_assist_card(uint8_t client_id, uint16_t card_id) {
   if (!ps) {
     throw runtime_error("player does not exist");
   }
-  ps->replace_assist_card_by_id(card_id);
+  if (card_id == 0xFFFF) {
+    ps->discard_set_assist_card();
+    this->check_for_destroyed_cards_and_send_6xB4x05_6xB4x02();
+    this->check_for_battle_end();
+
+  } else {
+    ps->replace_assist_card_by_id(card_id);
+  }
 }
 
 void Server::force_destroy_field_character(uint8_t client_id, size_t visible_index) {
@@ -2130,7 +2142,7 @@ void Server::handle_CAx13_update_map_during_setup_t(shared_ptr<Client> c, const
     // in the case of NTE, no values at all, since the Rules structure is
     // smaller). So, use the values from the last chosen map if applicable, or
     // the values from the $dicerange command if available.
-    uint8_t language = c ? c->language() : 1;
+    Language language = c ? c->language() : Language::ENGLISH;
     const Rules* map_rules = this->last_chosen_map ? &this->last_chosen_map->version(language)->map->default_rules : nullptr;
     auto& server_rules = this->map_and_rules->rules;
     // NTE can specify the DEF dice value range in its Rules struct, so we use
@@ -2519,8 +2531,8 @@ void Server::handle_CAx40_map_list_request(shared_ptr<Client> sender_c, const st
   }
 
   size_t num_players = l ? l->count_clients() : 1;
-  uint8_t language = sender_c ? sender_c->language() : 1;
-  const auto& list_data = this->options.map_index->get_compressed_list(num_players, language);
+  Language language = sender_c ? sender_c->language() : Language::ENGLISH;
+  const auto& list_data = this->options.map_index->get_compressed_list(num_players, language, this->options.is_nte());
 
   phosg::StringWriter w;
   uint32_t subcommand_size = (list_data.size() + sizeof(G_MapList_Ep3_6xB6x40) + 3) & (~3);
@@ -2546,15 +2558,16 @@ void Server::send_6xB6x41_to_all_clients() const {
       if (!c) {
         return;
       }
-      if (map_commands_by_language.size() <= c->language()) {
-        map_commands_by_language.resize(c->language() + 1);
+      size_t lang_index = static_cast<size_t>(c->language());
+      if (map_commands_by_language.size() <= lang_index) {
+        map_commands_by_language.resize(lang_index + 1);
       }
-      if (map_commands_by_language[c->language()].empty()) {
-        map_commands_by_language[c->language()] = this->prepare_6xB6x41_map_definition(
+      if (map_commands_by_language[lang_index].empty()) {
+        map_commands_by_language[lang_index] = this->prepare_6xB6x41_map_definition(
             this->last_chosen_map, c->language(), this->options.is_nte());
       }
-      this->log().info_f("Sending {} version of map {:08X}", char_for_language_code(c->language()), this->last_chosen_map->map_number);
-      send_command(c, 0x6C, 0x00, map_commands_by_language[c->language()]);
+      this->log().info_f("Sending {} version of map {:08X}", name_for_language(c->language()), this->last_chosen_map->map_number);
+      send_command(c, 0x6C, 0x00, map_commands_by_language[lang_index]);
     };
     for (const auto& c : l->clients) {
       send_to_client(c);
@@ -2571,15 +2584,14 @@ void Server::send_6xB6x41_to_all_clients() const {
       // in the playback lobby
       for (string& data : map_commands_by_language) {
         if (!data.empty()) {
-          this->battle_record->add_command(
-              BattleRecord::Event::Type::BATTLE_COMMAND, std::move(data));
+          this->battle_record->add_command(BattleRecord::Event::Type::BATTLE_COMMAND, std::move(data));
           break;
         }
       }
     }
 
   } else {
-    auto out_data = this->prepare_6xB6x41_map_definition(this->last_chosen_map, 1, false);
+    auto out_data = this->prepare_6xB6x41_map_definition(this->last_chosen_map, Language::ENGLISH, false);
     this->send(out_data.data(), out_data.size(), 0x6C, false);
   }
 }
@@ -2588,7 +2600,7 @@ void Server::handle_CAx41_map_request(shared_ptr<Client>, const string& data) {
   const auto& cmd = check_size_t<G_MapDataRequest_Ep3_CAx41>(data);
   this->send_debug_command_received_message(cmd.header.subsubcommand, "MAP DATA");
   if (!this->options.tournament || (this->options.tournament->get_map()->map_number == cmd.map_number)) {
-    this->last_chosen_map = this->options.map_index->for_number(cmd.map_number);
+    this->last_chosen_map = this->options.map_index->map_for_id(cmd.map_number);
     this->send_6xB6x41_to_all_clients();
   }
 }

src/Episode3/Server.hh

@@ -76,6 +76,7 @@ public:
     std::shared_ptr<RandomGenerator> rand_crypt;
     std::shared_ptr<const Tournament> tournament;
     std::array<std::vector<uint16_t>, 5> trap_card_ids;
+    std::shared_ptr<std::deque<std::string>> output_queue; // For replay testing
 
     inline bool is_nte() const {
       return (this->behavior_flags & BehaviorFlag::IS_TRIAL_EDITION);
@@ -265,7 +266,7 @@ public:
 
   G_UpdateDecks_Ep3_6xB4x07 prepare_6xB4x07_decks_update() const;
   G_SetPlayerNames_Ep3_6xB4x1C prepare_6xB4x1C_names_update() const;
-  static std::string prepare_6xB6x41_map_definition(std::shared_ptr<const MapIndex::Map> map, uint8_t language, bool is_nte);
+  static std::string prepare_6xB6x41_map_definition(std::shared_ptr<const MapIndex::Map> map, Language language, bool is_nte);
   void send_6xB6x41_to_all_clients() const;
   G_SetTrapTileLocations_Ep3_6xB4x50 prepare_6xB4x50_trap_tile_locations() const;
 

src/Episode3/Tournament.cc

@@ -18,7 +18,7 @@ Tournament::PlayerEntry::PlayerEntry(uint32_t account_id, const string& player_n
 Tournament::PlayerEntry::PlayerEntry(shared_ptr<Client> c)
     : account_id(c->login->account->account_id),
       client(c),
-      player_name(c->character()->disp.name.decode(c->language())) {}
+      player_name(c->character_file()->disp.name.decode(c->language())) {}
 
 Tournament::PlayerEntry::PlayerEntry(
     shared_ptr<const COMDeckDefinition> com_deck)
@@ -357,7 +357,7 @@ void Tournament::init() {
   bool is_registration_complete;
   if (!this->source_json.is_null()) {
     this->name = this->source_json.get_string("name");
-    this->map = this->map_index->for_number(this->source_json.get_int("map_number"));
+    this->map = this->map_index->map_for_id(this->source_json.get_int("map_number"));
     this->rules = Rules(this->source_json.at("rules"));
     this->flags = this->source_json.get_int("flags", 0x02);
     if (this->source_json.get_bool("is_2v2", false)) {
@@ -737,7 +737,7 @@ string Tournament::bracket_str() const {
     }
   };
 
-  auto en_vm = this->map->version(1);
+  auto en_vm = this->map->version(Language::ENGLISH);
   if (en_vm) {
     string map_name = en_vm->map->name.decode(en_vm->language);
     ret += std::format("  Map: {:08X} ({})\n", this->map->map_number, map_name);

src/FunctionCompiler.cc

@@ -96,11 +96,9 @@ string CompiledFunctionCode::generate_client_command(
     size_t suffix_size,
     uint32_t override_relocations_offset) const {
   if (this->arch == Architecture::POWERPC) {
-    return this->generate_client_command_t<true>(
-        label_writes, suffix_data, suffix_size, override_relocations_offset);
+    return this->generate_client_command_t<true>(label_writes, suffix_data, suffix_size, override_relocations_offset);
   } else if ((this->arch == Architecture::X86) || (this->arch == Architecture::SH4)) {
-    return this->generate_client_command_t<false>(
-        label_writes, suffix_data, suffix_size, override_relocations_offset);
+    return this->generate_client_command_t<false>(label_writes, suffix_data, suffix_size, override_relocations_offset);
   } else {
     throw logic_error("invalid architecture");
   }
@@ -110,18 +108,118 @@ bool CompiledFunctionCode::is_big_endian() const {
   return this->arch == Architecture::POWERPC;
 }
 
-shared_ptr<CompiledFunctionCode> compile_function_code(
+static unordered_map<uint32_t, std::string> preprocess_function_code(const std::string& text) {
+  auto parse_specific_version_list = +[](std::string&& text) -> vector<uint32_t> {
+    phosg::strip_whitespace(text);
+    vector<uint32_t> ret;
+    for (auto& vers_token : phosg::split(text, ' ')) {
+      phosg::strip_whitespace(vers_token);
+      if (vers_token.empty()) {
+        continue;
+      }
+      if (vers_token.size() != 4) {
+        throw std::runtime_error("invalid specific_version: " + vers_token);
+      }
+      ret.emplace_back(*reinterpret_cast<const be_uint32_t*>(vers_token.data()));
+    }
+    return ret;
+  };
+
+  // Find a .versions directive and populate specific_versions
+  vector<uint32_t> specific_versions;
+  auto lines = phosg::split(text, '\n');
+  for (auto& line : lines) {
+    if (line.starts_with(".versions ")) {
+      if (!specific_versions.empty()) {
+        throw std::runtime_error("multiple .versions directives in file");
+      }
+      specific_versions = parse_specific_version_list(line.substr(10));
+      if (specific_versions.empty()) {
+        throw std::runtime_error(".versions directive does not specify any versions");
+      }
+      line.clear();
+    }
+  }
+
+  // If there's no .versions directive, just return the text as-is
+  if (specific_versions.empty()) {
+    return {{0, std::move(text)}};
+  }
+
+  vector<deque<string>> version_lines;
+  version_lines.resize(specific_versions.size());
+
+  size_t line_num = 1;
+  vector<uint32_t> current_only_versions;
+  unordered_set<uint32_t> current_only_versions_set;
+  for (auto& line : lines) {
+    phosg::strip_whitespace(line);
+    if (line.starts_with(".only_versions ")) {
+      current_only_versions = parse_specific_version_list(line.substr(15));
+      current_only_versions_set.clear();
+      for (uint32_t specific_version : current_only_versions) {
+        current_only_versions_set.emplace(specific_version);
+      }
+
+    } else if (line == ".all_versions") {
+      current_only_versions.clear();
+      current_only_versions_set.clear();
+
+    } else {
+      size_t vers_offset = line.find("<VERS ");
+      if (vers_offset == string::npos) {
+        for (size_t vers_index = 0; vers_index < specific_versions.size(); vers_index++) {
+          if (current_only_versions.empty() || current_only_versions_set.count(specific_versions[vers_index])) {
+            version_lines[vers_index].emplace_back(line);
+          } else {
+            version_lines[vers_index].emplace_back("");
+          }
+        }
+
+      } else {
+        size_t token_index = 0;
+        for (size_t vers_index = 0; vers_index < specific_versions.size(); vers_index++) {
+          if (current_only_versions.empty() || current_only_versions_set.count(specific_versions[vers_index])) {
+            string version_line = line;
+            size_t vers_offset = line.find("<VERS ");
+            while (vers_offset != string::npos) {
+              size_t end_offset = version_line.find('>', vers_offset + 6);
+              if (end_offset == string::npos) {
+                throw runtime_error(std::format("(line {}) unterminated <VERS> replacement", line_num));
+              }
+              auto tokens = phosg::split(version_line.substr(vers_offset + 6, end_offset - vers_offset - 6), ' ');
+              if (tokens.size() <= token_index) {
+                throw runtime_error(std::format("(line {}) invalid <VERS> replacement", line_num));
+              }
+              version_line = version_line.substr(0, vers_offset) + tokens.at(token_index) + version_line.substr(end_offset + 1);
+              vers_offset = version_line.find("<VERS ");
+            }
+            version_lines[vers_index].emplace_back(version_line);
+            token_index++;
+          } else {
+            version_lines[vers_index].emplace_back("");
+          }
+        }
+      }
+    }
+
+    line_num++;
+  }
+
+  unordered_map<uint32_t, string> ret;
+  for (size_t z = 0; z < specific_versions.size(); z++) {
+    ret.emplace(specific_versions[z], phosg::join(version_lines.at(z), "\n"));
+  }
+  return ret;
+}
+
+static vector<shared_ptr<CompiledFunctionCode>> compile_function_code(
     CompiledFunctionCode::Architecture arch,
     const string& function_directory,
     const string& system_directory,
     const string& name,
-    const string& text) {
-  auto ret = make_shared<CompiledFunctionCode>();
-  ret->arch = arch;
-  ret->short_name = name;
-  ret->index = 0;
-  ret->hide_from_patches_menu = false;
-
+    const string& text,
+    bool raise_on_any_failure) {
   unordered_set<string> get_include_stack;
   function<string(const string&)> get_include = [&](const string& name) -> string {
     const char* arch_name_token;
@@ -177,83 +275,94 @@ shared_ptr<CompiledFunctionCode> compile_function_code(
     throw runtime_error("data not found for include: " + name + " (from " + asm_filename + " or " + bin_filename + ")");
   };
 
-  ResourceDASM::EmulatorBase::AssembleResult assembled;
-  if (arch == CompiledFunctionCode::Architecture::POWERPC) {
-    assembled = ResourceDASM::PPC32Emulator::assemble(text, get_include);
-  } else if (arch == CompiledFunctionCode::Architecture::X86) {
-    assembled = ResourceDASM::X86Emulator::assemble(text, get_include);
-  } else if (arch == CompiledFunctionCode::Architecture::SH4) {
-    assembled = ResourceDASM::SH4Emulator::assemble(text, get_include);
-  } else {
-    throw runtime_error("invalid architecture");
-  }
-  ret->code = std::move(assembled.code);
-  ret->label_offsets = std::move(assembled.label_offsets);
-  for (const auto& it : assembled.metadata_keys) {
-    if (it.first == "hide_from_patches_menu") {
-      ret->hide_from_patches_menu = true;
-    } else if (it.first == "index") {
-      if (it.second.size() != 1) {
-        throw runtime_error("invalid index value in .meta directive");
+  auto version_texts = preprocess_function_code(text);
+
+  vector<shared_ptr<CompiledFunctionCode>> ret;
+  for (const auto& [specific_version, version_text] : version_texts) {
+    try {
+      ResourceDASM::EmulatorBase::AssembleResult assembled;
+      if (arch == CompiledFunctionCode::Architecture::POWERPC) {
+        assembled = ResourceDASM::PPC32Emulator::assemble(version_text, get_include);
+      } else if (arch == CompiledFunctionCode::Architecture::X86) {
+        assembled = ResourceDASM::X86Emulator::assemble(version_text, get_include);
+      } else if (arch == CompiledFunctionCode::Architecture::SH4) {
+        assembled = ResourceDASM::SH4Emulator::assemble(version_text, get_include);
+      } else {
+        throw runtime_error("invalid architecture");
       }
-      ret->index = it.second[0];
-    } else if (it.first == "name") {
-      ret->long_name = it.second;
-    } else if (it.first == "description") {
-      ret->description = it.second;
-    } else {
-      throw runtime_error("unknown metadata key: " + it.first);
-    }
-  }
 
-  set<uint32_t> reloc_indexes;
-  for (const auto& it : ret->label_offsets) {
-    if (it.first.starts_with("reloc")) {
-      reloc_indexes.emplace(it.second / 4);
-    }
-  }
+      auto compiled = ret.emplace_back(make_shared<CompiledFunctionCode>());
+      compiled->arch = arch;
+      compiled->short_name = name;
+      compiled->specific_version = specific_version;
+      compiled->code = std::move(assembled.code);
+      compiled->label_offsets = std::move(assembled.label_offsets);
+      for (const auto& it : assembled.metadata_keys) {
+        if (it.first == "hide_from_patches_menu") {
+          compiled->hide_from_patches_menu = true;
+        } else if (it.first == "name") {
+          compiled->long_name = it.second;
+        } else if (it.first == "description") {
+          compiled->description = it.second;
+        } else if (it.first == "client_flag") {
+          compiled->client_flag = stoull(it.second, nullptr, 0);
+        } else {
+          throw runtime_error("unknown metadata key: " + it.first);
+        }
+      }
 
-  try {
-    ret->entrypoint_offset_offset = ret->label_offsets.at("entry_ptr");
-  } catch (const out_of_range&) {
-    throw runtime_error("code does not contain entry_ptr label");
-  }
+      set<uint32_t> reloc_indexes;
+      for (const auto& it : compiled->label_offsets) {
+        if (it.first.starts_with("reloc")) {
+          reloc_indexes.emplace(it.second / 4);
+        }
+      }
 
-  uint32_t prev_index = 0;
-  for (const auto& it : reloc_indexes) {
-    uint32_t delta = it - prev_index;
-    if (delta > 0xFFFF) {
-      throw runtime_error("relocation delta too far away");
+      try {
+        compiled->entrypoint_offset_offset = compiled->label_offsets.at("entry_ptr");
+      } catch (const out_of_range&) {
+        throw runtime_error("code does not contain entry_ptr label");
+      }
+
+      uint32_t prev_index = 0;
+      for (const auto& it : reloc_indexes) {
+        uint32_t delta = it - prev_index;
+        if (delta > 0xFFFF) {
+          throw runtime_error("relocation delta too far away");
+        }
+        compiled->relocation_deltas.emplace_back(delta);
+        prev_index = it;
+      }
+
+    } catch (const exception& e) {
+      string version_str = specific_version ? (" (" + str_for_specific_version(specific_version) + ")") : "";
+      if (raise_on_any_failure) {
+        throw;
+      }
+      function_compiler_log.warning_f("Failed to compile function {}{}: {}", name, version_str, e.what());
     }
-    ret->relocation_deltas.emplace_back(delta);
-    prev_index = it;
   }
 
   return ret;
 }
 
-FunctionCodeIndex::FunctionCodeIndex(const string& directory) {
+FunctionCodeIndex::FunctionCodeIndex(const string& directory, bool raise_on_any_failure) {
   string system_dir_path = directory.ends_with("/") ? (directory + "System") : (directory + "/System");
 
   uint32_t next_menu_item_id = 1;
   for (const auto& item : std::filesystem::directory_iterator(directory)) {
     string subdir_name = item.path().filename().string();
     string subdir_path = directory.ends_with("/") ? (directory + subdir_name) : (directory + "/" + subdir_name);
-    if (!std::filesystem::is_directory(subdir_path)) {
-      function_compiler_log.warning_f("Skipping {} (not a directory)", subdir_name);
-      continue;
-    }
 
-    for (const auto& item : std::filesystem::directory_iterator(subdir_path)) {
-      string filename = item.path().filename().string();
+    auto add_file = [&](string filename) -> void {
       try {
         if (!filename.ends_with(".s")) {
-          continue;
+          return;
         }
 
         string name = filename.substr(0, filename.size() - 2);
         if (name.ends_with(".inc")) {
-          continue;
+          return;
         }
 
         bool is_patch = name.ends_with(".patch");
@@ -299,50 +408,63 @@ FunctionCodeIndex::FunctionCodeIndex(const string& directory) {
 
         string path = subdir_path + "/" + filename;
         string text = phosg::load_file(path);
-        auto code = compile_function_code(arch, subdir_path, system_dir_path, name, text);
-        if (code->index != 0) {
-          if (!this->index_to_function.emplace(code->index, code).second) {
-            throw runtime_error(std::format(
-                "duplicate function index: {:08X}", code->index));
+        for (auto code : compile_function_code(arch, subdir_path, system_dir_path, name, text, raise_on_any_failure)) {
+          if (code->specific_version == 0) {
+            code->specific_version = specific_version;
+          }
+          code->source_path = path;
+          code->short_name = short_name;
+          this->name_to_function.emplace(name, code);
+          if (is_patch) {
+            code->menu_item_id = next_menu_item_id++;
+            this->menu_item_id_and_specific_version_to_patch_function.emplace(
+                static_cast<uint64_t>(code->menu_item_id) << 32 | code->specific_version, code);
+            this->name_and_specific_version_to_patch_function.emplace(
+                std::format("{}-{:08X}", code->short_name, code->specific_version), code);
           }
-        }
-        code->specific_version = specific_version;
-        code->source_path = path;
-        code->short_name = short_name;
-        this->name_to_function.emplace(name, code);
-        if (is_patch) {
-          code->menu_item_id = next_menu_item_id++;
-          this->menu_item_id_and_specific_version_to_patch_function.emplace(
-              static_cast<uint64_t>(code->menu_item_id) << 32 | specific_version, code);
-          this->name_and_specific_version_to_patch_function.emplace(
-              std::format("{}-{:08X}", short_name, specific_version), code);
-        }
 
-        string index_prefix = code->index ? std::format("{:02X} => ", code->index) : "";
-        string patch_prefix = is_patch ? std::format("[{:08X}/{:08X}] ", code->menu_item_id, code->specific_version) : "";
-        function_compiler_log.debug_f("Compiled function {}{}{} ({})",
-            index_prefix, patch_prefix, name, name_for_architecture(code->arch));
+          string patch_prefix = is_patch ? std::format("[{:08X}] ", code->menu_item_id) : "";
+          function_compiler_log.debug_f("Compiled function {}{} ({}; {})",
+              patch_prefix, name, str_for_specific_version(code->specific_version), name_for_architecture(code->arch));
+        }
 
       } catch (const exception& e) {
+        if (raise_on_any_failure) {
+          throw runtime_error(format("({}) {}", filename, e.what()));
+        }
         function_compiler_log.warning_f("Failed to compile function {}: {}", filename, e.what());
       }
+    };
+
+    if (std::filesystem::is_regular_file(subdir_path)) {
+      add_file(subdir_path);
+    } else if (std::filesystem::is_directory(subdir_path)) {
+      for (const auto& item : std::filesystem::directory_iterator(subdir_path)) {
+        string filename = item.path().filename().string();
+        add_file(filename);
+      }
+    } else {
+      function_compiler_log.warning_f("Skipping {} (unknown file type)", subdir_name);
+      continue;
     }
   }
 }
 
 shared_ptr<const Menu> FunctionCodeIndex::patch_switches_menu(
-    uint32_t specific_version, const std::unordered_set<std::string>& auto_patches_enabled) const {
+    uint32_t specific_version,
+    const std::unordered_set<std::string>& server_auto_patches_enabled,
+    const std::unordered_set<std::string>& client_auto_patches_enabled) const {
   auto suffix = std::format("-{:08X}", specific_version);
 
   auto ret = make_shared<Menu>(MenuID::PATCH_SWITCHES, "Patches");
   ret->items.emplace_back(PatchesMenuItemID::GO_BACK, "Go back", "Return to the\nmain menu", 0);
   for (const auto& it : this->name_and_specific_version_to_patch_function) {
     const auto& fn = it.second;
-    if (fn->hide_from_patches_menu || !it.first.ends_with(suffix)) {
+    if (fn->hide_from_patches_menu || !it.first.ends_with(suffix) || server_auto_patches_enabled.count(fn->short_name)) {
       continue;
     }
     string name;
-    name.push_back(auto_patches_enabled.count(fn->short_name) ? '*' : '-');
+    name.push_back(client_auto_patches_enabled.count(fn->short_name) ? '*' : '-');
     name += fn->long_name.empty() ? fn->short_name : fn->long_name;
     ret->items.emplace_back(fn->menu_item_id, name, fn->description, MenuItem::Flag::REQUIRES_SEND_FUNCTION_CALL_RUNS_CODE);
   }

src/FunctionCompiler.hh

@@ -25,15 +25,15 @@ struct CompiledFunctionCode {
   std::string code;
   std::vector<uint16_t> relocation_deltas;
   std::unordered_map<std::string, uint32_t> label_offsets;
-  uint32_t entrypoint_offset_offset;
+  uint32_t entrypoint_offset_offset = 0;
   std::string source_path; // Path to source file from newserv root
   std::string short_name; // Based on filename
   std::string long_name; // From .meta name directive
   std::string description; // From .meta description directive
-  uint8_t index; // 0 = unused (not registered in index_to_function)
-  uint32_t menu_item_id;
-  bool hide_from_patches_menu;
-  uint32_t specific_version;
+  uint64_t client_flag = 0; // From .meta client_flag directive
+  uint32_t menu_item_id = 0;
+  bool hide_from_patches_menu = false;
+  uint32_t specific_version = 0; // 0 = not a client-selectable patch
 
   bool is_big_endian() const;
 
@@ -52,15 +52,9 @@ struct CompiledFunctionCode {
 
 const char* name_for_architecture(CompiledFunctionCode::Architecture arch);
 
-std::shared_ptr<CompiledFunctionCode> compile_function_code(
-    CompiledFunctionCode::Architecture arch,
-    const std::string& directory,
-    const std::string& name,
-    const std::string& text);
-
 struct FunctionCodeIndex {
   FunctionCodeIndex() = default;
-  explicit FunctionCodeIndex(const std::string& directory);
+  FunctionCodeIndex(const std::string& directory, bool raise_on_any_failure);
 
   std::unordered_map<std::string, std::shared_ptr<CompiledFunctionCode>> name_to_function;
   std::unordered_map<uint8_t, std::shared_ptr<CompiledFunctionCode>> index_to_function;
@@ -68,7 +62,10 @@ struct FunctionCodeIndex {
   // Key here is e.g. "PATCHNAME-SPECIFICVERSION", with the latter in hex
   std::map<std::string, std::shared_ptr<CompiledFunctionCode>> name_and_specific_version_to_patch_function;
 
-  std::shared_ptr<const Menu> patch_switches_menu(uint32_t specific_version, const std::unordered_set<std::string>& auto_patches_enabled) const;
+  std::shared_ptr<const Menu> patch_switches_menu(
+      uint32_t specific_version,
+      const std::unordered_set<std::string>& server_auto_patches_enabled,
+      const std::unordered_set<std::string>& client_auto_patches_enabled) const;
   bool patch_menu_empty(uint32_t specific_version) const;
 
   std::shared_ptr<const CompiledFunctionCode> get_patch(const std::string& name, uint32_t specific_version) const;

src/GameServer.cc

@@ -96,7 +96,7 @@ vector<shared_ptr<Client>> GameServer::get_clients_by_identifier(const string& i
       continue;
     }
 
-    auto p = c->character(false, false);
+    auto p = c->character_file(false, false);
     if (p && p->disp.name.eq(ident, p->inventory.language)) {
       results.emplace_back(c);
       continue;
@@ -118,7 +118,9 @@ vector<shared_ptr<Client>> GameServer::get_clients_by_identifier(const string& i
 shared_ptr<Client> GameServer::create_client(shared_ptr<GameServerSocket> listen_sock, asio::ip::tcp::socket&& client_sock) {
   uint32_t addr = ipv4_addr_for_asio_addr(client_sock.remote_endpoint().address());
   if (this->state->banned_ipv4_ranges->check(addr)) {
-    client_sock.close();
+    if (client_sock.is_open()) {
+      client_sock.close();
+    }
     return nullptr;
   }
 
@@ -126,7 +128,7 @@ shared_ptr<Client> GameServer::create_client(shared_ptr<GameServerSocket> listen
       this->io_context,
       make_unique<asio::ip::tcp::socket>(std::move(client_sock)),
       listen_sock->version,
-      1,
+      Language::ENGLISH,
       "",
       phosg::TerminalFormat::FG_YELLOW,
       phosg::TerminalFormat::FG_GREEN);

src/HTTPServer.hh

@@ -4,6 +4,7 @@
 
 #include <memory>
 #include <string>
+#include <variant>
 
 #include "AsyncHTTPServer.hh"
 #include "ServerState.hh"
@@ -20,27 +21,13 @@ public:
   asio::awaitable<void> send_rare_drop_notification(std::shared_ptr<const phosg::JSON> message);
 
 protected:
+  struct RawResponse {
+    std::string content_type;
+    std::string data;
+  };
   std::shared_ptr<ServerState> state;
   std::unordered_set<std::shared_ptr<HTTPClient>> rare_drop_subscribers;
-
-  std::shared_ptr<phosg::JSON> generate_server_version() const;
-  std::shared_ptr<phosg::JSON> generate_account_json(std::shared_ptr<const Account> a) const;
-  std::shared_ptr<phosg::JSON> generate_client_json(
-      std::shared_ptr<const Client> c, std::shared_ptr<const ItemNameIndex> item_name_index) const;
-  std::shared_ptr<phosg::JSON> generate_lobby_json(
-      std::shared_ptr<const Lobby> l, std::shared_ptr<const ItemNameIndex> item_name_index) const;
-  std::shared_ptr<phosg::JSON> generate_accounts_json() const;
-  std::shared_ptr<phosg::JSON> generate_clients_json() const;
-  std::shared_ptr<phosg::JSON> generate_server_info_json() const;
-  std::shared_ptr<phosg::JSON> generate_lobbies_json() const;
-  std::shared_ptr<phosg::JSON> generate_summary_json() const;
-  std::shared_ptr<phosg::JSON> generate_all_json() const;
-
-  asio::awaitable<std::shared_ptr<phosg::JSON>> generate_ep3_cards_json(bool trial) const;
-  asio::awaitable<std::shared_ptr<phosg::JSON>> generate_common_tables_json() const;
-  std::shared_ptr<phosg::JSON> generate_rare_table_list_json() const;
-  asio::awaitable<std::shared_ptr<phosg::JSON>> generate_rare_table_json(const std::string& table_name) const;
-  asio::awaitable<std::shared_ptr<phosg::JSON>> generate_quest_list_json(std::shared_ptr<const QuestIndex> q);
+  HTTPRouter<std::variant<RawResponse, std::shared_ptr<const phosg::JSON>>> router;
 
   void require_GET(const HTTPRequest& req);
   phosg::JSON require_POST(const HTTPRequest& req);

src/IPStackSimulator.cc

@@ -129,10 +129,7 @@ void IPSSClient::TCPConnection::linearize_outbound_data(size_t size) {
 }
 
 IPSSClient::IPSSClient(
-    shared_ptr<IPStackSimulator> sim,
-    uint64_t network_id,
-    VirtualNetworkProtocol protocol,
-    asio::ip::tcp::socket&& sock)
+    shared_ptr<IPStackSimulator> sim, uint64_t network_id, VirtualNetworkProtocol protocol, asio::ip::tcp::socket&& sock)
     : io_context(sim->get_io_context()),
       sim(sim),
       network_id(network_id),
@@ -154,7 +151,9 @@ void IPSSClient::reschedule_idle_timeout() {
   this->idle_timeout_timer.async_wait([this, sim](std::error_code ec) {
     if (!ec) {
       sim->log.info_f("Idle timeout expired on N-{:X}", this->network_id);
-      this->sock.close();
+      if (this->sock.is_open()) {
+        this->sock.close();
+      }
     }
   });
 }
@@ -164,7 +163,7 @@ IPSSChannel::IPSSChannel(
     std::weak_ptr<IPSSClient> ipss_client,
     std::weak_ptr<IPSSClient::TCPConnection> tcp_conn,
     Version version,
-    uint8_t language,
+    Language language,
     const std::string& name,
     phosg::TerminalFormat terminal_send_color,
     phosg::TerminalFormat terminal_recv_color)
@@ -177,8 +176,7 @@ IPSSChannel::IPSSChannel(
 std::string IPSSChannel::default_name() const {
   auto ipc = this->ipss_client.lock();
   if (ipc) {
-    string addr_str = str_for_endpoint(ipc->sock.remote_endpoint());
-    return std::format("ipss:N-{}:{}", ipc->network_id, addr_str);
+    return std::format("ipss:N-{}:{}", ipc->network_id, str_for_endpoint(ipc->sock.remote_endpoint()));
   } else {
     return std::format("ipss:N-{}:__unknown_address__", ipc->network_id);
   }
@@ -211,9 +209,7 @@ void IPSSChannel::add_inbound_data(const void* data, size_t size) {
     data = reinterpret_cast<const uint8_t*>(data) + direct_size;
     size -= direct_size;
     this->recv_buf_size -= direct_size;
-    this->recv_buf = this->recv_buf_size
-        ? reinterpret_cast<uint8_t*>(this->recv_buf) + direct_size
-        : nullptr;
+    this->recv_buf = this->recv_buf_size ? (reinterpret_cast<uint8_t*>(this->recv_buf) + direct_size) : nullptr;
   }
 
   // If there is still data left after the above, add it to the pending inbound
@@ -336,6 +332,7 @@ uint64_t IPStackSimulator::tcp_conn_key_for_client_frame(const FrameInfo& fi) {
 string IPStackSimulator::str_for_ipv4_netloc(uint32_t addr, uint16_t port) {
   be_uint32_t be_addr = addr;
   char addr_str[INET_ADDRSTRLEN];
+  memset(addr_str, 0, sizeof(addr_str));
   if (!inet_ntop(AF_INET, &be_addr, addr_str, INET_ADDRSTRLEN)) {
     return std::format("<UNKNOWN>:{}", port);
   } else {
@@ -354,11 +351,12 @@ string IPStackSimulator::str_for_tcp_connection(
 asio::awaitable<void> IPStackSimulator::send_ethernet_tapserver_frame(
     shared_ptr<IPSSClient> c, FrameInfo::Protocol proto, const void* data, size_t size) const {
 
-  struct {
+  struct TapServerEthernetHeader {
     phosg::le_uint16_t frame_size;
     EthernetHeader ether;
-  } header;
-  static_assert(sizeof(header) == 0x10, "Ethernet tapserver header size is incorrect");
+  } __attribute__((packed));
+  static_assert(sizeof(TapServerEthernetHeader) == 0x10, "Ethernet tapserver header size is incorrect");
+  TapServerEthernetHeader header;
 
   header.ether.dest_mac = c->mac_addr;
   header.ether.src_mac = this->host_mac_address_bytes;
@@ -378,9 +376,7 @@ asio::awaitable<void> IPStackSimulator::send_ethernet_tapserver_frame(
   }
   header.frame_size = size + sizeof(EthernetHeader);
 
-  array<asio::const_buffer, 2> bufs{
-      asio::buffer(static_cast<const void*>(&header), sizeof(header)),
-      asio::buffer(data, size)};
+  array<asio::const_buffer, 2> bufs{asio::buffer(static_cast<const void*>(&header), sizeof(header)), asio::buffer(data, size)};
   co_await asio::async_write(c->sock, bufs, asio::use_awaitable);
 }
 
@@ -462,8 +458,7 @@ asio::awaitable<void> IPStackSimulator::on_client_frame(shared_ptr<IPSSClient> c
 
   FrameInfo fi(link_type, data, size);
   if (this->log.should_log(phosg::LogLevel::L_DEBUG)) {
-    string fi_header = fi.header_str();
-    this->log.debug_f("Frame header: {}", fi_header);
+    this->log.debug_f("Frame header: {}", fi.header_str());
   }
 
   if (fi.ether) {
@@ -477,8 +472,7 @@ asio::awaitable<void> IPStackSimulator::on_client_frame(shared_ptr<IPSSClient> c
     uint16_t stored_checksum = fi.stored_hdlc_checksum();
     if (expected_checksum != stored_checksum) {
       throw runtime_error(std::format(
-          "HDLC checksum is incorrect ({:04X} expected, {:04X} received)",
-          expected_checksum, stored_checksum));
+          "HDLC checksum is incorrect ({:04X} expected, {:04X} received)", expected_checksum, stored_checksum));
     }
   } else {
     throw runtime_error("frame is not Ethernet or HDLC");
@@ -500,8 +494,7 @@ asio::awaitable<void> IPStackSimulator::on_client_frame(shared_ptr<IPSSClient> c
     uint16_t expected_ipv4_checksum = fi.computed_ipv4_header_checksum();
     if (fi.ipv4->checksum != expected_ipv4_checksum) {
       throw runtime_error(std::format(
-          "IPv4 header checksum is incorrect ({:04X} expected, {:04X} received)",
-          expected_ipv4_checksum, fi.ipv4->checksum));
+          "IPv4 header checksum is incorrect ({:04X} expected, {:04X} received)", expected_ipv4_checksum, fi.ipv4->checksum));
     }
 
     if ((fi.ipv4->src_addr != c->ipv4_addr) && (fi.ipv4->src_addr != 0)) {
@@ -512,8 +505,7 @@ asio::awaitable<void> IPStackSimulator::on_client_frame(shared_ptr<IPSSClient> c
       uint16_t expected_udp_checksum = fi.computed_udp4_checksum();
       if (fi.udp->checksum != expected_udp_checksum) {
         throw runtime_error(std::format(
-            "UDP checksum is incorrect ({:04X} expected, {:04X} received)",
-            expected_udp_checksum, fi.udp->checksum));
+            "UDP checksum is incorrect ({:04X} expected, {:04X} received)", expected_udp_checksum, fi.udp->checksum));
       }
       co_await this->on_client_udp_frame(c, fi);
 
@@ -521,8 +513,7 @@ asio::awaitable<void> IPStackSimulator::on_client_frame(shared_ptr<IPSSClient> c
       uint16_t expected_tcp_checksum = fi.computed_tcp4_checksum();
       if (fi.tcp->checksum != expected_tcp_checksum) {
         throw runtime_error(std::format(
-            "TCP checksum is incorrect ({:04X} expected, {:04X} received)",
-            expected_tcp_checksum, fi.tcp->checksum));
+            "TCP checksum is incorrect ({:04X} expected, {:04X} received)", expected_tcp_checksum, fi.tcp->checksum));
       }
       co_await this->on_client_tcp_frame(c, fi);
 
@@ -988,8 +979,7 @@ asio::awaitable<void> IPStackSimulator::on_client_udp_frame(shared_ptr<IPSSClien
     r_ipv4.size = sizeof(IPv4Header) + sizeof(UDPHeader) + r_data.size();
     r_udp.size = sizeof(UDPHeader) + r_data.size();
     r_ipv4.checksum = FrameInfo::computed_ipv4_header_checksum(r_ipv4);
-    r_udp.checksum = FrameInfo::computed_udp4_checksum(
-        r_ipv4, r_udp, r_data.data(), r_data.size());
+    r_udp.checksum = FrameInfo::computed_udp4_checksum(r_ipv4, r_udp, r_data.data(), r_data.size());
 
     if (this->log.should_log(phosg::LogLevel::L_DEBUG)) {
       string remote_str = this->str_for_ipv4_netloc(fi.ipv4->src_addr, fi.udp->src_port);
@@ -1103,11 +1093,15 @@ asio::awaitable<void> IPStackSimulator::on_client_tcp_frame(shared_ptr<IPSSClien
         conn_str, conn->acked_server_seq, conn->next_client_seq);
 
   } else {
-    // This frame isn't a SYN, so a connection object should already exist
+    // This frame isn't a SYN, so a connection object should already exist;
+    // ignore the frame if there's no connection
     uint64_t key = this->tcp_conn_key_for_client_frame(fi);
     auto conn_it = c->tcp_connections.find(key);
     if (conn_it == c->tcp_connections.end()) {
-      throw runtime_error("non-SYN frame does not correspond to any open TCP connection");
+      if (this->log.debug_f("Ignoring non-SYN TCP frame with no active connection")) {
+        phosg::print_data(stderr, fi.payload, fi.payload_size);
+      }
+      co_return;
     }
     auto& conn = conn_it->second;
     bool conn_valid = true;
@@ -1233,11 +1227,9 @@ asio::awaitable<void> IPStackSimulator::on_client_tcp_frame(shared_ptr<IPSSClien
 
         // Send the new data to the server
         if (!conn->server_channel) {
-          this->log.warning_f("Client sent data on TCP connection {}, but server channel is missing",
-              conn_str);
+          this->log.warning_f("Client sent data on TCP connection {}, but server channel is missing", conn_str);
         } else if (!conn->server_channel->connected()) {
-          this->log.warning_f("Client sent data on TCP connection {}, but server channel is disconnected",
-              conn_str);
+          this->log.warning_f("Client sent data on TCP connection {}, but server channel is disconnected", conn_str);
         } else {
           conn->server_channel->add_inbound_data(payload, payload_size);
         }
@@ -1246,8 +1238,7 @@ asio::awaitable<void> IPStackSimulator::on_client_tcp_frame(shared_ptr<IPSSClien
         conn->next_client_seq += payload_size;
         conn->bytes_received += payload_size;
         if (conn->next_client_seq < payload_size) {
-          this->log.warning_f("Client sequence number has wrapped (next={:08X}, bytes={:X})",
-              fi.tcp->seq_num, payload_size);
+          this->log.warning_f("Client sequence number has wrapped (next={:08X}, bytes={:X})", fi.tcp->seq_num, payload_size);
         }
       }
 
@@ -1396,7 +1387,7 @@ asio::awaitable<void> IPStackSimulator::open_server_connection(
   }
   const auto& port_config = port_config_it->second;
 
-  conn->server_channel = make_shared<IPSSChannel>(this->shared_from_this(), c, conn, port_config->version, 1);
+  conn->server_channel = make_shared<IPSSChannel>(this->shared_from_this(), c, conn, port_config->version, Language::ENGLISH);
 
   if (!this->state->game_server.get()) {
     this->log.error_f("No server available for TCP connection {}", conn_str);
@@ -1425,7 +1416,9 @@ std::shared_ptr<IPSSClient> IPStackSimulator::create_client(
     std::shared_ptr<IPSSSocket> listen_sock, asio::ip::tcp::socket&& client_sock) {
   uint32_t addr = ipv4_addr_for_asio_addr(client_sock.remote_endpoint().address());
   if (this->state->banned_ipv4_ranges->check(addr)) {
-    client_sock.close();
+    if (client_sock.is_open()) {
+      client_sock.close();
+    }
     return nullptr;
   }
 

src/IPStackSimulator.hh

@@ -109,7 +109,7 @@ public:
       std::weak_ptr<IPSSClient> ipss_client,
       std::weak_ptr<IPSSClient::TCPConnection> tcp_conn,
       Version version,
-      uint8_t language,
+      Language language,
       const std::string& name = "",
       phosg::TerminalFormat terminal_send_color = phosg::TerminalFormat::END,
       phosg::TerminalFormat terminal_recv_color = phosg::TerminalFormat::END);

src/ImageEncoder.cc

@@ -35,7 +35,7 @@ struct GVRHeader {
   be_uint16_t height;
 } __packed_ws__(GVRHeader, 0x10);
 
-string encode_gvm(const phosg::Image& img, GVRDataFormat data_format, const string& internal_name, uint32_t global_index) {
+string encode_gvm(const phosg::ImageRGBA8888N& img, GVRDataFormat data_format, const string& internal_name, uint32_t global_index) {
   int8_t dimensions_field = -2;
   {
     size_t h = img.get_height();
@@ -71,7 +71,7 @@ string encode_gvm(const phosg::Image& img, GVRDataFormat data_format, const stri
   w.put<GVMFileHeader>({.signature = 0x47564D48, .header_size = 0x48, .flags = 0x000F, .num_files = 1});
   GVMFileEntry file_entry;
   file_entry.file_num = 0;
-  file_entry.name.encode(internal_name, 1);
+  file_entry.name.encode(internal_name, Language::ENGLISH);
   file_entry.data_format = data_format;
   file_entry.format_flags = 0;
   file_entry.dimensions = (dimensions_field << 4) | dimensions_field;
@@ -90,17 +90,16 @@ string encode_gvm(const phosg::Image& img, GVRDataFormat data_format, const stri
     for (size_t x = 0; x < img.get_width(); x += 4) {
       for (size_t yy = 0; yy < 4; yy++) {
         for (size_t xx = 0; xx < 4; xx++) {
-          uint64_t a, r, g, b;
-          img.read_pixel(x + xx, y + yy, &r, &g, &b, &a);
+          uint32_t c = img.read(x + xx, y + yy);
           switch (data_format) {
             case GVRDataFormat::RGB565:
-              w.put_u16b(encode_rgb565(r, g, b));
+              w.put_u16b(phosg::rgb565_for_rgba8888(c));
               break;
             case GVRDataFormat::RGB5A3:
-              w.put_u16b(encode_rgb5a3(r, g, b, a));
+              w.put_u16b(encode_rgb5a3(c));
               break;
             case GVRDataFormat::ARGB8888:
-              w.put_u32b(encode_argb8888(r, g, b, a));
+              w.put_u32b(phosg::argb8888_for_rgba8888(c));
               break;
             default:
               throw logic_error("cannot encode pixel format");
@@ -115,15 +114,15 @@ string encode_gvm(const phosg::Image& img, GVRDataFormat data_format, const stri
 
 static const array<uint32_t, 4> fon_colors = {0x000000FF, 0x555555FF, 0xAAAAAAFF, 0xFFFFFFFF};
 
-phosg::Image decode_fon(const string& data, size_t width) {
+phosg::ImageRGB888 decode_fon(const string& data, size_t width) {
   size_t num_pixels = data.size() * 4;
   size_t height = num_pixels / width;
-  phosg::Image ret(width, height);
+  phosg::ImageRGB888 ret(width, height);
 
   phosg::BitReader r(data);
   for (size_t y = 0; y < height; y++) {
     for (size_t x = 0; x < width; x++) {
-      ret.write_pixel(x, y, fon_colors[r.read(2)]);
+      ret.write(x, y, fon_colors[r.read(2)]);
     }
   }
   return ret;
@@ -133,11 +132,11 @@ constexpr size_t uabs(size_t a, size_t b) {
   return (a > b) ? (a - b) : (b - a);
 }
 
-string encode_fon(const phosg::Image& img) {
+string encode_fon(const phosg::ImageRGB888& img) {
   phosg::BitWriter w;
   for (size_t y = 0; y < img.get_height(); y++) {
     for (size_t x = 0; x < img.get_width(); x++) {
-      uint32_t color = img.read_pixel(x, y);
+      uint32_t color = img.read(x, y);
 
       size_t result_delta = 0x400;
       size_t result_index = 0;

src/ImageEncoder.hh

@@ -19,43 +19,29 @@ enum class GVRDataFormat : uint8_t {
   DXT1 = 0x0E,
 };
 
-std::string encode_gvm(const phosg::Image& img, GVRDataFormat data_format, const std::string& internal_name, uint32_t global_index);
-phosg::Image decode_fon(const std::string& data, size_t width);
-std::string encode_fon(const phosg::Image& img);
+std::string encode_gvm(
+    const phosg::ImageRGBA8888N& img, GVRDataFormat data_format, const std::string& internal_name, uint32_t global_index);
+phosg::ImageRGB888 decode_fon(const std::string& data, size_t width);
+std::string encode_fon(const phosg::ImageRGB888& img);
 
-constexpr uint16_t encode_rgb565(uint8_t r, uint8_t g, uint8_t b) {
-  return ((r << 8) & 0xF800) | ((g << 3) & 0x07E0) | ((b >> 3) & 0x001F);
-}
-
-constexpr uint16_t encode_rgb5a3(uint8_t r, uint8_t g, uint8_t b, uint8_t a) {
-  if ((a & 0xE0) == 0xE0) {
-    return 0x8000 | ((r << 7) & 0x7C00) | ((g << 2) & 0x03E0) | ((b >> 3) & 0x001F);
+constexpr uint16_t encode_rgb5a3(uint32_t c) {
+  if ((phosg::get_a(c) & 0xE0) == 0xE0) {
+    return 0x8000 | ((phosg::get_r(c) << 7) & 0x7C00) | ((phosg::get_g(c) << 2) & 0x03E0) | ((phosg::get_b(c) >> 3) & 0x001F);
   } else {
-    return ((a << 7) & 0x7000) | ((r << 4) & 0x0F00) | (g & 0x00F0) | ((b >> 4) & 0x000F);
+    return ((phosg::get_a(c) << 7) & 0x7000) | ((phosg::get_r(c) << 4) & 0x0F00) | (phosg::get_g(c) & 0x00F0) | ((phosg::get_b(c) >> 4) & 0x000F);
   }
 }
 
-constexpr uint32_t encode_argb8888(uint8_t r, uint8_t g, uint8_t b, uint8_t a) {
-  return (a << 24) | (r << 16) | (g << 8) | b;
-}
-
-constexpr uint16_t encode_argb8888_to_argb1555(uint32_t argb8888) {
-  // In:  aaaaaaaarrrrrrrrggggggggbbbbbbbb
-  // Out:                 arrrrrgggggbbbbb
-  return ((argb8888 >> 9) & 0x7C00) | ((argb8888 >> 6) & 0x03E0) | ((argb8888 >> 3) & 0x001F) | ((argb8888 >> 16) & 0x8000);
-}
-
-constexpr uint16_t encode_rgba8888_to_argb1555(uint32_t rgba8888) {
-  // In:  rrrrrrrrggggggggbbbbbbbbaaaaaaaa
-  // Out:                 arrrrrgggggbbbbb
-  return ((rgba8888 >> 17) & 0x7C00) | ((rgba8888 >> 14) & 0x03E0) | ((rgba8888 >> 11) & 0x001F) | ((rgba8888 << 8) & 0x8000);
-}
-
-constexpr uint32_t decode_argb1555_to_rgba8888(uint16_t argb1555) {
-  // In:                  arrrrrgggggbbbbb
-  // Out: rrrrrrrrggggggggbbbbbbbbaaaaaaaa
-  return ((argb1555 << 17) & 0xF8000000) | ((argb1555 << 12) & 0x07000000) |
-      ((argb1555 << 14) & 0x00F80000) | ((argb1555 << 9) & 0x00070000) |
-      ((argb1555 << 11) & 0x0000F800) | ((argb1555 << 6) & 0x00000700) |
-      ((argb1555 & 0x8000) ? 0x000000FF : 0x00000000);
+template <phosg::PixelFormat Format>
+bool has_any_transparent_pixels(const phosg::Image<Format>& img) {
+  if constexpr (phosg::Image<Format>::HAS_ALPHA) {
+    for (size_t y = 0; y < img.get_height(); y++) {
+      for (size_t x = 0; x < img.get_height(); x++) {
+        if (phosg::get_a(img.read(x, y)) != 0xFF) {
+          return true;
+        }
+      }
+    }
+  }
+  return false;
 }

src/IntegralExpression.hh

@@ -9,7 +9,6 @@
 #include <vector>
 
 #include "PlayerSubordinates.hh"
-#include "QuestScript.hh"
 #include "StaticGameData.hh"
 #include "TeamIndex.hh"
 

src/ItemCreator.cc

@@ -32,7 +32,7 @@ ItemCreator::ItemCreator(
     std::shared_ptr<const ItemData::StackLimits> stack_limits,
     Episode episode,
     GameMode mode,
-    uint8_t difficulty,
+    Difficulty difficulty,
     uint8_t section_id,
     std::shared_ptr<RandomGenerator> rand_crypt,
     shared_ptr<const BattleRules> restrictions)
@@ -342,7 +342,7 @@ bool ItemCreator::should_allow_meseta_drops() const {
 
 ItemData ItemCreator::check_rare_spec_and_create_rare_enemy_item(uint32_t enemy_type, uint8_t area_norm) {
   ItemData item;
-  if (this->are_rare_drops_allowed() && (enemy_type > 0) && (enemy_type < 0x58)) {
+  if (this->are_rare_drops_allowed() && (enemy_type > 0) && (enemy_type < 0x64)) {
     // Note: In the original implementation, enemies can only have one possible
     // rare drop. In our implementation, they can have multiple rare drops if
     // JSONRareItemSet is used (the other RareItemSet implementations never
@@ -1072,7 +1072,7 @@ void ItemCreator::generate_armor_shop_armors(vector<ItemData>& shop, size_t play
     item.data1[1] = 1;
     item.data1[2] = pt.pop();
 
-    if ((this->difficulty == 3) && (player_level > 99)) {
+    if ((this->difficulty == Difficulty::ULTIMATE) && (player_level > 99)) {
       if (player_level > 150) {
         item.data1[2] += 3;
       } else if (player_level >= 100) {
@@ -1116,7 +1116,7 @@ void ItemCreator::generate_armor_shop_shields(vector<ItemData>& shop, size_t pla
     item.data1[1] = 2;
     item.data1[2] = pt.pop();
 
-    if ((this->difficulty == 3) && (player_level > 99)) {
+    if ((this->difficulty == Difficulty::ULTIMATE) && (player_level > 99)) {
       if (player_level > 150) {
         item.data1[2] += 3;
       } else if (player_level >= 100) {
@@ -1360,7 +1360,7 @@ vector<ItemData> ItemCreator::generate_weapon_shop_contents(size_t player_level)
   }
 
   size_t table_index;
-  if (this->difficulty == 3) {
+  if (this->difficulty == Difficulty::ULTIMATE) {
     if (player_level < 11) {
       table_index = 0;
     } else if (player_level < 26) {
@@ -1654,8 +1654,7 @@ void ItemCreator::generate_weapon_shop_item_bonus1(
 
   } else {
     const auto* range = this->weapon_random_set->get_bonus_range(0, table_index);
-    item.data1[7] = bonus_values.at(max<size_t>(
-        this->rand_int(range->max + 1), range->min));
+    item.data1[7] = bonus_values.at(max<size_t>(this->rand_int(range->max + 1), range->min));
   }
 }
 
@@ -1700,8 +1699,7 @@ void ItemCreator::generate_weapon_shop_item_bonus2(ItemData& item, size_t player
 
   } else {
     const auto* range = this->weapon_random_set->get_bonus_range(1, table_index);
-    item.data1[9] = bonus_values.at(max<size_t>(
-        this->rand_int(range->max + 1), range->min));
+    item.data1[9] = bonus_values.at(max<size_t>(this->rand_int(range->max + 1), range->min));
   }
 }
 
@@ -1752,7 +1750,6 @@ ItemData ItemCreator::base_item_for_specialized_box(uint32_t param4, uint32_t pa
     case 0x04:
       item.data2d = ((param5 >> 0x10) & 0xFFFF) * 10;
       break;
-
     default:
       throw runtime_error("invalid item class");
   }

src/ItemCreator.hh

@@ -22,7 +22,7 @@ public:
       std::shared_ptr<const ItemData::StackLimits> stack_limits,
       Episode episode,
       GameMode mode,
-      uint8_t difficulty,
+      Difficulty difficulty,
       uint8_t section_id,
       std::shared_ptr<RandomGenerator> rand_crypt,
       std::shared_ptr<const BattleRules> restrictions = nullptr);
@@ -61,7 +61,7 @@ private:
   std::shared_ptr<const ItemData::StackLimits> stack_limits;
   Episode episode;
   GameMode mode;
-  uint8_t difficulty;
+  Difficulty difficulty;
   uint8_t section_id;
   std::shared_ptr<const RareItemSet> rare_item_set;
   std::shared_ptr<const ArmorRandomSet> armor_random_set;
@@ -96,7 +96,7 @@ private:
   //   [0x0B] - unit modifiers
   //   [0x0C] - common armor DFP bonuses
   //   [0x0D] - common armor EVP bonuses
-  //   [0x0E] - apparently unused
+  //   [0x0E] - unit stars
   //   [0x0F] - which common weapon special to generate
   //   [0x10] - apparently unused
   std::shared_ptr<RandomGenerator> rand_crypt;

src/ItemData.cc

@@ -254,9 +254,14 @@ size_t ItemData::max_stack_size(const StackLimits& limits) const {
   return limits.get(this->data1[0], this->data1[1]);
 }
 
-void ItemData::enforce_min_stack_size(const StackLimits& limits) {
-  if (this->stack_size(limits) == 0) {
-    this->data1[5] = 1;
+void ItemData::enforce_stack_size_limits(const StackLimits& limits) {
+  if (this->data1[0] == 0x03) {
+    size_t max_stack_size = this->max_stack_size(limits);
+    if (max_stack_size > 1) {
+      this->data1[5] = std::clamp<uint8_t>(this->data1[5], 1, max_stack_size);
+    } else {
+      this->data1[5] = 0;
+    }
   }
 }
 

src/ItemData.hh

@@ -81,15 +81,16 @@ struct ItemData {
 
   // QUICK ITEM FORMAT REFERENCE
   //           data1/0  data1/4  data1/8  data2
-  //   Weapon: 00ZZZZGG SSNNAABB AABBAABB 00000000
-  //   Armor:  0101ZZ00 FFTTDDDD EEEE0000 00000000
-  //   Shield: 0102ZZ00 FFTTDDDD EEEE0000 00000000
-  //   Unit:   0103ZZ00 FF00RRRR 00000000 00000000
-  //   Mag:    02ZZLLWW HHHHIIII JJJJKKKK YYQQPPVV
-  //   Tool:   03ZZZZUU 00CC0000 00000000 00000000
-  //   Meseta: 04000000 00000000 00000000 MMMMMMMM
-  // A = attribute type (for S-ranks, custom name)
-  // B = attribute amount (for S-ranks, custom name)
+  //   Weapon:    00ZZZZGG SSNNAABB AABBAABB 00000000
+  //   Armor:     0101ZZ00 FFTTDDDD EEEEXXXX 00000000
+  //   Shield:    0102ZZ00 FFTTDDDD EEEEXXXX 00000000
+  //   Unit:      0103ZZ00 FF00RRRR 0000XXXX 00000000
+  //   Mag:       02ZZLLWW HHHHIIII JJJJKKKK YYQQPPVV
+  //   Tool:      03ZZZZUU 00CC0000 0000XXXX 00000000
+  //   Tech disk: 0302&&UU %%CC0000 0000XXXX 00000000
+  //   Meseta:    04000000 00000000 00000000 MMMMMMMM
+  // A = attribute type (for S-ranks, custom name; last pair is kill count for some weapons)
+  // B = attribute amount (for S-ranks, custom name; last pair is kill count for some weapons)
   // C = stack size (for tools)
   // D = DEF bonus
   // E = EVP bonus
@@ -110,15 +111,18 @@ struct ItemData {
   // U = tool flags (40=present; unused if item is stackable)
   // V = mag color
   // W = photon blasts
+  // X = kill count (big-endian; high bit always set)
   // Y = mag synchro
   // Z = item ID
+  // & = technique level
+  // % = technique number
   // Note: PSO GC erroneously byteswaps data2 even when the item is a mag. This
   // makes it incompatible with little-endian versions of PSO (i.e. all other
   // versions). We manually byteswap data2 upon receipt and immediately before
   // sending where needed.
   // Related note: PSO V2 has an annoyingly complicated format for mags that
   // doesn't match the above table. We decode this upon receipt and encode it
-  // imemdiately before sending when interacting with V2 clients; see the
+  // immediately before sending when interacting with V2 clients; see the
   // implementation of decode_for_version() for details.
 
   union {
@@ -163,7 +167,7 @@ struct ItemData {
   bool is_stackable(const StackLimits& limits) const;
   size_t stack_size(const StackLimits& limits) const;
   size_t max_stack_size(const StackLimits& limits) const;
-  void enforce_min_stack_size(const StackLimits& limits);
+  void enforce_stack_size_limits(const StackLimits& limits);
 
   static bool is_common_consumable(uint32_t primary_identifier);
   bool is_common_consumable() const;

src/ItemNameIndex.cc

@@ -12,9 +12,16 @@ ItemNameIndex::ItemNameIndex(
       limits(limits) {
 
   for (uint32_t primary_identifier : item_parameter_table->compute_all_valid_primary_identifiers()) {
+    // 00000000 is a valid primary identifier but not a valid weapon; skip it
+    if (primary_identifier == 0x00000000) {
+      continue;
+    }
+
     const string* name = nullptr;
+    bool is_es_weapon = false;
     try {
       ItemData item = ItemData::from_primary_identifier(*this->limits, primary_identifier);
+      is_es_weapon = item.is_s_rank_weapon();
       name = &name_coll.at(item_parameter_table->get_item_id(item));
     } catch (const out_of_range&) {
     }
@@ -25,11 +32,14 @@ ItemNameIndex::ItemNameIndex(
       meta->name = *name;
       this->primary_identifier_index.emplace(meta->primary_identifier, meta);
       this->name_index.emplace(phosg::tolower(meta->name), meta);
+      if (is_es_weapon && ((primary_identifier & 0x0000FFFF) == 0x00000000)) {
+        this->es_name_index.emplace(phosg::tolower(meta->name), meta);
+      }
     }
   }
 }
 
-static const char* s_rank_name_characters = "\0ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_";
+static std::string s_rank_name_characters("\0ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_", 0x20);
 
 // clang-format off
 static const array<const char*, 0x29> name_for_weapon_special = {
@@ -97,7 +107,10 @@ const array<const char*, 0x11> name_for_s_rank_special = {
     "King\'s",
 };
 
-std::string ItemNameIndex::describe_item(const ItemData& item, bool include_color_escapes, bool hide_mag_stats) const {
+std::string ItemNameIndex::describe_item(const ItemData& item, uint8_t flags) const {
+  bool include_color_escapes = flags & ItemNameIndex::Flag::INCLUDE_PSO_COLOR_ESCAPES;
+  bool name_only = flags & ItemNameIndex::Flag::NAME_ONLY;
+
   if (item.data1[0] == 0x04) {
     return std::format("{}{} Meseta", include_color_escapes ? "$C7" : "", item.data2d);
   }
@@ -108,13 +121,14 @@ std::string ItemNameIndex::describe_item(const ItemData& item, bool include_colo
   bool is_unidentified = false;
   if ((item.data1[0] == 0x00) && (item.data1[4] != 0x00) && !item.is_s_rank_weapon()) {
     is_unidentified = item.data1[4] & 0x80;
-    bool is_present = item.data1[4] & 0x40;
     uint8_t special_id = item.data1[4] & 0x3F;
-    if (is_present) {
-      ret_tokens.emplace_back("Wrapped");
-    }
-    if (is_unidentified) {
-      ret_tokens.emplace_back("????");
+    if (!name_only) {
+      if (item.data1[4] & 0x40) {
+        ret_tokens.emplace_back("Wrapped");
+      }
+      if (is_unidentified) {
+        ret_tokens.emplace_back("????");
+      }
     }
     if (special_id) {
       try {
@@ -124,7 +138,7 @@ std::string ItemNameIndex::describe_item(const ItemData& item, bool include_colo
       }
     }
   }
-  if ((item.data1[0] == 0x00) && (item.data1[2] != 0x00) && item.is_s_rank_weapon()) {
+  if (!name_only && (item.data1[0] == 0x00) && (item.data1[2] != 0x00) && item.is_s_rank_weapon()) {
     try {
       ret_tokens.emplace_back(name_for_s_rank_special.at(item.data1[2]));
     } catch (const out_of_range&) {
@@ -135,9 +149,10 @@ std::string ItemNameIndex::describe_item(const ItemData& item, bool include_colo
   // Armors, shields, and units (0x01) can be wrapped, as can mags (0x02) and
   // non-stackable tools (0x03). However, each of these item classes has its
   // flags in a different location.
-  if (((item.data1[0] == 0x01) && (item.data1[4] & 0x40)) ||
-      ((item.data1[0] == 0x02) && (item.data2[2] & 0x40)) ||
-      ((item.data1[0] == 0x03) && !item.is_stackable(*this->limits) && (item.data1[3] & 0x40))) {
+  if (!name_only &&
+      (((item.data1[0] == 0x01) && (item.data1[4] & 0x40)) ||
+          ((item.data1[0] == 0x02) && (item.data2[2] & 0x40)) ||
+          ((item.data1[0] == 0x03) && !item.is_stackable(*this->limits) && (item.data1[3] & 0x40)))) {
     ret_tokens.emplace_back("Wrapped");
   }
 
@@ -168,7 +183,7 @@ std::string ItemNameIndex::describe_item(const ItemData& item, bool include_colo
 
   if (item.data1[0] == 0x00) {
     // For weapons, add the grind and bonuses, or S-rank name if applicable
-    if (item.data1[3] > 0) {
+    if (!name_only && item.data1[3] > 0) {
       ret_tokens.emplace_back(std::format("+{}", item.data1[3]));
     }
 
@@ -196,7 +211,7 @@ std::string ItemNameIndex::describe_item(const ItemData& item, bool include_colo
 
       string name;
       for (size_t x = 0; x < 8; x++) {
-        char ch = s_rank_name_characters[char_indexes[x]];
+        char ch = s_rank_name_characters.at(char_indexes[x]);
         if (ch == 0) {
           break;
         }
@@ -210,18 +225,22 @@ std::string ItemNameIndex::describe_item(const ItemData& item, bool include_colo
         }
       }
 
-    } else { // Not S-rank (extended name bits not set)
+    } else if (!name_only) { // Not S-rank (extended name bits not set)
+      size_t num_bonuses = 3;
+
+      if (item.data1[10] & 0x80) {
+        ret_tokens.emplace_back(std::format("K:{}", item.get_kill_count()));
+        num_bonuses = 2;
+      }
+
       parray<int8_t, 5> bonuses(0);
-      for (size_t x = 0; x < 3; x++) {
+      for (size_t x = 0; x < num_bonuses; x++) {
         uint8_t which = item.data1[6 + 2 * x];
         uint8_t value = item.data1[7 + 2 * x];
         if (which == 0) {
           continue;
         }
-        if (which & 0x80) {
-          uint16_t kill_count = ((which << 8) & 0x7F00) | (value & 0xFF);
-          ret_tokens.emplace_back(std::format("K:{}", kill_count));
-        } else if (which > 5) {
+        if (which > 5) {
           ret_tokens.emplace_back(std::format("!PC:{:02X}{:02X}", which, value));
         } else {
           bonuses[which - 1] = value;
@@ -258,7 +277,11 @@ std::string ItemNameIndex::describe_item(const ItemData& item, bool include_colo
         ret_tokens.emplace_back(std::format("!MD:{:04X}", modifier));
       }
 
-    } else { // Armor/shields
+      if (!name_only && (item.data1[10] & 0x80)) {
+        ret_tokens.emplace_back(std::format("K:{}", item.get_kill_count()));
+      }
+
+    } else if (!name_only) { // Armor/shields
       if (item.data1[5] > 0) {
         if (item.data1[5] == 1) {
           ret_tokens.emplace_back("(1 slot)");
@@ -276,7 +299,7 @@ std::string ItemNameIndex::describe_item(const ItemData& item, bool include_colo
       }
     }
 
-  } else if (!hide_mag_stats && (item.data1[0] == 0x02)) {
+  } else if (!name_only && (item.data1[0] == 0x02)) {
     // For mags, add tons of info
     ret_tokens.emplace_back(std::format("LV{}", item.data1[2]));
 
@@ -296,7 +319,7 @@ std::string ItemNameIndex::describe_item(const ItemData& item, bool include_colo
       }
     };
     ret_tokens.emplace_back(format_stat(def) + "/" + format_stat(pow) + "/" + format_stat(dex) + "/" + format_stat(mind));
-    ret_tokens.emplace_back(std::format("{}%", item.data2[0]));
+    ret_tokens.emplace_back(std::format("{}{}", item.data2[0], include_color_escapes ? "%%" : "%"));
     ret_tokens.emplace_back(std::format("{}IQ", item.data2[1]));
 
     uint8_t flags = item.data2[2];
@@ -383,7 +406,7 @@ ItemData ItemNameIndex::parse_item_description(const std::string& desc) const {
       }
     }
   }
-  ret.enforce_min_stack_size(*this->limits);
+  ret.enforce_stack_size_limits(*this->limits);
   return ret;
 }
 
@@ -400,6 +423,112 @@ ItemData ItemNameIndex::parse_item_description_phase(const std::string& descript
     return ret;
   }
 
+  if (desc.starts_with("es ")) {
+    auto parse_name = [&](const std::string& token) -> void {
+      if (token.size() > 8) {
+        throw runtime_error("s-rank name too long");
+      }
+
+      uint8_t char_indexes[8] = {0, 0, 0, 0, 0, 0, 0, 0};
+      for (size_t z = 0; z < token.size(); z++) {
+        char ch = toupper(token[z]);
+        size_t pos = s_rank_name_characters.find(ch);
+        if (pos == std::string::npos) {
+          throw runtime_error(std::format("s-rank name contains invalid character {:02X} ({})", ch, ch));
+        }
+        char_indexes[z] = pos;
+      }
+
+      ret.data1w[3] = phosg::bswap16(0x8000 | (char_indexes[1] & 0x1F) | ((char_indexes[0] & 0x1F) << 5));
+      ret.data1w[4] = phosg::bswap16(0x8000 | (char_indexes[4] & 0x1F) | ((char_indexes[3] & 0x1F) << 5) | ((char_indexes[2] & 0x1F) << 10));
+      ret.data1w[5] = phosg::bswap16(0x8000 | (char_indexes[7] & 0x1F) | ((char_indexes[6] & 0x1F) << 5) | ((char_indexes[5] & 0x1F) << 10));
+    };
+    auto parse_special = [&](const std::string& token) -> bool {
+      for (size_t z = 0; z < name_for_s_rank_special.size(); z++) {
+        if (name_for_s_rank_special[z] && (token == phosg::tolower(name_for_s_rank_special[z]))) {
+          ret.data1[2] = z;
+          return true;
+        }
+      }
+      return false;
+    };
+    auto parse_grind = [&](const std::string& token) -> bool {
+      if (token.starts_with('+')) {
+        ret.data1[3] = stoul(token.substr(1), nullptr, 0);
+        return true;
+      }
+      return false;
+    };
+    auto parse_type = [&](const std::string& token) -> void {
+      const auto& meta = this->es_name_index.at(token);
+      if (meta->primary_identifier & 0xFF00FFFF) {
+        throw std::runtime_error("ES weapon has invalid bits in primary identifier");
+      }
+      ret.data1[1] = meta->primary_identifier >> 16;
+    };
+
+    // Syntax: ES [NAME] [SPECIAL] TYPE [+GRIND]
+    auto tokens = phosg::split(desc, ' ');
+    switch (tokens.size()) {
+      case 0:
+      case 1:
+        throw std::runtime_error("ES weapon type is missing");
+
+      case 2:
+        // Must be ES TYPE
+        parse_name(tokens[1]);
+        break;
+
+      case 3:
+        // Any of the following:
+        // ES TYPE +N
+        // ES SPECIAL TYPE
+        // ES NAME TYPE
+        if (parse_grind(tokens[2])) {
+          parse_type(tokens[1]);
+        } else if (parse_special(tokens[1])) {
+          parse_type(tokens[2]);
+        } else {
+          parse_name(tokens[1]);
+          parse_type(tokens[2]);
+        }
+        break;
+      case 4:
+        // Any of the following:
+        // ES SPECIAL TYPE +N
+        // ES NAME TYPE +N
+        // ES NAME SPECIAL TYPE
+        if (parse_grind(tokens[3])) {
+          if (!parse_special(tokens[1])) {
+            parse_name(tokens[1]);
+          }
+          parse_type(tokens[2]);
+        } else {
+          parse_name(tokens[1]);
+          if (!parse_special(tokens[2])) {
+            throw std::runtime_error("invalid ES special");
+          }
+          parse_type(tokens[3]);
+        }
+        break;
+      case 5:
+        // Must be ES NAME SPECIAL TYPE +N
+        parse_name(tokens[1]);
+        if (!parse_special(tokens[2])) {
+          throw std::runtime_error("invalid ES special");
+        }
+        parse_type(tokens[3]);
+        if (!parse_grind(tokens[4])) {
+          throw std::runtime_error("invalid grind");
+        }
+        break;
+      default:
+        throw std::runtime_error("too many ES weapon tokens");
+    }
+
+    return ret;
+  }
+
   if (desc.starts_with("disk:")) {
     auto tokens = phosg::split(desc, ' ');
     tokens[0] = tokens[0].substr(5); // Trim off "disk:"
@@ -441,7 +570,6 @@ ItemData ItemNameIndex::parse_item_description_phase(const std::string& descript
     desc = desc.substr(z);
   }
 
-  // TODO: It'd be nice to be able to parse S-rank weapon specials here too.
   uint8_t weapon_special = 0;
   if (!skip_special) {
     for (size_t z = 0; z < name_for_weapon_special.size(); z++) {
@@ -494,6 +622,7 @@ ItemData ItemNameIndex::parse_item_description_phase(const std::string& descript
     // kill count if unsealable
     ret.data1[4] = weapon_special | (is_wrapped ? 0x40 : 0x00) | (is_unidentified ? 0x80 : 0x00);
 
+    bool kill_count_set = false;
     auto tokens = phosg::split(desc, ' ');
     for (auto& token : tokens) {
       if (token.empty()) {
@@ -504,23 +633,11 @@ ItemData ItemNameIndex::parse_item_description_phase(const std::string& descript
         ret.data1[3] = stoul(token, nullptr, 10);
 
       } else if (ret.is_s_rank_weapon()) {
-        if (token.size() > 8) {
-          throw runtime_error("s-rank name too long");
-        }
+        throw std::runtime_error("ES weapon must be prefixed with \"ES\"");
 
-        uint8_t char_indexes[8] = {0, 0, 0, 0, 0, 0, 0, 0};
-        for (size_t z = 0; z < token.size(); z++) {
-          char ch = toupper(token[z]);
-          const char* pos = strchr(s_rank_name_characters, ch);
-          if (!pos) {
-            throw runtime_error(std::format("s-rank name contains invalid character {:02X} ({})", ch, ch));
-          }
-          char_indexes[z] = (pos - s_rank_name_characters);
-        }
-
-        ret.data1w[3] = phosg::bswap16(0x8000 | (char_indexes[1] & 0x1F) | ((char_indexes[0] & 0x1F) << 5));
-        ret.data1w[4] = phosg::bswap16(0x8000 | (char_indexes[4] & 0x1F) | ((char_indexes[3] & 0x1F) << 5) | ((char_indexes[2] & 0x1F) << 10));
-        ret.data1w[5] = phosg::bswap16(0x8000 | (char_indexes[7] & 0x1F) | ((char_indexes[6] & 0x1F) << 5) | ((char_indexes[5] & 0x1F) << 10));
+      } else if (token.starts_with("k:")) {
+        ret.set_kill_count(stoul(token.substr(2), nullptr, 0));
+        kill_count_set = true;
 
       } else {
         auto p_tokens = phosg::split(token, '/');
@@ -544,7 +661,7 @@ ItemData ItemNameIndex::parse_item_description_phase(const std::string& descript
       }
     }
 
-    if (this->item_parameter_table->is_unsealable_item(ret)) {
+    if (this->item_parameter_table->is_unsealable_item(ret) && !kill_count_set) {
       ret.set_kill_count(0);
     }
 
@@ -557,7 +674,29 @@ ItemData ItemNameIndex::parse_item_description_phase(const std::string& descript
           {"+", 0x0002},
           {"++", 0x0004},
       });
-      ret.data1w[3] = modifiers.at(desc);
+      bool kill_count_set = false;
+      for (const auto& token : phosg::split(desc, ' ')) {
+        if (token == "--") {
+          ret.data1w[3] = 0xFFFC;
+        } else if (token == "-") {
+          ret.data1w[3] = 0xFFFE;
+        } else if (token == "+") {
+          ret.data1w[3] = 0x0002;
+        } else if (token == "++") {
+          ret.data1w[3] = 0x0004;
+        } else if (token.starts_with('+')) {
+          ret.data1w[3] = stoul(token.substr(1), nullptr, 0);
+        } else if (token.starts_with('-')) {
+          ret.data1w[3] = static_cast<uint16_t>(stol(token, nullptr, 0));
+        } else if (token.starts_with("k:")) {
+          ret.set_kill_count(stoul(token.substr(2), nullptr, 0));
+          kill_count_set = true;
+        }
+      }
+
+      if (this->item_parameter_table->is_unsealable_item(ret) && !kill_count_set) {
+        ret.set_kill_count(0);
+      }
 
     } else { // Armor/shield
       for (const auto& token : phosg::split(desc, ' ')) {
@@ -662,7 +801,8 @@ ItemData ItemNameIndex::parse_item_description_phase(const std::string& descript
 void ItemNameIndex::print_table(FILE* stream) const {
   auto pmt = this->item_parameter_table;
 
-  phosg::fwrite_fmt(stream, "WEAPON => ---ID--- TYPE SKIN POINTS FLAG ATPLO ATPHI ATPRQ MSTRQ ATARQ -MST- GND PH SP ATA SB PJ 1X 1Y 2X 2Y CL A1 A2 A3 A4 A5 TB BF V1 ST* USL ---DIVISOR--- NAME\n");
+  phosg::fwrite_fmt(stream, "WEAPONS\n");
+  phosg::fwrite_fmt(stream, "  CODE   => ---ID--- TYPE SKIN POINTS FLAG ATPLO ATPHI ATPRQ MSTRQ ATARQ -MST- GND PH SP ATA SB(S1:AMT1,S2:AMT2) PJ 1X 1Y 2X 2Y CL --A1-- A4 A5 TB BF V1 ST* USL ---DIVISOR--- NAME\n");
   for (size_t data1_1 = 0; data1_1 < pmt->num_weapon_classes; data1_1++) {
     uint8_t v1_replacement = pmt->get_weapon_v1_replacement(data1_1);
     float sale_divisor = pmt->get_sale_divisor(0x00, data1_1);
@@ -681,7 +821,8 @@ void ItemNameIndex::print_table(FILE* stream) const {
       item.data1[2] = data1_2;
       string name = this->describe_item(item);
 
-      phosg::fwrite_fmt(stream, "00{:02X}{:02X} => {:08X} {:04X} {:04X} {:6} {:04X} {:5} {:5} {:5} {:5} {:5} {:5} {:3} {:02X} {:02X} {:3} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:2}* {} {} {}\n",
+      auto& stat_boost = pmt->get_stat_boost(w.stat_boost_entry_index);
+      phosg::fwrite_fmt(stream, "  00{:02X}{:02X} => {:08X} {:04X} {:04X} {:6} {:04X} {:5} {:5} {:5} {:5} {:5} {:5} {:3} {:02X} {:02X} {:3} {:02X}({:02X}:{:04X},{:02X}:{:04X}) {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X}{:02X}{:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:2}* {} {} {}\n",
           data1_1,
           data1_2,
           w.base.id,
@@ -699,16 +840,20 @@ void ItemNameIndex::print_table(FILE* stream) const {
           w.photon,
           w.special,
           w.ata,
-          w.stat_boost,
+          w.stat_boost_entry_index,
+          stat_boost.stats[0],
+          stat_boost.amounts[0],
+          stat_boost.stats[1],
+          stat_boost.amounts[1],
           w.projectile,
           w.trail1_x,
           w.trail1_y,
           w.trail2_x,
           w.trail2_y,
           w.color,
-          w.unknown_a1,
-          w.unknown_a2,
-          w.unknown_a3,
+          w.unknown_a1[0],
+          w.unknown_a1[1],
+          w.unknown_a1[2],
           w.unknown_a4,
           w.unknown_a5,
           w.tech_boost,
@@ -721,7 +866,8 @@ void ItemNameIndex::print_table(FILE* stream) const {
     }
   }
 
-  phosg::fwrite_fmt(stream, "ARMOR  => ---ID--- TYPE SKIN POINTS -DFP- -EVP- BP BE FLAG LVL EFR ETH EIC EDK ELT DFR EVR SB TB FT A4 ST* ---DIVISOR--- NAME\n");
+  phosg::fwrite_fmt(stream, "ARMORS\n");
+  phosg::fwrite_fmt(stream, "  CODE   => ---ID--- TYPE SKIN POINTS -DFP- -EVP- BP BE FLAG LVL EFR ETH EIC EDK ELT DFR EVR SB(S1:AMT1,S2:AMT2) TB FT A4 ST* ---DIVISOR--- NAME\n");
   for (size_t data1_1 = 1; data1_1 < 3; data1_1++) {
     float sale_divisor = pmt->get_sale_divisor(0x01, data1_1);
     string divisor_str = std::format("{:g}", sale_divisor);
@@ -738,7 +884,8 @@ void ItemNameIndex::print_table(FILE* stream) const {
       item.data1[2] = data1_2;
       string name = this->describe_item(item);
 
-      phosg::fwrite_fmt(stream, "01{:02X}{:02X} => {:08X} {:04X} {:04X} {:6} {:5} {:5} {:02X} {:02X} {:04X} {:3} {:3} {:3} {:3} {:3} {:3} {:3} {:3} {:02X} {:02X} {:02X} {:02X} {:2}* {} {}\n",
+      auto& stat_boost = pmt->get_stat_boost(a.stat_boost_entry_index);
+      phosg::fwrite_fmt(stream, "  01{:02X}{:02X} => {:08X} {:04X} {:04X} {:6} {:5} {:5} {:02X} {:02X} {:04X} {:3} {:3} {:3} {:3} {:3} {:3} {:3} {:3} {:02X}({:02X}:{:04X},{:02X}:{:04X}) {:02X} {:02X} {:02X} {:2}* {} {}\n",
           data1_1,
           data1_2,
           a.base.id,
@@ -758,7 +905,11 @@ void ItemNameIndex::print_table(FILE* stream) const {
           a.elt,
           a.dfp_range,
           a.evp_range,
-          a.stat_boost,
+          a.stat_boost_entry_index,
+          stat_boost.stats[0],
+          stat_boost.amounts[0],
+          stat_boost.stats[1],
+          stat_boost.amounts[1],
           a.tech_boost,
           a.flags_type,
           a.unknown_a4,
@@ -768,7 +919,8 @@ void ItemNameIndex::print_table(FILE* stream) const {
     }
   }
 
-  phosg::fwrite_fmt(stream, "UNIT   => ---ID--- TYPE SKIN POINTS STAT COUNT ST-MOD ST* ---DIVISOR--- NAME\n");
+  phosg::fwrite_fmt(stream, "UNITS\n");
+  phosg::fwrite_fmt(stream, "  CODE   => ---ID--- TYPE SKIN POINTS STAT COUNT ST-MOD ST* ---DIVISOR--- NAME\n");
   {
     float sale_divisor = pmt->get_sale_divisor(0x01, 0x03);
     string divisor_str = std::format("{:g}", sale_divisor);
@@ -785,7 +937,7 @@ void ItemNameIndex::print_table(FILE* stream) const {
       item.data1[2] = data1_2;
       string name = this->describe_item(item);
 
-      phosg::fwrite_fmt(stream, "0103{:02X} => {:08X} {:04X} {:04X} {:6} {:04X} {:5} {:6} {:2}* {} {}\n",
+      phosg::fwrite_fmt(stream, "  0103{:02X} => {:08X} {:04X} {:04X} {:6} {:04X} {:5} {:6} {:2}* {} {}\n",
           data1_2,
           u.base.id,
           u.base.type,
@@ -800,7 +952,8 @@ void ItemNameIndex::print_table(FILE* stream) const {
     }
   }
 
-  phosg::fwrite_fmt(stream, "MAG    => ---ID--- TYPE SKIN POINTS FTBL PB AC E1 E2 E3 E4 C1 C2 C3 C4 FLAG ---DIVISOR--- NAME\n");
+  phosg::fwrite_fmt(stream, "MAGS\n");
+  phosg::fwrite_fmt(stream, "  CODE   => ---ID--- TYPE SKIN POINTS FTBL PB AC E1 E2 E3 E4 C1 C2 C3 C4 FLAG ---DIVISOR--- NAME\n");
   {
     size_t data1_1_limit = pmt->num_mags();
     for (size_t data1_1 = 0; data1_1 < data1_1_limit; data1_1++) {
@@ -816,7 +969,7 @@ void ItemNameIndex::print_table(FILE* stream) const {
       item.data1[2] = 0x00;
       string name = this->describe_item(item);
 
-      phosg::fwrite_fmt(stream, "02{:02X}00 => {:08X} {:04X} {:04X} {:6} {:04X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:04X} {} {}\n",
+      phosg::fwrite_fmt(stream, "  02{:02X}00 => {:08X} {:04X} {:04X} {:6} {:04X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:02X} {:04X} {} {}\n",
           data1_1,
           m.base.id,
           m.base.type,
@@ -839,7 +992,8 @@ void ItemNameIndex::print_table(FILE* stream) const {
     }
   }
 
-  phosg::fwrite_fmt(stream, "TOOL   => ---ID--- TYPE SKIN POINTS COUNT TECH -COST- ITEMFLAG ---DIVISOR--- NAME\n");
+  phosg::fwrite_fmt(stream, "TOOLS\n");
+  phosg::fwrite_fmt(stream, "  CODE   => ---ID--- TYPE SKIN POINTS COUNT TECH -COST- ITEMFLAG ---DIVISOR--- NAME\n");
   for (size_t data1_1 = 0; data1_1 < pmt->num_tool_classes; data1_1++) {
     float sale_divisor = pmt->get_sale_divisor(0x03, data1_1);
     string divisor_str = std::format("{:g}", sale_divisor);
@@ -856,7 +1010,7 @@ void ItemNameIndex::print_table(FILE* stream) const {
       item.set_tool_item_amount(*this->limits, 1);
       string name = this->describe_item(item);
 
-      phosg::fwrite_fmt(stream, "03{:02X}{:02X} => {:08X} {:04X} {:04X} {:6} {:5} {:04X} {:6} {:08X} {} {}\n",
+      phosg::fwrite_fmt(stream, "  03{:02X}{:02X} => {:08X} {:04X} {:04X} {:6} {:5} {:04X} {:6} {:08X} {} {}\n",
           data1_1,
           data1_2,
           t.base.id,
@@ -872,9 +1026,10 @@ void ItemNameIndex::print_table(FILE* stream) const {
     }
   }
 
-  phosg::fwrite_fmt(stream, "CLASS     =>  F GF RF  B GB RB  Z GZ RZ GR DB JL ZL SH RY RS AT RV MG\n");
+  phosg::fwrite_fmt(stream, "MAX TECH LEVELS\n");
+  phosg::fwrite_fmt(stream, "  CLASS     =>  F GF RF  B GB RB  Z GZ RZ GR DB JL ZL SH RY RS AT RV MG\n");
   for (size_t char_class = 0; char_class < 12; char_class++) {
-    phosg::fwrite_fmt(stream, "{:9} =>", name_for_char_class(char_class));
+    phosg::fwrite_fmt(stream, "  {:9} =>", name_for_char_class(char_class));
     for (size_t tech_num = 0; tech_num < 0x13; tech_num++) {
       uint8_t max_level = pmt->get_max_tech_level(char_class, tech_num) + 1;
       if (max_level == 0x00) {
@@ -886,30 +1041,40 @@ void ItemNameIndex::print_table(FILE* stream) const {
     phosg::fwrite_fmt(stream, "\n");
   }
 
+  phosg::fwrite_fmt(stream, "MAG FEED TABLES\n");
   for (size_t table_index = 0; table_index < 8; table_index++) {
     static const char* names[11] = {
         "Monomate", "Dimate", "Trimate", "Monofluid",
         "Difluid", "Trifluid", "Antidote", "Antiparalysis",
         "Sol Atomizer", "Moon Atomizer", "Star Atomizer"};
-    phosg::fwrite_fmt(stream, "TABLE {:02X}       => -DEF -POW -DEX MIND -IQ- SYNC\n", table_index);
+    phosg::fwrite_fmt(stream, "  TABLE {:02X}         => -DEF -POW -DEX MIND -IQ- SYNC\n", table_index);
     for (size_t which = 0; which < 11; which++) {
       const auto& res = pmt->get_mag_feed_result(table_index, which);
-      phosg::fwrite_fmt(stream, "{:14} => {:4} {:4} {:4} {:4} {:4} {:4}\n",
+      phosg::fwrite_fmt(stream, "    {:14} => {:4} {:4} {:4} {:4} {:4} {:4}\n",
           names[which], res.def, res.pow, res.dex, res.mind, res.iq, res.synchro);
     }
   }
 
-  phosg::fwrite_fmt(stream, "SPECIAL => TYPE COUNT ST*\n");
+  phosg::fwrite_fmt(stream, "SPECIAL DEFINITIONS\n");
+  phosg::fwrite_fmt(stream, "  SPECIAL => TYPE COUNT ST* NAME\n");
   for (size_t index = 0; index < pmt->num_specials; index++) {
     const auto& sp = pmt->get_special(index);
     uint8_t stars = pmt->get_special_stars(index);
-    phosg::fwrite_fmt(stream, "     {:02X} => {:04X} {:5} {:2}*\n", index, sp.type, sp.amount, stars);
+    const char* name = "";
+    if (index) {
+      try {
+        name = name_for_weapon_special.at(index);
+      } catch (const out_of_range&) {
+      }
+    }
+    phosg::fwrite_fmt(stream, "       {:02X} => {:04X} {:5} {:2}* {}\n", index, sp.type, sp.amount, stars, name);
   }
 
-  phosg::fwrite_fmt(stream, "---USE + -EQUIP => RESULT MLV GND LVL CLS\n");
+  phosg::fwrite_fmt(stream, "ITEM COMBINATIONS\n");
+  phosg::fwrite_fmt(stream, "  ---USE + -EQUIP => RESULT MLV GND LVL CLS\n");
   for (const auto& combo_list_it : pmt->get_all_item_combinations()) {
     for (const auto& combo : combo_list_it.second) {
-      phosg::fwrite_fmt(stream, "{:02X}{:02X}{:02X} + {:02X}{:02X}{:02X} => {:02X}{:02X}{:02X}",
+      phosg::fwrite_fmt(stream, "  {:02X}{:02X}{:02X} + {:02X}{:02X}{:02X} => {:02X}{:02X}{:02X}",
           combo.used_item[0], combo.used_item[1], combo.used_item[2],
           combo.equipped_item[0], combo.equipped_item[1], combo.equipped_item[2],
           combo.result_item[0], combo.result_item[1], combo.result_item[2]);
@@ -936,13 +1101,14 @@ void ItemNameIndex::print_table(FILE* stream) const {
     }
   }
 
+  phosg::fwrite_fmt(stream, "EVENT ITEMS\n");
   size_t num_events = pmt->num_events();
   for (size_t event_number = 0; event_number < num_events; event_number++) {
-    phosg::fwrite_fmt(stream, "EV {:3} => PRB\n", event_number);
+    phosg::fwrite_fmt(stream, "  EV {:3} => PRB\n", event_number);
     auto events_list = pmt->get_event_items(event_number);
     for (size_t z = 0; z < events_list.second; z++) {
       const auto& event_item = events_list.first[z];
-      phosg::fwrite_fmt(stream, "{:02X}{:02X}{:02X} => {:3}\n",
+      phosg::fwrite_fmt(stream, "  {:02X}{:02X}{:02X} => {:3}\n",
           event_item.item[0], event_item.item[1], event_item.item[2], event_item.probability);
     }
   }

src/ItemNameIndex.hh

@@ -38,7 +38,12 @@ public:
   inline bool exists(const ItemData& item) const {
     return this->primary_identifier_index.count(item.primary_identifier());
   }
-  std::string describe_item(const ItemData& item, bool include_color_escapes = false, bool hide_mag_stats = false) const;
+
+  enum Flag : uint8_t {
+    INCLUDE_PSO_COLOR_ESCAPES = 0x01,
+    NAME_ONLY = 0x02,
+  };
+  std::string describe_item(const ItemData& item, uint8_t flags = 0) const;
   ItemData parse_item_description(const std::string& description) const;
 
   void print_table(FILE* stream) const;
@@ -51,4 +56,5 @@ private:
 
   std::unordered_map<uint32_t, std::shared_ptr<const ItemMetadata>> primary_identifier_index;
   std::map<std::string, std::shared_ptr<const ItemMetadata>> name_index;
+  std::map<std::string, std::shared_ptr<const ItemMetadata>> es_name_index;
 };

src/ItemParameterTable.cc

@@ -4,6 +4,41 @@
 
 using namespace std;
 
+template <>
+ServerDropMode phosg::enum_for_name<ServerDropMode>(const char* name) {
+  if (!strcmp(name, "DISABLED")) {
+    return ServerDropMode::DISABLED;
+  } else if (!strcmp(name, "CLIENT")) {
+    return ServerDropMode::CLIENT;
+  } else if (!strcmp(name, "SERVER_SHARED")) {
+    return ServerDropMode::SERVER_SHARED;
+  } else if (!strcmp(name, "SERVER_PRIVATE")) {
+    return ServerDropMode::SERVER_PRIVATE;
+  } else if (!strcmp(name, "SERVER_DUPLICATE")) {
+    return ServerDropMode::SERVER_DUPLICATE;
+  } else {
+    throw runtime_error("invalid drop mode");
+  }
+}
+
+template <>
+const char* phosg::name_for_enum<ServerDropMode>(ServerDropMode value) {
+  switch (value) {
+    case ServerDropMode::DISABLED:
+      return "DISABLED";
+    case ServerDropMode::CLIENT:
+      return "CLIENT";
+    case ServerDropMode::SERVER_SHARED:
+      return "SERVER_SHARED";
+    case ServerDropMode::SERVER_PRIVATE:
+      return "SERVER_PRIVATE";
+    case ServerDropMode::SERVER_DUPLICATE:
+      return "SERVER_DUPLICATE";
+    default:
+      throw runtime_error("invalid drop mode");
+  }
+}
+
 ItemParameterTable::ItemParameterTable(shared_ptr<const string> data, Version version)
     : version(version),
       data(data),
@@ -175,7 +210,7 @@ ItemParameterTable::WeaponV4 ItemParameterTable::WeaponV1V2::to_v4() const {
   ret.photon = this->photon;
   ret.special = this->special;
   ret.ata = this->ata;
-  ret.stat_boost = this->stat_boost;
+  ret.stat_boost_entry_index = this->stat_boost_entry_index;
   return ret;
 }
 
@@ -195,7 +230,7 @@ ItemParameterTable::WeaponV4 ItemParameterTable::WeaponGCNTE::to_v4() const {
   ret.photon = this->photon;
   ret.special = this->special;
   ret.ata = this->ata;
-  ret.stat_boost = this->stat_boost;
+  ret.stat_boost_entry_index = this->stat_boost_entry_index;
   ret.projectile = this->projectile;
   ret.trail1_x = this->trail1_x;
   ret.trail1_y = this->trail1_y;
@@ -203,8 +238,6 @@ ItemParameterTable::WeaponV4 ItemParameterTable::WeaponGCNTE::to_v4() const {
   ret.trail2_y = this->trail2_y;
   ret.color = this->color;
   ret.unknown_a1 = this->unknown_a1;
-  ret.unknown_a2 = this->unknown_a2;
-  ret.unknown_a3 = this->unknown_a3;
   return ret;
 }
 
@@ -225,7 +258,7 @@ ItemParameterTable::WeaponV4 ItemParameterTable::WeaponV3T<BE>::to_v4() const {
   ret.photon = this->photon;
   ret.special = this->special;
   ret.ata = this->ata;
-  ret.stat_boost = this->stat_boost;
+  ret.stat_boost_entry_index = this->stat_boost_entry_index;
   ret.projectile = this->projectile;
   ret.trail1_x = this->trail1_x;
   ret.trail1_y = this->trail1_y;
@@ -233,8 +266,6 @@ ItemParameterTable::WeaponV4 ItemParameterTable::WeaponV3T<BE>::to_v4() const {
   ret.trail2_y = this->trail2_y;
   ret.color = this->color;
   ret.unknown_a1 = this->unknown_a1;
-  ret.unknown_a2 = this->unknown_a2;
-  ret.unknown_a3 = this->unknown_a3;
   ret.unknown_a4 = this->unknown_a4;
   ret.unknown_a5 = this->unknown_a5;
   ret.tech_boost = this->tech_boost;
@@ -277,7 +308,7 @@ ItemParameterTable::ArmorOrShieldV4 ItemParameterTable::ArmorOrShieldV1V2::to_v4
   ret.elt = this->elt;
   ret.dfp_range = this->dfp_range;
   ret.evp_range = this->evp_range;
-  ret.stat_boost = this->stat_boost;
+  ret.stat_boost_entry_index = this->stat_boost_entry_index;
   ret.tech_boost = this->tech_boost;
   ret.flags_type = this->flags_type;
   ret.unknown_a4 = this->unknown_a4;
@@ -303,7 +334,7 @@ ItemParameterTable::ArmorOrShieldV4 ItemParameterTable::ArmorOrShieldV3T<BE>::to
   ret.elt = this->elt;
   ret.dfp_range = this->dfp_range;
   ret.evp_range = this->evp_range;
-  ret.stat_boost = this->stat_boost;
+  ret.stat_boost_entry_index = this->stat_boost_entry_index;
   ret.tech_boost = this->tech_boost;
   ret.flags_type = this->flags_type;
   ret.unknown_a4 = this->unknown_a4;
@@ -522,25 +553,22 @@ const ItemParameterTable::ArmorOrShieldV4& ItemParameterTable::get_armor_or_shie
     }
     return ret;
   } catch (const std::out_of_range&) {
-    ArmorOrShieldV4 def_v4;
-
-    if (this->offsets_dc_protos) {
-      def_v4 = indirect_lookup_2d<ArmorOrShieldDCProtos, false>(this->r, this->offsets_dc_protos->armor_table, data1_1 - 1, data1_2).to_v4();
-    } else if (this->offsets_v1_v2) {
-      def_v4 = indirect_lookup_2d<ArmorOrShieldV1V2, false>(this->r, this->offsets_v1_v2->armor_table, data1_1 - 1, data1_2).to_v4();
-    } else if (this->offsets_gc_nte) {
-      def_v4 = indirect_lookup_2d<ArmorOrShieldV3BE, true>(this->r, this->offsets_gc_nte->armor_table, data1_1 - 1, data1_2).to_v4();
-    } else if (this->offsets_v3_le) {
-      def_v4 = indirect_lookup_2d<ArmorOrShieldV3, false>(this->r, this->offsets_v3_le->armor_table, data1_1 - 1, data1_2).to_v4();
-    } else if (this->offsets_v3_be) {
-      def_v4 = indirect_lookup_2d<ArmorOrShieldV3BE, true>(this->r, this->offsets_v3_be->armor_table, data1_1 - 1, data1_2).to_v4();
-    } else {
-      throw logic_error("table is not v2, v3, or v4");
+    while (data1_2 >= parsed_vec.size()) {
+      auto& def_v4 = parsed_vec.emplace_back();
+      if (this->offsets_dc_protos) {
+        def_v4 = indirect_lookup_2d<ArmorOrShieldDCProtos, false>(this->r, this->offsets_dc_protos->armor_table, data1_1 - 1, parsed_vec.size() - 1).to_v4();
+      } else if (this->offsets_v1_v2) {
+        def_v4 = indirect_lookup_2d<ArmorOrShieldV1V2, false>(this->r, this->offsets_v1_v2->armor_table, data1_1 - 1, parsed_vec.size() - 1).to_v4();
+      } else if (this->offsets_gc_nte) {
+        def_v4 = indirect_lookup_2d<ArmorOrShieldV3BE, true>(this->r, this->offsets_gc_nte->armor_table, data1_1 - 1, parsed_vec.size() - 1).to_v4();
+      } else if (this->offsets_v3_le) {
+        def_v4 = indirect_lookup_2d<ArmorOrShieldV3, false>(this->r, this->offsets_v3_le->armor_table, data1_1 - 1, parsed_vec.size() - 1).to_v4();
+      } else if (this->offsets_v3_be) {
+        def_v4 = indirect_lookup_2d<ArmorOrShieldV3BE, true>(this->r, this->offsets_v3_be->armor_table, data1_1 - 1, parsed_vec.size() - 1).to_v4();
+      } else {
+        throw logic_error("table is not v2, v3, or v4");
+      }
     }
-    if (data1_2 >= parsed_vec.size()) {
-      parsed_vec.resize(data1_2 + 1);
-    }
-    parsed_vec[data1_2] = def_v4;
     return parsed_vec[data1_2];
   }
 }
@@ -575,24 +603,22 @@ const ItemParameterTable::UnitV4& ItemParameterTable::get_unit(uint8_t data1_2)
     }
     return ret;
   } catch (const std::out_of_range&) {
-    UnitV4 def_v4;
-    if (this->offsets_dc_protos) {
-      def_v4 = indirect_lookup_2d<UnitDCProtos, false>(this->r, this->offsets_dc_protos->unit_table, 0, data1_2).to_v4();
-    } else if (this->offsets_v1_v2) {
-      def_v4 = indirect_lookup_2d<UnitV1V2, false>(this->r, this->offsets_v1_v2->unit_table, 0, data1_2).to_v4();
-    } else if (this->offsets_gc_nte) {
-      def_v4 = indirect_lookup_2d<UnitV3BE, true>(this->r, this->offsets_gc_nte->unit_table, 0, data1_2).to_v4();
-    } else if (this->offsets_v3_le) {
-      def_v4 = indirect_lookup_2d<UnitV3, false>(this->r, this->offsets_v3_le->unit_table, 0, data1_2).to_v4();
-    } else if (this->offsets_v3_be) {
-      def_v4 = indirect_lookup_2d<UnitV3BE, true>(this->r, this->offsets_v3_be->unit_table, 0, data1_2).to_v4();
-    } else {
-      throw logic_error("table is not v2, v3, or v4");
+    while (data1_2 >= this->parsed_units.size()) {
+      auto& def_v4 = this->parsed_units.emplace_back();
+      if (this->offsets_dc_protos) {
+        def_v4 = indirect_lookup_2d<UnitDCProtos, false>(this->r, this->offsets_dc_protos->unit_table, 0, this->parsed_units.size() - 1).to_v4();
+      } else if (this->offsets_v1_v2) {
+        def_v4 = indirect_lookup_2d<UnitV1V2, false>(this->r, this->offsets_v1_v2->unit_table, 0, this->parsed_units.size() - 1).to_v4();
+      } else if (this->offsets_gc_nte) {
+        def_v4 = indirect_lookup_2d<UnitV3BE, true>(this->r, this->offsets_gc_nte->unit_table, 0, this->parsed_units.size() - 1).to_v4();
+      } else if (this->offsets_v3_le) {
+        def_v4 = indirect_lookup_2d<UnitV3, false>(this->r, this->offsets_v3_le->unit_table, 0, this->parsed_units.size() - 1).to_v4();
+      } else if (this->offsets_v3_be) {
+        def_v4 = indirect_lookup_2d<UnitV3BE, true>(this->r, this->offsets_v3_be->unit_table, 0, this->parsed_units.size() - 1).to_v4();
+      } else {
+        throw logic_error("table is not v2, v3, or v4");
+      }
     }
-    if (data1_2 >= this->parsed_units.size()) {
-      this->parsed_units.resize(data1_2 + 1);
-    }
-    this->parsed_units[data1_2] = def_v4;
     return this->parsed_units[data1_2];
   }
 }
@@ -627,28 +653,26 @@ const ItemParameterTable::MagV4& ItemParameterTable::get_mag(uint8_t data1_1) co
     }
     return ret;
   } catch (const std::out_of_range&) {
-    MagV4 def_v4;
-    if (this->offsets_dc_protos) {
-      def_v4 = indirect_lookup_2d<MagV1, false>(this->r, this->offsets_dc_protos->mag_table, 0, data1_1).to_v4();
-    } else if (this->offsets_v1_v2) {
-      if (is_v1(this->version)) {
-        def_v4 = indirect_lookup_2d<MagV1, false>(this->r, this->offsets_v1_v2->mag_table, 0, data1_1).to_v4();
+    while (data1_1 >= this->parsed_mags.size()) {
+      auto& def_v4 = this->parsed_mags.emplace_back();
+      if (this->offsets_dc_protos) {
+        def_v4 = indirect_lookup_2d<MagV1, false>(this->r, this->offsets_dc_protos->mag_table, 0, this->parsed_mags.size() - 1).to_v4();
+      } else if (this->offsets_v1_v2) {
+        if (is_v1(this->version)) {
+          def_v4 = indirect_lookup_2d<MagV1, false>(this->r, this->offsets_v1_v2->mag_table, 0, this->parsed_mags.size() - 1).to_v4();
+        } else {
+          def_v4 = indirect_lookup_2d<MagV2, false>(this->r, this->offsets_v1_v2->mag_table, 0, this->parsed_mags.size() - 1).to_v4();
+        }
+      } else if (this->offsets_gc_nte) {
+        def_v4 = indirect_lookup_2d<MagV3BE, true>(this->r, this->offsets_gc_nte->mag_table, 0, this->parsed_mags.size() - 1).to_v4();
+      } else if (this->offsets_v3_le) {
+        def_v4 = indirect_lookup_2d<MagV3, false>(this->r, this->offsets_v3_le->mag_table, 0, this->parsed_mags.size() - 1).to_v4();
+      } else if (this->offsets_v3_be) {
+        def_v4 = indirect_lookup_2d<MagV3BE, true>(this->r, this->offsets_v3_be->mag_table, 0, this->parsed_mags.size() - 1).to_v4();
       } else {
-        def_v4 = indirect_lookup_2d<MagV2, false>(this->r, this->offsets_v1_v2->mag_table, 0, data1_1).to_v4();
+        throw logic_error("table is not v2, v3, or v4");
       }
-    } else if (this->offsets_gc_nte) {
-      def_v4 = indirect_lookup_2d<MagV3BE, true>(this->r, this->offsets_gc_nte->mag_table, 0, data1_1).to_v4();
-    } else if (this->offsets_v3_le) {
-      def_v4 = indirect_lookup_2d<MagV3, false>(this->r, this->offsets_v3_le->mag_table, 0, data1_1).to_v4();
-    } else if (this->offsets_v3_be) {
-      def_v4 = indirect_lookup_2d<MagV3BE, true>(this->r, this->offsets_v3_be->mag_table, 0, data1_1).to_v4();
-    } else {
-      throw logic_error("table is not v2, v3, or v4");
     }
-    if (data1_1 >= this->parsed_mags.size()) {
-      this->parsed_mags.resize(data1_1 + 1);
-    }
-    this->parsed_mags[data1_1] = def_v4;
     return this->parsed_mags[data1_1];
   }
 }
@@ -928,6 +952,38 @@ const ItemParameterTable::Special& ItemParameterTable::get_special(uint8_t speci
   }
 }
 
+const ItemParameterTable::StatBoost& ItemParameterTable::get_stat_boost(uint8_t entry_index) const {
+  if (this->offsets_dc_protos) {
+    return this->r.pget<StatBoost>(this->offsets_dc_protos->stat_boost_table + sizeof(StatBoost) * entry_index);
+  } else if (this->offsets_v1_v2) {
+    return this->r.pget<StatBoost>(this->offsets_v1_v2->stat_boost_table + sizeof(StatBoost) * entry_index);
+  } else if (this->offsets_v3_le) {
+    return this->r.pget<StatBoost>(this->offsets_v3_le->stat_boost_table + sizeof(StatBoost) * entry_index);
+  } else if (this->offsets_gc_nte) {
+    while (entry_index >= this->parsed_stat_boosts.size()) {
+      const auto& sb_be = this->r.pget<StatBoostBE>(this->offsets_gc_nte->stat_boost_table + sizeof(StatBoostBE) * this->parsed_stat_boosts.size());
+      auto& sb = this->parsed_stat_boosts.emplace_back();
+      sb.stats = sb_be.stats;
+      sb.amounts[0] = sb_be.amounts[0];
+      sb.amounts[1] = sb_be.amounts[1];
+    }
+    return this->parsed_stat_boosts[entry_index];
+  } else if (this->offsets_v3_be) {
+    while (entry_index >= this->parsed_stat_boosts.size()) {
+      const auto& sb_be = this->r.pget<StatBoostBE>(this->offsets_v3_be->stat_boost_table + sizeof(StatBoostBE) * this->parsed_stat_boosts.size());
+      auto& sb = this->parsed_stat_boosts.emplace_back();
+      sb.stats = sb_be.stats;
+      sb.amounts[0] = sb_be.amounts[0];
+      sb.amounts[1] = sb_be.amounts[1];
+    }
+    return this->parsed_stat_boosts[entry_index];
+  } else if (this->offsets_v4) {
+    return this->r.pget<StatBoost>(this->offsets_v4->stat_boost_table + sizeof(StatBoost) * entry_index);
+  } else {
+    throw logic_error("table is not v2, v3, or v4");
+  }
+}
+
 uint8_t ItemParameterTable::get_max_tech_level(uint8_t char_class, uint8_t tech_num) const {
   if (char_class >= 12) {
     throw out_of_range("invalid character class");

src/ItemParameterTable.hh

@@ -1,5 +1,7 @@
 #pragma once
 
+#include "WindowsPlatform.hh"
+
 #include <stdint.h>
 
 #include <array>
@@ -16,6 +18,26 @@
 #include "Types.hh"
 #include "Version.hh"
 
+// TODO: These don't really belong here, but putting them anywhere else creates
+// annoying dependency cycles. Find or make a better place for these.
+enum class ServerDropMode {
+  DISABLED = 0,
+  CLIENT = 1, // Not allowed for BB games
+  SERVER_SHARED = 2,
+  SERVER_PRIVATE = 3,
+  SERVER_DUPLICATE = 4,
+};
+enum class ProxyDropMode {
+  DISABLED = 0,
+  PASSTHROUGH,
+  INTERCEPT,
+};
+
+template <>
+ServerDropMode phosg::enum_for_name<ServerDropMode>(const char* name);
+template <>
+const char* phosg::name_for_enum<ServerDropMode>(ServerDropMode value);
+
 class ItemParameterTable {
 public:
   // TODO: This implementation is ugly. We should use real classes and virtual
@@ -71,7 +93,7 @@ public:
     /* 11 */ uint8_t photon = 0;
     /* 12 */ uint8_t special = 0;
     /* 13 */ uint8_t ata = 0;
-    /* 14 */ uint8_t stat_boost = 0; // TODO: This could be larger (16 or 32 bits)
+    /* 14 */ uint8_t stat_boost_entry_index = 0; // TODO: This could be larger (16 or 32 bits)
     /* 15 */ parray<uint8_t, 3> unknown_a9;
     /* 18 */
 
@@ -91,16 +113,14 @@ public:
     /* 17 */ uint8_t photon = 0;
     /* 18 */ uint8_t special = 0;
     /* 19 */ uint8_t ata = 0;
-    /* 1A */ uint8_t stat_boost = 0;
+    /* 1A */ uint8_t stat_boost_entry_index = 0;
     /* 1B */ uint8_t projectile = 0;
     /* 1C */ int8_t trail1_x = 0;
     /* 1D */ int8_t trail1_y = 0;
     /* 1E */ int8_t trail2_x = 0;
     /* 1F */ int8_t trail2_y = 0;
     /* 20 */ int8_t color = 0;
-    /* 21 */ uint8_t unknown_a1 = 0;
-    /* 22 */ uint8_t unknown_a2 = 0;
-    /* 23 */ uint8_t unknown_a3 = 0;
+    /* 21 */ parray<uint8_t, 3> unknown_a1 = 0;
     /* 24 */
 
     WeaponV4 to_v4() const;
@@ -120,16 +140,14 @@ public:
     /* 17 */ uint8_t photon = 0;
     /* 18 */ uint8_t special = 0;
     /* 19 */ uint8_t ata = 0;
-    /* 1A */ uint8_t stat_boost = 0;
+    /* 1A */ uint8_t stat_boost_entry_index = 0;
     /* 1B */ uint8_t projectile = 0;
     /* 1C */ int8_t trail1_x = 0;
     /* 1D */ int8_t trail1_y = 0;
     /* 1E */ int8_t trail2_x = 0;
     /* 1F */ int8_t trail2_y = 0;
     /* 20 */ int8_t color = 0;
-    /* 21 */ uint8_t unknown_a1 = 0;
-    /* 22 */ uint8_t unknown_a2 = 0;
-    /* 23 */ uint8_t unknown_a3 = 0;
+    /* 21 */ parray<uint8_t, 3> unknown_a1 = 0;
     /* 24 */ uint8_t unknown_a4 = 0;
     /* 25 */ uint8_t unknown_a5 = 0;
     /* 26 */ uint8_t tech_boost = 0;
@@ -162,16 +180,14 @@ public:
     /* 1B */ uint8_t photon = 0;
     /* 1C */ uint8_t special = 0;
     /* 1D */ uint8_t ata = 0;
-    /* 1E */ uint8_t stat_boost = 0;
+    /* 1E */ uint8_t stat_boost_entry_index = 0;
     /* 1F */ uint8_t projectile = 0;
     /* 20 */ int8_t trail1_x = 0;
     /* 21 */ int8_t trail1_y = 0;
     /* 22 */ int8_t trail2_x = 0;
     /* 23 */ int8_t trail2_y = 0;
     /* 24 */ int8_t color = 0;
-    /* 25 */ uint8_t unknown_a1 = 0;
-    /* 26 */ uint8_t unknown_a2 = 0;
-    /* 27 */ uint8_t unknown_a3 = 0;
+    /* 25 */ parray<uint8_t, 3> unknown_a1 = 0;
     /* 28 */ uint8_t unknown_a4 = 0;
     /* 29 */ uint8_t unknown_a5 = 0;
     /* 2A */ uint8_t tech_boost = 0;
@@ -201,7 +217,7 @@ public:
 
   template <typename BaseT, bool BE>
   struct ArmorOrShieldFinalT : ArmorOrShieldT<BaseT, BE> {
-    /* 14 */ uint8_t stat_boost = 0;
+    /* 14 */ uint8_t stat_boost_entry_index = 0;
     /* 15 */ uint8_t tech_boost = 0;
     // TODO: Figure out what this does. Only a few values appear to do anything:
     // Shields:
@@ -391,10 +407,30 @@ public:
 
   template <bool BE>
   struct StatBoostT {
-    uint8_t stat1 = 0;
-    uint8_t stat2 = 0;
-    U16T<BE> amount1 = 0;
-    U16T<BE> amount2 = 0;
+    // Only the first of these stat/amount pairs is used in most versions of
+    // the game. In DC 11/2000 Sega apparently changed the loop from
+    // `for (z = 0; z != 2; z++)` to `for (z = 0; z != 1; z++)`, so only the
+    // first stat/amount pair is used on all versions after DC NTE.
+    // Values for stats:
+    //   01 = ATP bonus
+    //   02 = ATA bonus
+    //   03 = EVP bonus
+    //   04 = DFP bonus
+    //   05 = MST bonus
+    //   06 = HP bonus
+    //   07 = LCK bonus
+    //   08 = all of the above bonuses except HP
+    //   09 = ATP penalty
+    //   0A = ATA penalty
+    //   0B = EVP penalty
+    //   0C = DFP penalty
+    //   0D = MST penalty
+    //   0E = HP penalty
+    //   0F = LCK penalty
+    //   10 = all of the above penalties except HP
+    //   Anything else (including 00) = no bonus or penalty
+    parray<uint8_t, 2> stats = 0;
+    parray<U16T<BE>, 2> amounts;
   } __attribute__((packed));
   using StatBoost = StatBoostT<false>;
   using StatBoostBE = StatBoostT<true>;
@@ -476,6 +512,7 @@ public:
   uint8_t get_item_stars(uint32_t id) const;
   uint8_t get_special_stars(uint8_t special) const;
   const Special& get_special(uint8_t special) const;
+  const StatBoost& get_stat_boost(uint8_t entry_index) const;
   uint8_t get_max_tech_level(uint8_t char_class, uint8_t tech_num) const;
   uint8_t get_weapon_v1_replacement(uint8_t data1_1) const;
 
@@ -624,6 +661,7 @@ protected:
   mutable std::vector<MagV4> parsed_mags;
   mutable std::unordered_map<uint16_t, ToolV4> parsed_tools;
   mutable std::vector<Special> parsed_specials;
+  mutable std::vector<StatBoost> parsed_stat_boosts;
 
   // Key is used_item. We can't index on (used_item, equipped_item) because
   // equipped_item may contain wildcards, and the matching order matters.
@@ -647,7 +685,7 @@ public:
       // This specifies which entry in ItemMagMotion.dat is used. The file is
       // just a list of 0x64-byte structures. 0xFF = no TItemMagSub is created
       uint8_t motion_table_entry = 0xFF;
-      parray<uint8_t, 5> unknown_a1;
+      parray<uint8_t, 5> unknown_a1 = 0;
     } __packed_ws__(Side, 0x06);
     parray<Side, 2> sides; // [0] = right side, [1] = left side
   } __packed_ws__(MotionReference, 0x0C);

src/Items.cc

@@ -16,7 +16,7 @@ void player_use_item(shared_ptr<Client> c, size_t item_index, shared_ptr<RandomG
   bool is_v3_or_later = is_v3(c->version()) || is_v4;
   bool should_delete_item = is_v3_or_later;
 
-  auto player = c->character();
+  auto player = c->character_file();
   auto& item = player->inventory.items[item_index];
   uint32_t primary_identifier = item.data.primary_identifier();
 
@@ -47,7 +47,7 @@ void player_use_item(shared_ptr<Client> c, size_t item_index, shared_ptr<RandomG
     weapon.data.data1[3] = min<uint8_t>(weapon.data.data1[3] + item.data.data1[2] + 1, weapon_def.max_grind);
 
   } else if ((primary_identifier & 0xFFFF0000) == 0x030B0000) { // Material
-    auto p = c->character();
+    auto p = c->character_file();
 
     using Type = PSOBBCharacterFile::MaterialType;
     Type type;
@@ -200,6 +200,16 @@ void player_use_item(shared_ptr<Client> c, size_t item_index, shared_ptr<RandomG
       }
     }
 
+  } else if (primary_identifier == 0x03170000) { // Unopened Hunters Report
+    // The unopened Hunters Report's rank is stored in the kill count field; using the unopened report copies the rank
+    // to data1[2] and replaces the inventory item with a new item with the same ID. The game also moves the item to
+    // the end of the inventory, so we do the same.
+    const auto& stack_limits = *s->item_stack_limits(c->version());
+    auto report_item = player->remove_item(item.data.id, 1, stack_limits);
+    report_item.data1[2] = report_item.get_kill_count();
+    player->add_item(report_item, stack_limits);
+    should_delete_item = false;
+
   } else {
     // Use item combinations table from ItemPMT
     bool combo_applied = false;
@@ -499,7 +509,7 @@ void apply_mag_feed_result(
 
 void player_feed_mag(std::shared_ptr<Client> c, size_t mag_item_index, size_t fed_item_index) {
   auto s = c->require_server_state();
-  auto player = c->character();
+  auto player = c->character_file();
   apply_mag_feed_result(
       player->inventory.items[mag_item_index].data,
       player->inventory.items[fed_item_index].data,

src/Lobby.cc

@@ -146,26 +146,9 @@ Lobby::Lobby(shared_ptr<ServerState> s, uint32_t id, bool is_game)
     : server_state(s),
       log(std::format("[{}:{:X}] ", is_game ? "Game" : "Lobby", id), lobby_log.min_level),
       lobby_id(id),
-      min_level(0),
-      max_level(0xFFFFFFFF),
-      next_game_item_id(0xCC000000),
-      allowed_versions(0x0000),
-      override_section_id(0xFF),
-      episode(Episode::NONE),
-      mode(GameMode::NORMAL),
-      difficulty(0),
-      base_exp_multiplier(1),
-      exp_share_multiplier(0.5),
-      challenge_exp_multiplier(1.0f),
       random_seed(phosg::random_object<uint32_t>()),
       rand_crypt(make_shared<DisabledRandomGenerator>()),
-      drop_mode(DropMode::CLIENT),
-      event(0),
-      block(0),
-      leader_id(0),
-      max_clients(12),
-      enabled_flags(0),
-      idle_timeout_usecs(0),
+      drop_mode(ServerDropMode::CLIENT),
       idle_timeout_timer(*s->io_context) {
   this->log.info_f("Created");
   if (is_game) {
@@ -186,6 +169,14 @@ void Lobby::reset_next_item_ids() {
   this->next_game_item_id = 0xCC000000;
 }
 
+uint8_t Lobby::area_for_floor(Version version, uint8_t floor) const {
+  if (this->quest) {
+    return this->quest->meta.area_for_floor.at(floor);
+  }
+  auto sdt = this->require_server_state()->set_data_table(version, this->episode, this->mode, this->difficulty);
+  return sdt->default_area_for_floor(this->episode, floor);
+}
+
 shared_ptr<ServerState> Lobby::require_server_state() const {
   auto s = this->server_state.lock();
   if (!s) {
@@ -214,62 +205,31 @@ void Lobby::create_item_creator(Version logic_version) {
     logic_version = leader_c ? leader_c->version() : Version::BB_V4;
   }
 
-  shared_ptr<const RareItemSet> rare_item_set;
-  shared_ptr<const CommonItemSet> common_item_set;
-  switch (logic_version) {
-    case Version::PC_PATCH:
-    case Version::BB_PATCH:
-    case Version::GC_EP3_NTE:
-    case Version::GC_EP3:
-      throw runtime_error("cannot create item creator for this base version");
-    case Version::DC_NTE:
-    case Version::DC_11_2000:
-    case Version::DC_V1:
-      // TODO: We should probably have a v1 common item set at some point too
-      common_item_set = s->common_item_set_v2;
-      rare_item_set = s->rare_item_sets.at("rare-table-v1");
-      break;
-    case Version::DC_V2:
-    case Version::PC_NTE:
-    case Version::PC_V2:
-      common_item_set = s->common_item_set_v2;
-      rare_item_set = s->rare_item_sets.at("rare-table-v2");
-      break;
-    case Version::GC_NTE:
-    case Version::GC_V3:
-    case Version::XB_V3:
-      common_item_set = s->common_item_set_v3_v4;
-      rare_item_set = s->rare_item_sets.at("rare-table-v3");
-      break;
-    case Version::BB_V4:
-      common_item_set = s->common_item_set_v3_v4;
-      rare_item_set = s->rare_item_sets.at("rare-table-v4");
-      break;
-    default:
-      throw logic_error("invalid lobby base version");
-  }
-
   shared_ptr<RandomGenerator> rand_crypt;
   if (s->use_psov2_rand_crypt) {
     rand_crypt = make_shared<PSOV2Encryption>(this->rand_crypt->seed());
   } else {
     rand_crypt = make_shared<MT19937Generator>(this->rand_crypt->seed());
   }
+  uint8_t effective_section_id = this->effective_section_id();
+  if (effective_section_id >= 10) {
+    effective_section_id = 0x00;
+  }
   this->item_creator = make_shared<ItemCreator>(
-      common_item_set,
-      rare_item_set,
+      s->common_item_set(logic_version, this->quest),
+      s->rare_item_set(logic_version, this->quest),
       s->armor_random_set,
       s->tool_random_set,
-      s->weapon_random_sets.at(this->difficulty),
+      s->weapon_random_set(this->difficulty),
       s->tekker_adjustment_set,
       s->item_parameter_table(logic_version),
       s->item_stack_limits(logic_version),
       this->episode,
       (this->mode == GameMode::SOLO) ? GameMode::NORMAL : this->mode,
       this->difficulty,
-      this->effective_section_id(),
+      effective_section_id,
       rand_crypt,
-      this->quest ? this->quest->battle_rules : nullptr);
+      this->quest ? this->quest->meta.battle_rules : nullptr);
 }
 
 uint8_t Lobby::effective_section_id() const {
@@ -281,9 +241,9 @@ uint8_t Lobby::effective_section_id() const {
   }
   auto leader = this->clients.at(this->leader_id);
   if (leader) {
-    return leader->character()->disp.visual.section_id;
+    return leader->character_file()->disp.visual.section_id;
   }
-  return 0;
+  return 0xFF;
 }
 
 uint16_t Lobby::quest_version_flags() const {
@@ -296,28 +256,29 @@ uint16_t Lobby::quest_version_flags() const {
   return ret;
 }
 
+uint8_t Lobby::client_extension_flags() const {
+  for (auto lc : this->clients) {
+    if (lc && !lc->check_flag(Client::Flag::HAS_ENEMY_DAMAGE_SYNC_PATCH)) {
+      return 0x01;
+    }
+  }
+  return 0x81;
+}
+
 void Lobby::load_maps() {
   auto rare_rates = this->rare_enemy_rates ? this->rare_enemy_rates : MapState::DEFAULT_RARE_ENEMIES;
 
   if (this->quest) {
+    this->log.info_f("Loading quest supermap");
+    auto supermap = this->quest->get_supermap(this->random_seed);
     this->map_state = make_shared<MapState>(
-        this->lobby_id,
-        this->difficulty,
-        this->event,
-        this->random_seed,
-        this->rare_enemy_rates,
-        this->rand_crypt,
-        this->quest->get_supermap(this->random_seed));
+        this->lobby_id, this->difficulty, this->event, this->random_seed, this->rare_enemy_rates, this->rand_crypt, supermap);
   } else {
+    this->log.info_f("Loading free play supermaps");
     auto s = this->require_server_state();
+    auto supermaps = s->supermaps_for_variations(this->episode, this->mode, this->difficulty, this->variations);
     this->map_state = make_shared<MapState>(
-        this->lobby_id,
-        this->difficulty,
-        this->event,
-        this->random_seed,
-        this->rare_enemy_rates,
-        this->rand_crypt,
-        s->supermaps_for_variations(this->episode, this->mode, this->difficulty, this->variations));
+        this->lobby_id, this->difficulty, this->event, this->random_seed, this->rare_enemy_rates, this->rand_crypt, supermaps);
   }
 
   if (this->check_flag(Lobby::Flag::DEBUG)) {
@@ -352,6 +313,7 @@ void Lobby::create_ep3_server() {
       .rand_crypt = this->rand_crypt,
       .tournament = tourn,
       .trap_card_ids = s->ep3_trap_card_ids,
+      .output_queue = nullptr,
   };
   if (is_nte) {
     options.behavior_flags |= Episode3::BehaviorFlag::IS_TRIAL_EDITION;
@@ -491,7 +453,7 @@ void Lobby::add_client(shared_ptr<Client> c, ssize_t required_client_id) {
 
   // If the lobby is recording a battle record, add the player join event
   if (this->battle_record) {
-    auto p = c->character();
+    auto p = c->character_file();
     PlayerLobbyDataDCGC lobby_data;
     lobby_data.player_tag = 0x00010000;
     lobby_data.guild_card_number = c->login->account->account_id;
@@ -633,7 +595,7 @@ shared_ptr<Client> Lobby::find_client(const string* identifier, uint64_t account
     if (account_id && lc->login && (lc->login->account->account_id == account_id)) {
       return lc;
     }
-    if (identifier && (lc->character()->disp.name.eq(*identifier, lc->language()))) {
+    if (identifier && (lc->character_file()->disp.name.eq(*identifier, lc->language()))) {
       return lc;
     }
   }
@@ -670,7 +632,7 @@ Lobby::JoinError Lobby::join_error_for_client(std::shared_ptr<Client> c, const s
       if (password && !this->password.empty() && (*password != this->password)) {
         return JoinError::INCORRECT_PASSWORD;
       }
-      auto p = c->character();
+      auto p = c->character_file();
       if (p->disp.stats.level < this->min_level) {
         return JoinError::LEVEL_TOO_LOW;
       }
@@ -764,7 +726,7 @@ void Lobby::on_item_id_generated_externally(uint32_t item_id) {
 }
 
 void Lobby::assign_inventory_and_bank_item_ids(shared_ptr<Client> c, bool consume_ids) {
-  auto p = c->character();
+  auto p = c->character_file();
   uint32_t orig_next_item_id = this->next_item_id_for_client.at(c->lobby_client_id);
   for (size_t z = 0; z < p->inventory.num_items; z++) {
     p->inventory.items[z].data.id = this->generate_item_id(c->lobby_client_id);
@@ -775,13 +737,15 @@ void Lobby::assign_inventory_and_bank_item_ids(shared_ptr<Client> c, bool consum
 
   if (c->log.info_f("Assigned inventory item IDs{}", consume_ids ? "" : " but did not mark IDs as used")) {
     c->print_inventory();
-    auto& bank = c->current_bank();
-    if (p->bank.num_items) {
-      bank.assign_ids(0x99000000 + (c->lobby_client_id << 20));
-      c->log.info_f("Assigned bank item IDs");
-      c->print_bank();
-    } else {
-      c->log.info_f("Bank is empty");
+    if ((c->version() == Version::BB_V4) && !c->has_overlay()) {
+      auto bank = c->bank_file();
+      if (!bank->items.empty()) {
+        bank->assign_ids(0x99000000 + (c->lobby_client_id << 20));
+        c->log.info_f("Assigned bank item IDs");
+        c->print_bank();
+      } else {
+        c->log.info_f("Bank is empty");
+      }
     }
   }
 }
@@ -875,38 +839,3 @@ bool Lobby::compare_shared(const shared_ptr<const Lobby>& a, const shared_ptr<co
 
   return a->name < b->name;
 }
-
-template <>
-Lobby::DropMode phosg::enum_for_name<Lobby::DropMode>(const char* name) {
-  if (!strcmp(name, "DISABLED")) {
-    return Lobby::DropMode::DISABLED;
-  } else if (!strcmp(name, "CLIENT")) {
-    return Lobby::DropMode::CLIENT;
-  } else if (!strcmp(name, "SERVER_SHARED")) {
-    return Lobby::DropMode::SERVER_SHARED;
-  } else if (!strcmp(name, "SERVER_PRIVATE")) {
-    return Lobby::DropMode::SERVER_PRIVATE;
-  } else if (!strcmp(name, "SERVER_DUPLICATE")) {
-    return Lobby::DropMode::SERVER_DUPLICATE;
-  } else {
-    throw runtime_error("invalid drop mode");
-  }
-}
-
-template <>
-const char* phosg::name_for_enum<Lobby::DropMode>(Lobby::DropMode value) {
-  switch (value) {
-    case Lobby::DropMode::DISABLED:
-      return "DISABLED";
-    case Lobby::DropMode::CLIENT:
-      return "CLIENT";
-    case Lobby::DropMode::SERVER_SHARED:
-      return "SERVER_SHARED";
-    case Lobby::DropMode::SERVER_PRIVATE:
-      return "SERVER_PRIVATE";
-    case Lobby::DropMode::SERVER_DUPLICATE:
-      return "SERVER_DUPLICATE";
-    default:
-      throw runtime_error("invalid drop mode");
-  }
-}

src/Lobby.hh

@@ -90,25 +90,18 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
     IS_OVERFLOW                     = 0x08000000,
     // clang-format on
   };
-  enum class DropMode {
-    DISABLED = 0,
-    CLIENT = 1, // Not allowed for BB games
-    SERVER_SHARED = 2,
-    SERVER_PRIVATE = 3,
-    SERVER_DUPLICATE = 4,
-  };
 
   std::weak_ptr<ServerState> server_state;
   phosg::PrefixedLogger log;
 
   uint32_t lobby_id;
 
-  uint32_t min_level;
-  uint32_t max_level;
+  uint32_t min_level = 0;
+  uint32_t max_level = 0xFFFFFFFF;
 
   // Game state
   std::array<uint32_t, 12> next_item_id_for_client;
-  uint32_t next_game_item_id;
+  uint32_t next_game_item_id = 0xCC000000;
   std::vector<FloorItemManager> floor_item_managers;
   std::shared_ptr<const MapState::RareEnemyRates> rare_enemy_rates;
   std::shared_ptr<MapState> map_state; // Always null for lobbies, never null for games
@@ -121,22 +114,22 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
   // Bits in allowed_versions specify who is allowed to join this game. The
   // bits are indexed as (1 << version), where version is a value from the
   // Version enum.
-  uint16_t allowed_versions;
-  uint8_t creator_section_id;
-  uint8_t override_section_id;
-  Episode episode;
-  GameMode mode;
-  uint8_t difficulty; // 0-3
-  uint16_t base_exp_multiplier;
-  float exp_share_multiplier;
-  float challenge_exp_multiplier;
+  uint16_t allowed_versions = 0x0000;
+  uint8_t creator_section_id = 0xFF;
+  uint8_t override_section_id = 0xFF;
+  Episode episode = Episode::NONE;
+  GameMode mode = GameMode::NORMAL;
+  Difficulty difficulty = Difficulty::NORMAL;
+  float base_exp_multiplier = 1.0f;
+  float exp_share_multiplier = 0.5f;
+  float challenge_exp_multiplier = 1.0f;
   std::string password;
   std::string name;
   // This seed is also sent to the client for rare enemy generation
-  uint32_t random_seed;
+  uint32_t random_seed = 0;
   std::shared_ptr<RandomGenerator> rand_crypt;
-  uint8_t allowed_drop_modes;
-  DropMode drop_mode;
+  uint8_t allowed_drop_modes = 0x1F;
+  ServerDropMode drop_mode = ServerDropMode::CLIENT;
   std::shared_ptr<ItemCreator> item_creator; // Always null for lobbies, never null for games
 
   struct ChallengeParameters {
@@ -171,11 +164,11 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
   std::shared_ptr<const G_SetEXResultValues_Ep3_6xB4x4B> ep3_ex_result_values;
 
   // Lobby stuff
-  uint8_t event;
-  uint8_t block;
-  uint8_t leader_id;
-  uint8_t max_clients;
-  uint32_t enabled_flags;
+  uint8_t event = 0;
+  uint8_t block = 0;
+  uint8_t leader_id = 0;
+  uint8_t max_clients = 12;
+  uint32_t enabled_flags = 0;
   std::shared_ptr<const Quest> quest;
   std::array<std::shared_ptr<Client>, 12> clients;
   // Keys in this map are client_id
@@ -183,7 +176,7 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
 
   // This is only used when the PERSISTENT flag is set and idle_timeout_usecs
   // is not zero
-  uint64_t idle_timeout_usecs;
+  uint64_t idle_timeout_usecs = 0;
   asio::steady_timer idle_timeout_timer;
 
   Lobby(std::shared_ptr<ServerState> s, uint32_t id, bool is_game);
@@ -208,11 +201,14 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
     this->enabled_flags ^= static_cast<uint32_t>(flag);
   }
 
+  uint8_t area_for_floor(Version version, uint8_t floor) const;
+
   std::shared_ptr<ServerState> require_server_state() const;
   std::shared_ptr<ChallengeParameters> require_challenge_params() const;
   void create_item_creator(Version logic_version = Version::UNKNOWN);
-  uint8_t effective_section_id() const;
+  uint8_t effective_section_id() const; // Returns 0xFF if not assigned (e.g. empty persistent game)
   uint16_t quest_version_flags() const;
+  uint8_t client_extension_flags() const;
   void load_maps();
   void create_ep3_server();
 
@@ -290,6 +286,6 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
 };
 
 template <>
-Lobby::DropMode phosg::enum_for_name<Lobby::DropMode>(const char* name);
+ServerDropMode phosg::enum_for_name<ServerDropMode>(const char* name);
 template <>
-const char* phosg::name_for_enum<Lobby::DropMode>(Lobby::DropMode value);
+const char* phosg::name_for_enum<ServerDropMode>(ServerDropMode value);

src/Map.cc

@@ -67,7 +67,7 @@ vector<string> SetDataTableBase::map_filenames_for_variations(
   return ret;
 }
 
-uint8_t SetDataTableBase::default_area_for_floor(Episode episode, uint8_t floor) const {
+uint8_t SetDataTableBase::default_area_for_floor(Version version, Episode episode, uint8_t floor) {
   // For some inscrutable reason, Pioneer 2's area number in Episode 4 is
   // discontiguous with all the rest. Why, Sega??
   static const array<uint8_t, 0x12> areas_ep1 = {
@@ -82,7 +82,7 @@ uint8_t SetDataTableBase::default_area_for_floor(Episode episode, uint8_t floor)
     case Episode::EP1:
       return areas_ep1.at(floor);
     case Episode::EP2: {
-      const auto& areas = ((this->version == Version::GC_NTE) ? areas_ep2_gc_nte : areas_ep2);
+      const auto& areas = ((version == Version::GC_NTE) ? areas_ep2_gc_nte : areas_ep2);
       return areas.at(floor);
     }
     case Episode::EP4:
@@ -92,6 +92,10 @@ uint8_t SetDataTableBase::default_area_for_floor(Episode episode, uint8_t floor)
   }
 }
 
+uint8_t SetDataTableBase::default_area_for_floor(Episode episode, uint8_t floor) const {
+  return this->default_area_for_floor(this->version, episode, floor);
+}
+
 SetDataTable::SetDataTable(Version version, const string& data) : SetDataTableBase(version) {
   if (is_big_endian(this->version)) {
     this->load_table_t<true>(data);
@@ -614,7 +618,8 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     //   param4 = source type:
     //     0 = use this set when advancing from a lower floor
     //     1 = use this set when returning from a higher floor
-    //     anything else = set is unused
+    //     anything else = set is unused (TODO: but maybe used by
+    //       TObjAreaWarpQuest?)
     {0x0000, F_V0_V4, 0x00007FFFFFFFFFFF, "TObjPlayerSet"},
     {0x0000, F_EP3, 0x0000000000008001, "TObjPlayerSet"},
 
@@ -934,7 +939,7 @@ static const vector<DATEntityDefinition> dat_object_definitions({
 
     // Radar collision. Params:
     //   param1 = radius
-    //   param4 = TODO
+    //   param4 = minimap segment ID
     {0x0017, F_V0_V4, 0x00004FFFFFF8FFFE, "TObjRaderCol"},
     {0x0017, F_EP3, 0x0000000000008000, "TObjRaderCol"},
 
@@ -972,7 +977,11 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     // In offline mode, this object constructs TObjWarpBossMulti instead.
     {0x0019, F_V0_V4, 0x00006FFC3FFC04A5, "TObjWarpBoss"},
 
-    // Sign board. This shows the loaded image from a quest (via load_pvr).
+    // Sign board. This shows the loaded image from a quest (via load_pvr). On
+    // the final version of PCv2, this object doesn't render at all; my guess
+    // is that Sega hardcoded the PVR filenames for the various events in the
+    // executable, then after those events ended, they just deleted the
+    // load_pvr code entirely, leaving this object nonfunctional.
     // Params:
     //   param1-3 = scale factors (x, y, z)
     {0x001A, F_V1_V4, 0x0000600000040001, "TObjSinBoard"},
@@ -982,7 +991,7 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     // that the object is not destroyed immediately if it's blue and the game
     // is Challenge mode. Params:
     //   param1 = player set ID (TODO: what exactly does this do? Looks like it
-    //     does nothing unless it's >= 2)
+    //     does nothing unless it's >= 2; see TObjPlayerSet)
     //   param4 = destination floor
     //   param6 = color (0 = blue, 1 = red)
     {0x001B, F_V1_V4, 0x00005000000078FE, "TObjAreaWarpQuest"},
@@ -997,10 +1006,10 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     // Params:
     //   param1 = TODO
     //   param2 = TODO
-    //   param3 = TODO
-    //   param4 = TODO
-    //   param5 = TODO
-    //   param6 = TODO
+    //   param3 = TODO (1 byte only)
+    //   param4 = TODO (1 byte only)
+    //   param5 = TODO (1 byte only)
+    //   param6 = TODO (1 byte only)
     {0x001D, F_V2_V4, 0x0000400000000002, "TEffStarLight2D_Base"},
 
     // VR Temple Beta / Barba Ray lens flare effect.
@@ -1009,7 +1018,6 @@ static const vector<DATEntityDefinition> dat_object_definitions({
 
     // Hides the area map when the player is near this object. Params:
     //   param1 = radius
-    // TODO: Test this.
     {0x001F, F_V2_V4, 0x00004FFC3FFB07FE, "TObjRaderHideCol"},
 
     // Item-detecting floor switch. Params:
@@ -1106,7 +1114,30 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     {0x0023, F_V2_V4, 0x00004FFC3FFB07FE, "TOAttackableCol"},
 
     // Damage effect. Params:
-    //   angle.x = effect type (in range [0x00, 0x14]; TODO: list these)
+    //   angle.x = effect type (in range [0x00, 0x14]; the following were
+    //     tested in Forest 1 and may have different effects in different
+    //     areas):
+    //     00 = fiery explosion with vertical camera shake
+    //     01 = vertical camera shake only
+    //     02 = bluish energy ball
+    //     03 = expanding white flash
+    //     04 = contracting white flash
+    //     05 = nothing
+    //     06 = rising fireball
+    //     07 = blue flash
+    //     08 = purple flash
+    //     09 = teal flash
+    //     0A = nothing
+    //     0B = big orange flash
+    //     0C = nothing
+    //     0D = big white flash
+    //     0E = black smoky flash
+    //     0F = nothing
+    //     10 = nothing
+    //     11 = smaller black smoke
+    //     12 = bluish-purple flash
+    //     13 = quick blue flash
+    //     14 = lavender-gray flash
     //   param1 = damage radius
     //   param2 = damage value, scaled by difficulty:
     //     Normal: param2 * 2
@@ -1124,23 +1155,23 @@ static const vector<DATEntityDefinition> dat_object_definitions({
 
     // Switch flag timer. This object watches for a switch flag to be
     // activated, then once it is, waits for a specified time, then disables
-    // that switch flag and activates up to three other switch flags. Note
-    // that this object just provides the timer functionality; the trigger
-    // switch flag must be activated by some other object. Params:
+    // that switch flag or activates up to three other switch flags. Note that
+    // this object just provides the timer functionality; the trigger switch
+    // flag must be activated by some other object. Params:
     //   angle.x = trigger mode:
     //     0 = disable watched switch flag when timer expires
-    //     any other value = enable up to 3 other switch flags when timer
+    //     any positive number = enable up to 3 other switch flags when timer
     //       expires and don't disable watched switch flag
     //   angle.y = if this is 1, play tick sound effect every second after
     //     activation (if any other value, no tick sound is played)
     //   angle.z = timer duration in frames
     //   param4 = switch flag to watch for activation in low 16 bits, switch
-    //     flag 1 to activate when timer expires (if angle.x = 0) in high 16
+    //     flag 1 to activate when timer expires (if angle.x != 0) in high 16
     //     bits (>= 0x100 if not needed)
     //   param5 = switch flag 2 to activate when timer expires (if
-    //     angle.x = 0) in high 16 bits (>= 0x100 if not needed)
+    //     angle.x != 0) in high 16 bits (>= 0x100 if not needed)
     //   param6 = switch flag 3 to activate when timer expires (if
-    //     angle.x = 0) in high 16 bits (>= 0x100 if not needed)
+    //     angle.x != 0) in high 16 bits (>= 0x100 if not needed)
     {0x0025, F_V2_V4, 0x00006FFC3FFF07FF, "TOSwitchTimer"},
 
     // Chat sensor. This object watches for chat messages said by players
@@ -1333,18 +1364,16 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     // be no parameters.
     {0x0054, F_V0_V4, 0x0000600000000001, "TObjCityDoor_Lobby"},
 
-    // Version of the main warp for Challenge mode? This object seems to
-    // behave similarly to boss teleporters; it shows the player a Yes/No
-    // confirmation menu and sends 6x6A to synchronize state. There is a
-    // global named last_set_mainwarp_value which is set to param4 when this
-    // object is constructed, but may be changed by a set_mainwarp quest
-    // opcode after that. If that happens, this object replaces its
-    // dest_floor with the floor specified in the last set_mainwarp quest
-    // opcode. Params:
+    // Version of the main warp for Challenge mode. This object looks like the
+    // main Ragol warp on Pioneer 2, but behaves similarly to boss teleporters:
+    // it shows the player a Yes/No confirmation menu and sends 6x6A to
+    // synchronize state. There is a global named last_set_mainwarp_value which
+    // is set to param4 when this object is constructed, but may be changed by
+    // a set_mainwarp quest opcode after that. If that happens, this object
+    // replaces its dest_floor with the floor specified in the last
+    // set_mainwarp quest opcode. Params:
     //   param4 = destination floor
     //   param5 = switch flag number
-    // TODO: This thing has a lot of code; figure out if there are any other
-    // parameters
     {0x0055, F_V2_V4, 0x0000600000040001, "TObjCityMainWarpChallenge"},
 
     // Episode 2 Lab door. Params:
@@ -1370,15 +1399,20 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     {0x0057, F_V3_V4, 0x0000600000040001, "TObjTradeCollision"},
     {0x0057, F_EP3, 0x0000000000000001, "TObjTradeCollision"},
 
-    // TODO: Describe this object. Presumably similar to TObjTradeCollision
-    // but enables the deck edit counter? Params:
+    // This object appears to be unused in both Ep3 NTE and the final release.
+    // It may have been an early version of Deck Edit or Entry Counter
+    // sequence, perhaps responsible for setting the players' statuses when
+    // they're near either of those counters. Params:
     //   param1 = radius
     {0x0058, F_EP3, 0x0000000000000001, "TObjDeckCollision"},
 
     // Forest door. Params:
+    //   param1-3 = x, y, z coordinates for unlock "cutscene" (see param6)
     //   param4 = switch flag number (low byte) and number to appear on door
-    //     (second-lowest byte, modulo 0x0A)
-    //   param6 = TODO (expected to be 0 or 1)
+    //     (second-lowest byte, modulo 10)
+    //   param6 = if set to 1, enables unlock "cutscene" - when the door is
+    //     unlocked, the camera will snap to the coordinates in param1-3,
+    //     pointing toward the door, for 2 seconds
     {0x0080, F_V0_V4, 0x0000400000000006, "TObjDoor"},
 
     // Forest switch. Params:
@@ -1387,9 +1421,13 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     {0x0081, F_V0_V4, 0x00004FF00078003E, "TObjDoorKey"},
 
     // Laser fence and square laser fence. Params:
-    //   param1 = color (range [0, 3])
+    //   param1 = color:
+    //     <=0 = red
+    //     1 = cyan
+    //     2 = green
+    //     >=3 = magenta
     //   param4 = switch flag number
-    //   param6 = model (TODO)
+    //   param6 = model (even value = short, odd value = long)
     {0x0082, F_V0_V4, 0x00004FF0000300FE, "TObjLazerFenceNorm"},
     {0x0083, F_V0_V4, 0x00004FF03FFB00FE, "TObjLazerFence4"},
 
@@ -1408,19 +1446,50 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     //   param1-3 = TODO
     {0x0086, F_V0_V4, 0x00004E0000000006, "TButterfly"},
 
-    // TODO: Describe this object. Params:
-    //   param1 = model number
+    // Small vehicle. Params:
+    //   param1 = model number (even value = crashed, odd value = intact)
     {0x0087, F_V0_V4, 0x0000400000000006, "TMotorcycle"},
 
     // Item box. Params:
     //   param1 = if positive, box is specialized to drop a specific item or
     //     type of item; if zero or negative, box drops any common item or
     //     none at all (and param3-6 are all ignored)
-    //   param3 = if zero, then bonuses, grinds, etc. are applied to the item
-    //     after it's generated; if nonzero, the item is not randomized at
-    //     all and drops exactly as specified in param4-6
-    //   param4-6 = item definition (see base_item_for_specialized_box in
-    //     ItemCreator.cc for how these values are decoded)
+    //   param3 = if zero, then only data1[0-1] are used and the rest of the
+    //     ItemData is cleared, then bonuses, grinds, etc. are applied to the
+    //     item; if nonzero, the item is not randomized at all and drops
+    //     exactly as specified in param4-6
+    //   param4-6 = item definition (see below)
+    // Not all fields in ItemData can be specified in the item definition here.
+    // The field order here does not match the field order in ItemData! The
+    // item definition is encoded here as follows:
+    //              -param4- -param5- -param6-
+    //   Weapon:    00wwZZSS GG--PPQQ PPQQPPQQ
+    //   Armor:     0100ZZTT 00VV---- --------
+    //   Shield:    0101ZZTT 00VV---- --------
+    //   Unit:      0102ZZ00 00VV---- --------
+    //   Mag:       02zz---- -------- --------
+    //   Tool:      03zzZZ-- -------- --------
+    //   Tech disk: 0302&&%% -------- --------
+    //   Meseta:    040000-- $$$$---- --------
+    //   - = ignored
+    //   G = weapon grind
+    //   P = attribute type (for S-ranks, custom name; last pair is kill count
+    //       for some weapons)
+    //   Q = attribute amount (for S-ranks, custom name; last pair is kill
+    //       count for some weapons)
+    //   S = weapon flags (80=untekked, 40=present) and special (low 6 bits)
+    //   T = slot count
+    //   U = tool flags (40=present; unused if item is stackable)
+    //   V = armor/shield/unit flags (40=present; low 4 bits are present color)
+    //   w = weapon class - 1 (second byte of item code, offset by 1, so e.g.
+    //       Mechgun is 07 here, not 08)
+    //   z = item class (second byte of item code)
+    //   Z = item subclass (third byte of item code)
+    //   & = technique level
+    //   % = technique number
+    //   $ = meseta amount, divided by 10 (so max possible amount is 655350)
+    // See base_item_for_specialized_box in ItemCreator.cc for newserv's
+    // implementation of decoding this format.
     // In the non-specialized case (param1 <= 0), param3-6 are still sent via
     // the 6xA2 command when the box is opened on v3 and later, and the
     // server may choose to use those parameters for some purpose. The client
@@ -1428,8 +1497,9 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     {0x0088, F_V0_V4, 0x00004FF0B00000FE, "TObjContainerBase2"},
     {0x0088, F_EP3, 0x0000000000000002, "TObjContainerBase2"},
 
-    // Elevated cylindrical tank. Params:
-    //   param1-3 = TODO
+    // Elevated cylindrical tank. There appear to be no parameters; param1-3
+    // are copied into the object instance as a Vector3F, but it appears that
+    // that vector is never used.
     {0x0089, F_V0_V4, 0x0000400000000006, "TObjTank"},
 
     // TODO: Describe this object. Params:
@@ -1452,7 +1522,7 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     //   param4 = base switch flag (the actual switch flags used are param4,
     //     param4 + 1, param4 + 2, etc.)
     //   param5 = number of switch flags (clamped to [1, 4])
-    //   param6 = TODO (only matters if this is zero or nonzero)
+    //   param6 = movement effect (0 = sparks + thud, 1 = grinding sound)
     {0x008C, F_V0_V1, 0x0000000000000006, "TObjContainerIdo"},
     {0x008C, F_V2_V4, 0x000040000000000E, "TObjContainerIdo"},
 
@@ -1462,8 +1532,9 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     //     negative = enabled when player is within 70 units
     //     range [0x00, 0xFF] = enabled by corresponding switch flag
     //     0x100 and above = never enabled
-    //   param5 = message number (used with message quest opcode; TODO: has
-    //     the same [100, 999] check as some other objects)
+    //   param5 = message number ("character ID" in qedit; if this is outside
+    //     the range [100, 999], the quest label in param6 is called in the
+    //     free play script instead of the quest script)
     //   param6 = quest label to call when activated
     {0x008D, F_V0_V4, 0x00004000000027FE, "TOCapsuleAncient01"},
 
@@ -1481,7 +1552,7 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     {0x008F, F_V0_V4, 0x0000400000000006, "TObjHashi"},
 
     // Generic switch. Visually, this is the type usually used for objects
-    // other than doors, such as lights, poison rooms, and the Forest 1
+    // other than doors, such as lights, poison rooms, and the Forest 2
     // bridge. Params:
     //   param1 = activation mode:
     //     negative = temporary (TODO: test this)
@@ -1545,9 +1616,13 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     // can be chosen via param6. If param6 is not 0, 1, or 2, no object is
     // created.
     // Params for TOHangceilingCave01Normal (param6 = 0):
-    //   param1 = TODO (radius delta? value is param1 + 29)
-    //   param2 = TODO (value is 1 - param2)
-    //   param3 = TODO (value is param3 + 100)
+    //   param1 = 1/4 time between crushes, in frames (value is param1 + 29, so
+    //     total time between crushes is (param1 + 29) * 4 frames)
+    //   param2 = wait time in frames after construction until first crush
+    //     (value is 1 - param2, so a more negative value means a longer wait
+    //     time)
+    //   param3 = base damage (value is param3 + 100; scaled by difficulty:
+    //     Normal = 1x, Hard = 2x, Very Hard = 3x, Ultimate = 6x)
     // Params for TOHangceilingCave01Key (param6 = 1):
     //   param1-3 = same as for TOHangceilingCave01Normal
     //   param4 = switch flag number (drops when switch flag is activated;
@@ -1666,11 +1741,11 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     {0x0101, F_V0_V1, 0x00000000000000C0, "TOKeyMachine01"},
     {0x0101, F_V2_V4, 0x00004FF0007B00C6, "TOKeyMachine01"},
 
-    // Mines single-switch door, or Ep4 test door if in Episode 4. Params (for
-    // both object types):
+    // Mines single-switch door, or Subterranean Desert door if in Episode 4.
+    // Params (for both object types):
     //   param4 = switch flag number
     {0x0102, F_V0_V4, 0x00000000000000C0, "TODoorMachine02"},
-    {0x0102, F_V4, 0x00004E0000000000, "__EP4_TEST_DOOR__"},
+    {0x0102, F_V4, 0x00004E0000000000, "__EP4_DOOR__"},
 
     // Large cryotube. There appear to be no parameters.
     {0x0103, F_V0_V4, 0x00004008000000C0, "TOCapsuleMachine01"},
@@ -1687,10 +1762,13 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     // appears that some may have different scale factors or offsets (TODO).
     {0x0106, F_V0_V4, 0x00004000000000C0, "TODragonflyMachine01"},
 
-    // Floating blue light. Params:
-    //   param4 = TODO
-    //   param5 = TODO
-    //   param6 = TODO
+    // Floating rotating blue light (often appears next to doors). The light
+    // rotates about its vertical axis and floats up and down sinusoidally.
+    // Params:
+    //   param4 = float cycles per second (value is param4 * 0.1 + 1.0)
+    //   param5 = max float distance (value is param5 * 0.1 + 0.5)
+    //   param6 = rotation speed in angle units per frame (0x10000 = 1 complete
+    //     rotation)
     {0x0107, F_V0_V4, 0x00004000000000C0, "TOLightMachine01"},
 
     // Self-destructing objects. Params:
@@ -1699,13 +1777,24 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     {0x0109, F_V0_V4, 0x00004000000000C0, "TOExplosiveMachine02"},
     {0x010A, F_V0_V4, 0x00004000000000C0, "TOExplosiveMachine03"},
 
-    // Spark machine. Params:
-    //   param1 = TODO (value is param1 - 0.98)
-    //   param2 = TODO (seems it only matters if this is <= 0 or not)
+    // Spark machine. This looks like it's intended to appear in the bridge
+    // room in Mine 1, to create an effect of the columnar machines sparking.
+    // This is implemented as four columns that randomly change their
+    // visibility, intended to be in the same position as the map geometry.
+    // Each column has an accumulator value, which is initially zero. Every
+    // frame, the value (param1 - 0.98) is added to each column's accumulator
+    // and a random number between 0 and 1 is chosen; if the random number is
+    // less than the column's accumulator, its visibility state is changed and
+    // its accumulator is reset to zero. In this manner, param1 can be thought
+    // of as the frequency of state changes - 0.98 would mean they never change
+    // state, 1.98 would mean they change every frame. Params:
+    //   param1 = state change accumulation per frame (value is param1 - 0.98)
+    //   param2 = if <= 0, only one column flickers and the others are always
+    //     visible; if > 0, all columns flicker
     {0x010B, F_V0_V4, 0x00004000000000C0, "TOSparkMachine01"},
 
-    // Large flashing box. Params:
-    //   param2 = TODO (seems it only matters if this is < 0 or not)
+    // Open stall with a spinning red light on either side. Params:
+    //   param2 = if > 0, a gray box is present in the left half of the stall
     {0x010C, F_V0_V4, 0x00004000000000C0, "TOHangerMachine01"},
 
     // Ruins entrance door (after Vol Opt). This object reads quest flags
@@ -1723,9 +1812,12 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     //     >= 0 in Challenge mode, the warp is destroyed immediately
     {0x0140, F_V0_V4, 0x0000400000000700, "TObjGoalWarpAncient"},
 
-    // Ruins intra-area warp. Same parameters as 0x0003 (TObjMapWarpForest),
-    // but also:
-    //   param5 = type (negative = one-way, zero or positive = normal)
+    // Ruins intra-area warp. Params:
+    //   param1-3 = destination (same as for TObjMapWarpForest)
+    //   param4 = destination angle (same as for TObjMapWarpForest)
+    //   param5 = if negative, no warp lines render (only the floor pad
+    //     appears) and the player cannot use the warp; if zero or positive,
+    //     the warp functions normally
     {0x0141, F_V0_V4, 0x0000400000000700, "TObjMapWarpAncient"},
 
     // Ruins switch. Same parameters as 0x00C0 (TOKeyCave01).
@@ -1759,7 +1851,9 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     //   param1 = activation radius delta (actual radius is param1 + 50)
     //   param4 = switch flag number
     //   param5 = if negative, sensor is always on
-    //   param6 = TODO (clamped to [0, 1] - model index?)
+    //   param6 = texture index; uses fs_obj_o_sensor01r if <= 0, uses
+    //     fs_obj_o_sensor02r if positive; the two texture files are identical
+    //     (at least on GC) so this has no user-visible effects
     {0x014C, F_V0_V4, 0x0000400000000700, "TOSensorAncient01"},
 
     // Ruins laser fence switch. Params:
@@ -1779,11 +1873,19 @@ static const vector<DATEntityDefinition> dat_object_definitions({
 
     // Ruins poison-spewing blob. This object is technically an item box, and
     // drops an item when destroyed. Unlike most other item boxes, it cannot
-    // be specialized (ignore_def is always true). Params:
-    //   param1 = TODO (value is param1 + 299)
-    //   param2 = TODO (value is param2 + 209)
-    //   param3 = TODO (value is param3 + 399)
-    //   param6 = TODO (value is param6 + 4)
+    // be specialized (ignore_def is always true). It cycles through the
+    // following 3 phases in order until it's destroyed:
+    //   Phase 0: idle (duration depends on param1 and players' positions)
+    //   Phase 1: preparing to spew poison (always lasts 60 frames)
+    //   Phase 2: spewing poison (duration specified by param2)
+    // Params:
+    //   param1 = maximum phase 0 duration in frames (value is param1 + 299;
+    //     it advances to phase 1 early if a player is within 80 units of it)
+    //   param2 = duration of phase 2 in frames (value is param2 + 209)
+    //   param3 = poison radius squared (value is param3 + 399, so if param3 =
+    //     1 for example, the poison radius is 20 units)
+    //   param6 = how often to create more particles during spewing phase (in
+    //     frames; value is param6 + 4)
     {0x0152, F_V0_V4, 0x00004E000F800700, "TContainerAncient01"},
 
     // Ruins falling trap. Trap power seems to be scaled by difficulty
@@ -1801,7 +1903,8 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     // Ruins pop-up trap. Params:
     //   param1 = trigger radius delta (value is (param1 / 2) + 30)
     //   param4 = delay (value is param4 + 30; clamped below to 0)
-    //   angle.z = TODO (seems it only matters if angle.z is > 0 or not)
+    //   angle.z = hide body (positive = only the blue highlights appear; zero
+    //     or negative = normal visibility)
     {0x0154, F_V0_V4, 0x0000400000000700, "TOTrapAncient02"},
 
     // Ruins crystal monument. Same parameters as TOCapsuleAncient01.
@@ -1877,7 +1980,7 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     //     2 = Gibarta
     //     3 = Megid
     //     anything else = Gifoie
-    //   angle.z = TODO (seems it only matters if angle.z is > 0 or not)
+    //   angle.z = hide body (same as for TOTrapAncient02)
     {0x0167, F_V2_V4, 0x0000400C3FF807C0, "TOTrapAncient02R"},
 
     // Flying white bird. Params:
@@ -2003,15 +2106,26 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     {0x018C, F_V3_V4, 0x0000400000008000, "TObjParticleLobby"},
     {0x018C, F_EP3, 0x0000000000008000, "TObjParticleLobby"},
 
-    // Episode 3 lobby battle table. Params:
-    //   param4 = player count (1-4); only 2 and 4 are used in-game
+    // Episode 3 lobby battle table. This object is responsible for the red
+    // panels on the floor next to the battle table that turn green when you
+    // step on them; it also shows the confirmation window and sends the
+    // necessary commands to the server. The actual table model and the non-lit
+    // parts of the floor panels are part of the lobby geometry, not this
+    // object. Params:
+    //   param4 = player count
+    //     1 = 1 player (doesn't work properly - there's no way to confirm)
+    //     2 = 2 players
+    //     3 = 3 players (unused, but works)
+    //     4 = 4 players
+    //     anything else = object doesn't load
     //   param5 = table number (used in E4 and E5 commands)
     {0x018D, F_EP3, 0x0000000000008000, "TObjLobbyTable"},
 
     // Episode 3 lobby jukebox. No parameters.
     {0x018E, F_EP3, 0x0000000000008000, "TObjJukeBox"},
 
-    // TODO: Describe this object. There appear to be no parameters.
+    // Spaceship overhead camera with red light and annoying gear noises. There
+    // appear to be no parameters.
     {0x0190, F_V2_V4, 0x0000400000610000, "TObjCamera"},
 
     // Short Spaceship wall. There appear to be no parameters.
@@ -2160,9 +2274,9 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     {0x020F, F_V3_V4, 0x0000400C3F800000, "TOTrapChainSawDamage"},
 
     // Laser detector trap. Params:
-    //   param3 = model number (<= for small laser, > 0 for large laser)
+    //   param3 = model number (<= 0 for small laser, > 0 for large laser)
     //   param4 = switch flag number (enables this flag when triggered)
-    //   param5-6: same as 0x020F (TOTrapChainSawDamage)
+    //   param5-6 = same as 0x020F (TOTrapChainSawDamage)
     {0x0210, F_V3_V4, 0x0000400C3F800000, "TOTrapChainSawKey"},
 
     // TODO: Describe this object. It's a subclass of TODragonfly and has the
@@ -2176,8 +2290,11 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     {0x0212, F_V3_V4, 0x000040080F800000, "__SEAGULL__"},
     {0x0212, F_EP3, 0x0000000000000002, "__SEAGULL__"},
 
-    // TODO: Describe this object. Params:
-    //   param4 = model number (clamped to [0, 2])
+    // Jungle wooden objects. Params:
+    //   param4 = model number:
+    //     0 or negative: long branch
+    //     1: shorter curved branch
+    //     2 or greater: small log
     {0x0213, F_V3_V4, 0x00004E040F000000, "TOJungleDesign"},
 
     // Fish. This object is not constructed in split-screen mode. Params:
@@ -2345,7 +2462,8 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     //   param6 low byte = TODO
     {0x02D0, F_EP3, 0x0000000000000002, "TObjKazariCard"},
 
-    // TODO: Describe these objects. There appear to be no parameters.
+    // Effects visible in the central column in the Morgue when cards
+    // transform. There appear to be no parameters.
     {0x02D1, F_EP3, 0x0000000000000001, "TObj_FloatingCardMaterial_Dark"},
     {0x02D2, F_EP3, 0x0000000000000001, "TObj_FloatingCardMaterial_Hero"},
 
@@ -2354,21 +2472,23 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     // for the lobby teleporter, or TShopGenerator for the battle counter).
     // TObjCardCityMapWarp takes no parameters.
     {0x02D3, F_EP3, 0x0000000000000001, "TObjCardCityMapWarp(0)"}, // Battle counter warp (blue lines)
-    {0x02D9, F_EP3, 0x0000000000000001, "TObjCardCityMapWarp(1)"}, // TODO
+    {0x02D9, F_EP3, 0x0000000000000001, "TObjCardCityMapWarp(1)"}, // Battle counter warp (green lines; unused)
     {0x02E3, F_EP3, 0x0000000000000001, "TObjCardCityMapWarp(2)"}, // Lobby warp (yellow lines)
 
     // Morgue doors. None of these take any parameters. Unsurprisingly, the
-    // _Closed variants don't open.
-    {0x02D4, F_EP3, 0x0000000000000001, "TObjCardCityDoor(0)"}, // Yellow (to deck edit room)
-    {0x02D5, F_EP3, 0x0000000000000001, "TObjCardCityDoor(1)"}, // Blue (to battle entry counter)
-    {0x02D8, F_EP3, 0x0000000000000001, "TObjCardCityDoor(2)"}, // TODO
-    {0x02DF, F_EP3, 0x0000000000000001, "TObjCardCityDoor(3)"}, // Solid (always closed in normal Morgue)
+    // _Closed variants don't open. The _Closed variants also have a red light
+    // in the center instead of a blue light. Curiously, the (3) variant is
+    // opaque when _Closed, unlike the other _Closed doors.
+    {0x02D4, F_EP3, 0x0000000000000001, "TObjCardCityDoor(0)"}, // Yellow V-pattern (to deck edit room)
+    {0x02D5, F_EP3, 0x0000000000000001, "TObjCardCityDoor(1)"}, // Blue V-pattern (to battle entry counter)
+    {0x02D8, F_EP3, 0x0000000000000001, "TObjCardCityDoor(2)"}, // Green V-pattern (unused)
+    {0x02DF, F_EP3, 0x0000000000000001, "TObjCardCityDoor(3)"}, // Blue X-pattern (to lobby teleporter)
     {0x02E0, F_EP3, 0x0000000000000001, "TObjCardCityDoor(4)"}, // Gray (to chief)
-    {0x02DC, F_EP3, 0x0000000000000001, "TObjCardCityDoor_Closed(0)"}, // TODO
-    {0x02DD, F_EP3, 0x0000000000000001, "TObjCardCityDoor_Closed(1)"}, // TODO
-    {0x02DE, F_EP3, 0x0000000000000001, "TObjCardCityDoor_Closed(2)"}, // TODO
-    {0x02E1, F_EP3, 0x0000000000000001, "TObjCardCityDoor_Closed(3)"}, // TODO
-    {0x02E2, F_EP3, 0x0000000000000001, "TObjCardCityDoor_Closed(4)"}, // TODO
+    {0x02DC, F_EP3, 0x0000000000000001, "TObjCardCityDoor_Closed(0)"}, // Yellow V-pattern (to deck edit room)
+    {0x02DD, F_EP3, 0x0000000000000001, "TObjCardCityDoor_Closed(1)"}, // Blue V-pattern (to battle entry counter)
+    {0x02DE, F_EP3, 0x0000000000000001, "TObjCardCityDoor_Closed(2)"}, // Green V-pattern (unused)
+    {0x02E1, F_EP3, 0x0000000000000001, "TObjCardCityDoor_Closed(3)"}, // Opaque gray X-pattern
+    {0x02E2, F_EP3, 0x0000000000000001, "TObjCardCityDoor_Closed(4)"}, // Gray (to chief)
 
     // Mortis Fons geyser. Params:
     //   param1-3 = TODO
@@ -2407,11 +2527,16 @@ static const vector<DATEntityDefinition> dat_object_definitions({
     //   param4 = index into location bit field (0-31 where 0 is the least-
     //     significant bit; see Episode3LobbyBanners in config.example.json
     //     for the bits' meanings)
-    //   param5 = TODO
+    //   param5 = per-axis mirror flags (the low 3 nybbles of this value
+    //     specify whether to invert each axis of the model; if any nybble is 1
+    //     then the model is inverted along the corresponding axis; the lowest
+    //     nybble corresponds to the x axis)
     {0x02E4, F_EP3, 0x0000000000008001, "TObjSinBoardCard"},
 
-    // TODO: Describe this object. Params:
-    //   param4 = model number (0 or 1)
+    // Morgue info screen. Params:
+    //   param4 = model number:
+    //     0 = tall, yellow, vertical-scrolling
+    //     1 = short, blue, horizontal-scrolling
     {0x02E5, F_EP3, 0x0000000000000001, "TObjCityMoji"},
 
     // Like TObjCardCityMapWarp(2) (the warp to the lobby from the Morgue)
@@ -2720,8 +2845,8 @@ static const vector<DATEntityDefinition> dat_enemy_definitions({
     {0x0110, F_EP3, 0x0000000000000001, "TObjNpcWalkingMeka_Hero"}, // Small talking robot in Morgue
     {0x0111, F_EP3, 0x0000000000000001, "TObjNpcWalkingMeka_Dark"}, // Small talking robot in Morgue
 
-    // Episode 3 scientist and aide NPCs. These NPC take all the same params as
-    // the NPCs defined above, but also:
+    // Episode 3 scientist and aide NPCs. These NPCs take all the same params
+    // as the NPCs defined above, but also:
     //   angle.x = model number (clamped to [0, 3] for scientists, [0, 2] for
     //     aides)
     // The two type values for scientists (00D4 and 00D5) are direct aliases
@@ -2927,7 +3052,7 @@ static const vector<DATEntityDefinition> dat_enemy_definitions({
     // Monest that has param1 = 3 and param2 = 10. This looks like just an
     // off-by-one error on Sega's part where they accidentally shifted the
     // parameters down by one place. As described above, this Monest expels
-    // 6 Mothmants, then no more after they are killed.
+    // 6 Mothmants immediately, then no more after those 6 are killed.
     {0x0042, F_V0_V4, 0x0000000000180006, "TObjEneBm3FlyNest"},
 
     // Savage Wolf or Barbarous Wolf. Params:
@@ -3380,7 +3505,7 @@ string MapFile::name_for_enemy_type(uint16_t type, Version version, uint8_t area
 
 string MapFile::ObjectSetEntry::str(Version version, uint8_t area) const {
   string name_str = MapFile::name_for_object_type(this->base_type, version, area);
-  return std::format("[ObjectSetEntry type={:04X} \"{}\" floor={:04X} group={:04X} room={:04X} a3={:04X} x={:g} y={:g} z={:g} x_angle={:08X} y_angle={:08X} z_angle={:08X} params=[{:g} {:g} {:g} {:08X} {:08X} {:08X}] unused={:08X}]",
+  return std::format("[ObjectSetEntry type={:04X} \"{}\" floor={:04X} group={:04X} room={:04X} a3={:04X} x={:g} y={:g} z={:g} x_angle={:08X} y_angle={:08X} z_angle={:08X} params=[{:g} {:g} {:g} {:08X} {:08X} {:08X}]]",
       this->base_type,
       name_str,
       this->floor,
@@ -3398,8 +3523,7 @@ string MapFile::ObjectSetEntry::str(Version version, uint8_t area) const {
       this->param3,
       this->param4,
       this->param5,
-      this->param6,
-      this->unused);
+      this->param6);
 }
 
 uint64_t MapFile::ObjectSetEntry::semantic_hash(uint8_t floor) const {
@@ -3420,7 +3544,7 @@ uint64_t MapFile::ObjectSetEntry::semantic_hash(uint8_t floor) const {
 
 string MapFile::EnemySetEntry::str(Version version, uint8_t area) const {
   auto type_name = MapFile::name_for_enemy_type(this->base_type, version, area);
-  return std::format("[EnemySetEntry type={:04X} \"{}\" num_children={:04X} floor={:04X} room={:04X} wave_number={:04X} wave_number2={:04X} a1={:04X} x={:g} y={:g} z={:g} x_angle={:08X} y_angle={:08X} z_angle={:08X} params=[{:g} {:g} {:g} {:g} {:g} {:04X} {:04X}] unused={:08X}]",
+  return std::format("[EnemySetEntry type={:04X} \"{}\" num_children={:04X} floor={:04X} room={:04X} wave_number={:04X} wave_number2={:04X} a1={:04X} x={:g} y={:g} z={:g} x_angle={:08X} y_angle={:08X} z_angle={:08X} params=[{:g} {:g} {:g} {:g} {:g} {:04X} {:04X}]]",
       this->base_type,
       type_name,
       this->num_children,
@@ -3441,8 +3565,7 @@ string MapFile::EnemySetEntry::str(Version version, uint8_t area) const {
       this->param4,
       this->param5,
       this->param6,
-      this->param7,
-      this->unused);
+      this->param7);
 }
 
 uint64_t MapFile::EnemySetEntry::semantic_hash(uint8_t floor) const {
@@ -4034,12 +4157,12 @@ string MapFile::disassemble_action_stream(const void* data, size_t size) {
       }
       case 0x0A: {
         uint16_t id = r.get_u16l();
-        ret.emplace_back(std::format("  0A {:04X}       enable_switch_flag      id={:04X}", id, id));
+        ret.emplace_back(std::format("  0A {:04X}       set_switch_flag         id={:04X}", id, id));
         break;
       }
       case 0x0B: {
         uint16_t id = r.get_u16l();
-        ret.emplace_back(std::format("  0B {:04X}       disable_switch_flag     id={:04X}", id, id));
+        ret.emplace_back(std::format("  0B {:04X}       clear_switch_flag       id={:04X}", id, id));
         break;
       }
       case 0x0C: {
@@ -4197,13 +4320,11 @@ string SuperMap::Enemy::id_str() const {
 }
 
 string SuperMap::Enemy::str() const {
-  string ret = std::format("[Enemy ES-{:02X}-{:03X}-{:03X} type={} child_index={:X} alias_enemy_index_delta={:X} is_default_rare_v123={} is_default_rare_bb={}",
-      this->floor,
-      this->super_set_id,
-      this->super_id,
+  string ret = std::format("[Enemy {} type={} child_index={:X} alias_target_ene={} is_default_rare_v123={} is_default_rare_bb={}",
+      this->id_str(),
       phosg::name_for_enum(this->type),
       this->child_index,
-      this->alias_enemy_index_delta,
+      (this->alias_target_ene ? this->alias_target_ene->id_str() : "(none)"),
       this->is_default_rare_v123 ? "true" : "false",
       this->is_default_rare_bb ? "true" : "false");
   for (Version v : ALL_NON_PATCH_VERSIONS) {
@@ -4343,16 +4464,18 @@ void SuperMap::link_object_version(std::shared_ptr<Object> obj, Version version,
 }
 
 shared_ptr<SuperMap::Enemy> SuperMap::add_enemy_and_children(
-    Version version,
-    uint8_t floor,
-    const MapFile::EnemySetEntry* set_entry) {
+    Version version, uint8_t floor, const MapFile::EnemySetEntry* set_entry) {
 
   shared_ptr<Enemy> head_ene = nullptr;
   size_t next_child_index = 0;
   auto add = [&](EnemyType type,
                  bool is_default_rare_v123 = false,
                  bool is_default_rare_bb = false,
-                 int16_t alias_enemy_index_delta = 0) -> void {
+                 std::shared_ptr<Enemy> alias_target_ene = nullptr) -> std::shared_ptr<Enemy> {
+    if (alias_target_ene && alias_target_ene->alias_target_ene) {
+      throw std::logic_error("alias may not point to an enemy that also is an alias");
+    }
+
     auto& entities = this->version(version);
 
     // TODO: It'd be nice to share some code between this function and
@@ -4367,7 +4490,7 @@ shared_ptr<SuperMap::Enemy> SuperMap::add_enemy_and_children(
     ene->type = type;
     ene->is_default_rare_v123 = is_default_rare_v123;
     ene->is_default_rare_bb = is_default_rare_bb;
-    ene->alias_enemy_index_delta = alias_enemy_index_delta;
+    ene->alias_target_ene = alias_target_ene;
     auto& ene_ver = ene->version(version);
     ene_ver.set_entry = set_entry;
     ene_ver.relative_enemy_index = entities.enemies.size();
@@ -4389,6 +4512,7 @@ shared_ptr<SuperMap::Enemy> SuperMap::add_enemy_and_children(
     // Add to room/group index
     uint64_t k = room_index_key(ene->floor, set_entry->room, set_entry->wave_number);
     entities.enemy_for_floor_room_and_wave_number.emplace(k, ene);
+    return ene;
   };
 
   // The following logic was originally based on the public version of
@@ -4632,26 +4756,28 @@ shared_ptr<SuperMap::Enemy> SuperMap::add_enemy_and_children(
     case 0x00C5: // Unnamed subclass of TObjEnemyCustom
       add(EnemyType::VOL_OPT_2);
       break;
-    case 0x00C8: // TBoss4DarkFalz
+    case 0x00C8: { // TBoss4DarkFalz
       if ((set_entry->num_children != 0) && (set_entry->num_children != 0x200)) {
         this->log.warning_f("DARK_FALZ has an unusual num_children (0x{:X})", set_entry->num_children);
       }
-      add(EnemyType::DARK_FALZ_3);
+      auto root_ene = add(EnemyType::DARK_FALZ_3);
       default_num_children = -1; // Skip adding children (because we do it here)
       for (size_t x = 0; x < 0x1FD; x++) {
         add(EnemyType::DARVANT);
       }
-      add(EnemyType::DARK_FALZ_3, false, false, -0x1FE);
-      add(EnemyType::DARK_FALZ_2, false, false, -0x1FF);
-      add(EnemyType::DARK_FALZ_1, false, false, -0x200);
+      add(EnemyType::DARK_FALZ_3, false, false, root_ene);
+      add(EnemyType::DARK_FALZ_2, false, false, root_ene);
+      add(EnemyType::DARK_FALZ_1, false, false, root_ene);
       break;
-    case 0x00CA: // TBoss6PlotFalz
-      add(EnemyType::OLGA_FLOW_2);
+    }
+    case 0x00CA: { // TBoss6PlotFalz
+      auto root_ene = add(EnemyType::OLGA_FLOW_2);
       default_num_children = -1; // Skip adding children (because we do it here)
       for (size_t x = 0; x < 0x200; x++) {
-        add(EnemyType::OLGA_FLOW_2, false, false, -(x + 1));
+        add(EnemyType::OLGA_FLOW_2, false, false, root_ene);
       }
       break;
+    }
     case 0x00CB: // TBoss7DeRolLeC
       add(EnemyType::BARBA_RAY);
       child_type = EnemyType::PIG_RAY;
@@ -4881,8 +5007,8 @@ static size_t get_action_stream_size(const void* data, size_t size) {
         r.skip(4);
         done = (cmd == 0x0D);
         break;
-      case 0x0A: // enable_switch_flag(uint16_t flag_num)
-      case 0x0B: // disable_switch_flag(uint16_t flag_num)
+      case 0x0A: // set_switch_flag(uint16_t flag_num)
+      case 0x0B: // clear_switch_flag(uint16_t flag_num)
         r.skip(2);
         break;
       default:
@@ -5365,25 +5491,16 @@ vector<shared_ptr<const SuperMap::Object>> SuperMap::doors_for_switch_flag(
   return ret;
 }
 
-shared_ptr<const SuperMap::Enemy> SuperMap::enemy_for_index(Version version, uint16_t enemy_id, bool follow_alias) const {
+shared_ptr<const SuperMap::Enemy> SuperMap::enemy_for_index(Version version, uint16_t enemy_index) const {
   const auto& entities = this->version(version);
 
   if (entities.enemies.empty()) {
     throw out_of_range("no enemies defined");
   }
-  if (enemy_id >= entities.enemies.size()) {
+  if (enemy_index >= entities.enemies.size()) {
     throw out_of_range("enemy ID out of range");
   }
-  auto& enemy = entities.enemies[enemy_id];
-  if (follow_alias && (enemy->alias_enemy_index_delta != 0)) {
-    uint16_t target_id = enemy_id + enemy->alias_enemy_index_delta;
-    if (target_id >= entities.enemies.size()) {
-      throw out_of_range("aliased enemy ID out of range");
-    }
-    return entities.enemies[target_id];
-  } else {
-    return enemy;
-  }
+  return entities.enemies[enemy_index];
 }
 
 shared_ptr<const SuperMap::Enemy> SuperMap::enemy_for_floor_type(Version version, uint8_t floor, EnemyType type) const {
@@ -5392,10 +5509,13 @@ shared_ptr<const SuperMap::Enemy> SuperMap::enemy_for_floor_type(Version version
   if (entities.enemies.empty()) {
     throw out_of_range("no enemies defined");
   }
-  // TODO: Linear search is bad here. Do something better, like binary search
-  // for the floor start and just linear search through the floor enemies.
-  for (auto& ene : entities.enemies) {
-    if ((ene->floor == floor) && (ene->type == type)) {
+  size_t start_z = entities.enemy_floor_start_indexes.at(floor);
+  size_t end_z = (floor < entities.enemy_floor_start_indexes.size() - 1)
+      ? entities.enemy_floor_start_indexes[floor + 1]
+      : entities.enemy_floor_start_indexes.size();
+  for (size_t z = start_z; z < end_z; z++) {
+    auto& ene = entities.enemies[z];
+    if ((ene->floor == floor) || (ene->type == type)) {
       return ene;
     }
   }
@@ -5819,7 +5939,7 @@ phosg::JSON MapState::RareEnemyRates::json() const {
   });
 }
 
-uint32_t MapState::RareEnemyRates::for_enemy_type(EnemyType type) const {
+uint32_t MapState::RareEnemyRates::get(EnemyType type) const {
   switch (type) {
     case EnemyType::HILDEBEAR:
       return this->hildeblue;
@@ -5935,7 +6055,7 @@ size_t MapState::EventIterator::num_entities_on_current_floor() const {
 
 MapState::MapState(
     uint64_t lobby_or_session_id,
-    uint8_t difficulty,
+    Difficulty difficulty,
     uint8_t event,
     uint32_t random_seed,
     std::shared_ptr<const RareEnemyRates> bb_rare_rates,
@@ -5985,7 +6105,7 @@ MapState::MapState(
 
 MapState::MapState(
     uint64_t lobby_or_session_id,
-    uint8_t difficulty,
+    Difficulty difficulty,
     uint8_t event,
     uint32_t random_seed,
     std::shared_ptr<const RareEnemyRates> bb_rare_rates,
@@ -6032,6 +6152,7 @@ void MapState::index_super_map(const FloorConfig& fc, shared_ptr<RandomGenerator
 
   for (const auto& ene : fc.super_map->all_enemies()) {
     auto& ene_st = this->enemy_states.emplace_back(make_shared<EnemyState>());
+
     if (ene->child_index == 0) {
       this->enemy_set_states.emplace_back(ene_st);
     }
@@ -6039,16 +6160,25 @@ void MapState::index_super_map(const FloorConfig& fc, shared_ptr<RandomGenerator
     ene_st->set_id = this->enemy_set_states.size() - 1;
     ene_st->super_ene = ene;
 
+    if (ene->alias_target_ene) {
+      ssize_t delta = ene->alias_target_ene->super_id - ene->super_id;
+      ene_st->alias_target_ene_st = this->enemy_states.at(ene_st->e_id + delta);
+      if (ene_st->alias_target_ene_st->super_ene != ene->alias_target_ene) {
+        throw std::logic_error("found incorrect alias target state for enemy");
+      }
+      if (ene_st->alias_target_ene_st->super_ene->alias_target_ene) {
+        throw std::runtime_error("target for enemy state alias is itself an alias");
+      }
+    }
+
     // Handle random rare enemies and difficulty-based effects
     EnemyType type;
     switch (ene->type) {
       case EnemyType::DARK_FALZ_3:
-        type = ((this->difficulty == 0) && (ene->alias_enemy_index_delta == 0))
-            ? EnemyType::DARK_FALZ_2
-            : EnemyType::DARK_FALZ_3;
+        type = (this->difficulty == Difficulty::NORMAL) ? EnemyType::DARK_FALZ_2 : EnemyType::DARK_FALZ_3;
         break;
       case EnemyType::DARVANT:
-        type = (this->difficulty == 3) ? EnemyType::DARVANT_ULTIMATE : EnemyType::DARVANT;
+        type = (this->difficulty == Difficulty::ULTIMATE) ? EnemyType::DARVANT_ULTIMATE : EnemyType::DARVANT;
         break;
       default:
         type = ene->type;
@@ -6057,7 +6187,7 @@ void MapState::index_super_map(const FloorConfig& fc, shared_ptr<RandomGenerator
     auto rare_type = type_definition_for_enemy(type).rare_type(fc.super_map->get_episode(), this->event, ene->floor);
     if ((type == EnemyType::MERICARAND) || (rare_type != type)) {
       unordered_map<uint32_t, float> det_cache;
-      uint32_t bb_rare_rate = this->bb_rare_rates->for_enemy_type(type);
+      uint32_t bb_rare_rate = this->bb_rare_rates->get(type);
       for (Version v : ALL_NON_PATCH_VERSIONS) {
         // Skip this version if the enemy doesn't exist there
         uint16_t relative_enemy_index = ene->version(v).relative_enemy_index;
@@ -6180,6 +6310,32 @@ uint16_t MapState::index_for_event_state(Version version, shared_ptr<const Event
       : (relative_index + this->floor_config(ev_st->super_ev->floor).base_indexes_for_version(version).base_event_index);
 }
 
+shared_ptr<MapState::ObjectState> MapState::object_state_for_index(Version version, uint16_t object_index) {
+  size_t dynamic_obj_base_index = this->dynamic_obj_base_index_for_version.at(static_cast<size_t>(version));
+  if (object_index < dynamic_obj_base_index) {
+    int8_t floor;
+    for (floor = this->floor_config_entries.size() - 1; floor >= 0; floor--) {
+      const auto& fc = this->floor_config_entries[floor];
+      size_t base_object_index = fc.base_indexes_for_version(version).base_object_index;
+      if (object_index >= base_object_index) {
+        if (!fc.super_map) {
+          throw out_of_range("there are no objects on the specified floor");
+        }
+        const auto& obj = fc.super_map->version(version).objects.at(object_index - base_object_index);
+        return this->object_states.at(fc.base_super_ids.base_object_index + obj->super_id);
+      }
+    }
+    throw out_of_range("the specified enemy does not exist");
+
+  } else {
+    size_t k_id_delta = object_index - dynamic_obj_base_index;
+    auto obj_st = make_shared<ObjectState>();
+    obj_st->k_id = this->dynamic_obj_base_k_id + k_id_delta;
+    obj_st->super_obj = nullptr;
+    return obj_st;
+  }
+}
+
 shared_ptr<MapState::ObjectState> MapState::object_state_for_index(Version version, uint8_t floor, uint16_t object_index) {
   size_t dynamic_obj_base_index = this->dynamic_obj_base_index_for_version.at(static_cast<size_t>(version));
   if (object_index < dynamic_obj_base_index) {
@@ -6227,6 +6383,22 @@ vector<shared_ptr<MapState::ObjectState>> MapState::door_states_for_switch_flag(
   return ret;
 }
 
+shared_ptr<MapState::EnemyState> MapState::enemy_state_for_index(Version version, uint16_t enemy_index) {
+  int8_t floor;
+  for (floor = this->floor_config_entries.size() - 1; floor >= 0; floor--) {
+    const auto& fc = this->floor_config_entries[floor];
+    size_t base_enemy_index = fc.base_indexes_for_version(version).base_enemy_index;
+    if (enemy_index >= base_enemy_index) {
+      if (!fc.super_map) {
+        throw out_of_range("there are no enemies on the specified floor");
+      }
+      const auto& ene = fc.super_map->version(version).enemies.at(enemy_index - base_enemy_index);
+      return this->enemy_states.at(fc.base_super_ids.base_enemy_index + ene->super_id);
+    }
+  }
+  throw out_of_range("the specified enemy does not exist");
+}
+
 shared_ptr<MapState::EnemyState> MapState::enemy_state_for_index(Version version, uint8_t floor, uint16_t enemy_index) {
   const auto& fc = this->floor_config(floor);
   size_t base_enemy_index = fc.base_indexes_for_version(version).base_enemy_index;
@@ -6387,18 +6559,18 @@ void MapState::import_enemy_states_from_sync(Version from_version, const SyncEne
       const auto& entry = entries[enemy_index];
       const auto& ene = entities.enemies.at(enemy_index - base_indexes.base_enemy_index);
       auto& ene_st = this->enemy_states.at(fc.base_super_ids.base_enemy_index + ene->super_id);
-      if (ene_st->super_ene != ene) {
-        throw logic_error("super enemy link is incorrect");
-      }
-      if (ene_st->game_flags != entry.flags) {
-        this->log.warning_f("({:04X} => E-{:03X}) Flags from client ({:08X}) do not match game flags from map ({:08X})",
-            enemy_index, ene_st->e_id, entry.flags, ene_st->game_flags);
-        ene_st->game_flags = entry.flags;
-      }
-      if (ene_st->total_damage != entry.total_damage) {
-        this->log.warning_f("({:04X} => E-{:03X}) Total damage from client ({}) does not match total damage from map ({})",
-            enemy_index, ene_st->e_id, entry.total_damage, ene_st->total_damage);
-        ene_st->total_damage = entry.total_damage;
+      // Only set the state if it's not an alias
+      if (ene_st->super_ene == ene) {
+        if (ene_st->game_flags != entry.flags) {
+          this->log.warning_f("({:04X} => E-{:03X}) Flags from client ({:08X}) do not match game flags from map ({:08X})",
+              enemy_index, ene_st->e_id, entry.flags, ene_st->game_flags);
+          ene_st->game_flags = entry.flags;
+        }
+        if (ene_st->total_damage != entry.total_damage) {
+          this->log.warning_f("({:04X} => E-{:03X}) Total damage from client ({}) does not match total damage from map ({})",
+              enemy_index, ene_st->e_id, entry.total_damage, ene_st->total_damage);
+          ene_st->total_damage = entry.total_damage;
+        }
       }
     }
   }
@@ -6671,7 +6843,7 @@ void MapState::print(FILE* stream) const {
   phosg::fwrite_fmt(stream, "BB rare rates: {}\n", rare_rates_str);
 
   phosg::fwrite_fmt(stream, "Base indexes:\n");
-  phosg::fwrite_fmt(stream, "  FL DCTE----------- DCPR----------- DCV1----------- DCV2----------- PCTE----------- PCV2----------- GCTE----------- GCV3----------- GCEP3TE-------- GCEP3---------- XBV3----------- BBV4-----------\n");
+  phosg::fwrite_fmt(stream, "  FL DC-NTE--------- DC-11-2000----- DC-V1---------- DC-V2---------- PC-NTE--------- PC-V2---------- GC-NTE--------- GC-V3---------- GC-EP3-NTE----- GC-EP3--------- XB-V3---------- BB-V4----------\n");
   phosg::fwrite_fmt(stream, "  FL KST EST ESS EVT KST EST ESS EVT KST EST ESS EVT KST EST ESS EVT KST EST ESS EVT KST EST ESS EVT KST EST ESS EVT KST EST ESS EVT KST EST ESS EVT KST EST ESS EVT KST EST ESS EVT KST EST ESS EVT\n");
   for (size_t floor = 0; floor < this->floor_config_entries.size(); floor++) {
     auto fc = this->floor_config_entries[floor];

src/Map.hh

@@ -67,6 +67,7 @@ public:
   std::vector<std::string> map_filenames_for_variations(
       Episode episode, GameMode mode, const Variations& variations, FilenameType type) const;
 
+  static uint8_t default_area_for_floor(Version version, Episode episode, uint8_t floor);
   uint8_t default_area_for_floor(Episode episode, uint8_t floor) const;
 
 protected:
@@ -186,7 +187,7 @@ public:
     /* 34 */ le_int32_t param4 = 0;
     /* 38 */ le_int32_t param5 = 0;
     /* 3C */ le_int32_t param6 = 0;
-    /* 40 */ le_uint32_t unused = 0; // Reserved for pointer in client's memory; unused by server
+    /* 40 */ le_uint32_t unused_obj_ptr = 0; // Reserved for pointer in client's memory; unused by server
     /* 44 */
 
     uint64_t semantic_hash(uint8_t floor) const;
@@ -214,7 +215,7 @@ public:
     /* 3C */ le_float param5 = 0.0f;
     /* 40 */ le_int16_t param6 = 0;
     /* 42 */ le_int16_t param7 = 0;
-    /* 44 */ le_uint32_t unused = 0; // Reserved for pointer in client's memory; unused by server
+    /* 44 */ le_uint32_t unused_obj_ptr = 0; // Reserved for pointer in client's memory; unused by server
     /* 48 */
 
     uint64_t semantic_hash(uint8_t floor) const;
@@ -490,7 +491,7 @@ public:
     size_t super_id = 0;
     size_t super_set_id = 0;
     uint16_t child_index = 0;
-    int16_t alias_enemy_index_delta = 0; // 0 = no alias
+    std::shared_ptr<Enemy> alias_target_ene; // May be null
     EnemyType type = EnemyType::UNKNOWN;
     bool is_default_rare_v123 = false;
     bool is_default_rare_bb = false;
@@ -584,7 +585,7 @@ public:
   std::vector<std::shared_ptr<const Object>> doors_for_switch_flag(
       Version version, uint8_t floor, uint8_t switch_flag) const;
 
-  std::shared_ptr<const Enemy> enemy_for_index(Version version, uint16_t enemy_index, bool follow_alias) const;
+  std::shared_ptr<const Enemy> enemy_for_index(Version version, uint16_t enemy_index) const;
   std::shared_ptr<const Enemy> enemy_for_floor_type(Version version, uint8_t floor, EnemyType type) const;
   std::vector<std::shared_ptr<const Enemy>> enemies_for_floor_room_wave(
       Version version, uint8_t floor, uint16_t room, uint16_t wave_number) const;
@@ -672,7 +673,7 @@ public:
     RareEnemyRates(uint32_t enemy_rate, uint32_t mericarand_rate, uint32_t boss_rate);
     explicit RareEnemyRates(const phosg::JSON& json);
 
-    uint32_t for_enemy_type(EnemyType type) const;
+    uint32_t get(EnemyType type) const;
 
     std::string str() const;
     phosg::JSON json() const;
@@ -709,6 +710,7 @@ public:
   };
 
   struct EnemyState {
+    std::shared_ptr<EnemyState> alias_target_ene_st; // Null for most enemies
     std::shared_ptr<const SuperMap::Enemy> super_ene;
     enum Flag {
       LAST_HIT_MASK = 0x0003,
@@ -744,7 +746,7 @@ public:
     inline void set_mericarand_variant_flag(Version version) {
       this->mericarand_variant_flags |= (1 << static_cast<size_t>(version));
     }
-    inline EnemyType type(Version version, Episode episode, uint8_t event) const {
+    inline EnemyType type(Version version, Episode episode, Difficulty difficulty, uint8_t event) const {
       if (this->super_ene->type == EnemyType::MERICARAND) {
         if (this->is_rare(version)) {
           return ((this->mericarand_variant_flags >> static_cast<size_t>(version)) & 1)
@@ -753,6 +755,10 @@ public:
         } else {
           return EnemyType::MERICAROL;
         }
+      } else if (this->super_ene->type == EnemyType::DARK_FALZ_3) {
+        return ((difficulty == Difficulty::NORMAL) && !this->super_ene->alias_target_ene)
+            ? EnemyType::DARK_FALZ_2
+            : EnemyType::DARK_FALZ_3;
       } else {
         return this->is_rare(version)
             ? type_definition_for_enemy(this->super_ene->type).rare_type(episode, event, this->super_ene->floor)
@@ -873,7 +879,7 @@ public:
 
   phosg::PrefixedLogger log;
   std::vector<FloorConfig> floor_config_entries;
-  uint8_t difficulty = 0;
+  Difficulty difficulty = Difficulty::NORMAL;
   uint8_t event = 0;
   uint32_t random_seed = 0;
   std::shared_ptr<const RareEnemyRates> bb_rare_rates;
@@ -888,7 +894,7 @@ public:
   // Constructor for free play
   MapState(
       uint64_t lobby_or_session_id,
-      uint8_t difficulty,
+      Difficulty difficulty,
       uint8_t event,
       uint32_t random_seed, // For client-matched rare enemies (non-BB)
       std::shared_ptr<const RareEnemyRates> bb_rare_rates,
@@ -897,7 +903,7 @@ public:
   // Constructor for quests
   MapState(
       uint64_t lobby_or_session_id,
-      uint8_t difficulty,
+      Difficulty difficulty,
       uint8_t event,
       uint32_t random_seed, // For client-matched rare enemies (non-BB)
       std::shared_ptr<const RareEnemyRates> bb_rare_rates,
@@ -939,12 +945,14 @@ public:
   uint16_t set_index_for_enemy_state(Version version, std::shared_ptr<const EnemyState> ene_st) const;
   uint16_t index_for_event_state(Version version, std::shared_ptr<const EventState> evt_st) const;
 
+  std::shared_ptr<ObjectState> object_state_for_index(Version version, uint16_t object_index);
   std::shared_ptr<ObjectState> object_state_for_index(Version version, uint8_t floor, uint16_t object_index);
   std::vector<std::shared_ptr<ObjectState>> object_states_for_floor_room_group(
       Version version, uint8_t floor, uint16_t room, uint16_t group);
   std::vector<std::shared_ptr<ObjectState>> door_states_for_switch_flag(
       Version version, uint8_t floor, uint8_t switch_flag);
 
+  std::shared_ptr<EnemyState> enemy_state_for_index(Version version, uint16_t enemy_index);
   std::shared_ptr<EnemyState> enemy_state_for_index(Version version, uint8_t floor, uint16_t enemy_index);
   std::shared_ptr<EnemyState> enemy_state_for_set_index(Version version, uint8_t floor, uint16_t enemy_set_index);
   std::shared_ptr<EnemyState> enemy_state_for_floor_type(Version version, uint8_t floor, EnemyType type);

src/Menu.hh

@@ -21,9 +21,10 @@ constexpr uint32_t LOBBY = 0x33000033;
 constexpr uint32_t GAME = 0x44000044;
 constexpr uint32_t QUEST_EP1 = 0x55010155;
 constexpr uint32_t QUEST_EP2 = 0x55020255;
+constexpr uint32_t QUEST_EP3 = 0x55030355;
 // See the decsription of the A2 command in CommandFormats.hh for why these
 // menu IDs don't fit the rest of the pattern.
-constexpr uint32_t QUEST_CATEGORIES_EP1 = 0x01000001;
+constexpr uint32_t QUEST_CATEGORIES_EP1_EP3_EP4 = 0x01000001;
 constexpr uint32_t QUEST_CATEGORIES_EP2 = 0x02000002;
 constexpr uint32_t PROXY_DESTINATIONS = 0x77000077;
 constexpr uint32_t PROGRAMS = 0x88000088;

src/PSOEncryption.hh

@@ -356,6 +356,36 @@ inline std::string decrypt_v2_registry_value(const std::string& s) {
   return decrypt_v2_registry_value(s.data(), s.size());
 }
 
+template <bool BE>
+std::string decrypt_pr1_data(const void* data, size_t size) {
+  if (size < 4) {
+    throw std::runtime_error("not enough data for PR1 footer");
+  }
+  phosg::StringReader r(data, size);
+  std::string ret = r.read(size - 4);
+  PSOV2Encryption crypt(r.get<U32T<BE>>());
+  if constexpr (BE) {
+    crypt.encrypt_big_endian(ret.data(), ret.size());
+  } else {
+    crypt.decrypt(ret.data(), ret.size());
+  }
+  return ret;
+}
+
+template <bool BE>
+std::string encrypt_pr1_data(const void* data, size_t size, uint32_t seed) {
+  phosg::StringWriter w;
+  w.write(data, size);
+  w.put<U32T<BE>>(seed);
+  PSOV2Encryption crypt(seed);
+  if constexpr (BE) {
+    crypt.encrypt_big_endian(w.str().data(), size);
+  } else {
+    crypt.encrypt(w.str().data(), size);
+  }
+  return std::move(w.str());
+}
+
 struct DecryptedPR2 {
   std::string compressed_data;
   size_t decompressed_size;

src/PatchDownloadSession.cc

@@ -0,0 +1,285 @@
+#include "PatchDownloadSession.hh"
+
+#include <ctype.h>
+#include <errno.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+
+#include <algorithm>
+#include <iostream>
+#include <phosg/Encoding.hh>
+#include <phosg/Filesystem.hh>
+#include <phosg/Network.hh>
+#include <phosg/Random.hh>
+#include <phosg/Strings.hh>
+#include <phosg/Time.hh>
+
+#include "Loggers.hh"
+#include "PSOProtocol.hh"
+#include "ReceiveCommands.hh"
+#include "ReceiveSubcommands.hh"
+#include "SendCommands.hh"
+
+using namespace std;
+
+PatchDownloadSession::PatchDownloadSession(
+    std::shared_ptr<asio::io_context> io_context,
+    const std::string& remote_host,
+    uint16_t remote_port,
+    const std::string& output_dir,
+    Version version,
+    const std::string& username,
+    const std::string& password,
+    const std::string& email,
+    bool show_command_data)
+    : remote_host(remote_host),
+      remote_port(remote_port),
+      output_dir(output_dir),
+      version(version),
+      username(username),
+      password(password),
+      email(email),
+      show_command_data(show_command_data),
+      log(std::format("[PatchDownloadSession:{}] ", phosg::name_for_enum(version)), proxy_server_log.min_level),
+      io_context(io_context),
+      current_file(nullptr, +[](FILE* f) -> void { fclose(f); }) {
+  if (this->output_dir.empty()) {
+    this->output_dir = ".";
+  }
+  if (!is_patch(this->version)) {
+    throw std::logic_error("invalid version in PatchDownloadSession");
+  }
+}
+
+asio::awaitable<void> PatchDownloadSession::run() {
+  string netloc_str = std::format("{}:{}", this->remote_host, this->remote_port);
+  this->log.info_f("Connecting to {}", netloc_str);
+  auto sock = make_unique<asio::ip::tcp::socket>(co_await async_connect_tcp(this->remote_host, this->remote_port));
+  this->channel = SocketChannel::create(
+      this->io_context,
+      std::move(sock),
+      this->version,
+      Language::ENGLISH,
+      netloc_str,
+      this->show_command_data ? phosg::TerminalFormat::FG_GREEN : phosg::TerminalFormat::END,
+      this->show_command_data ? phosg::TerminalFormat::FG_YELLOW : phosg::TerminalFormat::END);
+  this->log.info_f("Server channel connected");
+
+  while (this->channel->connected()) {
+    auto msg = co_await this->channel->recv();
+    co_await this->on_message(msg);
+  }
+}
+
+void PatchDownloadSession::check_path_token(const std::string& token) {
+  if (token == "..") {
+    throw std::runtime_error("parent directory token is not allowed");
+  }
+  if ((token.find('/') != string::npos) || (token.find('\\') != string::npos)) {
+    throw std::runtime_error("directory token contains path separator");
+  }
+}
+
+std::string PatchDownloadSession::resolve_filename(const std::string& filename) const {
+  check_path_token(filename);
+  string path = this->output_dir;
+  for (const auto& dir_name : this->dir_path) {
+    path.push_back('/');
+    path += dir_name;
+  }
+  if (!filename.empty()) {
+    path.push_back('/');
+    path += filename;
+  }
+  return path;
+}
+
+asio::awaitable<void> PatchDownloadSession::on_message(Channel::Message& msg) {
+  switch (msg.command) {
+    case 0x02: {
+      const auto& cmd = msg.check_size_t<S_ServerInit_Patch_02>();
+      if (cmd.copyright.decode() != "Patch Server. Copyright SonicTeam, LTD. 2001") {
+        throw std::runtime_error("incorrect copyright message");
+      }
+      this->channel->crypt_in = make_shared<PSOV2Encryption>(cmd.server_key);
+      this->channel->crypt_out = make_shared<PSOV2Encryption>(cmd.client_key);
+      this->channel->send(0x02);
+      this->log.info_f("Enabled encryption");
+      break;
+    }
+
+    case 0x04: {
+      if (!msg.data.empty()) {
+        throw std::runtime_error("invalid login request command");
+      }
+      C_Login_Patch_04 cmd;
+      cmd.username.encode(this->username);
+      cmd.password.encode(this->password);
+      cmd.email_address.encode(this->email);
+      this->channel->send(0x04, 0x00, &cmd, sizeof(cmd));
+      this->log.info_f("Sent login credentials");
+      break;
+    }
+
+    case 0x05: {
+      this->log.info_f("Server sent disconnect command");
+      this->channel->disconnect();
+      break;
+    }
+
+    case 0x06: {
+      if (this->current_file) {
+        throw std::runtime_error("protocol violation: previous file was not closed before open file command");
+      }
+      const auto& cmd = msg.check_size_t<S_OpenFile_Patch_06>();
+      this->current_file_bytes_remaining = cmd.size;
+      auto filename = this->resolve_filename(cmd.filename.decode());
+      this->current_file = phosg::fopen_unique(filename, "wb");
+      this->log.info_f("Opened file {}", filename);
+      break;
+    }
+
+    case 0x07: {
+      if (!this->current_file) {
+        throw std::runtime_error("protocol violation: no file is open; cannot write data");
+      }
+
+      const auto& cmd = msg.check_size_t<S_WriteFileHeader_Patch_07>(0xFFFF);
+      const void* data = msg.data.data() + sizeof(cmd);
+      if (cmd.chunk_size > msg.data.size() - sizeof(cmd)) {
+        throw std::runtime_error("protocol violation: write command size is invalid");
+      }
+      if (cmd.chunk_size > this->current_file_bytes_remaining) {
+        throw std::runtime_error("protocol violation: chunk would exceed file size specified in open command");
+      }
+      if (phosg::crc32(data, cmd.chunk_size) != cmd.chunk_checksum) {
+        throw std::runtime_error("protocol violation: write command checksum is invalid");
+      }
+
+      phosg::fwritex(this->current_file.get(), data, cmd.chunk_size);
+      this->current_file_bytes_remaining -= cmd.chunk_size;
+      this->log.info_f("Wrote {} to file", phosg::format_size(cmd.chunk_size));
+      break;
+    }
+
+    case 0x08: {
+      if (!this->current_file) {
+        throw std::runtime_error("protocol violation: no file is open; cannot close it");
+      }
+      this->current_file.reset();
+      this->log.info_f("Closed file");
+      break;
+    }
+
+    case 0x09: {
+      if (this->current_file) {
+        throw std::runtime_error("protocol violation: cannot enter directory with a file open");
+      }
+
+      const auto& cmd = msg.check_size_t<S_EnterDirectory_Patch_09>();
+      string dirname = cmd.name.decode();
+      check_path_token(dirname);
+      this->dir_path.emplace_back(std::move(dirname));
+      std::filesystem::create_directories(this->resolve_filename(""));
+      this->log.info_f("Entered directory {}", dirname);
+      break;
+    }
+
+    case 0x0A: {
+      if (this->current_file) {
+        throw std::runtime_error("protocol violation: cannot exit directory with a file open");
+      }
+      if (this->dir_path.empty()) {
+        throw std::runtime_error("protocol violation: cannot exit directory with empty directory stack");
+      }
+      this->dir_path.pop_back();
+      this->log.info_f("Left directory");
+      break;
+    }
+
+    case 0x0B:
+      if (this->current_file) {
+        throw std::runtime_error("protocol violation: cannot start patch session when file is already open");
+      }
+      this->dir_path.clear();
+      this->log.info_f("Started patch session");
+      break;
+
+    case 0x0C: {
+      const auto& cmd = msg.check_size_t<S_FileChecksumRequest_Patch_0C>();
+      auto filename = this->resolve_filename(cmd.filename.decode());
+      uint32_t checksum = 0, size = 0;
+      try {
+        auto data = phosg::load_file(filename);
+        checksum = phosg::crc32(data.data(), data.size());
+        size = data.size();
+      } catch (const phosg::cannot_open_file&) {
+      }
+      this->pending_checksum_results.emplace_back(C_FileInformation_Patch_0F{cmd.request_id, checksum, size});
+      this->log.info_f("Checked file {}", filename);
+      break;
+    }
+
+    case 0x0D:
+      for (const auto& it : this->pending_checksum_results) {
+        this->channel->send(0x0F, 0x00, &it, sizeof(it));
+      }
+      this->pending_checksum_results.clear();
+      this->channel->send(0x10);
+      this->log.info_f("Sent all checksum results");
+      break;
+
+    case 0x11: {
+      const auto& cmd = msg.check_size_t<S_StartFileDownloads_Patch_11>();
+      this->log.info_f("{} files ({}) to download", cmd.num_files.load(), phosg::format_size(cmd.total_bytes));
+      break;
+    }
+
+    case 0x12:
+      this->log.info_f("Patch session succeeded");
+      this->channel->disconnect();
+      break;
+
+    case 0x13: {
+      phosg::strip_trailing_zeroes(msg.data);
+      if (msg.data.size() & 1) {
+        msg.data.push_back(0);
+      }
+      this->log.info_f("Message from server:\n{}", strip_color(tt_utf16_to_utf8(msg.data)));
+      break;
+    }
+
+    case 0x14: {
+      const auto& cmd = msg.check_size_t<S_Reconnect_Patch_14>();
+
+      auto new_ep = make_endpoint_ipv4(cmd.address, cmd.port);
+      string netloc_str = str_for_endpoint(new_ep);
+      this->log.info_f("Connecting to {}", netloc_str);
+      auto sock = make_unique<asio::ip::tcp::socket>(co_await async_connect_tcp(new_ep));
+
+      auto old_channel = this->channel;
+      auto new_channel = SocketChannel::create(
+          this->io_context,
+          std::move(sock),
+          this->channel->version,
+          this->channel->language,
+          netloc_str,
+          this->channel->terminal_send_color,
+          this->channel->terminal_recv_color);
+      this->channel = new_channel;
+      old_channel->disconnect();
+      this->log.info_f("Server channel connected");
+      break;
+    }
+
+    case 0x15:
+      this->log.error_f("Server rejected login credentials");
+      this->channel->disconnect();
+      break;
+
+    default:
+      throw std::runtime_error("invalid command");
+  }
+}

src/PatchDownloadSession.hh

@@ -0,0 +1,61 @@
+#pragma once
+
+#include <asio.hpp>
+#include <map>
+#include <memory>
+#include <phosg/Filesystem.hh>
+#include <string>
+#include <unordered_map>
+#include <unordered_set>
+#include <vector>
+
+#include "Channel.hh"
+#include "CommandFormats.hh"
+#include "PSOEncryption.hh"
+#include "PSOProtocol.hh"
+
+class PatchDownloadSession {
+public:
+  PatchDownloadSession(
+      std::shared_ptr<asio::io_context> io_context,
+      const std::string& remote_host,
+      uint16_t remote_port,
+      const std::string& output_dir,
+      Version version,
+      const std::string& username,
+      const std::string& password,
+      const std::string& email,
+      bool show_command_data);
+  PatchDownloadSession(const PatchDownloadSession&) = delete;
+  PatchDownloadSession(PatchDownloadSession&&) = delete;
+  PatchDownloadSession& operator=(const PatchDownloadSession&) = delete;
+  PatchDownloadSession& operator=(PatchDownloadSession&&) = delete;
+  virtual ~PatchDownloadSession() = default;
+
+  asio::awaitable<void> run();
+
+protected:
+  // Config (must be set by caller)
+  std::string remote_host;
+  uint16_t remote_port;
+  std::string output_dir;
+  Version version;
+  std::string username;
+  std::string password;
+  std::string email;
+  bool show_command_data;
+
+  // State (set during session)
+  phosg::PrefixedLogger log;
+  std::shared_ptr<asio::io_context> io_context;
+  std::shared_ptr<Channel> channel;
+  std::vector<std::string> dir_path;
+  std::unique_ptr<FILE, void (*)(FILE*)> current_file;
+  size_t current_file_bytes_remaining = 0;
+  std::vector<C_FileInformation_Patch_0F> pending_checksum_results;
+
+  static void check_path_token(const std::string& token);
+  std::string resolve_filename(const std::string& filename) const;
+
+  asio::awaitable<void> on_message(Channel::Message& msg);
+};

src/PatchFileIndex.cc

@@ -109,8 +109,8 @@ PatchFileIndex::PatchFileIndex(const string& root_dir)
         if (!compute_crc32s_message.empty()) {
           auto data = f->load_data(); // Sets f->size
           f->crc32 = phosg::crc32(data->data(), f->size);
-          for (size_t x = 0; x < data->size(); x += 0x4000) {
-            size_t chunk_bytes = min<size_t>(f->size - x, 0x4000);
+          for (size_t x = 0; x < data->size(); x += this->CHUNK_SIZE) {
+            size_t chunk_bytes = min<size_t>(f->size - x, this->CHUNK_SIZE);
             f->chunk_crcs.emplace_back(phosg::crc32(data->data() + x, chunk_bytes));
           }
 

src/PatchFileIndex.hh

@@ -9,6 +9,8 @@
 #include <vector>
 
 struct PatchFileIndex {
+  static constexpr size_t CHUNK_SIZE = 0x6000;
+
   explicit PatchFileIndex(const std::string& root_dir);
 
   struct File {

src/PlayerFilesManager.cc

@@ -1,122 +0,0 @@
-#include "PlayerFilesManager.hh"
-
-#include <stdio.h>
-#include <string.h>
-#include <wchar.h>
-
-#include <phosg/Filesystem.hh>
-#include <phosg/Hash.hh>
-#include <stdexcept>
-
-#include "FileContentsCache.hh"
-#include "ItemData.hh"
-#include "Loggers.hh"
-#include "PSOEncryption.hh"
-#include "PSOProtocol.hh"
-#include "StaticGameData.hh"
-#include "Text.hh"
-#include "Version.hh"
-
-using namespace std;
-
-PlayerFilesManager::PlayerFilesManager(std::shared_ptr<asio::io_context> io_context)
-    : io_context(io_context),
-      clear_expired_files_timer(*this->io_context) {
-  this->schedule_callback();
-}
-
-std::shared_ptr<PSOBBBaseSystemFile> PlayerFilesManager::get_system(const std::string& filename) {
-  try {
-    return this->loaded_system_files.at(filename);
-  } catch (const out_of_range&) {
-    return nullptr;
-  }
-}
-
-std::shared_ptr<PSOBBCharacterFile> PlayerFilesManager::get_character(const std::string& filename) {
-  try {
-    return this->loaded_character_files.at(filename);
-  } catch (const out_of_range&) {
-    return nullptr;
-  }
-}
-
-std::shared_ptr<PSOBBGuildCardFile> PlayerFilesManager::get_guild_card(const std::string& filename) {
-  try {
-    return this->loaded_guild_card_files.at(filename);
-  } catch (const out_of_range&) {
-    return nullptr;
-  }
-}
-
-std::shared_ptr<PlayerBank200> PlayerFilesManager::get_bank(const std::string& filename) {
-  try {
-    return this->loaded_bank_files.at(filename);
-  } catch (const out_of_range&) {
-    return nullptr;
-  }
-}
-
-void PlayerFilesManager::set_system(const std::string& filename, std::shared_ptr<PSOBBBaseSystemFile> file) {
-  if (!this->loaded_system_files.emplace(filename, file).second) {
-    throw runtime_error("Guild Card file already loaded: " + filename);
-  }
-}
-
-void PlayerFilesManager::set_character(const std::string& filename, std::shared_ptr<PSOBBCharacterFile> file) {
-  if (!this->loaded_character_files.emplace(filename, file).second) {
-    throw runtime_error("character file already loaded: " + filename);
-  }
-}
-
-void PlayerFilesManager::set_guild_card(const std::string& filename, std::shared_ptr<PSOBBGuildCardFile> file) {
-  if (!this->loaded_guild_card_files.emplace(filename, file).second) {
-    throw runtime_error("Guild Card file already loaded: " + filename);
-  }
-}
-
-void PlayerFilesManager::set_bank(const std::string& filename, std::shared_ptr<PlayerBank200> file) {
-  if (!this->loaded_bank_files.emplace(filename, file).second) {
-    throw runtime_error("bank file already loaded: " + filename);
-  }
-}
-
-void PlayerFilesManager::schedule_callback() {
-  this->clear_expired_files_timer.expires_after(std::chrono::seconds(30));
-  this->clear_expired_files_timer.async_wait(bind(&PlayerFilesManager::clear_expired_files, this));
-}
-
-template <typename KeyT, typename ValueT>
-size_t erase_unused(std::unordered_map<KeyT, std::shared_ptr<ValueT>>& m) {
-  size_t ret = 0;
-  for (auto it = m.begin(); it != m.end();) {
-    if (it->second.use_count() <= 1) {
-      it = m.erase(it);
-      ret++;
-    } else {
-      it++;
-    }
-  }
-  return ret;
-}
-
-void PlayerFilesManager::clear_expired_files() {
-  size_t num_deleted = erase_unused(this->loaded_system_files);
-  if (num_deleted) {
-    player_data_log.info_f("Cleared {} expired system file(s)", num_deleted);
-  }
-  num_deleted = erase_unused(this->loaded_character_files);
-  if (num_deleted) {
-    player_data_log.info_f("Cleared {} expired character file(s)", num_deleted);
-  }
-  num_deleted = erase_unused(this->loaded_guild_card_files);
-  if (num_deleted) {
-    player_data_log.info_f("Cleared {} expired Guild Card file(s)", num_deleted);
-  }
-  num_deleted = erase_unused(this->loaded_bank_files);
-  if (num_deleted) {
-    player_data_log.info_f("Cleared {} expired bank file(s)", num_deleted);
-  }
-
-  this->schedule_callback();
-}

src/PlayerFilesManager.hh

@@ -1,48 +0,0 @@
-#pragma once
-
-#include <inttypes.h>
-#include <stddef.h>
-
-#include <array>
-#include <asio.hpp>
-#include <phosg/Encoding.hh>
-#include <string>
-#include <utility>
-#include <vector>
-
-#include "Episode3/DataIndexes.hh"
-#include "ItemCreator.hh"
-#include "ItemNameIndex.hh"
-#include "LevelTable.hh"
-#include "PlayerSubordinates.hh"
-#include "SaveFileFormats.hh"
-#include "Text.hh"
-#include "Version.hh"
-
-class PlayerFilesManager {
-public:
-  explicit PlayerFilesManager(std::shared_ptr<asio::io_context> io_context);
-  ~PlayerFilesManager() = default;
-
-  std::shared_ptr<PSOBBBaseSystemFile> get_system(const std::string& filename);
-  std::shared_ptr<PSOBBCharacterFile> get_character(const std::string& filename);
-  std::shared_ptr<PSOBBGuildCardFile> get_guild_card(const std::string& filename);
-  std::shared_ptr<PlayerBank200> get_bank(const std::string& filename);
-
-  void set_system(const std::string& filename, std::shared_ptr<PSOBBBaseSystemFile> file);
-  void set_character(const std::string& filename, std::shared_ptr<PSOBBCharacterFile> file);
-  void set_guild_card(const std::string& filename, std::shared_ptr<PSOBBGuildCardFile> file);
-  void set_bank(const std::string& filename, std::shared_ptr<PlayerBank200> file);
-
-private:
-  std::shared_ptr<asio::io_context> io_context;
-  asio::steady_timer clear_expired_files_timer;
-
-  std::unordered_map<std::string, std::shared_ptr<PSOBBBaseSystemFile>> loaded_system_files;
-  std::unordered_map<std::string, std::shared_ptr<PSOBBCharacterFile>> loaded_character_files;
-  std::unordered_map<std::string, std::shared_ptr<PSOBBGuildCardFile>> loaded_guild_card_files;
-  std::unordered_map<std::string, std::shared_ptr<PlayerBank200>> loaded_bank_files;
-
-  void schedule_callback();
-  void clear_expired_files();
-};

src/PlayerInventory.cc

@@ -0,0 +1,116 @@
+#include "PlayerInventory.hh"
+
+void PlayerBank::load(FILE* f) {
+  le_uint32_t num_items;
+  le_uint32_t meseta;
+  phosg::freadx(f, &num_items, sizeof(num_items));
+  phosg::freadx(f, &meseta, sizeof(meseta));
+  this->meseta = meseta;
+  this->items.reserve(num_items);
+  while (this->items.size() < num_items) {
+    auto& item = this->items.emplace_back();
+    phosg::freadx(f, &item, sizeof(item));
+  }
+}
+
+void PlayerBank::save(FILE* f) const {
+  le_uint32_t num_items = this->items.size();
+  le_uint32_t meseta = this->meseta;
+  phosg::fwritex(f, &num_items, sizeof(num_items));
+  phosg::fwritex(f, &meseta, sizeof(meseta));
+  for (const auto& item : this->items) {
+    phosg::fwritex(f, &item, sizeof(item));
+  }
+}
+
+uint32_t PlayerBank::bb_checksum() const {
+  le_uint32_t num_items = this->items.size();
+  le_uint32_t meseta = this->meseta;
+  uint32_t ret = phosg::crc32(&num_items, sizeof(num_items));
+  ret = phosg::crc32(&meseta, sizeof(meseta), ret);
+  for (const auto& item : this->items) {
+    ret = phosg::crc32(&item, sizeof(item), ret);
+  }
+  return ret;
+}
+
+void PlayerBank::add_item(const ItemData& item, const ItemData::StackLimits& limits) {
+  uint32_t primary_identifier = item.primary_identifier();
+
+  if (primary_identifier == 0x04000000) {
+    this->meseta += item.data2d;
+    if (this->meseta > this->max_meseta) {
+      this->meseta = this->max_meseta;
+    }
+    return;
+  }
+
+  size_t combine_max = item.max_stack_size(limits);
+  if (combine_max > 1) {
+    size_t y;
+    for (y = 0; y < this->items.size(); y++) {
+      if (this->items[y].data.primary_identifier() == primary_identifier) {
+        break;
+      }
+    }
+
+    if (y < this->items.size()) {
+      uint8_t new_count = this->items[y].data.data1[5] + item.data1[5];
+      if (new_count > combine_max) {
+        throw std::runtime_error("stack size would exceed limit");
+      }
+      this->items[y].data.data1[5] = new_count;
+      this->items[y].amount = new_count;
+      return;
+    }
+  }
+
+  if (this->items.size() >= this->max_items) {
+    throw std::runtime_error("no free space in bank");
+  }
+
+  auto& new_item = this->items.emplace_back();
+  new_item.data = item;
+  new_item.amount = (item.max_stack_size(limits) > 1) ? item.data1[5] : 1;
+  new_item.present = 1;
+}
+
+ItemData PlayerBank::remove_item(uint32_t item_id, uint32_t amount, const ItemData::StackLimits& limits) {
+  size_t index = this->find_item(item_id);
+  auto& bank_item = this->items[index];
+
+  ItemData ret = bank_item.data;
+  if (amount && (bank_item.data.stack_size(limits) > 1) && (amount < bank_item.data.data1[5])) {
+    ret.data1[5] = amount;
+    bank_item.data.data1[5] -= amount;
+    bank_item.amount -= amount;
+  } else {
+    this->items.erase(this->items.begin() + index);
+  }
+  return ret;
+}
+
+size_t PlayerBank::find_item(uint32_t item_id) {
+  for (size_t x = 0; x < this->items.size(); x++) {
+    if (this->items[x].data.id == item_id) {
+      return x;
+    }
+  }
+  throw std::out_of_range("item not present");
+}
+
+void PlayerBank::sort() {
+  std::sort(this->items.begin(), this->items.end());
+}
+
+void PlayerBank::assign_ids(uint32_t base_id) {
+  for (size_t z = 0; z < this->items.size(); z++) {
+    this->items[z].data.id = base_id + z;
+  }
+}
+
+void PlayerBank::enforce_stack_limits(std::shared_ptr<const ItemData::StackLimits> stack_limits) {
+  for (auto& item : this->items) {
+    item.data.enforce_stack_size_limits(*stack_limits);
+  }
+}

src/PlayerInventory.hh

@@ -77,6 +77,10 @@ struct PlayerInventoryItemT {
     ret.data.id.store_raw(phosg::bswap32(ret.data.id.load_raw()));
     return ret;
   }
+
+  bool is_equipped() const {
+    return (this->flags & 8);
+  }
 } __attribute__((packed));
 using PlayerInventoryItem = PlayerInventoryItemT<false>;
 using PlayerInventoryItemBE = PlayerInventoryItemT<true>;
@@ -112,7 +116,7 @@ struct PlayerInventoryT {
   /* 0000 */ uint8_t num_items = 0;
   /* 0001 */ uint8_t hp_from_materials = 0;
   /* 0002 */ uint8_t tp_from_materials = 0;
-  /* 0003 */ uint8_t language = 0;
+  /* 0003 */ Language language = Language::JAPANESE;
   /* 0004 */ parray<PlayerInventoryItemT<BE>, 30> items;
   /* 034C */
 
@@ -175,11 +179,11 @@ struct PlayerInventoryT {
     }
   }
 
-  void equip_item_id(uint32_t item_id, EquipSlot slot, bool allow_overwrite) {
-    this->equip_item_index(this->find_item(item_id), slot, allow_overwrite);
+  void equip_item_id(uint32_t item_id, EquipSlot slot) {
+    this->equip_item_index(this->find_item(item_id), slot);
   }
 
-  void equip_item_index(size_t index, EquipSlot slot, bool allow_overwrite) {
+  void equip_item_index(size_t index, EquipSlot slot) {
     auto& item = this->items[index];
 
     if ((slot == EquipSlot::UNKNOWN) || !item.data.can_be_equipped_in_slot(slot)) {
@@ -187,11 +191,7 @@ struct PlayerInventoryT {
     }
 
     if (this->has_equipped_item(slot)) {
-      if (allow_overwrite) {
-        this->unequip_item_slot(slot);
-      } else {
-        throw std::runtime_error("equip slot is already in use");
-      }
+      this->unequip_item_slot(slot);
     }
 
     item.flags |= 0x00000008;
@@ -265,14 +265,14 @@ struct PlayerInventoryT {
       // issue - its inventory format matches the rest of the versions.
       this->hp_from_materials = 0;
       this->tp_from_materials = 0;
-      this->language = 0;
+      this->language = Language::JAPANESE;
     } else if ((v != Version::PC_NTE) && (v != Version::PC_V2)) {
-      if (this->language > 4) {
-        this->language = 0;
+      if (static_cast<size_t>(this->language) > 4) {
+        this->language = Language::JAPANESE;
       }
     } else {
-      if (this->language > 7) {
-        this->language = 0;
+      if (static_cast<size_t>(this->language) > 7) {
+        this->language = Language::JAPANESE;
       }
     }
 
@@ -283,6 +283,12 @@ struct PlayerInventoryT {
     }
   }
 
+  void enforce_stack_limits(std::shared_ptr<const ItemData::StackLimits> stack_limits) {
+    for (size_t z = 0; z < std::min<uint8_t>(this->num_items, this->items.size()); z++) {
+      this->items[z].data.enforce_stack_size_limits(*stack_limits);
+    }
+  }
+
   operator PlayerInventoryT<!BE>() const {
     PlayerInventoryT<!BE> ret;
     ret.num_items = this->num_items;
@@ -305,95 +311,6 @@ struct PlayerBankT {
   /* 0008 */ parray<PlayerBankItemT<BE>, SlotCount> items;
   /* 05A8 for 60 items (v1/v2), 12C8 for 200 items (v3/v4) */
 
-  uint32_t checksum() const {
-    return phosg::crc32(this, 2 * sizeof(U32T<BE>) + sizeof(PlayerBankItemT<BE>) * std::min<size_t>(SlotCount, this->num_items));
-  }
-
-  void add_item(const ItemData& item, const ItemData::StackLimits& limits) {
-    uint32_t primary_identifier = item.primary_identifier();
-
-    if (primary_identifier == 0x04000000) {
-      this->meseta += item.data2d;
-      if (this->meseta > 999999) {
-        this->meseta = 999999;
-      }
-      return;
-    }
-
-    size_t combine_max = item.max_stack_size(limits);
-    if (combine_max > 1) {
-      size_t y;
-      for (y = 0; y < this->num_items; y++) {
-        if (this->items[y].data.primary_identifier() == primary_identifier) {
-          break;
-        }
-      }
-
-      if (y < this->num_items) {
-        uint8_t new_count = this->items[y].data.data1[5] + item.data1[5];
-        if (new_count > combine_max) {
-          throw std::runtime_error("stack size would exceed limit");
-        }
-        this->items[y].data.data1[5] = new_count;
-        this->items[y].amount = new_count;
-        return;
-      }
-    }
-
-    if (this->num_items >= SlotCount) {
-      throw std::runtime_error("no free space in bank");
-    }
-    auto& last_item = this->items[this->num_items];
-    last_item.data = item;
-    last_item.amount = (item.max_stack_size(limits) > 1) ? item.data1[5] : 1;
-    last_item.present = 1;
-    this->num_items++;
-  }
-
-  ItemData remove_item(uint32_t item_id, uint32_t amount, const ItemData::StackLimits& limits) {
-    size_t index = this->find_item(item_id);
-    auto& bank_item = this->items[index];
-
-    ItemData ret;
-    if (amount && (bank_item.data.stack_size(limits) > 1) && (amount < bank_item.data.data1[5])) {
-      ret = bank_item.data;
-      ret.data1[5] = amount;
-      bank_item.data.data1[5] -= amount;
-      bank_item.amount -= amount;
-      return ret;
-    }
-
-    ret = bank_item.data;
-    this->num_items--;
-    for (size_t x = index; x < this->num_items; x++) {
-      this->items[x] = this->items[x + 1];
-    }
-    auto& last_item = this->items[this->num_items];
-    last_item.amount = 0;
-    last_item.present = 0;
-    last_item.data.clear();
-    return ret;
-  }
-
-  size_t find_item(uint32_t item_id) {
-    for (size_t x = 0; x < this->num_items; x++) {
-      if (this->items[x].data.id == item_id) {
-        return x;
-      }
-    }
-    throw std::out_of_range("item not present");
-  }
-
-  void sort() {
-    std::sort(this->items.data(), this->items.data() + this->num_items);
-  }
-
-  void assign_ids(uint32_t base_id) {
-    for (size_t z = 0; z < this->num_items; z++) {
-      this->items[z].data.id = base_id + z;
-    }
-  }
-
   void decode_from_client(Version v) {
     for (size_t z = 0; z < this->items.size(); z++) {
       this->items[z].data.decode_for_version(v);
@@ -423,3 +340,46 @@ using PlayerBank200BE = PlayerBankT<200, true>;
 check_struct_size(PlayerBank60, 0x05A8);
 check_struct_size(PlayerBank200, 0x12C8);
 check_struct_size(PlayerBank200BE, 0x12C8);
+
+struct PlayerBank {
+  uint32_t max_meseta = 999999;
+  uint32_t max_items = 200;
+  uint32_t meseta = 0;
+  std::vector<PlayerBankItem> items;
+
+  PlayerBank() = default;
+
+  template <size_t SrcSlotCount, bool SrcBE>
+  PlayerBank(const PlayerBankT<SrcSlotCount, SrcBE>& src)
+      : max_meseta(999999), max_items(SrcSlotCount), meseta(src.meseta) {
+    this->items.reserve(src.num_items);
+    for (size_t z = 0; z < src.num_items; z++) {
+      this->items.emplace_back(src.items[z]);
+    }
+  }
+
+  template <size_t DestSlotCount, bool DestBE>
+  operator PlayerBankT<DestSlotCount, DestBE>() const {
+    PlayerBankT<DestSlotCount, DestBE> ret;
+    ret.num_items = std::min<size_t>(ret.items.size(), this->items.size());
+    ret.meseta = this->meseta;
+    for (size_t z = 0; z < ret.num_items; z++) {
+      ret.items[z] = this->items[z];
+    }
+    return ret;
+  }
+
+  void load(FILE* f);
+  void save(FILE* f) const;
+
+  uint32_t bb_checksum() const;
+
+  void add_item(const ItemData& item, const ItemData::StackLimits& limits);
+  ItemData remove_item(uint32_t item_id, uint32_t amount, const ItemData::StackLimits& limits);
+  size_t find_item(uint32_t item_id);
+  void sort();
+
+  void assign_ids(uint32_t base_id);
+
+  void enforce_stack_limits(std::shared_ptr<const ItemData::StackLimits> stack_limits);
+};

src/PlayerSubordinates.cc

@@ -50,7 +50,7 @@ void GuildCardBB::clear() {
   this->team_name.clear();
   this->description.clear();
   this->present = 0;
-  this->language = 0;
+  this->language = Language::JAPANESE;
   this->section_id = 0;
   this->char_class = 0;
 }
@@ -218,11 +218,11 @@ PlayerRecordsChallengeBB::PlayerRecordsChallengeBB(const PlayerRecordsChallengeD
       grave_x(rec.grave_x),
       grave_y(rec.grave_y),
       grave_z(rec.grave_z),
-      grave_team(rec.grave_team.decode(), 1),
-      grave_message(rec.grave_message.decode(), 1),
+      grave_team(rec.grave_team.decode(), Language::ENGLISH),
+      grave_message(rec.grave_message.decode(), Language::ENGLISH),
       unknown_m5(0),
       unknown_t6(0),
-      rank_title(rec.rank_title.decode(), 1),
+      rank_title(rec.rank_title.decode(), Language::ENGLISH),
       unknown_l7(0) {}
 
 PlayerRecordsChallengeBB::PlayerRecordsChallengeBB(const PlayerRecordsChallengePC& rec)
@@ -242,11 +242,11 @@ PlayerRecordsChallengeBB::PlayerRecordsChallengeBB(const PlayerRecordsChallengeP
       grave_x(rec.grave_x),
       grave_y(rec.grave_y),
       grave_z(rec.grave_z),
-      grave_team(rec.grave_team.decode(), 1),
-      grave_message(rec.grave_message.decode(), 1),
+      grave_team(rec.grave_team.decode(), Language::ENGLISH),
+      grave_message(rec.grave_message.decode(), Language::ENGLISH),
       unknown_m5(0),
       unknown_t6(0),
-      rank_title(rec.rank_title.decode(), 1),
+      rank_title(rec.rank_title.decode(), Language::ENGLISH),
       unknown_l7(0) {}
 
 PlayerRecordsChallengeBB::operator PlayerRecordsChallengeDC() const {
@@ -309,22 +309,7 @@ PlayerRecordsChallengeBB::operator PlayerRecordsChallengePC() const {
   return ret;
 }
 
-QuestFlagsV1& QuestFlagsV1::operator=(const QuestFlags& other) {
-  this->data[0] = other.data[0];
-  this->data[1] = other.data[1];
-  this->data[2] = other.data[2];
-  return *this;
-}
-
-QuestFlagsV1::operator QuestFlags() const {
-  QuestFlags ret;
-  ret.data[0] = this->data[0];
-  ret.data[1] = this->data[1];
-  ret.data[2] = this->data[2];
-  return ret;
-}
-
-const QuestFlagsForDifficulty QuestFlagsForDifficulty::BB_QUEST_FLAG_APPLY_MASK{{
+const QuestFlagsForDifficulty BB_QUEST_FLAG_APPLY_MASK{{
     // clang-format off
     /* 0000 */ 0x00, 0x3F, 0xFF, 0xE3, 0xE0, 0xFF, 0xFF, 0x00,
     /* 0040 */ 0x03, 0xFF, 0xFF, 0xFF, 0xF0, 0x00, 0x00, 0x00,

src/PlayerSubordinates.hh

@@ -32,7 +32,21 @@ struct PlayerVisualConfigT {
   /* 10 */ parray<uint8_t, 8> unknown_a2;
   /* 18 */ U32T<BE> name_color = 0xFFFFFFFF; // ARGB
   /* 1C */ uint8_t extra_model = 0;
-  /* 1D */ parray<uint8_t, 0x0F> unused;
+  // Some NPCs can crash the client if the character's class is incorrect. To
+  // handle this, we save the affected fields in the unused bytes after
+  // extra_model. This is a newserv-specific extension; it appears the
+  // following 15 bytes were simply never used by Sega.
+  /* 1D */ uint8_t npc_saved_data_type = 0;
+  /* 1E */ uint8_t npc_saved_costume = 0;
+  /* 1F */ uint8_t npc_saved_skin = 0;
+  /* 20 */ uint8_t npc_saved_face = 0;
+  /* 21 */ uint8_t npc_saved_head = 0;
+  /* 22 */ uint8_t npc_saved_hair = 0;
+  /* 23 */ uint8_t npc_saved_hair_r = 0;
+  /* 24 */ uint8_t npc_saved_hair_g = 0;
+  /* 25 */ uint8_t npc_saved_hair_b = 0;
+  /* 26 */ parray<uint8_t, 2> unused;
+  /* 28 */ F32T<BE> npc_saved_proportion_y = 0.0;
   // See compute_name_color_checksum for details on how this is computed. If the
   // value is incorrect, V1 and V2 will ignore the name_color field and use the
   // default color instead. This field is ignored on GC; on BB (and presumably
@@ -81,6 +95,72 @@ struct PlayerVisualConfigT {
     this->name_color_checksum = this->compute_name_color_checksum(this->name_color);
   }
 
+  void backup_npc_saved_fields() {
+    if (this->npc_saved_data_type == 0x8E) {
+      return;
+    }
+
+    // Restore old-format data if needed before backing up again
+    this->restore_npc_saved_fields();
+
+    this->npc_saved_data_type = 0x8E;
+    this->npc_saved_costume = this->costume;
+    this->npc_saved_skin = this->skin;
+    this->npc_saved_face = this->face;
+    this->npc_saved_head = this->head;
+    this->npc_saved_hair = this->hair;
+    this->npc_saved_hair_r = this->hair_r;
+    this->npc_saved_hair_g = this->hair_g;
+    this->npc_saved_hair_b = this->hair_b;
+    this->npc_saved_proportion_y = this->proportion_y;
+    this->costume = 0;
+    this->skin = 0;
+    this->face = 0;
+    this->head = 0;
+    this->hair = 0;
+    this->hair_r = 0;
+    this->hair_g = 0;
+    this->hair_b = 0;
+    this->proportion_y = 0;
+  }
+
+  void restore_npc_saved_fields() {
+    switch (this->npc_saved_data_type) {
+      case 0x00:
+        break;
+      case 0x8D: // Old format
+        this->char_class = this->npc_saved_costume;
+        this->head = this->npc_saved_skin;
+        this->hair = this->npc_saved_face;
+        break;
+      case 0x8E: // New format
+        this->costume = this->npc_saved_costume;
+        this->skin = this->npc_saved_skin;
+        this->face = this->npc_saved_face;
+        this->head = this->npc_saved_head;
+        this->hair = this->npc_saved_hair;
+        this->hair_r = this->npc_saved_hair_r;
+        this->hair_g = this->npc_saved_hair_g;
+        this->hair_b = this->npc_saved_hair_b;
+        this->proportion_y = this->npc_saved_proportion_y;
+        break;
+      default:
+        throw std::runtime_error("unknown saved NPC data format");
+    }
+
+    this->npc_saved_data_type = 0;
+    this->npc_saved_costume = 0;
+    this->npc_saved_skin = 0;
+    this->npc_saved_face = 0;
+    this->npc_saved_head = 0;
+    this->npc_saved_hair = 0;
+    this->npc_saved_hair_r = 0;
+    this->npc_saved_hair_g = 0;
+    this->npc_saved_hair_b = 0;
+    this->unused.clear(0);
+    this->npc_saved_proportion_y = 0.0;
+  }
+
   void enforce_lobby_join_limits_for_version(Version v) {
     struct ClassMaxes {
       uint16_t costume;
@@ -248,7 +328,7 @@ struct PlayerDispDataDCPCV3T {
     this->visual.enforce_lobby_join_limits_for_version(v);
   }
 
-  PlayerDispDataBB to_bb(uint8_t to_language, uint8_t from_language) const;
+  PlayerDispDataBB to_bb(Language to_language, Language from_language) const;
 } __attribute__((packed));
 using PlayerDispDataDCPCV3 = PlayerDispDataDCPCV3T<false>;
 using PlayerDispDataDCPCV3BE = PlayerDispDataDCPCV3T<true>;
@@ -281,7 +361,7 @@ struct PlayerDispDataBB {
   }
 
   template <bool BE>
-  PlayerDispDataDCPCV3T<BE> to_dcpcv3(uint8_t to_language, uint8_t from_language) const {
+  PlayerDispDataDCPCV3T<BE> to_dcpcv3(Language to_language, Language from_language) const {
     PlayerDispDataDCPCV3T<BE> ret;
     ret.stats = this->stats;
     ret.visual = this->visual;
@@ -297,7 +377,7 @@ struct PlayerDispDataBB {
 } __packed_ws__(PlayerDispDataBB, 0x190);
 
 template <bool BE>
-PlayerDispDataBB PlayerDispDataDCPCV3T<BE>::to_bb(uint8_t to_language, uint8_t from_language) const {
+PlayerDispDataBB PlayerDispDataDCPCV3T<BE>::to_bb(Language to_language, Language from_language) const {
   PlayerDispDataBB bb;
   bb.stats = this->stats;
   bb.visual = this->visual;
@@ -318,7 +398,7 @@ struct GuildCardDCNTE {
   /* 20 */ pstring<TextEncoding::MARKED, 0x48> description;
   /* 68 */ parray<uint8_t, 0x0F> unused2;
   /* 77 */ uint8_t present = 0;
-  /* 78 */ uint8_t language = 0;
+  /* 78 */ Language language = Language::JAPANESE;
   /* 79 */ uint8_t section_id = 0;
   /* 7A */ uint8_t char_class = 0;
   /* 7B */
@@ -333,7 +413,7 @@ struct GuildCardDC {
   /* 20 */ pstring<TextEncoding::MARKED, 0x48> description;
   /* 68 */ parray<uint8_t, 0x11> unused2;
   /* 79 */ uint8_t present = 0;
-  /* 7A */ uint8_t language = 0;
+  /* 7A */ Language language = Language::JAPANESE;
   /* 7B */ uint8_t section_id = 0;
   /* 7C */ uint8_t char_class = 0;
   /* 7D */
@@ -348,7 +428,7 @@ struct GuildCardPC {
   /* 08 */ pstring<TextEncoding::UTF16, 0x18> name;
   /* 38 */ pstring<TextEncoding::UTF16, 0x5A> description;
   /* EC */ uint8_t present = 0;
-  /* ED */ uint8_t language = 0;
+  /* ED */ Language language = Language::JAPANESE;
   /* EE */ uint8_t section_id = 0;
   /* EF */ uint8_t char_class = 0;
   /* F0 */
@@ -375,11 +455,11 @@ struct GuildCardGCT {
   /* 04:04 */ U32T<BE> guild_card_number = 0;
   /* 08:08 */ pstring<TextEncoding::ASCII, 0x18> name;
   /* 20:20 */ pstring<TextEncoding::MARKED, DescriptionLength> description;
-  /* 8C:8C */ uint8_t present = 0;
-  /* 8D:8D */ uint8_t language = 0;
-  /* 8E:8E */ uint8_t section_id = 0;
-  /* 8F:8F */ uint8_t char_class = 0;
-  /* 90:90 */
+  /* A0:8C */ uint8_t present = 0;
+  /* A1:8D */ Language language = Language::JAPANESE;
+  /* A2:8E */ uint8_t section_id = 0;
+  /* A3:8F */ uint8_t char_class = 0;
+  /* A4:90 */
 
   operator GuildCardBB() const;
 } __attribute__((packed));
@@ -400,7 +480,7 @@ struct GuildCardXB {
   /* 0010 */ pstring<TextEncoding::ASCII, 0x18> name;
   /* 0028 */ pstring<TextEncoding::MARKED, 0x200> description;
   /* 0228 */ uint8_t present = 0;
-  /* 0229 */ uint8_t language = 0;
+  /* 0229 */ Language language = Language::JAPANESE;
   /* 022A */ uint8_t section_id = 0;
   /* 022B */ uint8_t char_class = 0;
   /* 022C */
@@ -414,7 +494,7 @@ struct GuildCardBB {
   /* 0034 */ pstring<TextEncoding::UTF16_ALWAYS_MARKED, 0x10> team_name;
   /* 0054 */ pstring<TextEncoding::UTF16, 0x58> description;
   /* 0104 */ uint8_t present = 0;
-  /* 0105 */ uint8_t language = 0;
+  /* 0105 */ Language language = Language::JAPANESE;
   /* 0106 */ uint8_t section_id = 0;
   /* 0107 */ uint8_t char_class = 0;
   /* 0108 */
@@ -483,6 +563,8 @@ struct XBNetworkLocation {
   /* 04 */ le_uint32_t external_ipv4_address = 0x23232323;
   /* 08 */ le_uint16_t port = 9500;
   /* 0A */ parray<uint8_t, 6> mac_address = 0x77;
+  // The remainder of this struct appears to be private/opaque in the XDK (and
+  // newserv doesn't use it either)
   /* 10 */ le_uint32_t sg_ip_address = 0x0B0B0B0B;
   /* 14 */ le_uint32_t spi = 0xCCCCCCCC;
   /* 18 */ le_uint64_t account_id = 0xFFFFFFFFFFFFFFFF;
@@ -694,13 +776,13 @@ struct PlayerRecordsChallengeBB {
         grave_x(rec.grave_x),
         grave_y(rec.grave_y),
         grave_z(rec.grave_z),
-        grave_team(rec.grave_team.decode(), 1),
-        grave_message(rec.grave_message.decode(), 1),
+        grave_team(rec.grave_team.decode(), Language::ENGLISH),
+        grave_message(rec.grave_message.decode(), Language::ENGLISH),
         unknown_m5(rec.unknown_m5),
         ep1_online_award_state(rec.ep1_online_award_state),
         ep2_online_award_state(rec.ep2_online_award_state),
         ep1_offline_award_state(rec.ep1_offline_award_state),
-        rank_title(rec.rank_title.decode(), 1),
+        rank_title(rec.rank_title.decode(), Language::ENGLISH),
         unknown_l7(rec.unknown_l7) {
     for (size_t z = 0; z < std::min<size_t>(this->unknown_t6.size(), rec.unknown_t6.size()); z++) {
       this->unknown_t6[z] = rec.unknown_t6[z];
@@ -728,8 +810,8 @@ struct PlayerRecordsChallengeBB {
     ret.grave_x = this->grave_x;
     ret.grave_y = this->grave_y;
     ret.grave_z = this->grave_z;
-    ret.grave_team.encode(this->grave_team.decode(), 1);
-    ret.grave_message.encode(this->grave_message.decode(), 1);
+    ret.grave_team.encode(this->grave_team.decode(), Language::ENGLISH);
+    ret.grave_message.encode(this->grave_message.decode(), Language::ENGLISH);
     ret.unknown_m5 = this->unknown_m5;
     for (size_t z = 0; z < std::min<size_t>(ret.unknown_t6.size(), this->unknown_t6.size()); z++) {
       ret.unknown_t6[z] = this->unknown_t6[z];
@@ -737,7 +819,7 @@ struct PlayerRecordsChallengeBB {
     ret.ep1_online_award_state = this->ep1_online_award_state;
     ret.ep2_online_award_state = this->ep2_online_award_state;
     ret.ep1_offline_award_state = this->ep1_offline_award_state;
-    ret.rank_title.encode(this->rank_title.decode(), 1);
+    ret.rank_title.encode(this->rank_title.decode(), Language::ENGLISH);
     ret.unknown_l7 = this->unknown_l7;
     return ret;
   }
@@ -774,38 +856,53 @@ check_struct_size(PlayerRecordsBattle, 0x18);
 check_struct_size(PlayerRecordsBattleBE, 0x18);
 
 template <typename DestT, typename SrcT = DestT>
-DestT convert_player_disp_data(const SrcT&, uint8_t, uint8_t) {
+DestT convert_player_disp_data(const SrcT&, Language, Language) {
   static_assert(phosg::always_false<DestT, SrcT>::v,
       "unspecialized convert_player_disp_data should never be called");
 }
 
 template <>
-inline PlayerDispDataDCPCV3 convert_player_disp_data<PlayerDispDataDCPCV3>(const PlayerDispDataDCPCV3& src, uint8_t, uint8_t) {
+inline PlayerDispDataDCPCV3 convert_player_disp_data<PlayerDispDataDCPCV3>(const PlayerDispDataDCPCV3& src, Language, Language) {
   return src;
 }
 
 template <>
 inline PlayerDispDataDCPCV3 convert_player_disp_data<PlayerDispDataDCPCV3, PlayerDispDataBB>(
-    const PlayerDispDataBB& src, uint8_t to_language, uint8_t from_language) {
+    const PlayerDispDataBB& src, Language to_language, Language from_language) {
   return src.to_dcpcv3<false>(to_language, from_language);
 }
 
 template <>
 inline PlayerDispDataBB convert_player_disp_data<PlayerDispDataBB, PlayerDispDataDCPCV3>(
-    const PlayerDispDataDCPCV3& src, uint8_t to_language, uint8_t from_language) {
+    const PlayerDispDataDCPCV3& src, Language to_language, Language from_language) {
   return src.to_bb(to_language, from_language);
 }
 
 template <>
 inline PlayerDispDataBB convert_player_disp_data<PlayerDispDataBB>(
-    const PlayerDispDataBB& src, uint8_t, uint8_t) {
+    const PlayerDispDataBB& src, Language, Language) {
   return src;
 }
 
-struct QuestFlagsForDifficulty {
-  static const QuestFlagsForDifficulty BB_QUEST_FLAG_APPLY_MASK;
+template <size_t NumFlags>
+struct FlagsArray {
+  parray<uint8_t, (NumFlags >> 3)> data = 0;
 
-  parray<uint8_t, 0x80> data;
+  FlagsArray() = default;
+  FlagsArray(const FlagsArray& other) = default;
+  FlagsArray(FlagsArray&& other) = default;
+  FlagsArray& operator=(const FlagsArray& other) = default;
+  FlagsArray& operator=(FlagsArray&& other) = default;
+
+  FlagsArray(std::initializer_list<uint8_t> init_items) : data(init_items) {}
+
+  template <size_t OtherNumFlags>
+  explicit FlagsArray(const FlagsArray<OtherNumFlags>& other) : data(other.data) {}
+  template <size_t OtherNumFlags>
+  FlagsArray& operator=(const FlagsArray<OtherNumFlags>& other) {
+    this->data = other.data;
+    return *this;
+  }
 
   inline bool get(uint16_t flag_index) const {
     size_t byte_index = flag_index >> 3;
@@ -822,6 +919,7 @@ struct QuestFlagsForDifficulty {
     uint8_t mask = 0x80 >> (flag_index & 7);
     this->data[byte_index] &= (~mask);
   }
+
   inline void update_all(bool set) {
     if (set) {
       this->data.clear(0xFF);
@@ -829,50 +927,66 @@ struct QuestFlagsForDifficulty {
       this->data.clear(0x00);
     }
   }
-} __packed_ws__(QuestFlagsForDifficulty, 0x80);
+} __attribute__((packed));
 
-struct QuestFlags {
-  parray<QuestFlagsForDifficulty, 4> data;
+template <size_t NumFlagsPerTable, size_t NumTables, typename TableIndexT = size_t>
+struct FlagsTable {
+  parray<FlagsArray<NumFlagsPerTable>, NumTables> data;
 
-  inline bool get(uint8_t difficulty, uint16_t flag_index) const {
-    return this->data[difficulty].get(flag_index);
+  FlagsTable() = default;
+  FlagsTable(const FlagsTable& other) = default;
+  FlagsTable(FlagsTable&& other) = default;
+  FlagsTable& operator=(const FlagsTable& other) = default;
+  FlagsTable& operator=(FlagsTable&& other) = default;
+
+  template <size_t OtherNumFlagsPerTable, size_t OtherNumTables>
+  explicit FlagsTable(const FlagsTable<OtherNumFlagsPerTable, OtherNumTables, TableIndexT>& other) : data(other.data) {}
+  template <size_t OtherNumFlagsPerTable, size_t OtherNumTables>
+  FlagsTable& operator=(const FlagsTable<OtherNumFlagsPerTable, OtherNumTables, TableIndexT>& other) {
+    this->data = other.data;
+    return *this;
   }
-  inline void set(uint8_t difficulty, uint16_t flag_index) {
-    this->data[difficulty].set(flag_index);
+
+  inline FlagsArray<NumFlagsPerTable>& array(TableIndexT which) {
+    return this->data[static_cast<size_t>(which)];
   }
-  inline void clear(uint8_t difficulty, uint16_t flag_index) {
-    this->data[difficulty].clear(flag_index);
+  inline const FlagsArray<NumFlagsPerTable>& array(TableIndexT which) const {
+    return this->data[static_cast<size_t>(which)];
   }
-  inline void update_all(uint8_t difficulty, bool set) {
-    this->data[difficulty].update_all(set);
+
+  inline bool get(TableIndexT array_index, size_t flag_index) const {
+    return this->array(array_index).get(flag_index);
+  }
+  inline void set(TableIndexT array_index, size_t flag_index) {
+    this->array(array_index).set(flag_index);
+  }
+  inline void clear(TableIndexT array_index, size_t flag_index) {
+    this->array(array_index).clear(flag_index);
+  }
+  inline void update_all(TableIndexT array_index, bool set) {
+    this->array(array_index).update_all(set);
   }
   inline void update_all(bool set) {
-    for (size_t z = 0; z < 4; z++) {
+    for (size_t z = 0; z < this->data.size(); z++) {
       this->update_all(z, set);
     }
   }
-} __packed_ws__(QuestFlags, 0x200);
+} __attribute__((packed));
 
-struct QuestFlagsV1 {
-  parray<QuestFlagsForDifficulty, 3> data;
+using QuestFlagsForDifficulty = FlagsArray<0x400>;
+using QuestFlagsV1 = FlagsTable<0x400, 3, Difficulty>;
+using QuestFlags = FlagsTable<0x400, 4, Difficulty>;
+using Ep3SeqVars = FlagsArray<0x2000>;
+using SwitchFlagsV1 = FlagsTable<0x100, 0x10>;
+using SwitchFlags = FlagsTable<0x100, 0x12>;
+static_assert(sizeof(QuestFlagsForDifficulty) == 0x80);
+static_assert(sizeof(QuestFlagsV1) == 0x180);
+static_assert(sizeof(QuestFlags) == 0x200);
+static_assert(sizeof(Ep3SeqVars) == 0x400);
+static_assert(sizeof(SwitchFlagsV1) == 0x200);
+static_assert(sizeof(SwitchFlags) == 0x240);
 
-  QuestFlagsV1& operator=(const QuestFlags& other);
-  operator QuestFlags() const;
-} __packed_ws__(QuestFlagsV1, 0x180);
-
-struct SwitchFlags {
-  parray<parray<uint8_t, 0x20>, 0x12> data;
-
-  inline bool get(uint8_t floor, uint16_t flag_num) const {
-    return this->data[floor][flag_num >> 3] & (0x80 >> (flag_num & 7));
-  }
-  inline void set(uint8_t floor, uint16_t flag_num) {
-    this->data[floor][flag_num >> 3] |= (0x80 >> (flag_num & 7));
-  }
-  inline void clear(uint8_t floor, uint16_t flag_num) {
-    this->data[floor][flag_num >> 3] &= ~(0x80 >> (flag_num & 7));
-  }
-} __packed_ws__(SwitchFlags, 0x240);
+extern const QuestFlagsForDifficulty BB_QUEST_FLAG_APPLY_MASK;
 
 struct BattleRules {
   enum class TechDiskMode : uint8_t {

src/ProxyCommands.cc

@@ -236,7 +236,7 @@ static asio::awaitable<HandlerResult> S_G_9A(shared_ptr<Client> c, Channel::Mess
   // right after the client config data
   c->proxy_session->server_channel->send(
       0x9E, 0x01, &cmd,
-      cmd.is_extended ? sizeof(C_LoginExtended_GC_9E) : sizeof(C_Login_GC_9E));
+      cmd.is_extended ? sizeof(C_LoginExtended_GC_9E) : sizeof(C_Login_PC_GC_9E));
 
   co_return HandlerResult::SUPPRESS;
 }
@@ -424,10 +424,7 @@ static asio::awaitable<HandlerResult> S_V123_04(shared_ptr<Client> c, Channel::M
   // If there was previously a guild card number, assume we got the lobby server
   // init text instead of the port map init text.
   memcpy(c->proxy_session->remote_client_config_data.data(),
-      had_guild_card_number
-          ? "t Lobby Server. Copyright SEGA E"
-          : "t Port Map. Copyright SEGA Enter",
-      0x20);
+      had_guild_card_number ? "t Lobby Server. Copyright SEGA E" : "t Port Map. Copyright SEGA Enter", 0x20);
   memcpy(c->proxy_session->remote_client_config_data.data(), &cmd.client_config,
       min<size_t>(msg.data.size() - offsetof(S_UpdateClientConfig_V3_04, client_config),
           c->proxy_session->remote_client_config_data.bytes()));
@@ -526,6 +523,12 @@ constexpr on_message_t S_P_81 = &S_81<SC_SimpleMail_PC_81>;
 constexpr on_message_t S_B_81 = &S_81<SC_SimpleMail_BB_81>;
 
 static asio::awaitable<HandlerResult> S_88(shared_ptr<Client> c, Channel::Message& msg) {
+  // If the client isn't in the lobby, suppress the command (Ep3 can crash if
+  // it receives this while loading; other versions probably also will crash)
+  if (!c->proxy_session->is_in_lobby) {
+    co_return HandlerResult::SUPPRESS;
+  }
+
   bool modified = false;
   if (c->login && c->login->account->account_id != c->proxy_session->remote_guild_card_number) {
     size_t expected_size = sizeof(S_ArrowUpdateEntry_88) * msg.flag;
@@ -678,6 +681,12 @@ static asio::awaitable<HandlerResult> C_B3(shared_ptr<Client> c, Channel::Messag
   }
 }
 
+static asio::awaitable<HandlerResult> C_B_E0(shared_ptr<Client> c, Channel::Message&) {
+  auto ret = c->proxy_session->bb_client_sent_E0 ? HandlerResult::FORWARD : HandlerResult::SUPPRESS;
+  c->proxy_session->bb_client_sent_E0 = true;
+  co_return ret;
+}
+
 static asio::awaitable<HandlerResult> S_B_E2(shared_ptr<Client> c, Channel::Message& msg) {
   if (c->check_flag(Client::Flag::PROXY_SAVE_FILES)) {
     string output_filename = std::format("system.{}.psosys", phosg::now());
@@ -779,6 +788,9 @@ static asio::awaitable<HandlerResult> S_19_U_14(shared_ptr<Client> c, Channel::M
   if (is_patch(c->version())) {
     auto& cmd = msg.check_size_t<S_Reconnect_Patch_14>();
     new_ep = make_endpoint_ipv4(cmd.address, cmd.port);
+  } else if (msg.flag == 6 && msg.data.size() >= sizeof(S_ReconnectIPv6_Extension_19)) {
+    auto& cmd = msg.check_size_t<S_ReconnectIPv6_Extension_19>(0xFFFF);
+    new_ep = make_endpoint_ipv6(cmd.address.data(), cmd.port);
   } else {
     // This weird maximum size is here to properly handle the version-split
     // command that some servers (including newserv) use on port 9100
@@ -838,13 +850,12 @@ static asio::awaitable<HandlerResult> SC_6x60_6xA2(shared_ptr<Client> c, Channel
     co_return HandlerResult::FORWARD;
   }
 
-  using DropMode = ProxySession::DropMode;
   switch (c->proxy_session->drop_mode) {
-    case DropMode::DISABLED:
+    case ProxyDropMode::DISABLED:
       co_return HandlerResult::SUPPRESS;
-    case DropMode::PASSTHROUGH:
+    case ProxyDropMode::PASSTHROUGH:
       co_return HandlerResult::FORWARD;
-    case DropMode::INTERCEPT:
+    case ProxyDropMode::INTERCEPT:
       break;
     default:
       throw logic_error("invalid drop mode");
@@ -861,7 +872,13 @@ static asio::awaitable<HandlerResult> SC_6x60_6xA2(shared_ptr<Client> c, Channel
 
   G_SpecializableItemDropRequest_6xA2 cmd = normalize_drop_request(msg.data.data(), msg.data.size());
   auto rec = reconcile_drop_request_with_map(
-      c, cmd, c->proxy_session->lobby_episode, c->proxy_session->lobby_event, c->proxy_session->map_state, false);
+      c,
+      cmd,
+      c->proxy_session->lobby_episode,
+      c->proxy_session->lobby_difficulty,
+      c->proxy_session->lobby_event,
+      c->proxy_session->map_state,
+      false);
 
   ItemCreator::DropResult res;
   if (rec.obj_st) {
@@ -884,7 +901,7 @@ static asio::awaitable<HandlerResult> SC_6x60_6xA2(shared_ptr<Client> c, Channel
     c->log.info_f("No item was created");
   } else {
     auto s = c->require_server_state();
-    string name = s->describe_item(c->version(), res.item, false);
+    string name = s->describe_item(c->version(), res.item);
     c->log.info_f("Entity {:04X} (area {:02X}) created item {}", cmd.entity_index, cmd.effective_area, name);
     res.item.id = c->proxy_session->next_item_id++;
     c->log.info_f("Creating item {:08X} at {:02X}:{:g},{:g} for all clients",
@@ -910,10 +927,43 @@ static asio::awaitable<HandlerResult> S_6x(shared_ptr<Client> c, Channel::Messag
       c->log.warning_f("Blocking invalid subcommand from server");
       co_return HandlerResult::SUPPRESS;
 
+    case 0x16:
+    case 0x84: {
+      const auto& cmd = msg.check_size_t<G_VolOptBossActions_6x16>(0xFFFF);
+      if (cmd.entity_index_count > 6) {
+        c->log.warning_f("Blocking subcommand 6x16/6x84 with invalid entity index count");
+        co_return HandlerResult::SUPPRESS;
+      }
+      for (size_t z = 0; z < cmd.entity_index_table.size(); z++) {
+        if (cmd.entity_index_table[z] >= 6) {
+          c->log.warning_f("Blocking subcommand 6x16/6x84 with invalid entity index");
+          co_return HandlerResult::SUPPRESS;
+        }
+      }
+      break;
+    }
+
+    case 0x17: {
+      const auto& cmd = msg.check_size_t<G_SetEntityPositionAndAngle_6x17>();
+      if (cmd.header.entity_id == c->lobby_client_id) {
+        c->log.warning_f("Blocking subcommand 6x17 targeting local client");
+        co_return HandlerResult::SUPPRESS;
+      }
+      break;
+    }
+
+    case 0x2F: {
+      const auto& cmd = msg.check_size_t<G_ChangePlayerHP_6x2F>();
+      if (cmd.client_id == c->lobby_client_id) {
+        c->log.warning_f("Blocking subcommand 6x2F targeting local player");
+        co_return HandlerResult::SUPPRESS;
+      }
+      break;
+    }
+
     case 0x46: {
-      const auto& cmd = msg.check_size_t<G_AttackFinished_6x46>(
-          offsetof(G_AttackFinished_6x46, targets), sizeof(G_AttackFinished_6x46));
-      if (cmd.target_count > min<size_t>(cmd.header.size - 2, cmd.targets.size())) {
+      const auto& header = msg.check_size_t<G_AttackFinished_Header_6x46>(0xFFFF);
+      if (header.target_count > min<size_t>(header.header.size - sizeof(G_AttackFinished_Header_6x46) / 4, 10)) {
         c->log.warning_f("Blocking subcommand 6x46 with invalid count");
         co_return HandlerResult::SUPPRESS;
       }
@@ -921,9 +971,8 @@ static asio::awaitable<HandlerResult> S_6x(shared_ptr<Client> c, Channel::Messag
     }
 
     case 0x47: {
-      const auto& cmd = msg.check_size_t<G_CastTechnique_6x47>(
-          offsetof(G_CastTechnique_6x47, targets), sizeof(G_CastTechnique_6x47));
-      if (cmd.target_count > min<size_t>(cmd.header.size - 2, cmd.targets.size())) {
+      const auto& header = msg.check_size_t<G_CastTechnique_Header_6x47>(0xFFFF);
+      if (header.target_count > min<size_t>(header.header.size - sizeof(G_CastTechnique_Header_6x47) / 4, 10)) {
         c->log.warning_f("Blocking subcommand 6x47 with invalid count");
         co_return HandlerResult::SUPPRESS;
       }
@@ -931,9 +980,8 @@ static asio::awaitable<HandlerResult> S_6x(shared_ptr<Client> c, Channel::Messag
     }
 
     case 0x49: {
-      const auto& cmd = msg.check_size_t<G_ExecutePhotonBlast_6x49>(
-          offsetof(G_ExecutePhotonBlast_6x49, targets), sizeof(G_ExecutePhotonBlast_6x49));
-      if (cmd.target_count > min<size_t>(cmd.header.size - 3, cmd.targets.size())) {
+      const auto& header = msg.check_size_t<G_ExecutePhotonBlast_Header_6x49>(0xFFFF);
+      if (header.target_count > min<size_t>(header.header.size - sizeof(G_ExecutePhotonBlast_Header_6x49) / 4, 10)) {
         c->log.warning_f("Blocking subcommand 6x49 with invalid count");
         co_return HandlerResult::SUPPRESS;
       }
@@ -952,6 +1000,42 @@ static asio::awaitable<HandlerResult> S_6x(shared_ptr<Client> c, Channel::Messag
     case 0xA2:
       co_return co_await SC_6x60_6xA2(c, msg);
 
+    case 0x6A: {
+      auto& cmd = msg.check_size_t<G_SetBossWarpFlags_6x6A>();
+      if (c->proxy_session->map_state) {
+        shared_ptr<MapState::ObjectState> obj_st;
+        try {
+          obj_st = c->proxy_session->map_state->object_state_for_index(c->version(), cmd.header.entity_id - 0x4000);
+        } catch (const exception& e) {
+          c->log.warning_f("Invalid object reference ({})", e.what());
+        }
+
+        if (!obj_st || !obj_st->super_obj) {
+          c->log.warning_f("Blocking subcommand 6x6A with missing object");
+          co_return HandlerResult::SUPPRESS;
+        }
+        auto set_entry = obj_st->super_obj->version(c->version()).set_entry;
+        if (!set_entry) {
+          c->log.warning_f("Blocking subcommand 6x6A with missing set entry");
+          co_return HandlerResult::SUPPRESS;
+        }
+        if (set_entry->base_type != 0x0019 && set_entry->base_type != 0x0055) {
+          c->log.warning_f("Blocking subcommand 6x6A with incorrect object type");
+          co_return HandlerResult::SUPPRESS;
+        }
+      }
+      break;
+    }
+
+    case 0x7D: {
+      const auto& cmd = msg.check_size_t<G_SetBattleModeData_6x7D>();
+      if ((cmd.what == 3 || cmd.what == 4) && cmd.params[0] >= 4) {
+        c->log.warning_f("Blocking subcommand 6x7D with invalid client ID");
+        co_return HandlerResult::SUPPRESS;
+      }
+      break;
+    }
+
     case 0xB3:
     case 0xB4:
     case 0xB5: {
@@ -1372,6 +1456,14 @@ static asio::awaitable<HandlerResult> S_G_B9(shared_ptr<Client> c, Channel::Mess
   co_return (c->version() == Version::GC_EP3) ? HandlerResult::FORWARD : HandlerResult::SUPPRESS;
 }
 
+static asio::awaitable<HandlerResult> C_G_B9(shared_ptr<Client> c, Channel::Message&) {
+  if (c->proxy_session->suppress_next_ep3_media_update_confirmation) {
+    c->proxy_session->suppress_next_ep3_media_update_confirmation = false;
+    co_return HandlerResult::SUPPRESS;
+  }
+  co_return HandlerResult::FORWARD;
+}
+
 static asio::awaitable<HandlerResult> S_G_EF(shared_ptr<Client> c, Channel::Message& msg) {
   if (is_ep3(c->version())) {
     if (c->check_flag(Client::Flag::PROXY_EP3_INFINITE_MESETA_ENABLED)) {
@@ -1419,10 +1511,11 @@ template <typename CmdT>
 static asio::awaitable<HandlerResult> S_65_67_68_EB(shared_ptr<Client> c, Channel::Message& msg) {
   if (msg.command == 0x67) {
     c->proxy_session->clear_lobby_players(12);
+    c->proxy_session->is_in_lobby = true;
     c->proxy_session->is_in_game = false;
     c->proxy_session->is_in_quest = false;
     c->floor = 0x0F;
-    c->proxy_session->lobby_difficulty = 0;
+    c->proxy_session->lobby_difficulty = Difficulty::NORMAL;
     c->proxy_session->lobby_section_id = 0;
     c->proxy_session->lobby_mode = GameMode::NORMAL;
     c->proxy_session->lobby_episode = Episode::EP1;
@@ -1547,7 +1640,7 @@ template <typename CmdT>
 static asio::awaitable<HandlerResult> S_64(shared_ptr<Client> c, Channel::Message& msg) {
   CmdT* cmd;
   S_JoinGame_Ep3_64* cmd_ep3 = nullptr;
-  if (c->sub_version >= 0x40) {
+  if ((c->sub_version >= 0x40) && is_v3(c->version())) {
     cmd = &msg.check_size_t<CmdT>(sizeof(S_JoinGame_Ep3_64));
     cmd_ep3 = &msg.check_size_t<S_JoinGame_Ep3_64>();
   } else if (c->version() == Version::XB_V3) {
@@ -1562,9 +1655,10 @@ static asio::awaitable<HandlerResult> S_64(shared_ptr<Client> c, Channel::Messag
 
   c->proxy_session->clear_lobby_players(4);
   c->floor = 0;
+  c->proxy_session->is_in_lobby = false;
   c->proxy_session->is_in_game = true;
   c->proxy_session->is_in_quest = false;
-  if constexpr (sizeof(cmd) > sizeof(S_JoinGame_DCNTE_64)) {
+  if constexpr (sizeof(*cmd) > sizeof(S_JoinGame_DCNTE_64)) {
     c->proxy_session->lobby_event = cmd->event;
     c->proxy_session->lobby_difficulty = cmd->difficulty;
     c->proxy_session->lobby_section_id = cmd->section_id;
@@ -1594,8 +1688,8 @@ static asio::awaitable<HandlerResult> S_64(shared_ptr<Client> c, Channel::Messag
 
   } else {
     c->proxy_session->lobby_event = 0;
-    c->proxy_session->lobby_difficulty = 0;
-    c->proxy_session->lobby_section_id = c->character()->disp.visual.section_id;
+    c->proxy_session->lobby_difficulty = Difficulty::NORMAL;
+    c->proxy_session->lobby_section_id = c->character_file()->disp.visual.section_id;
     c->proxy_session->lobby_mode = GameMode::NORMAL;
     c->proxy_session->lobby_random_seed = phosg::random_object<uint32_t>();
   }
@@ -1666,10 +1760,11 @@ static asio::awaitable<HandlerResult> S_E8(shared_ptr<Client> c, Channel::Messag
   auto& cmd = msg.check_size_t<S_JoinSpectatorTeam_Ep3_E8>();
 
   c->floor = 0;
+  c->proxy_session->is_in_lobby = false;
   c->proxy_session->is_in_game = true;
   c->proxy_session->is_in_quest = false;
   c->proxy_session->lobby_event = cmd.event;
-  c->proxy_session->lobby_difficulty = 0;
+  c->proxy_session->lobby_difficulty = Difficulty::NORMAL;
   c->proxy_session->lobby_section_id = cmd.section_id;
   c->proxy_session->lobby_mode = GameMode::NORMAL;
   c->proxy_session->lobby_random_seed = 0;
@@ -1755,10 +1850,11 @@ static asio::awaitable<HandlerResult> S_66_69_E9(shared_ptr<Client> c, Channel::
 
 static asio::awaitable<HandlerResult> C_98(shared_ptr<Client> c, Channel::Message& msg) {
   c->floor = 0x0F;
+  c->proxy_session->is_in_lobby = false;
   c->proxy_session->is_in_game = false;
   c->proxy_session->is_in_quest = false;
   c->proxy_session->lobby_event = 0;
-  c->proxy_session->lobby_difficulty = 0;
+  c->proxy_session->lobby_difficulty = Difficulty::NORMAL;
   c->proxy_session->lobby_section_id = 0;
   c->proxy_session->lobby_episode = Episode::EP1;
   c->proxy_session->lobby_mode = GameMode::NORMAL;
@@ -1926,9 +2022,9 @@ asio::awaitable<HandlerResult> C_6x(shared_ptr<Client> c, Channel::Message& msg)
     case 0x4A:
     case 0x4B:
     case 0x4C:
-      if (!is_v1(c->version()) && c->check_flag(Client::Flag::INFINITE_HP_ENABLED)) {
-        send_player_stats_change(c->channel, c->lobby_client_id, PlayerStatsChange::ADD_HP, 2550);
-        send_player_stats_change(c->proxy_session->server_channel, c->lobby_client_id, PlayerStatsChange::ADD_HP, 2550);
+      if (c->check_flag(Client::Flag::INFINITE_HP_ENABLED)) {
+        co_await send_change_player_hp(c, c->lobby_client_id, PlayerHPChange::MAXIMIZE_HP, 0);
+        send_change_player_hp(c->proxy_session->server_channel, c->lobby_client_id, PlayerHPChange::MAXIMIZE_HP, 0);
       }
       break;
 
@@ -1940,13 +2036,20 @@ asio::awaitable<HandlerResult> C_6x(shared_ptr<Client> c, Channel::Message& msg)
       c->pos = msg.check_size_t<G_SetPosition_6x3F>().pos;
       break;
 
-    case 0x40:
-      c->pos = msg.check_size_t<G_WalkToPosition_6x40>().pos;
+    case 0x40: {
+      const auto& cmd = msg.check_size_t<G_WalkToPosition_6x40>();
+      c->pos.x = cmd.pos.x;
+      c->pos.z = cmd.pos.z;
       break;
+    }
 
     case 0x41:
-      c->pos = msg.check_size_t<G_MoveToPosition_6x41_6x42>().pos;
+    case 0x42: {
+      const auto& cmd = msg.check_size_t<G_MoveToPosition_6x41_6x42>();
+      c->pos.x = cmd.pos.x;
+      c->pos.z = cmd.pos.z;
       break;
+    }
 
     case 0x48:
       if (!is_v1(c->version()) && c->check_flag(Client::Flag::INFINITE_TP_ENABLED)) {
@@ -2211,7 +2314,7 @@ static on_message_t handlers[0x100][NUM_VERSIONS][2] = {
 /* B6 */ {{S_invalid,     nullptr}, {S_invalid,     nullptr}, {S_invalid,     nullptr},  {S_invalid,     nullptr},     {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,     nullptr},      {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,        nullptr},      {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,    nullptr}},
 /* B7 */ {{S_invalid,     nullptr}, {S_invalid,     nullptr}, {S_invalid,     nullptr},  {S_invalid,     nullptr},     {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,     nullptr},      {S_invalid,        nullptr},     {S_G_B7,           nullptr},      {S_G_B7,           nullptr},      {S_G_B7,           nullptr},      {S_invalid,     nullptr},      {S_invalid,    nullptr}},
 /* B8 */ {{S_invalid,     nullptr}, {S_invalid,     nullptr}, {S_invalid,     nullptr},  {S_invalid,     nullptr},     {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,     nullptr},      {S_invalid,        nullptr},     {S_G_B8,           nullptr},      {S_G_B8,           nullptr},      {S_G_B8,           nullptr},      {S_invalid,     nullptr},      {nullptr,      nullptr}},
-/* B9 */ {{S_invalid,     nullptr}, {S_invalid,     nullptr}, {S_invalid,     nullptr},  {S_invalid,     nullptr},     {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,     nullptr},      {S_invalid,        nullptr},     {S_G_B9,           nullptr},      {S_G_B9,           nullptr},      {S_G_B9,           nullptr},      {S_invalid,     nullptr},      {S_invalid,    nullptr}},
+/* B9 */ {{S_invalid,     nullptr}, {S_invalid,     nullptr}, {S_invalid,     nullptr},  {S_invalid,     nullptr},     {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,     nullptr},      {S_invalid,        nullptr},     {S_G_B9,           C_G_B9},       {S_G_B9,           C_G_B9},       {S_G_B9,           C_G_B9},       {S_invalid,     nullptr},      {S_invalid,    nullptr}},
 /* BA */ {{S_invalid,     nullptr}, {S_invalid,     nullptr}, {S_invalid,     nullptr},  {S_invalid,     nullptr},     {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,     nullptr},      {S_invalid,        nullptr},     {S_G_BA,           nullptr},      {S_G_BA,           nullptr},      {S_G_BA,           nullptr},      {S_invalid,     nullptr},      {S_invalid,    nullptr}},
 /* BB */ {{S_invalid,     nullptr}, {S_invalid,     nullptr}, {S_invalid,     nullptr},  {S_invalid,     nullptr},     {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,     nullptr},      {S_invalid,        nullptr},     {nullptr,          nullptr},      {nullptr,          nullptr},      {nullptr,          nullptr},      {S_invalid,     nullptr},      {S_invalid,    nullptr}},
 /* BC */ {{S_invalid,     nullptr}, {S_invalid,     nullptr}, {S_invalid,     nullptr},  {S_invalid,     nullptr},     {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,     nullptr},      {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,        nullptr},      {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,    nullptr}},
@@ -2253,7 +2356,7 @@ static on_message_t handlers[0x100][NUM_VERSIONS][2] = {
 /* DE */ {{S_invalid,     nullptr}, {S_invalid,     nullptr}, {S_invalid,     nullptr},  {S_invalid,     nullptr},     {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,     nullptr},      {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,        nullptr},      {S_invalid,        nullptr},      {S_invalid,     nullptr},      {nullptr,      nullptr}},
 /* DF */ {{S_invalid,     nullptr}, {S_invalid,     nullptr}, {S_invalid,     nullptr},  {S_invalid,     nullptr},     {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,     nullptr},      {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,        nullptr},      {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,    nullptr}},
 // CMD     S_PC_PATCH     C          S_BB_PATCH     C          S_DC_NTE       C           S_DC_12_2000   C              S_DC_V1           C              S_DC_V2           C               S_PC_NTE       C               S_PC_V2        C               S_GC_NTE          C              S_GC_V3           C               S_GC_EP3_NTE      C               S_GC_EP3          C               S_XB_V3        C               S_BB_V4       C
-/* E0 */ {{S_invalid,     nullptr}, {S_invalid,     nullptr}, {S_invalid,     nullptr},  {S_invalid,     nullptr},     {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,     nullptr},      {S_invalid,        nullptr},     {nullptr,          nullptr},      {nullptr,          nullptr},      {nullptr,          nullptr},      {S_invalid,     nullptr},      {nullptr,      nullptr}},
+/* E0 */ {{S_invalid,     nullptr}, {S_invalid,     nullptr}, {S_invalid,     nullptr},  {S_invalid,     nullptr},     {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,     nullptr},      {S_invalid,        nullptr},     {nullptr,          nullptr},      {nullptr,          nullptr},      {nullptr,          nullptr},      {S_invalid,     nullptr},      {nullptr,      C_B_E0}},
 /* E1 */ {{S_invalid,     nullptr}, {S_invalid,     nullptr}, {S_invalid,     nullptr},  {S_invalid,     nullptr},     {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,     nullptr},      {S_invalid,        nullptr},     {nullptr,          nullptr},      {nullptr,          nullptr},      {nullptr,          nullptr},      {S_invalid,     nullptr},      {nullptr,      nullptr}},
 /* E2 */ {{S_invalid,     nullptr}, {S_invalid,     nullptr}, {S_invalid,     nullptr},  {S_invalid,     nullptr},     {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,     nullptr},      {S_invalid,        nullptr},     {nullptr,          nullptr},      {nullptr,          nullptr},      {nullptr,          nullptr},      {S_invalid,     nullptr},      {S_B_E2,       nullptr}},
 /* E3 */ {{S_invalid,     nullptr}, {S_invalid,     nullptr}, {S_invalid,     nullptr},  {S_invalid,     nullptr},     {S_invalid,        nullptr},     {S_invalid,        nullptr},      {S_invalid,     nullptr},      {S_invalid,     nullptr},      {S_invalid,        nullptr},     {nullptr,          nullptr},      {nullptr,          nullptr},      {nullptr,          nullptr},      {S_invalid,     nullptr},      {nullptr,      nullptr}},

src/ProxySession.cc

@@ -21,50 +21,17 @@ ProxySession::~ProxySession() {
   this->num_proxy_sessions--;
 }
 
-void ProxySession::set_drop_mode(shared_ptr<ServerState> s, Version version, int64_t override_random_seed, DropMode new_mode) {
+void ProxySession::set_drop_mode(
+    shared_ptr<ServerState> s, Version version, int64_t override_random_seed, ProxyDropMode new_mode) {
   this->drop_mode = new_mode;
-  if (this->drop_mode == DropMode::INTERCEPT) {
-    shared_ptr<const RareItemSet> rare_item_set;
-    shared_ptr<const CommonItemSet> common_item_set;
-    switch (version) {
-      case Version::PC_PATCH:
-      case Version::BB_PATCH:
-      case Version::GC_EP3_NTE:
-      case Version::GC_EP3:
-        throw runtime_error("cannot create item creator for this base version");
-      case Version::DC_NTE:
-      case Version::DC_11_2000:
-      case Version::DC_V1:
-        // TODO: We should probably have a v1 common item set at some point too
-        common_item_set = s->common_item_set_v2;
-        rare_item_set = s->rare_item_sets.at("rare-table-v1");
-        break;
-      case Version::DC_V2:
-      case Version::PC_NTE:
-      case Version::PC_V2:
-        common_item_set = s->common_item_set_v2;
-        rare_item_set = s->rare_item_sets.at("rare-table-v2");
-        break;
-      case Version::GC_NTE:
-      case Version::GC_V3:
-      case Version::XB_V3:
-        common_item_set = s->common_item_set_v3_v4;
-        rare_item_set = s->rare_item_sets.at("rare-table-v3");
-        break;
-      case Version::BB_V4:
-        common_item_set = s->common_item_set_v3_v4;
-        rare_item_set = s->rare_item_sets.at("rare-table-v4");
-        break;
-      default:
-        throw logic_error("invalid lobby base version");
-    }
+  if (this->drop_mode == ProxyDropMode::INTERCEPT) {
     auto rand_crypt = make_shared<MT19937Generator>((override_random_seed >= 0) ? override_random_seed : this->lobby_random_seed);
     this->item_creator = make_shared<ItemCreator>(
-        common_item_set,
-        rare_item_set,
+        s->common_item_set(version, nullptr),
+        s->rare_item_set(version, nullptr),
         s->armor_random_set,
         s->tool_random_set,
-        s->weapon_random_sets.at(this->lobby_difficulty),
+        s->weapon_random_set(this->lobby_difficulty),
         s->tekker_adjustment_set,
         s->item_parameter_table(version),
         s->item_stack_limits(version),

src/ProxySession.hh

@@ -22,36 +22,34 @@ struct ProxySession {
   uint32_t remote_ip_crc = 0;
   bool received_reconnect = false;
   bool enable_remote_ip_crc_patch = false;
+  bool bb_client_sent_E0 = false;
 
   struct LobbyPlayer {
     uint32_t guild_card_number = 0;
     uint64_t xb_user_id = 0;
     std::string name;
-    uint8_t language = 0;
+    Language language = Language::JAPANESE;
     uint8_t section_id = 0;
     uint8_t char_class = 0;
   };
   std::vector<LobbyPlayer> lobby_players;
+  bool is_in_lobby = false;
   bool is_in_game = false;
   bool is_in_quest = false;
   uint8_t leader_client_id = 0;
   uint8_t lobby_event = 0;
-  uint8_t lobby_difficulty = 0;
+  Difficulty lobby_difficulty = Difficulty::NORMAL;
   uint8_t lobby_section_id = 0;
   GameMode lobby_mode = GameMode::NORMAL;
   Episode lobby_episode = Episode::EP1;
   uint32_t lobby_random_seed = 0;
   uint64_t server_ping_start_time = 0;
+  bool suppress_next_ep3_media_update_confirmation = false;
 
   int64_t remote_guild_card_number = -1;
   parray<uint8_t, 0x28> remote_client_config_data;
 
-  enum class DropMode {
-    DISABLED = 0,
-    PASSTHROUGH,
-    INTERCEPT,
-  };
-  DropMode drop_mode = DropMode::PASSTHROUGH;
+  ProxyDropMode drop_mode = ProxyDropMode::PASSTHROUGH;
   std::shared_ptr<std::string> quest_dat_data;
   std::shared_ptr<ItemCreator> item_creator;
   std::shared_ptr<MapState> map_state;
@@ -79,7 +77,7 @@ struct ProxySession {
   };
   std::unordered_map<std::string, SavingFile> saving_files;
 
-  void set_drop_mode(std::shared_ptr<ServerState> s, Version version, int64_t override_random_seed, DropMode new_mode);
+  void set_drop_mode(std::shared_ptr<ServerState> s, Version version, int64_t override_random_seed, ProxyDropMode new_mode);
 
   void clear_lobby_players(size_t num_slots);
 };

src/Quest.cc

@@ -194,93 +194,119 @@ struct PSODownloadQuestHeader {
 } __packed_ws__(PSODownloadQuestHeader, 8);
 
 void VersionedQuest::assert_valid() const {
-  if (this->category_id == 0xFFFFFFFF) {
+  if (this->meta.category_id == 0xFFFFFFFF) {
     throw runtime_error("category ID is not set");
   }
-  if (this->quest_number == 0xFFFFFFFF) {
+  if (this->meta.quest_number == 0xFFFFFFFF) {
     throw runtime_error("quest number is not set");
   }
   if (this->version == Version::UNKNOWN) {
     throw runtime_error("version is not set");
   }
-  if (this->language == 0xFF) {
+  if (this->language == Language::UNKNOWN) {
     throw runtime_error("language is not set");
   }
-  if (this->episode == Episode::NONE) {
-    throw runtime_error("episode is not set");
+  switch (this->meta.episode) {
+    case Episode::EP1:
+      for (size_t floor = 0; floor < this->meta.area_for_floor.size(); floor++) {
+        uint8_t area = this->meta.area_for_floor[floor];
+        if (area >= 0x12) {
+          throw runtime_error("Episode 1 quest specifies invalid area");
+        }
+      }
+      break;
+    case Episode::EP2:
+      if (is_v1_or_v2(this->version)) {
+        throw runtime_error("v1 or v2 quest specifies Episode 2");
+      }
+      for (size_t floor = 0; floor < this->meta.area_for_floor.size(); floor++) {
+        uint8_t area = this->meta.area_for_floor[floor];
+        if ((area < 0x12) || (area >= 0x24)) {
+          throw runtime_error("Episode 2 quest specifies invalid area");
+        }
+      }
+      break;
+    case Episode::EP3:
+      if (!is_ep3(this->version)) {
+        throw runtime_error("non-Ep3 quest specifies Episode 3");
+      }
+      for (size_t floor = 0; floor < this->meta.area_for_floor.size(); floor++) {
+        if (this->meta.area_for_floor[floor] != 0xFF) {
+          throw runtime_error("Episode 3 quest specifies floor overrides");
+        }
+      }
+      break;
+    case Episode::EP4:
+      if (!is_v4(this->version)) {
+        throw runtime_error("non-v4 quest specifies Episode 4");
+      }
+      for (size_t floor = 0; floor < this->meta.area_for_floor.size(); floor++) {
+        uint8_t area = this->meta.area_for_floor[floor];
+        if (area != 0xFF && (area < 0x24 || area >= 0x2F)) {
+          throw runtime_error("Episode 4 quest specifies invalid floor");
+        }
+      }
+      break;
+    case Episode::NONE:
+      throw runtime_error("episode is not set");
+    default:
+      throw runtime_error("episode is not valid");
   }
-  if (this->max_players == 0) {
+  if (this->meta.max_players == 0) {
     throw runtime_error("max players is not set");
   }
   if (!this->bin_contents) {
     throw runtime_error("bin file is missing");
   }
-  if (!is_ep3(this->version) && !this->dat_contents) {
+  if (!this->dat_contents) {
     throw runtime_error("dat file is missing");
   }
-  if (!is_ep3(this->version) && !this->map_file) {
+  if (!this->map_file) {
     throw runtime_error("parsed map file is missing");
   }
+  if (this->meta.common_item_set_name.empty() != !this->meta.common_item_set) {
+    throw runtime_error("common item set name/pointer mismatch");
+  }
+  if (this->meta.rare_item_set_name.empty() != !this->meta.rare_item_set) {
+    throw runtime_error("rare item set name/pointer mismatch");
+  }
+  if (this->meta.allowed_drop_modes &&
+      !(this->meta.allowed_drop_modes & (1 << static_cast<size_t>(this->meta.default_drop_mode)))) {
+    throw runtime_error("default drop mode is not allowed");
+  }
 }
 
 string VersionedQuest::bin_filename() const {
-  if (this->episode == Episode::EP3) {
-    return std::format("m{:06}p_e.bin", this->quest_number);
-  } else {
-    return std::format("quest{}.bin", this->quest_number);
-  }
+  return std::format("quest{}.bin", this->meta.quest_number);
 }
 
 string VersionedQuest::dat_filename() const {
-  if (this->episode == Episode::EP3) {
-    throw logic_error("Episode 3 quests do not have .dat files");
-  } else {
-    return std::format("quest{}.dat", this->quest_number);
-  }
+  return std::format("quest{}.dat", this->meta.quest_number);
 }
 
 string VersionedQuest::pvr_filename() const {
-  if (this->episode == Episode::EP3) {
-    throw logic_error("Episode 3 quests do not have .pvr files");
-  } else {
-    return std::format("quest{}.pvr", this->quest_number);
-  }
+  return std::format("quest{}.pvr", this->meta.quest_number);
 }
 
 string VersionedQuest::xb_filename() const {
-  if (this->episode == Episode::EP3) {
-    throw logic_error("Episode 3 quests do not have Xbox filenames");
-  } else {
-    return std::format("quest{}_{}.dat", this->quest_number, static_cast<char>(tolower(char_for_language_code(this->language))));
-  }
+  return std::format("quest{}_{}.dat",
+      this->meta.quest_number, static_cast<char>(tolower(char_for_language(this->language))));
 }
 
 string VersionedQuest::encode_qst() const {
   unordered_map<string, shared_ptr<const string>> files;
-  files.emplace(std::format("quest{}.bin", this->quest_number), this->bin_contents);
-  files.emplace(std::format("quest{}.dat", this->quest_number), this->dat_contents);
+  files.emplace(std::format("quest{}.bin", this->meta.quest_number), this->bin_contents);
+  files.emplace(std::format("quest{}.dat", this->meta.quest_number), this->dat_contents);
   if (this->pvr_contents) {
-    files.emplace(std::format("quest{}.pvr", this->quest_number), this->pvr_contents);
+    files.emplace(std::format("quest{}.pvr", this->meta.quest_number), this->pvr_contents);
   }
-  string xb_filename = std::format("quest{}_{}.dat", quest_number, static_cast<char>(tolower(char_for_language_code(language))));
-  return encode_qst_file(files, this->name, this->quest_number, xb_filename, this->version, this->is_dlq_encoded);
+  string xb_filename = std::format("quest{}_{}.dat",
+      this->meta.quest_number, static_cast<char>(tolower(char_for_language(language))));
+  return encode_qst_file(files, this->meta.name, this->meta.quest_number, xb_filename, this->version, this->is_dlq_encoded);
 }
 
 Quest::Quest(shared_ptr<const VersionedQuest> initial_version)
-    : quest_number(initial_version->quest_number),
-      category_id(initial_version->category_id),
-      episode(initial_version->episode),
-      allow_start_from_chat_command(initial_version->allow_start_from_chat_command),
-      joinable(initial_version->joinable),
-      max_players(initial_version->max_players),
-      lock_status_register(initial_version->lock_status_register),
-      name(initial_version->name),
-      supermap(nullptr),
-      battle_rules(initial_version->battle_rules),
-      challenge_template_index(initial_version->challenge_template_index),
-      description_flag(initial_version->description_flag),
-      available_expression(initial_version->available_expression),
-      enabled_expression(initial_version->enabled_expression) {
+    : meta(initial_version->meta), supermap(nullptr) {
   this->add_version(initial_version);
 }
 
@@ -289,125 +315,55 @@ phosg::JSON Quest::json() const {
   for (const auto& [_, vq] : this->versions) {
     versions_json.emplace_back(phosg::JSON::dict({
         {"Version", phosg::name_for_enum(vq->version)},
-        {"Language", name_for_language_code(vq->language)},
-        {"ShortDescription", vq->short_description},
-        {"LongDescription", vq->long_description},
+        {"Language", ::name_for_language(vq->language)},
+        {"Name", vq->meta.name},
+        {"ShortDescription", vq->meta.short_description},
+        {"LongDescription", vq->meta.long_description},
         {"BINFileSize", vq->bin_contents ? vq->bin_contents->size() : phosg::JSON(nullptr)},
         {"DATFileSize", vq->dat_contents ? vq->dat_contents->size() : phosg::JSON(nullptr)},
         {"PVRFileSize", vq->pvr_contents ? vq->pvr_contents->size() : phosg::JSON(nullptr)},
     }));
   }
 
-  auto battle_rules_json = this->battle_rules ? this->battle_rules->json() : nullptr;
-  auto challenge_template_index_json = (this->challenge_template_index >= 0)
-      ? this->challenge_template_index
-      : phosg::JSON(nullptr);
   return phosg::JSON::dict({
-      {"Number", this->quest_number},
-      {"CategoryID", this->category_id},
-      {"Episode", name_for_episode(this->episode)},
-      {"AllowStartFromChatCommand", this->allow_start_from_chat_command},
-      {"Joinable", this->joinable},
-      {"MaxPlayers", this->max_players},
-      {"LockStatusRegister", (this->lock_status_register >= 0) ? this->lock_status_register : phosg::JSON(nullptr)},
-      {"Name", this->name},
-      {"BattleRules", std::move(battle_rules_json)},
-      {"ChallengeTemplateIndex", std::move(challenge_template_index_json)},
-      {"DescriptionFlag", this->description_flag},
-      {"AvailableExpression", this->available_expression ? this->available_expression->str() : phosg::JSON(nullptr)},
-      {"EnabledExpression", this->available_expression ? this->available_expression->str() : phosg::JSON(nullptr)},
+      {"Metadata", this->meta.json()},
       {"Versions", std::move(versions_json)},
   });
 }
 
-uint32_t Quest::versions_key(Version v, uint8_t language) {
-  return (static_cast<uint32_t>(v) << 8) | language;
+uint32_t Quest::versions_key(Version v, Language language) {
+  return (static_cast<uint32_t>(v) << 8) | static_cast<uint8_t>(language);
+}
+
+const string& Quest::name_for_language(Language language) const {
+  size_t lang_index = static_cast<size_t>(language);
+  if (!this->names_by_language.at(lang_index).empty()) {
+    return this->names_by_language[lang_index];
+  }
+  constexpr size_t english_lang_index = static_cast<size_t>(Language::ENGLISH);
+  if (!this->names_by_language[english_lang_index].empty()) {
+    return this->names_by_language[english_lang_index];
+  }
+  for (const string& name : this->names_by_language) {
+    if (!name.empty()) {
+      return name;
+    }
+  }
+  return this->meta.name;
 }
 
 void Quest::add_version(shared_ptr<const VersionedQuest> vq) {
-  if (this->quest_number != vq->quest_number) {
-    throw logic_error(std::format(
-        "incorrect versioned quest number (existing: {:08X}, new: {:08X})",
-        this->quest_number, vq->quest_number));
+  this->meta.assert_compatible(vq->meta);
+  if (this->meta.create_item_mask_entries.empty()) {
+    this->meta.create_item_mask_entries = vq->meta.create_item_mask_entries;
   }
-  if (this->category_id != vq->category_id) {
-    throw runtime_error(std::format(
-        "quest version is in a different category (existing: {:08X}, new: {:08X})",
-        this->category_id, vq->category_id));
-  }
-  if (this->episode != vq->episode) {
-    throw runtime_error(std::format(
-        "quest version is in a different episode (existing: {}, new: {})",
-        name_for_episode(this->episode), name_for_episode(vq->episode)));
-  }
-  if (this->allow_start_from_chat_command != vq->allow_start_from_chat_command) {
-    throw runtime_error(std::format(
-        "quest version has a different allow_start_from_chat_command state (existing: {}, new: {})",
-        this->allow_start_from_chat_command ? "true" : "false", vq->allow_start_from_chat_command ? "true" : "false"));
-  }
-  if (this->joinable != vq->joinable) {
-    throw runtime_error(std::format(
-        "quest version has a different joinability state (existing: {}, new: {})",
-        this->joinable ? "true" : "false", vq->joinable ? "true" : "false"));
-  }
-  if (this->max_players != vq->max_players) {
-    throw runtime_error(std::format(
-        "quest version has a different maximum player count (existing: {}, new: {})",
-        this->max_players, vq->max_players));
-  }
-  if (this->lock_status_register != vq->lock_status_register) {
-    throw runtime_error(std::format(
-        "quest version has a different lock status register (existing: {:04X}, new: {:04X})",
-        this->lock_status_register, vq->lock_status_register));
-  }
-  if (!this->battle_rules != !vq->battle_rules) {
-    throw runtime_error(std::format(
-        "quest version has a different battle rules presence state (existing: {}, new: {})",
-        this->battle_rules ? "present" : "absent", vq->battle_rules ? "present" : "absent"));
-  }
-  if (this->battle_rules && (*this->battle_rules != *vq->battle_rules)) {
-    string existing_str = this->battle_rules->json().serialize();
-    string new_str = vq->battle_rules->json().serialize();
-    throw runtime_error(std::format(
-        "quest version has different battle rules (existing: {}, new: {})",
-        existing_str, new_str));
-  }
-  if (this->challenge_template_index != vq->challenge_template_index) {
-    throw runtime_error(std::format(
-        "quest version has different challenge template index (existing: {}, new: {})",
-        this->challenge_template_index, vq->challenge_template_index));
-  }
-  if (this->description_flag != vq->description_flag) {
-    throw runtime_error(std::format(
-        "quest version has different description flag (existing: {:02X}, new: {:02X})",
-        this->description_flag, vq->description_flag));
-  }
-  if (!this->available_expression != !vq->available_expression) {
-    throw runtime_error(std::format(
-        "quest version has available expression but root quest does not, or vice versa (existing: {}, new: {})",
-        this->available_expression ? "present" : "absent", vq->available_expression ? "present" : "absent"));
-  }
-  if (this->available_expression && *this->available_expression != *vq->available_expression) {
-    string existing_str = this->available_expression->str();
-    string new_str = vq->available_expression->str();
-    throw runtime_error(std::format(
-        "quest version has a different available expression (existing: {}, new: {})",
-        existing_str, new_str));
-  }
-  if (!this->enabled_expression != !vq->enabled_expression) {
-    throw runtime_error(std::format(
-        "quest version has enabled expression but root quest does not, or vice versa (existing: {}, new: {})",
-        this->enabled_expression ? "present" : "absent", vq->enabled_expression ? "present" : "absent"));
-  }
-  if (this->enabled_expression && *this->enabled_expression != *vq->enabled_expression) {
-    string existing_str = this->enabled_expression->str();
-    string new_str = vq->enabled_expression->str();
-    throw runtime_error(std::format(
-        "quest version has a different enabled expression (existing: {}, new: {})",
-        existing_str, new_str));
-  }
-
   this->versions.emplace(this->versions_key(vq->version, vq->language), vq);
+
+  size_t lang_index = static_cast<size_t>(vq->language);
+  auto& name_by_language = this->names_by_language.at(lang_index);
+  if (name_by_language.empty()) {
+    name_by_language = vq->meta.name;
+  }
 }
 
 std::shared_ptr<const SuperMap> Quest::get_supermap(int64_t random_seed) const {
@@ -419,7 +375,7 @@ std::shared_ptr<const SuperMap> Quest::get_supermap(int64_t random_seed) const {
   bool any_map_file_present = false;
   array<shared_ptr<const MapFile>, NUM_VERSIONS> map_files;
   for (Version v : ALL_NON_PATCH_VERSIONS) {
-    auto vq = this->version(v, 1);
+    auto vq = this->version(v, Language::ENGLISH);
     if (vq && vq->map_file) {
       auto map_file = vq->map_file;
       if (map_file->has_random_sections()) {
@@ -438,27 +394,27 @@ std::shared_ptr<const SuperMap> Quest::get_supermap(int64_t random_seed) const {
     return nullptr;
   }
 
-  auto supermap = make_shared<SuperMap>(this->episode, map_files);
+  auto supermap = make_shared<SuperMap>(this->meta.episode, map_files);
   if (save_to_cache) {
     this->supermap = supermap;
   }
   static_game_data_log.info_f("Constructed {} supermap for quest {} ({})",
-      save_to_cache ? "cacheable" : "temporary", this->quest_number, this->name);
+      save_to_cache ? "cacheable" : "temporary", this->meta.quest_number, this->meta.name);
 
   return supermap;
 }
 
-bool Quest::has_version(Version v, uint8_t language) const {
+bool Quest::has_version(Version v, Language language) const {
   return this->versions.count(this->versions_key(v, language));
 }
 
 bool Quest::has_version_any_language(Version v) const {
-  uint32_t k = this->versions_key(v, 0);
+  uint32_t k = this->versions_key(v, Language::JAPANESE);
   auto it = this->versions.lower_bound(k);
   return ((it != this->versions.end()) && ((it->first & 0xFF00) == k));
 }
 
-shared_ptr<const VersionedQuest> Quest::version(Version v, uint8_t language) const {
+shared_ptr<const VersionedQuest> Quest::version(Version v, Language language) const {
   // Return the requested version, if it exists
   try {
     return this->versions.at(this->versions_key(v, language));
@@ -467,13 +423,13 @@ shared_ptr<const VersionedQuest> Quest::version(Version v, uint8_t language) con
 
   // Return the English version, if it exists
   try {
-    return this->versions.at(this->versions_key(v, 1));
+    return this->versions.at(this->versions_key(v, Language::ENGLISH));
   } catch (const out_of_range&) {
   }
 
   // Return the first language, if it exists
-  auto it = this->versions.lower_bound(this->versions_key(v, 0));
-  if ((it == this->versions.end()) || ((it->first & 0xFF00) != this->versions_key(v, 0))) {
+  auto it = this->versions.lower_bound(this->versions_key(v, Language::JAPANESE));
+  if ((it == this->versions.end()) || ((it->first & 0xFF00) != this->versions_key(v, Language::JAPANESE))) {
     return nullptr;
   }
   return it->second;
@@ -482,7 +438,9 @@ shared_ptr<const VersionedQuest> Quest::version(Version v, uint8_t language) con
 QuestIndex::QuestIndex(
     const string& directory,
     shared_ptr<const QuestCategoryIndex> category_index,
-    bool is_ep3)
+    const unordered_map<string, shared_ptr<const CommonItemSet>>& common_item_sets,
+    const unordered_map<string, shared_ptr<const RareItemSet>>& rare_item_sets,
+    bool raise_on_any_failure)
     : directory(directory),
       category_index(category_index) {
 
@@ -492,7 +450,7 @@ QuestIndex::QuestIndex(
   };
   struct BINFileData {
     string filename;
-    unique_ptr<QuestMetadata> metadata;
+    shared_ptr<const AssembledQuestScript> assembled;
     shared_ptr<const string> data;
   };
   struct DATFileData {
@@ -506,12 +464,6 @@ QuestIndex::QuestIndex(
   map<string, FileData> json_files;
   map<string, uint32_t> categories;
   for (const auto& cat : this->category_index->categories) {
-    // Don't index Ep3 download categories for non-Ep3 quest indexing, and vice
-    // versa
-    if (is_ep3 != cat->check_flag(QuestMenuType::EP3_DOWNLOAD)) {
-      continue;
-    }
-
     auto add_file = [&](map<string, FileData>& files, const string& basename, const string& filename, string&& value, bool check_chunk_size) {
       if (categories.emplace(basename, cat->category_id).first->second != cat->category_id) {
         throw runtime_error("file " + basename + " exists in multiple categories");
@@ -528,7 +480,7 @@ QuestIndex::QuestIndex(
       }
     };
 
-    auto add_bin_file = [&](const string& basename, const string& filename, string&& data, const QuestMetadata* metadata) {
+    auto add_bin_file = [&](const string& basename, const string& filename, string&& data, shared_ptr<AssembledQuestScript> assembled) {
       if (categories.emplace(basename, cat->category_id).first->second != cat->category_id) {
         throw runtime_error("bin file " + basename + " exists in multiple categories");
       }
@@ -540,9 +492,7 @@ QuestIndex::QuestIndex(
       auto& entry = emplace_ret.first->second;
       entry.filename = filename;
       entry.data = data_ptr;
-      if (metadata) {
-        entry.metadata = make_unique<QuestMetadata>(*metadata);
-      }
+      entry.assembled = assembled;
       if (!(data_ptr->size() & 0x3FF)) {
         data_ptr->push_back(0x00);
       }
@@ -573,7 +523,7 @@ QuestIndex::QuestIndex(
       }
 
       string file_path = cat_path + "/" + filename;
-      unique_ptr<AssembledQuestScript> assembled;
+      shared_ptr<AssembledQuestScript> assembled;
       try {
         string orig_filename = filename;
         string file_data;
@@ -588,7 +538,7 @@ QuestIndex::QuestIndex(
           filename.resize(filename.size() - 4);
         } else if (filename.ends_with(".bin.txt")) {
           string include_dir = phosg::dirname(file_path);
-          assembled = make_unique<AssembledQuestScript>(assemble_quest_script(
+          assembled = make_shared<AssembledQuestScript>(assemble_quest_script(
               phosg::load_file(file_path),
               {include_dir, "system/quests/includes"},
               {include_dir, "system/quests/includes", "system/client-functions/System"}));
@@ -614,9 +564,9 @@ QuestIndex::QuestIndex(
         if (extension == "json") {
           add_file(json_files, file_basename, orig_filename, std::move(file_data), false);
         } else if (extension == "bin" || extension == "mnm") {
-          add_bin_file(file_basename, orig_filename, std::move(file_data), assembled ? &assembled->metadata : nullptr);
+          add_bin_file(file_basename, orig_filename, std::move(file_data), assembled);
         } else if (extension == "bind" || extension == "mnmd") {
-          add_bin_file(file_basename, orig_filename, prs_compress_optimal(file_data), assembled ? &assembled->metadata : nullptr);
+          add_bin_file(file_basename, orig_filename, prs_compress_optimal(file_data), assembled);
         } else if (extension == "dat") {
           add_dat_file(file_basename, orig_filename, std::move(file_data));
         } else if (extension == "datd") {
@@ -639,6 +589,9 @@ QuestIndex::QuestIndex(
         }
 
       } catch (const exception& e) {
+        if (raise_on_any_failure) {
+          throw runtime_error(format("({}) {}", filename, e.what()));
+        }
         static_game_data_log.warning_f("({}) Failed to load quest file: ({})", filename, e.what());
       }
     }
@@ -669,28 +622,18 @@ QuestIndex::QuestIndex(
         version_token = std::move(filename_tokens[1]);
         language_token = std::move(filename_tokens[2]);
       }
-      vq->category_id = categories.at(basename);
-
-      // Find the quest's metadata. If the quest was assembled (that is, if it
-      // came from a .bin.txt file), use the metadata from the source file;
-      // otherwise, figure it out from the already-assembled code
-      if (entry.metadata) {
-        vq->quest_number = entry.metadata->quest_number;
-        vq->version = ::is_ep3(entry.metadata->version) ? Version::GC_V3 : entry.metadata->version;
-        vq->language = entry.metadata->language;
-        vq->episode = entry.metadata->episode;
-        vq->joinable = entry.metadata->joinable;
-        vq->max_players = entry.metadata->max_players;
-        vq->name = entry.metadata->name;
-        vq->short_description = entry.metadata->short_description;
-        vq->long_description = entry.metadata->long_description;
+      vq->meta.category_id = categories.at(basename);
 
+      if (entry.assembled) {
+        vq->meta.quest_number = entry.assembled->quest_number;
+        vq->version = entry.assembled->version;
+        vq->language = entry.assembled->language;
       } else {
         // Get the number from the first token
         if (quest_number_token.empty()) {
           throw runtime_error("quest number token is missing");
         }
-        vq->quest_number = strtoull(quest_number_token.c_str() + 1, nullptr, 10);
+        vq->meta.quest_number = strtoull(quest_number_token.c_str() + 1, nullptr, 10);
 
         // Get the version from the second token
         static const unordered_map<string, Version> name_to_version({
@@ -713,148 +656,46 @@ QuestIndex::QuestIndex(
         if (language_token.size() != 1) {
           throw runtime_error("language token is not a single character");
         }
-        vq->language = language_code_for_char(language_token[0]);
-
-        auto bin_decompressed = prs_decompress(*entry.data);
-        switch (vq->version) {
-          case Version::DC_NTE: {
-            if (bin_decompressed.size() < sizeof(PSOQuestHeaderDCNTE)) {
-              throw invalid_argument("file is too small for header");
-            }
-            auto* header = reinterpret_cast<const PSOQuestHeaderDCNTE*>(bin_decompressed.data());
-            vq->episode = Episode::EP1;
-            vq->max_players = 4;
-            vq->name = header->name.decode(vq->language);
-            if (vq->quest_number == 0xFFFFFFFF) {
-              vq->quest_number = phosg::fnv1a64(vq->name);
-            }
-            break;
-          }
-
-          case Version::DC_11_2000:
-          case Version::DC_V1:
-          case Version::DC_V2: {
-            if (bin_decompressed.size() < sizeof(PSOQuestHeaderDC)) {
-              throw invalid_argument("file is too small for header");
-            }
-            auto* header = reinterpret_cast<const PSOQuestHeaderDC*>(bin_decompressed.data());
-            vq->episode = Episode::EP1;
-            vq->max_players = 4;
-            if (vq->quest_number == 0xFFFFFFFF) {
-              vq->quest_number = header->quest_number;
-            }
-            vq->name = header->name.decode(vq->language);
-            vq->short_description = header->short_description.decode(vq->language);
-            vq->long_description = header->long_description.decode(vq->language);
-            break;
-          }
-
-          case Version::PC_NTE:
-          case Version::PC_V2: {
-            if (bin_decompressed.size() < sizeof(PSOQuestHeaderPC)) {
-              throw invalid_argument("file is too small for header");
-            }
-            auto* header = reinterpret_cast<const PSOQuestHeaderPC*>(bin_decompressed.data());
-            vq->episode = Episode::EP1;
-            vq->max_players = 4;
-            if (vq->quest_number == 0xFFFFFFFF) {
-              vq->quest_number = header->quest_number;
-            }
-            vq->name = header->name.decode(vq->language);
-            vq->short_description = header->short_description.decode(vq->language);
-            vq->long_description = header->long_description.decode(vq->language);
-            break;
-          }
-
-          case Version::GC_EP3_NTE:
-          case Version::GC_EP3: {
-            // Note: This codepath handles Episode 3 download quests, which are not
-            // the same as Episode 3 quest scripts. The latter are only used offline
-            // in story mode, but can be disassembled with disassemble_quest_script.
-            // It's unfortunate that Version::GC_EP3 is used here for Episode 3
-            // download quests (maps) and there for offline story mode scripts, but
-            // it's probably not worth refactoring this logic, at least right now.
-            if (bin_decompressed.size() != sizeof(Episode3::MapDefinition)) {
-              throw invalid_argument("file is incorrect size");
-            }
-            auto* map = reinterpret_cast<const Episode3::MapDefinition*>(bin_decompressed.data());
-            vq->episode = Episode::EP3;
-            vq->max_players = 4;
-            if (vq->quest_number == 0xFFFFFFFF) {
-              vq->quest_number = map->map_number;
-            }
-            vq->name = map->name.decode(vq->language);
-            vq->short_description = map->quest_name.decode(vq->language);
-            vq->long_description = map->description.decode(vq->language);
-            break;
-          }
-
-          case Version::XB_V3:
-          case Version::GC_NTE:
-          case Version::GC_V3: {
-            if (bin_decompressed.size() < sizeof(PSOQuestHeaderGC)) {
-              throw invalid_argument("file is too small for header");
-            }
-            auto* header = reinterpret_cast<const PSOQuestHeaderGC*>(bin_decompressed.data());
-            vq->episode = find_quest_episode_from_script(
-                bin_decompressed.data(), bin_decompressed.size(), vq->version);
-            vq->max_players = 4;
-            if (vq->quest_number == 0xFFFFFFFF) {
-              vq->quest_number = header->quest_number;
-            }
-            vq->name = header->name.decode(vq->language);
-            vq->short_description = header->short_description.decode(vq->language);
-            vq->long_description = header->long_description.decode(vq->language);
-            break;
-          }
-
-          case Version::BB_V4: {
-            if (bin_decompressed.size() < sizeof(PSOQuestHeaderBB)) {
-              throw invalid_argument("file is too small for header");
-            }
-            auto* header = reinterpret_cast<const PSOQuestHeaderBB*>(bin_decompressed.data());
-            vq->episode = find_quest_episode_from_script(
-                bin_decompressed.data(), bin_decompressed.size(), vq->version);
-            vq->joinable |= header->joinable;
-            vq->max_players = 4;
-            if (vq->quest_number == 0xFFFFFFFF) {
-              vq->quest_number = header->quest_number;
-            }
-            vq->name = header->name.decode(vq->language);
-            vq->short_description = header->short_description.decode(vq->language);
-            vq->long_description = header->long_description.decode(vq->language);
-            break;
-          }
-
-          default:
-            throw logic_error("invalid quest game version");
-        }
+        vq->language = language_for_char(language_token[0]);
       }
 
-      // Find the corresponding dat and pvr files
+      auto bin_decompressed = prs_decompress(*entry.data);
+      populate_quest_metadata_from_script(vq->meta, bin_decompressed.data(), bin_decompressed.size(), vq->version, vq->language);
+
+      // If the quest was assembled (that is, if it came from a .bin.txt file),
+      // the metadata from the source file overrides any automatically-detected
+      // values from above
+      if (entry.assembled) {
+        vq->meta.quest_number = entry.assembled->quest_number;
+        vq->meta.episode = entry.assembled->episode;
+        vq->meta.joinable = entry.assembled->joinable;
+        vq->meta.max_players = entry.assembled->max_players;
+        vq->meta.name = entry.assembled->name;
+        vq->meta.short_description = entry.assembled->short_description;
+        vq->meta.long_description = entry.assembled->long_description;
+      }
+
+      // Find the corresponding dat and pvr files with the same basename as the
+      // bin file; if not found, look for them without the language suffix
       const DATFileData* dat_filedata = nullptr;
       const FileData* pvr_filedata = nullptr;
-      if (!::is_ep3(vq->version)) {
-        // Look for dat and pvr files with the same basename as the bin file; if
-        // not found, look for them without the language suffix
+      try {
+        dat_filedata = &dat_files.at(basename);
+      } catch (const out_of_range&) {
         try {
-          dat_filedata = &dat_files.at(basename);
+          dat_filedata = &dat_files.at(quest_number_token + "-" + version_token);
         } catch (const out_of_range&) {
-          try {
-            dat_filedata = &dat_files.at(quest_number_token + "-" + version_token);
-          } catch (const out_of_range&) {
-            throw runtime_error("no dat file found for bin file " + basename);
-          }
+          throw runtime_error("no dat file found for bin file " + basename);
         }
+      }
+      try {
+        pvr_filedata = &pvr_files.at(basename);
+      } catch (const out_of_range&) {
         try {
-          pvr_filedata = &pvr_files.at(basename);
+          pvr_filedata = &pvr_files.at(quest_number_token + "-" + version_token);
         } catch (const out_of_range&) {
-          try {
-            pvr_filedata = &pvr_files.at(quest_number_token + "-" + version_token);
-          } catch (const out_of_range&) {
-            // pvr files aren't required (and most quests do not have them), so
-            // don't fail if it's missing
-          }
+          // pvr files aren't required (and most quests do not have them), so
+          // don't fail if it's missing
         }
       }
       vq->bin_contents = entry.data;
@@ -883,42 +724,60 @@ QuestIndex::QuestIndex(
       if (json_filedata) {
         auto metadata_json = phosg::JSON::parse(*json_filedata->data);
         try {
-          vq->battle_rules = make_shared<BattleRules>(metadata_json.at("BattleRules"));
+          vq->meta.description_flag = metadata_json.at("DescriptionFlag").as_int();
         } catch (const out_of_range&) {
         }
         try {
-          vq->challenge_template_index = metadata_json.at("ChallengeTemplateIndex").as_int();
+          vq->meta.available_expression = make_shared<IntegralExpression>(metadata_json.get_string("AvailableIf"));
         } catch (const out_of_range&) {
         }
         try {
-          vq->description_flag = metadata_json.at("DescriptionFlag").as_int();
+          vq->meta.enabled_expression = make_shared<IntegralExpression>(metadata_json.get_string("EnabledIf"));
         } catch (const out_of_range&) {
         }
         try {
-          vq->available_expression = make_shared<IntegralExpression>(metadata_json.get_string("AvailableIf"));
+          vq->meta.allow_start_from_chat_command = metadata_json.get_bool("AllowStartFromChatCommand");
         } catch (const out_of_range&) {
         }
         try {
-          vq->enabled_expression = make_shared<IntegralExpression>(metadata_json.get_string("EnabledIf"));
+          vq->meta.joinable = metadata_json.get_bool("Joinable");
         } catch (const out_of_range&) {
         }
         try {
-          vq->allow_start_from_chat_command = metadata_json.get_bool("AllowStartFromChatCommand");
+          vq->meta.lock_status_register = metadata_json.get_int("LockStatusRegister");
         } catch (const out_of_range&) {
         }
         try {
-          vq->joinable = metadata_json.get_bool("Joinable");
+          vq->meta.enemy_exp_overrides = QuestMetadata::parse_enemy_exp_overrides(metadata_json.at("EnemyEXPOverrides"));
         } catch (const out_of_range&) {
         }
         try {
-          vq->lock_status_register = metadata_json.get_int("LockStatusRegister");
+          vq->meta.common_item_set_name = metadata_json.at("CommonItemSetName").as_string();
+        } catch (const out_of_range&) {
+        }
+        if (!vq->meta.common_item_set_name.empty()) {
+          vq->meta.common_item_set = common_item_sets.at(vq->meta.common_item_set_name);
+        }
+        try {
+          vq->meta.rare_item_set_name = metadata_json.at("RareItemSetName").as_string();
+        } catch (const out_of_range&) {
+        }
+        if (!vq->meta.rare_item_set_name.empty()) {
+          vq->meta.rare_item_set = rare_item_sets.at(vq->meta.rare_item_set_name);
+        }
+        try {
+          vq->meta.allowed_drop_modes = metadata_json.at("AllowedDropModes").as_int();
+        } catch (const out_of_range&) {
+        }
+        try {
+          vq->meta.default_drop_mode = phosg::enum_for_name<ServerDropMode>(metadata_json.at("DefaultDropMode").as_string());
         } catch (const out_of_range&) {
         }
       }
 
       vq->assert_valid();
 
-      auto category_name = this->category_index->at(vq->category_id)->name;
+      auto category_name = this->category_index->at(vq->meta.category_id)->name;
       string filenames_str = entry.filename;
       if (dat_filedata) {
         filenames_str += std::format("/{}", dat_filedata->filename);
@@ -929,32 +788,35 @@ QuestIndex::QuestIndex(
       if (json_filedata) {
         filenames_str += std::format("/{}", json_filedata->filename);
       }
-      auto q_it = this->quests_by_number.find(vq->quest_number);
+      auto q_it = this->quests_by_number.find(vq->meta.quest_number);
       if (q_it != this->quests_by_number.end()) {
         q_it->second->add_version(vq);
         static_game_data_log.debug_f("({}) Added {} {} version of quest {} ({})",
             filenames_str,
             phosg::name_for_enum(vq->version),
-            char_for_language_code(vq->language),
-            vq->quest_number,
-            vq->name);
+            char_for_language(vq->language),
+            vq->meta.quest_number,
+            vq->meta.name);
       } else {
         auto q = make_shared<Quest>(vq);
-        this->quests_by_number.emplace(vq->quest_number, q);
-        this->quests_by_name.emplace(vq->name, q);
-        this->quests_by_category_id_and_number[q->category_id].emplace(vq->quest_number, q);
+        this->quests_by_number.emplace(vq->meta.quest_number, q);
+        this->quests_by_name.emplace(vq->meta.name, q);
+        this->quests_by_category_id_and_number[q->meta.category_id].emplace(vq->meta.quest_number, q);
         static_game_data_log.debug_f("({}) Created {} {} quest {} ({}) ({}, {} ({}), {})",
             filenames_str,
             phosg::name_for_enum(vq->version),
-            char_for_language_code(vq->language),
-            vq->quest_number,
-            vq->name,
-            name_for_episode(vq->episode),
+            char_for_language(vq->language),
+            vq->meta.quest_number,
+            vq->meta.name,
+            name_for_episode(vq->meta.episode),
             category_name,
-            vq->category_id,
-            vq->joinable ? "joinable" : "not joinable");
+            vq->meta.category_id,
+            vq->meta.joinable ? "joinable" : "not joinable");
       }
     } catch (const exception& e) {
+      if (raise_on_any_failure) {
+        throw runtime_error(format("({}) {}", basename, e.what()));
+      }
       static_game_data_log.warning_f("({}) Failed to index quest file: {}", basename, e.what());
     }
   }
@@ -983,9 +845,6 @@ phosg::JSON QuestIndex::json() const {
       {"Categories", std::move(categories_json)},
       {"Quests", std::move(quests_json)},
   });
-  // std::map<uint32_t, std::shared_ptr<Quest>> quests_by_number;
-  // std::map<std::string, std::shared_ptr<Quest>> quests_by_name;
-  // std::map<uint32_t, std::map<uint32_t, std::shared_ptr<Quest>>> quests_by_category_id_and_number;
 }
 
 shared_ptr<const Quest> QuestIndex::get(uint32_t quest_number) const {
@@ -1033,7 +892,7 @@ vector<pair<QuestIndex::IncludeState, shared_ptr<const Quest>>> QuestIndex::filt
     return ret;
   }
   for (auto it : category_it->second) {
-    if ((effective_episode != Episode::NONE) && (it.second->episode != effective_episode)) {
+    if ((effective_episode != Episode::NONE) && (it.second->meta.episode != effective_episode)) {
       continue;
     }
     bool all_required_versions_present = true;
@@ -1082,25 +941,18 @@ string encode_download_quest_data(const string& compressed_data, size_t decompre
   data.resize((data.size() + 3) & (~3));
 
   PSOV2Encryption encr(encryption_seed);
-  encr.encrypt(data.data() + sizeof(PSODownloadQuestHeader),
-      data.size() - sizeof(PSODownloadQuestHeader));
+  encr.encrypt(data.data() + sizeof(PSODownloadQuestHeader), data.size() - sizeof(PSODownloadQuestHeader));
   data.resize(original_size);
 
   return data;
 }
 
-shared_ptr<VersionedQuest> VersionedQuest::create_download_quest(uint8_t override_language) const {
+shared_ptr<VersionedQuest> VersionedQuest::create_download_quest(Language override_language) const {
   // The download flag needs to be set in the bin header, or else the client
   // will ignore it when scanning for download quests in an offline game. To set
   // this flag, we need to decompress the quest's .bin file, set the flag, then
   // recompress it again.
 
-  // This function should not be used for Episode 3 quests (they should be sent
-  // to the client as-is, without any encryption or other preprocessing)
-  if (this->episode == Episode::EP3 || is_ep3(this->version)) {
-    throw logic_error("Episode 3 quests cannot be converted to download quests");
-  }
-
   string decompressed_bin = prs_decompress(*this->bin_contents);
 
   void* data_ptr = decompressed_bin.data();
@@ -1118,7 +970,7 @@ shared_ptr<VersionedQuest> VersionedQuest::create_download_quest(uint8_t overrid
       if (decompressed_bin.size() < sizeof(PSOQuestHeaderDC)) {
         throw runtime_error("bin file is too small for header");
       }
-      if (override_language != 0xFF) {
+      if (override_language != Language::UNKNOWN) {
         reinterpret_cast<PSOQuestHeaderDC*>(data_ptr)->language = override_language;
       }
       break;
@@ -1127,7 +979,7 @@ shared_ptr<VersionedQuest> VersionedQuest::create_download_quest(uint8_t overrid
       if (decompressed_bin.size() < sizeof(PSOQuestHeaderPC)) {
         throw runtime_error("bin file is too small for header");
       }
-      if (override_language != 0xFF) {
+      if (override_language != Language::UNKNOWN) {
         reinterpret_cast<PSOQuestHeaderPC*>(data_ptr)->language = override_language;
       }
       break;
@@ -1137,7 +989,7 @@ shared_ptr<VersionedQuest> VersionedQuest::create_download_quest(uint8_t overrid
       if (decompressed_bin.size() < sizeof(PSOQuestHeaderGC)) {
         throw runtime_error("bin file is too small for header");
       }
-      if (override_language != 0xFF) {
+      if (override_language != Language::UNKNOWN) {
         reinterpret_cast<PSOQuestHeaderGC*>(data_ptr)->language = override_language;
       }
       break;

src/Quest.hh

@@ -8,10 +8,14 @@
 #include <unordered_map>
 #include <vector>
 
+#include "CommonItemSet.hh"
 #include "IntegralExpression.hh"
+#include "ItemParameterTable.hh"
 #include "Map.hh"
 #include "PlayerSubordinates.hh"
+#include "QuestMetadata.hh"
 #include "QuestScript.hh"
+#include "RareItemSet.hh"
 #include "StaticGameData.hh"
 #include "TeamIndex.hh"
 
@@ -31,7 +35,6 @@ enum class QuestMenuType {
   SOLO = 3,
   GOVERNMENT = 4,
   DOWNLOAD = 5,
-  EP3_DOWNLOAD = 6,
   // 7 can't be used as a menu type (it enables the per-episode filter)
 };
 
@@ -64,29 +67,16 @@ struct QuestCategoryIndex {
 };
 
 struct VersionedQuest {
+  QuestMetadata meta;
+
   // Most of these default values are intentionally invalid; we use these
   // values to check if each field was parsed during quest indexing.
-  uint32_t category_id = 0xFFFFFFFF;
-  uint32_t quest_number = 0xFFFFFFFF;
   Version version = Version::UNKNOWN;
-  uint8_t language = 0xFF;
-  Episode episode = Episode::NONE;
-  bool joinable = false;
-  uint8_t max_players = 0x00;
-  std::string name;
-  std::string short_description;
-  std::string long_description;
+  Language language = Language::UNKNOWN;
   std::shared_ptr<const std::string> bin_contents;
   std::shared_ptr<const std::string> dat_contents;
   std::shared_ptr<const MapFile> map_file;
   std::shared_ptr<const std::string> pvr_contents;
-  std::shared_ptr<const BattleRules> battle_rules;
-  ssize_t challenge_template_index = -1;
-  uint8_t description_flag = 0x00;
-  std::shared_ptr<const IntegralExpression> available_expression;
-  std::shared_ptr<const IntegralExpression> enabled_expression;
-  bool allow_start_from_chat_command = false;
-  int16_t lock_status_register = -1;
   bool is_dlq_encoded = false;
 
   void assert_valid() const;
@@ -96,26 +86,15 @@ struct VersionedQuest {
   std::string pvr_filename() const;
   std::string xb_filename() const;
 
-  std::shared_ptr<VersionedQuest> create_download_quest(uint8_t override_language = 0xFF) const;
+  std::shared_ptr<VersionedQuest> create_download_quest(Language override_language = Language::UNKNOWN) const;
   std::string encode_qst() const;
 };
 
 struct Quest {
-  uint32_t quest_number;
-  uint32_t category_id;
-  Episode episode;
-  bool allow_start_from_chat_command;
-  bool joinable;
-  uint8_t max_players;
-  int16_t lock_status_register;
-  std::string name;
+  QuestMetadata meta;
   mutable std::shared_ptr<const SuperMap> supermap;
-  std::shared_ptr<const BattleRules> battle_rules;
-  ssize_t challenge_template_index;
-  uint8_t description_flag;
-  std::shared_ptr<const IntegralExpression> available_expression;
-  std::shared_ptr<const IntegralExpression> enabled_expression;
   std::map<uint32_t, std::shared_ptr<const VersionedQuest>> versions;
+  std::array<std::string, 8> names_by_language;
 
   Quest() = delete;
   explicit Quest(std::shared_ptr<const VersionedQuest> initial_version);
@@ -128,12 +107,14 @@ struct Quest {
 
   std::shared_ptr<const SuperMap> get_supermap(int64_t random_seed) const;
 
-  void add_version(std::shared_ptr<const VersionedQuest> vq);
-  bool has_version(Version v, uint8_t language) const;
-  bool has_version_any_language(Version v) const;
-  std::shared_ptr<const VersionedQuest> version(Version v, uint8_t language) const;
+  const std::string& name_for_language(Language language) const;
 
-  static uint32_t versions_key(Version v, uint8_t language);
+  void add_version(std::shared_ptr<const VersionedQuest> vq);
+  bool has_version(Version v, Language language) const;
+  bool has_version_any_language(Version v) const;
+  std::shared_ptr<const VersionedQuest> version(Version v, Language language) const;
+
+  static uint32_t versions_key(Version v, Language language);
 };
 
 struct QuestIndex {
@@ -151,7 +132,12 @@ struct QuestIndex {
   std::map<std::string, std::shared_ptr<Quest>> quests_by_name;
   std::map<uint32_t, std::map<uint32_t, std::shared_ptr<Quest>>> quests_by_category_id_and_number;
 
-  QuestIndex(const std::string& directory, std::shared_ptr<const QuestCategoryIndex> category_index, bool is_ep3);
+  QuestIndex(
+      const std::string& directory,
+      std::shared_ptr<const QuestCategoryIndex> category_index,
+      const std::unordered_map<std::string, std::shared_ptr<const CommonItemSet>>& common_item_sets,
+      const std::unordered_map<std::string, std::shared_ptr<const RareItemSet>>& rare_item_sets,
+      bool raise_on_any_failure);
   phosg::JSON json() const;
 
   std::shared_ptr<const Quest> get(uint32_t quest_number) const;

src/QuestMetadata.cc

@@ -0,0 +1,298 @@
+#include "QuestMetadata.hh"
+
+using namespace std;
+
+void QuestMetadata::assign_default_areas(Version version, Episode episode) {
+  for (size_t z = 0; z < 0x12; z++) {
+    this->area_for_floor[z] = SetDataTableBase::default_area_for_floor(version, episode, z);
+  }
+}
+
+void QuestMetadata::assert_compatible(const QuestMetadata& other) const {
+  if (this->quest_number != other.quest_number) {
+    throw logic_error(std::format(
+        "incorrect versioned quest number (existing: {:08X}, new: {:08X})",
+        this->quest_number, other.quest_number));
+  }
+  if (this->category_id != other.category_id) {
+    throw runtime_error(std::format(
+        "quest version is in a different category (existing: {:08X}, new: {:08X})",
+        this->category_id, other.category_id));
+  }
+  if (this->episode != other.episode) {
+    throw runtime_error(std::format(
+        "quest version is in a different episode (existing: {}, new: {})",
+        name_for_episode(this->episode), name_for_episode(other.episode)));
+  }
+  if (this->allow_start_from_chat_command != other.allow_start_from_chat_command) {
+    throw runtime_error(std::format(
+        "quest version has a different allow_start_from_chat_command state (existing: {}, new: {})",
+        this->allow_start_from_chat_command ? "true" : "false", other.allow_start_from_chat_command ? "true" : "false"));
+  }
+  if (this->joinable != other.joinable) {
+    throw runtime_error(std::format(
+        "quest version has a different joinability state (existing: {}, new: {})",
+        this->joinable ? "true" : "false", other.joinable ? "true" : "false"));
+  }
+  if (this->max_players != other.max_players) {
+    throw runtime_error(std::format(
+        "quest version has a different maximum player count (existing: {}, new: {})",
+        this->max_players, other.max_players));
+  }
+  if (this->lock_status_register != other.lock_status_register) {
+    throw runtime_error(std::format(
+        "quest version has a different lock status register (existing: {:04X}, new: {:04X})",
+        this->lock_status_register, other.lock_status_register));
+  }
+  if (this->enemy_exp_overrides != other.enemy_exp_overrides) {
+    throw runtime_error("quest version has different enemy EXP overrides");
+  }
+  if (!this->create_item_mask_entries.empty() &&
+      !other.create_item_mask_entries.empty() &&
+      this->create_item_mask_entries != other.create_item_mask_entries) {
+    string this_str, other_str;
+    for (const auto& item : this->create_item_mask_entries) {
+      if (!this_str.empty()) {
+        this_str += ", ";
+      }
+      this_str += item.str();
+    }
+    for (const auto& item : other.create_item_mask_entries) {
+      if (!other_str.empty()) {
+        other_str += ", ";
+      }
+      other_str += item.str();
+    }
+    throw runtime_error(std::format(
+        "quest version has a different set of create item masks (existing: {}, new: {})", this_str, other_str));
+  }
+  if (!this->battle_rules != !other.battle_rules) {
+    throw runtime_error(std::format(
+        "quest version has a different battle rules presence state (existing: {}, new: {})",
+        this->battle_rules ? "present" : "absent", other.battle_rules ? "present" : "absent"));
+  }
+  if (this->battle_rules && (*this->battle_rules != *other.battle_rules)) {
+    string existing_str = this->battle_rules->json().serialize();
+    string new_str = other.battle_rules->json().serialize();
+    throw runtime_error(std::format(
+        "quest version has different battle rules (existing: {}, new: {})",
+        existing_str, new_str));
+  }
+  if (this->challenge_template_index != other.challenge_template_index) {
+    throw runtime_error(std::format(
+        "quest version has different challenge template index (existing: {}, new: {})",
+        this->challenge_template_index, other.challenge_template_index));
+  }
+  if (this->challenge_exp_multiplier != other.challenge_exp_multiplier) {
+    throw runtime_error(std::format(
+        "quest version has different challenge EXP multiplier (existing: {}, new: {})",
+        this->challenge_exp_multiplier, other.challenge_exp_multiplier));
+  }
+  if (this->challenge_difficulty != other.challenge_difficulty) {
+    throw runtime_error(std::format(
+        "quest version has different challenge difficulty (existing: {}, new: {})",
+        name_for_difficulty(this->challenge_difficulty), name_for_difficulty(other.challenge_difficulty)));
+  }
+  for (size_t z = 0; z < this->area_for_floor.size(); z++) {
+    const auto& this_fa = this->area_for_floor[z];
+    const auto& other_fa = other.area_for_floor[z];
+    if (this_fa != other_fa) {
+      throw runtime_error(std::format(
+          "quest version has different area on floor 0x{:02X} (existing: {}, new: {})",
+          z, phosg::format_data_string(this->area_for_floor.data(), 0x12), phosg::format_data_string(other.area_for_floor.data(), 0x12)));
+    }
+  }
+  if (this->description_flag != other.description_flag) {
+    throw runtime_error(std::format(
+        "quest version has different description flag (existing: {:02X}, new: {:02X})",
+        this->description_flag, other.description_flag));
+  }
+  if (!this->available_expression != !other.available_expression) {
+    throw runtime_error(std::format(
+        "quest version has available expression but root quest does not, or vice versa (existing: {}, new: {})",
+        this->available_expression ? "present" : "absent", other.available_expression ? "present" : "absent"));
+  }
+  if (this->available_expression && *this->available_expression != *other.available_expression) {
+    string existing_str = this->available_expression->str();
+    string new_str = other.available_expression->str();
+    throw runtime_error(std::format(
+        "quest version has a different available expression (existing: {}, new: {})",
+        existing_str, new_str));
+  }
+  if (!this->enabled_expression != !other.enabled_expression) {
+    throw runtime_error(std::format(
+        "quest version has enabled expression but root quest does not, or vice versa (existing: {}, new: {})",
+        this->enabled_expression ? "present" : "absent", other.enabled_expression ? "present" : "absent"));
+  }
+  if (this->enabled_expression && *this->enabled_expression != *other.enabled_expression) {
+    string existing_str = this->enabled_expression->str();
+    string new_str = other.enabled_expression->str();
+    throw runtime_error(std::format(
+        "quest version has a different enabled expression (existing: {}, new: {})",
+        existing_str, new_str));
+  }
+  if (this->common_item_set_name != other.common_item_set_name) {
+    throw runtime_error(std::format(
+        "quest version has different common table name (existing: {}, new: {})",
+        this->common_item_set_name, other.common_item_set_name));
+  }
+  if (this->common_item_set != other.common_item_set) {
+    throw runtime_error("quest version has different common table");
+  }
+  if (this->rare_item_set_name != other.rare_item_set_name) {
+    throw runtime_error(std::format(
+        "quest version has different rare table name (existing: {}, new: {})",
+        this->rare_item_set_name, other.rare_item_set_name));
+  }
+  if (this->rare_item_set != other.rare_item_set) {
+    throw runtime_error("quest version has different rare table");
+  }
+  if (this->allowed_drop_modes != other.allowed_drop_modes) {
+    throw runtime_error(format("quest version has different allowed drop modes (existing: {:02X}, new: {:02X})",
+        this->allowed_drop_modes, other.allowed_drop_modes));
+  }
+  if (this->default_drop_mode != other.default_drop_mode) {
+    throw runtime_error(format("quest version has different default drop mode (existing: {}, new: {})",
+        phosg::name_for_enum(this->default_drop_mode), phosg::name_for_enum(other.default_drop_mode)));
+  }
+}
+
+phosg::JSON QuestMetadata::json() const {
+  auto floors_json = phosg::JSON::list();
+  for (const auto& fa : this->area_for_floor) {
+    floors_json.emplace_back(fa);
+  }
+  auto enemy_exp_overrides_json = phosg::JSON::dict();
+  for (const auto& [key, exp_override] : this->enemy_exp_overrides) {
+    auto difficulty = static_cast<Difficulty>((key >> 24) & 3);
+    auto floor = static_cast<uint8_t>((key >> 16) & 0xFF);
+    auto enemy_type = static_cast<EnemyType>(key & 0xFFFF);
+    auto key_str = std::format("{}:0x{:02X}:{}", name_for_difficulty(difficulty), floor, phosg::name_for_enum(enemy_type));
+    enemy_exp_overrides_json.emplace(key_str, exp_override);
+  }
+
+  auto create_item_mask_entries_json = phosg::JSON::list();
+  for (const auto& item : this->create_item_mask_entries) {
+    create_item_mask_entries_json.emplace_back(item.str());
+  }
+
+  return phosg::JSON::dict({
+      {"CategoryID", this->category_id},
+      {"Number", this->quest_number},
+      {"Episode", name_for_episode(this->episode)},
+      {"FloorAssignments", std::move(floors_json)},
+      {"Joinable", this->joinable},
+      {"MaxPlayers", this->max_players},
+      {"BattleRules", this->battle_rules ? this->battle_rules->json() : phosg::JSON(nullptr)},
+      {"ChallengeTemplateIndex", (this->challenge_template_index >= 0) ? this->challenge_template_index : phosg::JSON(nullptr)},
+      {"ChallengeEXPMultiplier", (this->challenge_exp_multiplier >= 0) ? this->challenge_exp_multiplier : phosg::JSON(nullptr)},
+      {"ChallengeDifficulty", (this->challenge_difficulty != Difficulty::UNKNOWN) ? name_for_difficulty(this->challenge_difficulty) : phosg::JSON(nullptr)},
+      {"DescriptionFlag", this->description_flag},
+      {"AvailableExpression", this->available_expression ? this->available_expression->str() : phosg::JSON(nullptr)},
+      {"EnabledExpression", this->available_expression ? this->available_expression->str() : phosg::JSON(nullptr)},
+      {"CommonItemSetName", this->common_item_set_name.empty() ? phosg::JSON(nullptr) : this->common_item_set_name},
+      {"RareItemSetName", this->rare_item_set_name.empty() ? phosg::JSON(nullptr) : this->rare_item_set_name},
+      {"AllowedDropModes", this->allowed_drop_modes},
+      {"DefaultDropMode", phosg::name_for_enum(this->default_drop_mode)},
+      {"AllowStartFromChatCommand", this->allow_start_from_chat_command},
+      {"LockStatusRegister", (this->lock_status_register >= 0) ? this->lock_status_register : phosg::JSON(nullptr)},
+      {"EnemyEXPOverrides", std::move(enemy_exp_overrides_json)},
+      {"CreateItemMasks", std::move(create_item_mask_entries_json)},
+  });
+}
+
+QuestMetadata::CreateItemMask::CreateItemMask(const std::string& s) {
+  phosg::StringReader r(s);
+  for (size_t z = 0; z < 12 && !r.eof(); z++) {
+    auto& range = this->data1_ranges[z];
+    char c = r.get_s8();
+    if (c == '[') {
+      c = r.get_s8();
+      range.min = (phosg::value_for_hex_char(c) << 4) | phosg::value_for_hex_char(r.get_s8());
+      if (r.get_s8() != '-') {
+        throw std::runtime_error("invalid range spec");
+      }
+      c = r.get_s8();
+      range.max = (phosg::value_for_hex_char(c) << 4) | phosg::value_for_hex_char(r.get_s8());
+      if (r.get_s8() != ']') {
+        throw std::runtime_error("invalid range spec");
+      }
+    } else {
+      range.min = (phosg::value_for_hex_char(c) << 4) | phosg::value_for_hex_char(r.get_s8());
+      range.max = range.min;
+    }
+  }
+}
+
+std::string QuestMetadata::CreateItemMask::str() const {
+  std::string ret;
+  for (size_t z = 0; z < 12; z++) {
+    const auto& r = this->data1_ranges[z];
+    if (r.min == r.max) {
+      ret += std::format("{:02X}", r.min);
+    } else {
+      ret += std::format("[{:02X}-{:02X}]", r.min, r.max);
+    }
+  }
+  return ret;
+}
+
+bool QuestMetadata::CreateItemMask::match(const ItemData& item) const {
+  for (size_t z = 0; z < 12; z++) {
+    const auto& r = this->data1_ranges[z];
+    uint8_t v = item.data1[z];
+    if (v < r.min || v > r.max) {
+      return false;
+    }
+  }
+  return true;
+}
+
+uint32_t QuestMetadata::CreateItemMask::primary_identifier() const {
+  uint32_t ret = 0;
+  for (size_t z = 0; z < 3; z++) {
+    const auto& r = this->data1_ranges[z];
+    if (r.min != r.max) {
+      throw std::runtime_error("create item mask is ambiguous; cannot compute primary identifier");
+    }
+    ret = (ret << 8) | r.min;
+  }
+  return ret << 8;
+}
+
+std::unordered_map<uint32_t, uint32_t> QuestMetadata::parse_enemy_exp_overrides(const phosg::JSON& json) {
+  try {
+    std::unordered_map<uint32_t, uint32_t> ret;
+    for (const auto& [key, exp_value_json] : json.as_dict()) {
+      // Key is like "Difficulty:Floor:EnemyType" or "Difficulty:EnemyType"
+      auto key_tokens = phosg::split(key, ':');
+
+      static const unordered_map<string, Difficulty> difficulty_keys(
+          {{"Normal", Difficulty::NORMAL}, {"Hard", Difficulty::HARD}, {"VeryHard", Difficulty::VERY_HARD}, {"Ultimate", Difficulty::ULTIMATE}});
+
+      Difficulty difficulty = Difficulty::NORMAL;
+      EnemyType enemy_type = EnemyType::UNKNOWN;
+      uint8_t floor = 0xFF;
+      if (key_tokens.size() == 2) {
+        enemy_type = phosg::enum_for_name<EnemyType>(key_tokens[1]);
+      } else if (key_tokens.size() == 3) {
+        floor = stoul(key_tokens[1], nullptr, 0);
+        enemy_type = phosg::enum_for_name<EnemyType>(key_tokens[2]);
+      } else {
+        throw runtime_error("malformatted key: " + key);
+      }
+      difficulty = difficulty_keys.at(key_tokens[0]);
+      if (floor == 0xFF) {
+        for (size_t floor = 0; floor < 0x12; floor++) {
+          ret.emplace(QuestMetadata::exp_override_key(difficulty, floor, enemy_type), exp_value_json->as_int());
+        }
+      } else {
+        ret.emplace(QuestMetadata::exp_override_key(difficulty, floor, enemy_type), exp_value_json->as_int());
+      }
+    }
+    return ret;
+
+  } catch (const exception& e) {
+    throw std::runtime_error(std::format("invalid enemy EXP overrides: ", e.what()));
+  }
+}

src/QuestMetadata.hh

@@ -0,0 +1,87 @@
+#pragma once
+
+#include <stdint.h>
+
+#include <map>
+#include <memory>
+#include <string>
+#include <unordered_map>
+#include <vector>
+
+#include "CommonItemSet.hh"
+#include "EnemyType.hh"
+#include "IntegralExpression.hh"
+#include "Map.hh"
+#include "PlayerSubordinates.hh"
+#include "RareItemSet.hh"
+#include "StaticGameData.hh"
+
+struct QuestMetadata {
+  // This structure contains configuration that should be the same across all
+  // versions of the quest, except for the name and description strings. This
+  // is used in both the Quest and VersionedQuest structures; in Quest, the
+  // name and description are used only internally.
+  uint32_t category_id = 0xFFFFFFFF;
+  uint32_t quest_number = 0xFFFFFFFF;
+  Episode episode = Episode::NONE;
+  std::array<uint8_t, 0x12> area_for_floor;
+  bool joinable = false;
+  uint8_t max_players = 0x00;
+  std::shared_ptr<const BattleRules> battle_rules;
+  ssize_t challenge_template_index = -1;
+  float challenge_exp_multiplier = -1.0f;
+  Difficulty challenge_difficulty = Difficulty::UNKNOWN;
+  uint8_t description_flag = 0x00;
+  std::shared_ptr<const IntegralExpression> available_expression;
+  std::shared_ptr<const IntegralExpression> enabled_expression;
+  std::string common_item_set_name; // blank = use default
+  std::string rare_item_set_name; // blank = use default
+  std::shared_ptr<const CommonItemSet> common_item_set;
+  std::shared_ptr<const RareItemSet> rare_item_set;
+  uint8_t allowed_drop_modes = 0x00; // 0 = use server default
+  ServerDropMode default_drop_mode = ServerDropMode::CLIENT; // Ignored if allowed_drop_modes == 0
+  bool allow_start_from_chat_command = false;
+  int16_t lock_status_register = -1;
+  std::unordered_map<uint32_t, uint32_t> enemy_exp_overrides;
+
+  // Item create allowances (only used on BB)
+  struct CreateItemMask {
+    struct Range {
+      uint8_t min = 0x00;
+      uint8_t max = 0x00;
+
+      bool operator==(const Range& other) const = default;
+      bool operator!=(const Range& other) const = default;
+    };
+    std::array<Range, 12> data1_ranges;
+
+    CreateItemMask() = default;
+    CreateItemMask(const CreateItemMask& other) = default;
+    CreateItemMask(CreateItemMask&& other) = default;
+    CreateItemMask& operator=(const CreateItemMask& other) = default;
+    CreateItemMask& operator=(CreateItemMask&& other) = default;
+    bool operator==(const CreateItemMask& other) const = default;
+    bool operator!=(const CreateItemMask& other) const = default;
+
+    explicit CreateItemMask(const std::string& s); // Inverse of str()
+    std::string str() const;
+
+    bool match(const ItemData& item) const;
+    uint32_t primary_identifier() const; // Raises if any of data1[0-2] are ambiguous
+  };
+  std::vector<CreateItemMask> create_item_mask_entries;
+
+  std::string name;
+  std::string short_description;
+  std::string long_description;
+
+  static std::unordered_map<uint32_t, uint32_t> parse_enemy_exp_overrides(const phosg::JSON& json);
+  static inline uint32_t exp_override_key(Difficulty difficulty, uint8_t floor, EnemyType enemy_type) {
+    return (static_cast<uint32_t>(difficulty) << 24) | (static_cast<uint32_t>(floor) << 16) | static_cast<uint32_t>(enemy_type);
+  }
+
+  void assign_default_areas(Version version, Episode episode);
+  void assert_compatible(const QuestMetadata& other) const;
+  phosg::JSON json() const;
+  std::string areas_str() const;
+};

src/QuestScript.hh

@@ -5,6 +5,7 @@
 #include <phosg/Encoding.hh>
 #include <phosg/Tools.hh>
 
+#include "QuestMetadata.hh"
 #include "StaticGameData.hh"
 #include "Text.hh"
 #include "Version.hh"
@@ -19,13 +20,25 @@ struct PSOQuestHeaderDCNTE {
   /* 0020 */
 } __packed_ws__(PSOQuestHeaderDCNTE, 0x20);
 
+struct PSOQuestHeaderDC112000 {
+  /* 0000 */ le_uint32_t code_offset = 0;
+  /* 0004 */ le_uint32_t function_table_offset = 0;
+  /* 0008 */ le_uint32_t size = 0;
+  /* 000C */ le_uint16_t unknown_a1 = 0;
+  /* 000E */ le_uint16_t unknown_a2 = 0;
+  /* 0010 */ pstring<TextEncoding::MARKED, 0x20> name;
+  /* 0030 */ pstring<TextEncoding::MARKED, 0x80> short_description;
+  /* 00B0 */ pstring<TextEncoding::MARKED, 0x120> long_description;
+  /* 01D0 */
+} __packed_ws__(PSOQuestHeaderDC112000, 0x1D0);
+
 struct PSOQuestHeaderDC { // Same format for DC v1 and v2
   /* 0000 */ le_uint32_t code_offset = 0;
   /* 0004 */ le_uint32_t function_table_offset = 0;
   /* 0008 */ le_uint32_t size = 0;
   /* 000C */ le_uint16_t unknown_a1 = 0;
   /* 000E */ le_uint16_t unknown_a2 = 0;
-  /* 0010 */ uint8_t language = 0;
+  /* 0010 */ Language language = Language::JAPANESE;
   /* 0011 */ uint8_t unknown_a3 = 0;
   /* 0012 */ le_uint16_t quest_number = 0; // 0xFFFF for challenge quests
   /* 0014 */ pstring<TextEncoding::MARKED, 0x20> name;
@@ -40,7 +53,7 @@ struct PSOQuestHeaderPC {
   /* 0008 */ le_uint32_t size = 0;
   /* 000C */ le_uint16_t unknown_a1 = 0;
   /* 000E */ le_uint16_t unknown_a2 = 0;
-  /* 0010 */ uint8_t language = 0;
+  /* 0010 */ Language language = Language::JAPANESE;
   /* 0011 */ uint8_t unknown_a3 = 0;
   /* 0012 */ le_uint16_t quest_number = 0; // 0xFFFF for challenge quests
   /* 0014 */ pstring<TextEncoding::UTF16, 0x20> name;
@@ -57,7 +70,7 @@ struct PSOQuestHeaderGC {
   /* 0008 */ le_uint32_t size = 0;
   /* 000C */ le_uint16_t unknown_a1 = 0;
   /* 000E */ le_uint16_t unknown_a2 = 0;
-  /* 0010 */ uint8_t language = 0;
+  /* 0010 */ Language language = Language::JAPANESE;
   /* 0011 */ uint8_t unknown_a3 = 0;
   // Note: The GC client byteswaps this field, then loads it as a byte, so
   // technically the high byte of this is what the client uses as the quest
@@ -70,7 +83,21 @@ struct PSOQuestHeaderGC {
   /* 01D4 */
 } __packed_ws__(PSOQuestHeaderGC, 0x1D4);
 
-struct PSOQuestHeaderBB {
+struct CreateItemMaskEntry {
+  parray<le_int32_t, 12> data1_fields;
+  le_uint32_t present = 0;
+  le_uint32_t unknown_a3 = 0;
+
+  bool is_valid() const {
+    return (this->data1_fields[0] || this->data1_fields[1] || this->data1_fields[2]);
+  }
+
+  CreateItemMaskEntry() = default;
+  CreateItemMaskEntry(const QuestMetadata::CreateItemMask& mask);
+  operator QuestMetadata::CreateItemMask() const;
+} __packed_ws__(CreateItemMaskEntry, 0x38);
+
+struct PSOQuestHeaderBBBase {
   /* 0000 */ le_uint32_t code_offset = 0;
   /* 0004 */ le_uint32_t function_table_offset = 0;
   /* 0008 */ le_uint32_t size = 0;
@@ -86,7 +113,13 @@ struct PSOQuestHeaderBB {
   /* 0058 */ pstring<TextEncoding::UTF16, 0x80> short_description;
   /* 0158 */ pstring<TextEncoding::UTF16, 0x120> long_description;
   /* 0398 */
-} __packed_ws__(PSOQuestHeaderBB, 0x398);
+} __packed_ws__(PSOQuestHeaderBBBase, 0x0398);
+
+struct PSOQuestHeaderBB : PSOQuestHeaderBBBase {
+  /* 0398 */ parray<uint8_t, 0x94> unknown_a5;
+  /* 042C */ parray<CreateItemMaskEntry, 0x40> create_item_mask_entries;
+  /* 122C */
+} __packed_ws__(PSOQuestHeaderBB, 0x122C);
 
 void check_opcode_definitions();
 
@@ -96,14 +129,15 @@ std::string disassemble_quest_script(
     const void* data,
     size_t size,
     Version version,
-    uint8_t override_language = 0xFF,
+    Language override_language = Language::UNKNOWN,
     bool reassembly_mode = false,
     bool use_qedit_names = false);
 
-struct QuestMetadata {
+struct AssembledQuestScript {
+  std::string data;
   int64_t quest_number = -1;
   Version version = Version::UNKNOWN;
-  uint8_t language = 0xFF;
+  Language language = Language::UNKNOWN;
   Episode episode = Episode::NONE;
   bool joinable = false;
   uint8_t max_players = 0x00;
@@ -111,13 +145,9 @@ struct QuestMetadata {
   std::string short_description;
   std::string long_description;
 };
-struct AssembledQuestScript {
-  std::string data;
-  QuestMetadata metadata;
-};
 AssembledQuestScript assemble_quest_script(
     const std::string& text,
     const std::vector<std::string>& script_include_directories,
     const std::vector<std::string>& native_include_directories);
 
-Episode find_quest_episode_from_script(const void* data, size_t size, Version version);
+void populate_quest_metadata_from_script(QuestMetadata& meta, const void* data, size_t size, Version version, Language language);

src/RareItemSet.cc

@@ -28,11 +28,10 @@ string RareItemSet::ExpandedDrop::str(shared_ptr<const ItemNameIndex> name_index
 }
 
 uint32_t RareItemSet::expand_rate(uint8_t pc) {
-  // To compute the actual drop rare drop rate from pc, first decode pc into
-  // shift and value:
+  // To compute the actual rare drop rate from pc, first decode pc:
   //   pc = bits SSSSSVVV
-  //     shift = S - 4 (so shift is 0-27)
-  //     value = V + 7 (so value is 7-14)
+  //   shift = S - 4 (so shift is 0-27)
+  //   value = V + 7 (so value is 7-14)
   // Then, take the value 0x00000002, shift it left by shift (0-27), and
   // multiply the result by value (7-14) to get the actual drop rate. The result
   // is a probability out of 0xFFFFFFFF (so 0x40000000 means the item will drop
@@ -237,12 +236,11 @@ RareItemSet::SpecCollection RareItemSet::ParsedRELData::as_collection() const {
 }
 
 RareItemSet::RareItemSet(const AFSArchive& afs, bool is_v1) {
-  const array<GameMode, 4> modes = {GameMode::NORMAL, GameMode::BATTLE, GameMode::CHALLENGE, GameMode::SOLO};
-  for (GameMode mode : modes) {
-    for (size_t difficulty = 0; difficulty < 4; difficulty++) {
+  for (GameMode mode : ALL_GAME_MODES_V4) {
+    for (Difficulty difficulty : ALL_DIFFICULTIES_V234) {
       for (size_t section_id = 0; section_id < 10; section_id++) {
         try {
-          size_t index = difficulty * 10 + section_id;
+          size_t index = static_cast<size_t>(difficulty) * 10 + section_id;
           ParsedRELData rel(afs.get_reader(index), false, is_v1);
           this->collections.emplace(
               this->key_for_params(mode, Episode::EP1, difficulty, section_id),
@@ -254,7 +252,7 @@ RareItemSet::RareItemSet(const AFSArchive& afs, bool is_v1) {
   }
 }
 
-string RareItemSet::gsl_entry_name_for_table(GameMode mode, Episode episode, uint8_t difficulty, uint8_t section_id) {
+string RareItemSet::gsl_entry_name_for_table(GameMode mode, Episode episode, Difficulty difficulty, uint8_t section_id) {
   return std::format("ItemRT{}{}{}{}.rel",
       ((mode == GameMode::CHALLENGE) ? "c" : ""),
       ((episode == Episode::EP2) ? "l" : ""),
@@ -263,11 +261,9 @@ string RareItemSet::gsl_entry_name_for_table(GameMode mode, Episode episode, uin
 }
 
 RareItemSet::RareItemSet(const GSLArchive& gsl, bool is_big_endian) {
-  const array<Episode, 2> episodes = {Episode::EP1, Episode::EP2};
-  const array<GameMode, 4> modes = {GameMode::NORMAL, GameMode::BATTLE, GameMode::CHALLENGE, GameMode::SOLO};
-  for (GameMode mode : modes) {
-    for (Episode episode : episodes) {
-      for (size_t difficulty = 0; difficulty < 4; difficulty++) {
+  for (GameMode mode : ALL_GAME_MODES_V23) {
+    for (Episode episode : ALL_EPISODES_V3) {
+      for (Difficulty difficulty : ALL_DIFFICULTIES_V234) {
         for (size_t section_id = 0; section_id < 10; section_id++) {
           try {
             string filename = this->gsl_entry_name_for_table(mode, episode, difficulty, section_id);
@@ -286,15 +282,15 @@ RareItemSet::RareItemSet(const GSLArchive& gsl, bool is_big_endian) {
 RareItemSet::RareItemSet(const string& rel_data, bool is_big_endian) {
   // Tables are 0x280 bytes in size in this format, laid out sequentially
   phosg::StringReader r(rel_data);
-  array<Episode, 3> episodes = {Episode::EP1, Episode::EP2, Episode::EP4};
-  for (size_t ep_index = 0; ep_index < episodes.size(); ep_index++) {
-    for (size_t difficulty = 0; difficulty < 4; difficulty++) {
+  for (Episode episode : ALL_EPISODES_V4) {
+    for (Difficulty difficulty : ALL_DIFFICULTIES_V234) {
       for (size_t section_id = 0; section_id < 10; section_id++) {
         try {
-          size_t index = (ep_index * 40) + difficulty * 10 + section_id;
+          size_t ep_index = (episode == Episode::EP1) ? 0 : ((episode == Episode::EP2) ? 1 : 2);
+          size_t index = (ep_index * 40) + static_cast<size_t>(difficulty) * 10 + section_id;
           ParsedRELData rel(r.sub(0x280 * index, 0x280), is_big_endian, false);
           this->collections.emplace(
-              this->key_for_params(GameMode::NORMAL, episodes[ep_index], difficulty, section_id),
+              this->key_for_params(GameMode::NORMAL, episode, difficulty, section_id),
               rel.as_collection());
         } catch (const out_of_range&) {
         }
@@ -315,9 +311,9 @@ RareItemSet::RareItemSet(const phosg::JSON& json, shared_ptr<const ItemNameIndex
       Episode episode = episode_keys.at(episode_it.first);
 
       for (const auto& difficulty_it : episode_it.second->as_dict()) {
-        static const unordered_map<string, uint8_t> difficulty_keys(
-            {{"Normal", 0}, {"Hard", 1}, {"VeryHard", 2}, {"Ultimate", 3}});
-        uint8_t difficulty = difficulty_keys.at(difficulty_it.first);
+        static const unordered_map<string, Difficulty> difficulty_keys(
+            {{"Normal", Difficulty::NORMAL}, {"Hard", Difficulty::HARD}, {"VeryHard", Difficulty::VERY_HARD}, {"Ultimate", Difficulty::ULTIMATE}});
+        Difficulty difficulty = difficulty_keys.at(difficulty_it.first);
 
         for (const auto& section_id_it : difficulty_it.second->as_dict()) {
           uint8_t section_id = section_id_for_name(section_id_it.first);
@@ -386,7 +382,10 @@ RareItemSet::RareItemSet(const phosg::JSON& json, shared_ptr<const ItemNameIndex
 
 std::string RareItemSet::serialize_afs(bool is_v1) const {
   vector<string> files;
-  for (uint8_t difficulty = 0; difficulty < (is_v1 ? 3 : 4); difficulty++) {
+  for (Difficulty difficulty : ALL_DIFFICULTIES_V234) {
+    if (is_v1 && (difficulty == Difficulty::ULTIMATE)) {
+      continue;
+    }
     for (uint8_t section_id = 0; section_id < 10; section_id++) {
       ParsedRELData rel(this->get_collection(GameMode::NORMAL, Episode::EP1, difficulty, section_id));
       files.emplace_back(rel.serialize(false, is_v1));
@@ -398,9 +397,8 @@ std::string RareItemSet::serialize_afs(bool is_v1) const {
 std::string RareItemSet::serialize_gsl(bool big_endian) const {
   unordered_map<string, string> files;
 
-  static const std::array<Episode, 2> episodes = {Episode::EP1, Episode::EP2};
-  for (Episode episode : episodes) {
-    for (uint8_t difficulty = 0; difficulty < 4; difficulty++) {
+  for (Episode episode : ALL_EPISODES_V3) {
+    for (Difficulty difficulty : ALL_DIFFICULTIES_V234) {
       for (uint8_t section_id = 0; section_id < 10; section_id++) {
         try {
           string filename = this->gsl_entry_name_for_table(GameMode::NORMAL, episode, difficulty, section_id);
@@ -413,7 +411,7 @@ std::string RareItemSet::serialize_gsl(bool big_endian) const {
     }
   }
 
-  for (uint8_t difficulty = 0; difficulty < 4; difficulty++) {
+  for (Difficulty difficulty : ALL_DIFFICULTIES_V234) {
     for (uint8_t section_id = 0; section_id < 10; section_id++) {
       try {
         string filename = this->gsl_entry_name_for_table(GameMode::CHALLENGE, Episode::EP1, difficulty, section_id);
@@ -430,8 +428,9 @@ std::string RareItemSet::serialize_gsl(bool big_endian) const {
 string RareItemSet::serialize_html(
     GameMode mode,
     Episode episode,
-    uint8_t difficulty,
-    shared_ptr<const ItemNameIndex> name_index) const {
+    Difficulty difficulty,
+    shared_ptr<const ItemNameIndex> name_index,
+    shared_ptr<const CommonItemSet> common_item_set) const {
 
   struct ZoneTypes {
     const char* name;
@@ -684,7 +683,7 @@ string RareItemSet::serialize_html(
     blocks.emplace_back("</tr>");
   };
 
-  auto add_specs_row = [&](const char* loc_name, bool is_box, const array<vector<ExpandedDrop>, 10>& specs_lists) -> void {
+  auto add_specs_row = [&](const EnemyTypeDefinition* type_def, const char* loc_name, bool is_box, const array<vector<ExpandedDrop>, 10>& specs_lists) -> void {
     bool any_list_nonempty = false;
     for (const auto& specs_list : specs_lists) {
       any_list_nonempty |= !specs_list.empty();
@@ -704,6 +703,16 @@ string RareItemSet::serialize_html(
 
         auto frac = phosg::reduce_fraction<uint64_t>(spec.probability, 0x100000000);
 
+        std::string exact_token = std::format("Exact rate: {} / {}", frac.first, frac.second);
+        if (common_item_set && type_def && type_def->rt_index != 0xFF) {
+          auto table = common_item_set->get_table(episode, mode, difficulty, section_id);
+          uint8_t dar = table->enemy_type_drop_probs.at(type_def->rt_index);
+          exact_token += std::format(" (DAR: {}%)", dar);
+          frac.first *= dar;
+          frac.second *= 100;
+          frac = phosg::reduce_fraction<uint64_t>(frac.first, frac.second);
+        }
+
         ItemData example_item = spec.data;
         if (example_item.can_be_encoded_in_rel_rare_table()) {
           // Apparently Return to Ragol has a patch that allows it to use the
@@ -722,16 +731,14 @@ string RareItemSet::serialize_html(
         }
 
         string hex = example_item.short_hex();
-        string desc = name_index->describe_item(example_item, false, true);
+        string desc = name_index->describe_item(example_item, ItemNameIndex::Flag::NAME_ONLY);
         tokens.emplace_back(std::format("<span class=\"item\" title=\"Hex: {}\">{}</span>", hex, desc));
 
         float denom = static_cast<float>(frac.second) / static_cast<double>(frac.first);
         string denom_token = (floor(denom) == denom)
-            ? std::format("1 / {:g}", denom)
-            : std::format("1 / %.02f", denom);
-        tokens.emplace_back(std::format(
-            "<span class=\"rate\" title=\"Exact rate: {} / {}\">{}</span>",
-            frac.first, frac.second, denom_token));
+            ? std::format("1 / {:.0f}", denom)
+            : std::format("1 / {:.02f}", denom);
+        tokens.emplace_back(std::format("<span class=\"rate\" title=\"{}\">{}</span>", exact_token, denom_token));
       }
       if (!blocks.empty()) {
         blocks.emplace_back(phosg::join(tokens, "<br />"));
@@ -754,8 +761,8 @@ string RareItemSet::serialize_html(
         specs_lists[section_id] = this->get_enemy_specs(mode, episode, difficulty, section_id, rt_index);
       }
       const auto& type_def = type_definition_for_enemy(type);
-      const char* name = (difficulty == 3 && type_def.ultimate_name) ? type_def.ultimate_name : type_def.in_game_name;
-      add_specs_row(name, false, specs_lists);
+      const char* name = (difficulty == Difficulty::ULTIMATE && type_def.ultimate_name) ? type_def.ultimate_name : type_def.in_game_name;
+      add_specs_row(&type_def, name, false, specs_lists);
     }
     for (uint8_t floor : zone_type.floors) {
       const auto& floor_def = FloorDefinition::get(episode, floor);
@@ -767,7 +774,7 @@ string RareItemSet::serialize_html(
         specs_lists[section_id] = this->get_box_specs(mode, episode, difficulty, section_id, floor_def.drop_area_norm);
       }
       auto loc_name = std::format("{} (box)", floor_def.in_game_name);
-      add_specs_row(loc_name.c_str(), true, specs_lists);
+      add_specs_row(nullptr, loc_name.c_str(), true, specs_lists);
     }
   }
   blocks.emplace_back("</table></div></body></html>");
@@ -777,13 +784,11 @@ string RareItemSet::serialize_html(
 
 phosg::JSON RareItemSet::json(shared_ptr<const ItemNameIndex> name_index) const {
   auto modes_dict = phosg::JSON::dict();
-  static const array<GameMode, 4> modes = {GameMode::NORMAL, GameMode::BATTLE, GameMode::CHALLENGE, GameMode::SOLO};
-  for (const auto& mode : modes) {
+  for (const auto& mode : ALL_GAME_MODES_V4) {
     auto episodes_dict = phosg::JSON::dict();
-    static const array<Episode, 3> episodes = {Episode::EP1, Episode::EP2, Episode::EP4};
-    for (const auto& episode : episodes) {
+    for (const auto& episode : ALL_EPISODES_V4) {
       auto difficulty_dict = phosg::JSON::dict();
-      for (uint8_t difficulty = 0; difficulty < 4; difficulty++) {
+      for (const auto& difficulty : ALL_DIFFICULTIES_V234) {
         auto section_id_dict = phosg::JSON::dict();
         for (uint8_t section_id = 0; section_id < 10; section_id++) {
           auto collection_dict = phosg::JSON::dict();
@@ -876,7 +881,7 @@ void RareItemSet::print_collection(
     FILE* stream,
     GameMode mode,
     Episode episode,
-    uint8_t difficulty,
+    Difficulty difficulty,
     uint8_t section_id,
     shared_ptr<const ItemNameIndex> name_index) const {
   const SpecCollection* collection;
@@ -920,11 +925,9 @@ void RareItemSet::print_collection(
 }
 
 void RareItemSet::print_all_collections(FILE* stream, std::shared_ptr<const ItemNameIndex> name_index) const {
-  static const array<GameMode, 4> modes = {GameMode::NORMAL, GameMode::BATTLE, GameMode::CHALLENGE, GameMode::SOLO};
-  static const array<Episode, 3> episodes = {Episode::EP1, Episode::EP2, Episode::EP4};
-  for (GameMode mode : modes) {
-    for (Episode episode : episodes) {
-      for (uint8_t difficulty = 0; difficulty < 4; difficulty++) {
+  for (GameMode mode : ALL_GAME_MODES_V4) {
+    for (Episode episode : ALL_EPISODES_V4) {
+      for (Difficulty difficulty : ALL_DIFFICULTIES_V234) {
         for (uint8_t section_id = 0; section_id < 10; section_id++) {
           try {
             this->print_collection(stream, mode, episode, difficulty, section_id, name_index);
@@ -937,7 +940,7 @@ void RareItemSet::print_all_collections(FILE* stream, std::shared_ptr<const Item
 }
 
 std::vector<RareItemSet::ExpandedDrop> RareItemSet::get_enemy_specs(
-    GameMode mode, Episode episode, uint8_t difficulty, uint8_t secid, uint8_t rt_index) const {
+    GameMode mode, Episode episode, Difficulty difficulty, uint8_t secid, uint8_t rt_index) const {
   try {
     return this->get_collection(mode, episode, difficulty, secid).rt_index_to_specs.at(rt_index);
   } catch (const out_of_range&) {
@@ -947,7 +950,7 @@ std::vector<RareItemSet::ExpandedDrop> RareItemSet::get_enemy_specs(
 }
 
 std::vector<RareItemSet::ExpandedDrop> RareItemSet::get_box_specs(
-    GameMode mode, Episode episode, uint8_t difficulty, uint8_t secid, uint8_t area_norm) const {
+    GameMode mode, Episode episode, Difficulty difficulty, uint8_t secid, uint8_t area_norm) const {
   try {
     return this->get_collection(mode, episode, difficulty, secid).box_area_norm_to_specs.at(area_norm);
   } catch (const out_of_range&) {
@@ -956,7 +959,7 @@ std::vector<RareItemSet::ExpandedDrop> RareItemSet::get_box_specs(
   }
 }
 
-bool RareItemSet::has_entries_for_game_config(GameMode mode, Episode episode, uint8_t difficulty) const {
+bool RareItemSet::has_entries_for_game_config(GameMode mode, Episode episode, Difficulty difficulty) const {
   for (uint8_t section_id = 0; section_id < 10; section_id++) {
     if (this->collections.count(this->key_for_params(mode, episode, difficulty, section_id))) {
       return true;
@@ -966,19 +969,19 @@ bool RareItemSet::has_entries_for_game_config(GameMode mode, Episode episode, ui
 }
 
 const RareItemSet::SpecCollection& RareItemSet::get_collection(
-    GameMode mode, Episode episode, uint8_t difficulty, uint8_t secid) const {
+    GameMode mode, Episode episode, Difficulty difficulty, uint8_t secid) const {
   return this->collections.at(this->key_for_params(mode, episode, difficulty, secid));
 }
 
-uint16_t RareItemSet::key_for_params(GameMode mode, Episode episode, uint8_t difficulty, uint8_t secid) {
-  if (difficulty > 3) {
+uint16_t RareItemSet::key_for_params(GameMode mode, Episode episode, Difficulty difficulty, uint8_t secid) {
+  if (static_cast<size_t>(difficulty) > 3) {
     throw logic_error("incorrect difficulty");
   }
   if (secid > 10) {
     throw logic_error("incorrect section id");
   }
 
-  uint16_t key = ((difficulty & 3) << 4) | (secid & 0x0F);
+  uint16_t key = ((static_cast<size_t>(difficulty) & 3) << 4) | (secid & 0x0F);
   switch (mode) {
     case GameMode::NORMAL:
       break;

src/RareItemSet.hh

@@ -9,6 +9,7 @@
 #include <string>
 
 #include "AFSArchive.hh"
+#include "CommonItemSet.hh"
 #include "GSLArchive.hh"
 #include "ItemNameIndex.hh"
 #include "StaticGameData.hh"
@@ -33,19 +34,20 @@ public:
   ~RareItemSet() = default;
 
   std::vector<ExpandedDrop> get_enemy_specs(
-      GameMode mode, Episode episode, uint8_t difficulty, uint8_t secid, uint8_t rt_index) const;
+      GameMode mode, Episode episode, Difficulty difficulty, uint8_t secid, uint8_t rt_index) const;
   std::vector<ExpandedDrop> get_box_specs(
-      GameMode mode, Episode episode, uint8_t difficulty, uint8_t secid, uint8_t area_norm) const;
+      GameMode mode, Episode episode, Difficulty difficulty, uint8_t secid, uint8_t area_norm) const;
 
-  bool has_entries_for_game_config(GameMode mode, Episode episode, uint8_t difficulty) const;
+  bool has_entries_for_game_config(GameMode mode, Episode episode, Difficulty difficulty) const;
 
   std::string serialize_afs(bool is_v1) const;
   std::string serialize_gsl(bool big_endian) const;
   std::string serialize_html(
       GameMode mode,
       Episode episode,
-      uint8_t difficulty,
-      std::shared_ptr<const ItemNameIndex> name_index = nullptr) const;
+      Difficulty difficulty,
+      std::shared_ptr<const ItemNameIndex> name_index = nullptr,
+      std::shared_ptr<const CommonItemSet> common_item_set = nullptr) const;
   phosg::JSON json(std::shared_ptr<const ItemNameIndex> name_index = nullptr) const;
 
   void multiply_all_rates(double factor);
@@ -54,7 +56,7 @@ public:
       FILE* stream,
       GameMode mode,
       Episode episode,
-      uint8_t difficulty,
+      Difficulty difficulty,
       uint8_t section_id,
       std::shared_ptr<const ItemNameIndex> name_index = nullptr) const;
   void print_all_collections(FILE* stream, std::shared_ptr<const ItemNameIndex> name_index = nullptr) const;
@@ -111,10 +113,10 @@ protected:
 
   std::unordered_map<uint16_t, SpecCollection> collections;
 
-  const SpecCollection& get_collection(GameMode mode, Episode episode, uint8_t difficulty, uint8_t secid) const;
+  const SpecCollection& get_collection(GameMode mode, Episode episode, Difficulty difficulty, uint8_t secid) const;
 
-  static std::string gsl_entry_name_for_table(GameMode mode, Episode episode, uint8_t difficulty, uint8_t section_id);
-  static uint16_t key_for_params(GameMode mode, Episode episode, uint8_t difficulty, uint8_t secid);
+  static std::string gsl_entry_name_for_table(GameMode mode, Episode episode, Difficulty difficulty, uint8_t section_id);
+  static uint16_t key_for_params(GameMode mode, Episode episode, Difficulty difficulty, uint8_t secid);
 
   static uint32_t expand_rate(uint8_t pc);
   static uint8_t compress_rate(uint32_t probability);