add remote address to command log messages

This adds archive links to updated Teth clients to make set up work with newserv more seamlessly. The Teth clients linked:

- Have map files for Coren
- Have fixed unitxt_e.prs and unitxt_j.prs files so no "Revival Curiass" error
- Connect to 127.0.0.1 by default for the common use case of local server on same machine
- Japanese client has had the version string changed to "JTethVer12513" so it doesn't get overriden to English

I will probably update these archives in the future to use the English files properly with the English client but for now this will make things just work in the mean time hopefully.

CMakeLists.txt

@@ -27,10 +27,12 @@ link_directories(${LOCAL_LIB_DIR})
 find_path     (LIBEVENT_INCLUDE_DIR NAMES event.h)
 find_library  (LIBEVENT_LIBRARY     NAMES event)
 find_library  (LIBEVENT_CORE        NAMES event_core)
+find_library  (LIBEVENT_PTHREADS    NAMES event_pthreads)
 set (LIBEVENT_INCLUDE_DIRS ${LIBEVENT_INCLUDE_DIR})
 set (LIBEVENT_LIBRARIES
         ${LIBEVENT_LIBRARY}
-        ${LIBEVENT_CORE})
+        ${LIBEVENT_CORE}
+        ${LIBEVENT_PTHREADS})
 
 find_package(phosg REQUIRED)
 find_package(Iconv REQUIRED)
@@ -56,6 +58,7 @@ add_custom_target(
 
 set(SOURCES
     ${CMAKE_CURRENT_SOURCE_DIR}/src/Revision.cc
+    src/Account.cc
     src/AFSArchive.cc
     src/BattleParamsIndex.cc
     src/BMLArchive.cc
@@ -81,25 +84,28 @@ set(SOURCES
     src/Episode3/RulerServer.cc
     src/Episode3/Server.cc
     src/Episode3/Tournament.cc
+    src/EventUtils.cc
     src/FileContentsCache.cc
     src/FunctionCompiler.cc
     src/GSLArchive.cc
     src/GVMEncoder.cc
+    src/HTTPServer.cc
     src/IPFrameInfo.cc
     src/IPStackSimulator.cc
+    src/IPV4RangeSet.cc
     src/ItemCreator.cc
     src/ItemData.cc
     src/ItemNameIndex.cc
     src/ItemParameterTable.cc
     src/Items.cc
     src/LevelTable.cc
-    src/License.cc
     src/Lobby.cc
     src/Loggers.cc
     src/Main.cc
     src/Map.cc
     src/Menu.cc
     src/NetworkAddresses.cc
+    src/PatchServer.cc
     src/PatchFileIndex.cc
     src/PlayerFilesManager.cc
     src/PlayerSubordinates.cc
@@ -109,7 +115,7 @@ set(SOURCES
     src/PSOGCObjectGraph.cc
     src/PSOProtocol.cc
     src/Quest.cc
-    src/QuestAvailabilityExpression.cc
+    src/IntegralExpression.cc
     src/QuestScript.cc
     src/RareItemSet.cc
     src/ReceiveCommands.cc
@@ -121,9 +127,7 @@ set(SOURCES
     src/Server.cc
     src/ServerShell.cc
     src/ServerState.cc
-    src/Shell.cc
     src/StaticGameData.cc
-    src/StepGraph.cc
     src/TeamIndex.cc
     src/Text.cc
     src/TextIndex.cc
@@ -140,6 +144,9 @@ target_include_directories(newserv PUBLIC ${LIBEVENT_INCLUDE_DIR} ${Iconv_INCLUD
 target_link_libraries(newserv phosg ${LIBEVENT_LIBRARIES} ${Iconv_LIBRARIES} pthread)
 add_dependencies(newserv newserv-Revision-cc)
 
+# target_compile_options(newserv PRIVATE -fsanitize=address)
+# target_link_options(newserv PRIVATE -fsanitize=address)
+
 if(resource_file_FOUND)
     target_compile_definitions(newserv PUBLIC HAVE_RESOURCE_FILE)
     target_link_libraries(newserv resource_file)

README.md

@@ -1,4 +1,4 @@
-# newserv <img align="right" src="s-newserv.png" />
+# newserv <img align="right" src="static/s-newserv.png" />
 
 newserv is a game server, proxy, and reverse-engineering tool for Phantasy Star Online (PSO).
 
@@ -9,12 +9,16 @@ Feel free to submit GitHub issues if you find bugs or have feature requests. I'd
 See TODO.md for a list of known issues and future work I've curated, or go to the GitHub issue tracker for issues and requests submitted by the community.
 
 **Table of contents**
+* Background
+    * [History](#history)
+    * [Other server projects](#other-server-projects)
 * [Compatibility](#compatibility)
 * Setup
     * [Server setup](#server-setup)
     * [Client patch directories for PC and BB](#client-patch-directories)
     * [How to connect](#how-to-connect)
 * Features and configuration
+    * [User accounts](#user-accounts)
     * [Installing quests](#installing-quests)
     * [Item tables and drop modes](#item-tables-and-drop-modes)
     * [Cross-version play](#cross-version-play)
@@ -24,28 +28,65 @@ See TODO.md for a list of known issues and future work I've curated, or go to th
     * [Chat commands](#chat-commands)
 * [Non-server features](#non-server-features)
 
+# History
+
+The history of this project essentially mirrors my development as a software engineer from the beginning of my hobby until now. If you don't care about the story, skip to the "Compatibility" or "Setup" sections below.
+
+I originally purchased PSO GC when I heard about PSUL, and wanted to play around with running homebrew on my GameCube. This pathway eventually led to [GCARS-CS](https://github.com/fuzziqersoftware/gcars-cs), but that's another story.
+
+<img align="left" src="static/s-khyps.png" /> After playing PSO for a while, both offline and online, I wrote a proxy called Khyps sometime in 2003. This was back in the days of the official Sega servers, where vulnerabilities weren't addressed in a timely manner or at all. It was common for malicious players using their own proxies or Action Replay codes (a story for another time) to send invalid commands that the servers would blindly forward, and cause the receiving clients to crash. These crashes were more than simply inconvenient; they could also corrupt your save data, destroying the hours of work you may have put into hunting items and leveling up your character.
+
+For a while it was essentially necessary to use a proxy to go online at all, so the proxy could block these invalid commands. Khyps was designed primarily with this function in mind, though it also implemented some convenient cheats, like the ability to give yourself or other players infinite HP and allow you to teleport to different places without using an in-game teleporter.
+
+<img align="left" src="static/s-khyller.png" /> After Khyps I took on the larger challenge of writing a server, which resulted in Khyller sometime in 2005. This was the first server of any type I had ever written. This project eventually evolved into a full-featured environment supporting all versions of the game that I had access to - at the time, PC, GC, and BB. (However, I suspect from reading the ancient source files that Khyller's BB support was very buggy.) As Khyller evolved, the code became increasingly cumbersome, littered with debugging filth that I never cleaned up and odd coding patterns I had picked up over the years. My understanding of the C++ language was woefully incomplete as well (as opposed to now, when it is still incomplete but not woefully so), which resulted in Khyller being essentially a C project that had a couple of classes in it.
+
+<img align="left" src="static/s-aeon.png" /> Sometime in 2006 or 2007, I abandoned Khyller and rebuilt the entire thing from scratch, resulting in Aeon. Aeon was substantially cleaner in code than Khyller but still fairly hard to work with, and it lacked a few of the more arcane features I had originally written (for example, the ability to convert any quest into a download quest). In addition, the code still had some stability problems... it turns out that Aeon's concurrency primitives were simply incorrect. I had derived the concept of a mutex myself, before taking any real computer engineering classes, but had implemented it incorrectly. I made the race window as small as possible, but Aeon would still randomly crash after running seemingly fine for a few days.
+
+At the time of its inception, Aeon was also called newserv, and you may find some beta releases floating around the Internet with filenames like `newserv-b3.zip`. I had released betas 1, 2, and 3 before I released the entire source of beta 5 and stopped working on the project when I went to college. This was around the time when I switched from writing software primarily on Windows to primarily on macOS and Linux, so Aeon beta 5 was the last server I wrote that specifically targeted Windows. (newserv, which you're looking at now, is a bit tedious to compile on Windows but does work.)
+
+<img align="left" src="static/s-newserv.png" /> After a long hiatus from PSO and much professional and personal development in my technical abilities, I was reminiscing sometime in October 2018 by reading my old code archives. Somehow inspired when I came across Aeon, I spent a weekend and a couple more evenings rewriting the entire project again, cleaning up ancient patterns I had used eleven years ago, replacing entire modules with simple STL containers, and eliminating even more support files in favor of configuration autodetection. The code is now suitably modern and stable, and I'm not embarrassed by its existence, as I am by Aeon beta 5's source code and my archive of Khyller (which, thankfully, no one else ever saw).
+
+## Other server projects
+
+Independently of this project, there are many other PSO servers out there. Those that I know of that are (or were) public are listed here in approximate chronological order:
+
+* (Early 2000s) **[Schtserv](https://schtserv.com/)**: The first public-access PSO server; written in Delphi by Schthack. Still active and popular as of this writing (early 2024). Schtserv is also the only other unofficial server to support all versions of PSO, including Episode 3.
+* (2005) **Khyller**: An early attempt of mine to support PSO PC, GC, and BB. See above for more details.
+* (2006) **Aeon**: My second attempt. Better than Khyller, but still unreliable.
+* (2008) **Tethealla**: A fairly extensive implementation of PSOBB, written in C by Sodaboy. The public version of Tethealla has been [officially disowned](https://www.pioneer2.net/community/threads/tethealla-server-forums-removal.26365/) (as it is now more than 15 years old), but closed-source development continues. [Ephinea](https://ephinea.pioneer2.net/), currently the most popular PSOBB server, is the continuation of this project. Several other modern PSOBB servers are forks of the initial public version of Tethealla as well.
+* (2008) **[Sylverant](https://sylverant.net/)** [(source)](https://sourceforge.net/projects/sylverant/): The second public-access PSO server; written in C by BlueCrab. Still active and popular as of this writing (early 2024).
+* (2015) **[Archon](https://github.com/dcrodman/archon)**: A PSOBB server written in Go by Drew Rodman.
+* (2015) **[Idola](https://github.com/HybridEidolon/idolapsoserv)**: A PSOBB server written in Rust by HybridEidolon. Functionality status unknown; the project has been archived.
+* (2017) **[Aselia](https://github.com/Solybum/Aselia)**: A PSOBB server written written in C# by Soly. It seems this was planned to be open-source at some point, but that has not (yet) happened.
+* (2018) **newserv**: This project right here.
+* (2019) **[Mechonis](https://gitlab.com/sora3087/mechonis)**: A PSOBB server with a microservice architecture written in TypeScript by TrueVision.
+* (2021) **[Phantasmal World](https://github.com/DaanVandenBosch/phantasmal-world)**: A set of PSO tools, including a web-based model viewer and quest builder, and a PSO server, written by Daan Vanden Bosch.
+* (2021) **[Elseware](http://git.sharnoth.com/jake/elseware)**: A PSOBB server written in Rust by Jake.
+* (2022) **[PSOSERVER](https://github.com/Sancaros/PSOSERVER)**: A server for all versions, written in C by Sancaros and based on Sylverant and newserv.
+
 # Compatibility
 
-newserv supports several versions of PSO, including various development prototypes. Specifically:
-| Version        | Lobbies      | Games        | Proxy        |
-|----------------|--------------|--------------|--------------|
-| DC NTE         | Yes          | Yes          | No           |
-| DC 11/2000     | Yes          | Yes          | No           |
-| DC 12/2000     | Yes          | Yes          | Yes          |
-| DC 01/2001     | Yes          | Yes          | Yes          |
-| DC V1          | Yes          | Yes          | Yes          |
-| DC 08/2001     | Yes          | Yes          | Yes          |
-| DC V2          | Yes          | Yes          | Yes          |
-| PC NTE         | Yes (3)      | Yes          | No           |
-| PC             | Yes          | Yes          | Yes          |
-| GC Ep1&2 NTE   | Yes          | Yes          | Yes          |
-| GC Ep1&2       | Yes          | Yes          | Yes          |
-| GC Ep1&2 Plus  | Yes          | Yes          | Yes          |
-| GC Ep3 NTE     | Yes          | Yes (1)      | Yes          |
-| GC Ep3         | Yes          | Yes          | Yes          |
-| Xbox Ep1&2     | Yes          | Yes          | Yes          |
-| BB (vanilla)   | Yes          | Yes (2)      | Yes          |
-| BB (Tethealla) | Yes          | Yes (2)      | Yes          |
+newserv supports all known versions of PSO, including development prototypes. Specifically:
+| Version         | Lobbies  | Games    | Proxy    |
+|-----------------|----------|----------|----------|
+| DC NTE          | Yes      | Yes      | No       |
+| DC 11/2000      | Yes      | Yes      | No       |
+| DC 12/2000      | Yes      | Yes      | Yes      |
+| DC 01/2001      | Yes      | Yes      | Yes      |
+| DC V1           | Yes      | Yes      | Yes      |
+| DC 08/2001      | Yes      | Yes      | Yes      |
+| DC V2           | Yes      | Yes      | Yes      |
+| PC NTE          | Yes (3)  | Yes      | No       |
+| PC              | Yes      | Yes      | Yes      |
+| GC Ep1&2 NTE    | Yes      | Yes      | Yes      |
+| GC Ep1&2        | Yes      | Yes      | Yes      |
+| GC Ep1&2 Plus   | Yes      | Yes      | Yes      |
+| GC Ep3 NTE      | Yes      | Yes (1)  | Yes      |
+| GC Ep3          | Yes      | Yes      | Yes      |
+| Xbox Ep1&2 Beta | Yes      | Yes      | Yes      |
+| Xbox Ep1&2      | Yes      | Yes      | Yes      |
+| BB (vanilla)    | Yes      | Yes (2)  | Yes      |
+| BB (Tethealla)  | Yes      | Yes (2)  | Yes      |
 
 *Notes:*
 1. *Ep3 NTE battles are not well-tested; some things may not work. See notes/ep3-nte-differences.txt for a list of known differences between NTE and the final version. NTE and non-NTE players cannot battle each other.*
@@ -91,7 +132,10 @@ newserv implements a patch server for PSO PC and PSO BB game data. Any file or d
 For Blue Burst set up, the below is mandatory for a smooth experience:
 
 1. Browse to your chosen client's data directory.
-2. Copy all the map_*.dat files and the data.gsl file and place them in `system/patch-bb/data`
+2. Copy all the map_*.dat files, unitxt_* files and the data.gsl file and place them in `system/patch-bb/data`.
+3. If you're using game files from the Tethealla client, make a copy of unitxt_j.prs inside system/patch-bb/data and name it unitxt_e.prs. (If unitxt_e.prs already exists, replace it with the copied file.)
+
+If you do not have a BB client, or using a Tethealla client from another source, Tethealla clients that are compatible with newserv can be found here: [English](https://web.archive.org/web/20240402011115/https://ragol.org/files/bb/TethVer12513_English.zip) / [Japanese](https://web.archive.org/web/20240402013127/https://ragol.org/files/bb/TethVer12513_Japanese.zip). These clients connect to 127.0.0.1 (localhost) automatically.
 
 For BB clients, newserv reads some files out of the patch data to implement game logic, so it's important that certain game files are synchronized between the server and the client. newserv contains defaults for these files in the system/maps/bb-v4 directory, but if these don't match the client's copies of the files, odd behavior will occur in games.
 
@@ -143,6 +187,17 @@ You can make PSO connect to newserv by setting its default gateway and DNS serve
 
 If you have PSO Plus or Episode III, it won't want to connect to a server on the same local network as the GameCube itself, as determined by the GameCube's IP address and subnet mask. In the old days, one way to get around this was to create a fake network adapter on the server (or use an existing real one) that has an IP address on a different subnet, tell the GameCube that the server is the default gateway (as above), and have the server reply to the DNS request with its non-local IP address. To do this with newserv, just set LocalAddress in the config file to a different interface. For example, if the GameCube is on the 192.168.0.x network and your other adapter has address 10.0.1.6, set newserv's LocalAddress to 10.0.1.6 and set PSO's DNS server and default gateway addresses to the server's 192.168.0.x address. This may not work on modern systems or on non-Windows machines - I haven't tested it in many years.
 
+### PSO GC on a Wii or Wii U
+
+Using a Wii or Wii U to connect to newserv requires the Wii or vWii to be softmodded. How to do this is beyond the scope of this document.
+
+Nintendont includes BBA emulation and is compatible with all PSO GameCube versions except Episodes I&II Trial Edition. To use Nintendont, enable BBA emulation in Nintendont's settings and follow the instructions in the above section (PSO GC on a real GameCube).
+
+Devolution includes modem emulation and is compatible with all PSO GameCube versions including Episodes I&II Trial Edition. newserv can act as a PPP server, which Devolution can directly cnnect to. To do this:
+1. Enable the PPPRawListen option according to the comments in config.json.
+2. Start newserv.
+3. In the game's network settings, set the username and password to anything (they cannot be blank), and set the phone number to the number that newserv outputs to the console during startup. (It will be near the end of all the startup log messages.) If your Wii is on the same network as newserv, use the local number; otherwise, use the external number.
+
 ### PSO GC on Dolphin
 
 If you're using the HLE BBA type, set the BBA's DNS server address to newserv's IP address and it should work. (If newserv is on the same machine as Dolphin, you will need to use an action replay code directed at 127.0.0.1 to connect, as PSO rejects DNS queries from the same IP address.) Set PSO's network settings the same as listed below.
@@ -159,9 +214,9 @@ If you're using a version of Dolphin with tapserver support, you can make it con
 
 The PSO BB client has been modified and distributed in many different forms. newserv supports most, but not all, of the common distributions. Unlike other versions, it's important that the client and server have the same map files, so make sure to set up the patch directory based on the client you'll be using with newserv. (See the "Client patch directories" section for instructions on setting this up.)
 
-The original Japanese and US versions of PSO BB should work, but you'll have to modify your hosts file or edit psobb.exe to point to your newserv instance. The original versions are packed, so this is a more involved process than simply opening the executable in a hex editor and finding/replacing some strings.
+The original Japanese and US versions of PSO BB should work (the last Japanese release can be found [here](https://archive.org/details/psobb_jp_setup_12511_20240109/)), but you'll have to modify your hosts file or edit psobb.exe to point to your newserv instance. The original versions are packed with various versions of ASProtect, so this is a more involved process than simply opening the executable in a hex editor and finding/replacing some strings.
 
-Alternatively, you can use the Tethealla client (https://archive.org/details/psobb-tethealla-client); you can find the connection addresses starting at 0x56D724 in psobb.exe. Overwrite these addresses with your server's hostname or IP address, and you should be able to connect.
+Alternatively, you can use the Tethealla client ([English](https://web.archive.org/web/20240402011115/https://ragol.org/files/bb/TethVer12513_English.zip) / [Japanese](https://web.archive.org/web/20240402013127/https://ragol.org/files/bb/TethVer12513_Japanese.zip)); you can find the connection addresses starting at 0x56D724 in psobb.exe. Overwrite these addresses with your server's hostname or IP address, and you should be able to connect. If the server is on the same PC as the client and you don't plan to have any external players, these Tethealla clients will automatically connect to the server without any modifications.
 
 ### Connecting external clients
 
@@ -171,6 +226,23 @@ For GC clients, you'll have to use newserv's built-in DNS server or set up your
 
 # Server feature configuration
 
+## User accounts
+
+By default, newserv does not require users to pre-register before playing; the server will instead automatically create an account the first time each player connects. These accounts have no special permissions. You can view, create, edit, and delete user accounts in the server's shell (run `help` in the shell to see how to do this).
+
+A license is a set of credentials that a player can use to log in. There are six types of licenses:
+* *DC NTE licenses* consist of a 16-character serial number and 16-character access key.
+* *DC licenses* consist of an 8-character hex serial number and an 8-character access key.
+* *PC licenses* are the same format as DC licenses, but are used for PC v2.
+* *GC licenses* consist of a 10-digit decimal serial number, a 12-character access key, and a password of up to 8 characters.
+* *XB licenses* consist of a gamertag of up to 16 characters, a 16-character hex user ID, and a 16-character hex account ID.
+* *BB licenses* consist of a username of up to 16 characters and a password of up to 16 characters.
+Each account may have multiple licenses. To add a license to an account, use `add-license` in the shell.
+
+On BB, character data is scoped to the license, but system and Guild Card data is scoped to the account. That is, an account with multiple BB licenses can have more than 4 characters (up to 4 per license), but they will all share the same team membership and Guild Card lists.
+
+You may want to give your account elevated privileges. To do so, run `update-account ACCOUNT-ID flags=root` (replacing ACCOUNT-ID with your actual account-id). You can also use update-account to edit other parts of the account; see the help text for more information.
+
 ## Installing quests
 
 newserv automatically finds quests in the subdirectories of the system/quests/ directory. To install your own quests, or to use quests you've saved using the proxy's Save Files option, just put them in one of the subdirectories there and name them appropriately. The subdirectories and their behaviors (e.g. in which game modes they should appear and for which PSO versions) is defined in the QuestCategories field in config.json.
@@ -238,7 +310,7 @@ There are five different available behaviors for item drops:
 
 In the `SERVER_PRIVATE` and `SERVER_DUPLICATE` modes, there is no incentive to pick up items before another player, since other players cannot pick up the items you see dropped from boxes and enemies. However, if you pick up an item and drop it later, it can then be seen and picked up by any player.
 
-The drop mode can be changed at any time during a game with the `$dropmode` chat command. If the mode is changed after some items have already been dropped, the existing items retain their visibility (that is, they still can't be picked up by other players since they were dropped before the mode was changed). You can configure which drop modes are used by default, and which modes players are allowed to choose, in config.json. See the comments above the AllowedDropModes and DefaultDropMode keys.
+The drop mode can be changed at any time during a game with the `$dropmode` chat command. If the mode is changed after some items have already been dropped, the existing items retain their visibility (that is, items dropped in private mode still can't be picked up by other players since they were dropped before the mode was changed). You can configure which drop modes are used by default, and which modes players are allowed to choose, in config.json. See the comments above the AllowedDropModes and DefaultDropMode keys.
 
 In the server drop modes, the item tables used to generate common items are in the `system/item-tables/ItemPT-*` files. (The V2 files are used for V1 as well.) The rare item tables are in the `rare-table-*.json` files. Unlike the original formats, it's possible to make each enemy drop multiple different rare items at different rates, though the default tables never do this.
 
@@ -293,23 +365,52 @@ Like quests, Episode 3 card definitions, maps, and quests are cached in memory.
 
 ## Memory patches, client functions, and DOL files
 
-Everything in this section requires resource_dasm to be installed, so newserv can use the assemblers and disassemblers from its libresource_file library. If resource_dasm is not installed, newserv will still build and run, but these features will not be available.
+*Everything in this section requires resource_dasm to be installed, so newserv can use the assemblers and disassemblers from its libresource_file library. If resource_dasm is not installed, newserv will still build and run, but these features will not be available.*
 
-In addition, these features are only supported for the following game versions:
-* PSO GameCube Episodes 1&2 Trial Edition
-* PSO GameCube Episodes 1&2 JP, USA, and EU but not Plus
-* PSO GameCube Episodes 1&2 Plus JP v1.4 but not v1.5
-* PSO GameCube Episode 3 Trial Edition
-* PSO GameCube Episode 3 JP
-* PSO GameCube Episode 3 USA (experimental; must be manually enabled in config.json)
-* PSO Xbox (all versions)
-* PSO BB
+You can put assembly files in the system/client-functions directory with filenames like PatchName.VERS.patch.s and they will appear in the Patches menu for clients that support client functions. Client functions are written in SH-4, PowerPC, or x86 assembly and are compiled when newserv is started. The assembly system's features are documented in the comments in system/client-functions/System/WriteMemory.ppc.s.
+
+The VERS token in client function filenames refers to the specific version of the game that the client function applies to. Some versions do not support receiving client functions at all. The specific versions are:
+
+| Game              | VERS | Supported |
+|-------------------|------|-----------|
+| PSO DC NTE        | 1OJ1 | No        |
+| PSO DC 11/2000    | 1OJ2 | No        |
+| PSO DC 12/2000    | 1OJ3 | No        |
+| PSO DC 01/2001    | 1OJ4 | No        |
+| PSO DC v1 JP      | 1OJF | No        |
+| PSO DC v1 US      | 1OEF | No        |
+| PSO DC v1 EU      | 1OPF | No        |
+| PSO DC 08/2001    | 2OJ5 | Yes       |
+| PSO DC v2 JP      | 2OJF | Yes       |
+| PSO DC v2 US      | 2OEF | Yes       |
+| PSO DC v2 EU      | 2OPF | Yes       |
+| PSO PC (v2)       | 2OJW | No        |
+| PSO GC NTE        | 3OJT | Yes       |
+| PSO GC v1.2 JP    | 3OJ2 | Yes       |
+| PSO GC v1.3 JP    | 3OJ3 | Yes       |
+| PSO GC v1.4 JP    | 3OJ4 | Yes       |
+| PSO GC v1.5 JP    | 3OJ5 | No        |
+| PSO GC v1.0 US    | 3OE0 | Yes       |
+| PSO GC v1.1 US    | 3OE1 | Yes       |
+| PSO GC v1.2 US    | 3OE2 | No        |
+| PSO GC v1.0 EU    | 3OP0 | Yes       |
+| PSO GC Ep3 NTE    | 3SJT | Yes       |
+| PSO GC Ep3 JP     | 3SJ0 | Yes       |
+| PSO GC Ep3 US     | 3SE0 | No        |
+| PSO GC Ep3 EU     | 3SP0 | No        |
+| PSO Xbox Beta     | 4OJB | Yes       |
+| PSO Xbox JP Disc  | 4OJD | Yes       |
+| PSO Xbox JP TU    | 4OJU | Yes       |
+| PSO Xbox US Disc  | 4OED | Yes       |
+| PSO Xbox US TU    | 4OEU | Yes       |
+| PSO Xbox EU Disc  | 4OPD | Yes       |
+| PSO Xbox EU TU    | 4OPU | Yes       |
+| PSO BB JP 1.25.13 | 51OC | Yes       |
+| PSO BB Tethealla  | 51OC | Yes       |
 
 *Note: newserv uses the shorter GameCube versioning convention, where discs labeled DOL-XXXX-0-0Y are version 1.Y. The PSO community seems to use the convention 1.0Y in some places instead, but these are the same version. For example, the version that newserv calls v1.4 is the same as v1.04, and is labeled DOL-GPOJ-0-04 on the underside of the disc.*
 
-You can put memory patches in the system/client-functions directory with filenames like PatchName.patch.s and they will appear in the Patches menu for PSO GC, XB, and BB clients that support patching. Memory patches are written in PowerPC or x86 assembly and are compiled when newserv is started. The assembly system's features are documented in the comments in system/client-functions/WriteMemory.ppc.s.
-
-newserv comes with a set of patches for GC Episodes 1&2 based on AR codes originally made by Ralf at GC-Forever. Many of them were originally posted in [this thread](https://www.gc-forever.com/forums/viewtopic.php?f=38&t=2050).
+newserv comes with a set of patches for some of the above versions, based on AR codes originally made by Ralf at GC-Forever and Aleron Ives. Many of them were originally posted in [this thread](https://www.gc-forever.com/forums/viewtopic.php?f=38&t=2050).
 
 You can also put DOL files in the system/dol directory, and they will appear in the Programs menu for GC clients. Selecting a DOL file there will load the file into the GameCube's memory and run it, just like the old homebrew loaders (PSUL and PSOload) did. For this to work, ReadMemoryWord.ppc.s, WriteMemory.ppc.s, and RunDOL.ppc.s must be present in the system/client-functions directory. This has been tested on Dolphin but not on a real GameCube, so results may vary.
 
@@ -334,7 +435,7 @@ There are many options available when starting a proxy session. All options are
 * **Block pings**: blocks automatic pings sent by the client, and responds to ping commands from the server automatically. This works around a bug in Sylverant's login server.
 * **Infinite HP**: automatically heals you whenever you get hit. An attack that kills you in one hit will still kill you, however.
 * **Infinite TP**: automatically restores your TP whenever you use any technique.
-* **Switch assist**: attempts to unlock doors that require two players in a one-player game.
+* **Switch assist**: attempts to unlock doors that require two or four players in a one-player game.
 * **Infinite Meseta** (Episode 3 only): gives you 1,000,000 Meseta, regardless of the value sent by the remote server.
 * **Block events**: disables holiday events sent by the remote server.
 * **Block patches**: prevents any B2 (patch) commands from reaching the client.
@@ -363,12 +464,16 @@ Some commands only work on the game server and not on the proxy server. The chat
     * `$si` (game server only): Shows basic information about the server.
     * `$ping`: Shows round-trip ping time from the server to you. On the proxy server, shows the ping time from you to the proxy and from the proxy to the server.
     * `$matcount` (game server only): Shows how many of each type of material you've used.
-    * `$itemnotifs <mode>`: Enables item drop notification messages. The modes are `off`, `rare`, and `on`, which should be self-explanatory. If the game has private drops enabled, you will only see a notification if the dropped item is visible to you; you won't be notified of other players' rare drops.
+    * `$itemnotifs <mode>`: Enables item drop notification messages. If the game has private drops enabled, you will only see a notification if the dropped item is visible to you; you won't be notified of other players' drops. The modes are:
+        * `off`: No notifications are shown.
+        * `rare`: You are notified when a rare item drops.
+        * `on`: You are notified when any item drops, except Meseta.
+        * `every`: You are notified when any item drops, including Meseta.
     * `$what` (game server only): Shows the type, name, and stats of the nearest item on the ground.
     * `$where` (game server only): Shows your current floor number and coordinates. Mainly useful for debugging.
 
 * Debugging commands
-    * `$debug` (game server only): Enable or disable debug. You need the DEBUG permission in your user license to use this command. Enabling debug does a few things:
+    * `$debug` (game server only): Enable or disable debug. You need the DEBUG flag in your user account to use this command. Enabling debug does a few things:
         * You'll see in-game messages from the server when you take certain actions, like killing an enemy in BB.
         * You'll see the rare seed value and floor variations when you join a game.
         * You'll be placed into the highest available slot in lobbies and games instead of the lowest, unless you're joining a BB solo-mode game.
@@ -376,25 +481,28 @@ Some commands only work on the game server and not on the proxy server. The chat
         * The rest of the commands in this section are enabled on the game server. (They are always enabled on the proxy server.)
     * `$quest <number>` (game server only): Load a quest by quest number. Can be used to load battle or challenge quests with only one player present.
     * `$qcall <function-id>`: Call a quest function on your client.
-    * `$qcheck <flag-num>` (game server only): Show the value of a quest flag. This command can be used without debug mode enabled.
-    * `$qset <flag-num>` or `$qclear <flag-num>`: Set or clear a quest flag for everyone in the game.
+    * `$qcheck <flag-num>` (game server only): Show the value of a quest flag. This command can be used without debug mode enabled. If you're in a game, show the value of the flag in that game; if you're in the lobby, show the saved value of that quest flag for your character (BB only).
+    * `$qset <flag-num>` or `$qclear <flag-num>`: Set or clear a quest flag for everyone in the game. If you're in the lobby and on BB, set or clear the saved value of a quest flag in your character file.
     * `$qgread <flag-num>` (game server only): Get the value of a quest counter ("global flag"). This command can be used without debug mode enabled.
     * `$qgwrite <flag-num> <value>` (game server only): Set the value of a quest counter ("global flag") for yourself.
     * `$qsync <reg-num> <value>`: Set a quest register's value for yourself only. `<reg-num>` should be either rXX (e.g. r60) or fXX (e.g. f60); if the latter, `<value>` is parsed as a floating-point value instead of as an integer.
     * `$qsyncall <reg-num> <value>`: Set a quest register's value for everyone in the game. `<reg-num>` should be either rXX (e.g. r60) or fXX (e.g. f60); if the latter, `<value>` is parsed as a floating-point value instead of as an integer.
+    * `$swset [floor] <flag-num>` and `$swclear [floor] <flag-num>`: Set or clear a switch flag. If floor is not given, sets or clears the flag on your current floor.
+    * `$swsetall`: Sets all switch flags on your current floor. This unlocks all doors, disables all laser fences, triggers all light/poison switches, etc.
     * `$gc` (game server only): Send your own Guild Card to yourself.
     * `$sc <data>`: Send a command to yourself.
-    * `$ss <data>` (proxy server only): Send a command to the remote server.
+    * `$ss <data>`: Send a command to the remote server (if in a proxy session) or to the game server.
+    * `$sb <data>`: Send a command to yourself, and to the remote server or game server.
     * `$meseta <amount>` (game server only; Episode 3 only): Add the given amount to your Meseta total.
     * `$auction` (Episode 3 only): Bring up the CARD Auction menu, regardless of how many players are in the game or if you have a VIP card.
     * `$ep3battledebug` (game server only; Episode 3 only): Enable or disable TCard00_Select. If enabled, the game will enter the debug menu when you start a battle.
 
 * Personal state commands
     * `$arrow <color-id>`: Changes your lobby arrow color.
-    * `$secid <section-id>`: Sets your override section ID. After running this command, any games you create will use your override section ID for rare drops instead of your character's actual section ID. To revert to your actual section id, run `$secid` with no name after it. On the proxy server, this will not work if the remote server controls item drops (e.g. on BB, or on Schtserv with server drops enabled). If the server does not allow cheat mode anywhere (that is, "CheatModeBehavior" is "Off" in config.json), this command does nothing.
+    * `$secid <section-id>`: Sets your override section ID. After running this command, any games you create will use your override section ID for rare drops instead of your character's actual section ID. If you're in a game and you are the leader of the game, this also immediately changes the item tables used by the server when creating items. To revert to your actual section id, run `$secid` with no name after it. On the proxy server, this will not work if the remote server controls item drops (e.g. on BB, or on Schtserv with server drops enabled). If the server does not allow cheat mode anywhere (that is, "CheatModeBehavior" is "Off" in config.json), this command does nothing.
     * `$rand <seed>`: Sets your override random seed (specified as a 32-bit hex value). This will make any games you create use the given seed for rare enemies. This also makes item drops deterministic in Blue Burst games hosted by newserv. On the proxy server, this command can cause desyncs with other players in the same game, since they will not see the overridden random seed. To remove the override, run `$rand` with no arguments. If the server does not allow cheat mode anywhere (that is, "CheatModeBehavior" is "Off" in config.json), this command does nothing.
     * `$ln [name-or-type]`: Sets the lobby number. Visible only to you. This command exists because some non-lobby maps can be loaded as lobbies with invalid lobby numbers. See the "GC lobby types" and "Ep3 lobby types" entries in the information menu for acceptable values here. Note that non-lobby maps do not have a lobby counter, so there's no way to exit the lobby without using either `$ln` again or `$exit`. On the game server, `$ln` reloads the lobby immediately; on the proxy server, it doesn't take effect until you load another lobby yourself (which means you'll like have to use `$exit` to escape). Run this command with no argument to return to the default lobby.
-    * `$swa`: Enables or disables switch assist. When enabled, the server will attempt to automatically unlock two-player doors in non-quest games if you step on both switches sequentially.
+    * `$swa`: Enables or disables switch assist. When enabled, the server will attempt to automatically unlock two-player and four-player doors in non-quest games if you step on all the required switches sequentially.
     * `$exit`: If you're in a lobby, sends you to the main menu (which ends your proxy session, if you're in one). If you're in a game or spectator team, sends you to the lobby (but does not end your proxy session if you're in one). Does nothing if you're in a non-Episode 3 game and no quest is in progress.
     * `$patch <name>`: Run a patch on your client. `<name>` must exactly match the name of a patch on the server.
 
@@ -402,7 +510,7 @@ Some commands only work on the game server and not on the proxy server. The chat
     * `$savechar <slot>`: Saves your current character data on the server in the specified slot (each serial number has 4 slots, numbered 1-4). These slots are separate from BB character slots; using this command does not affect BB characters.
     * `$loadchar <slot>` (v1 and v2 only): Loads your character data from the specified slot. The changes will be undone if you join a game - to save your changes, disconnect from the lobby.
     * `$bbchar <username> <password> <slot>`: Use this command when playing on a non-BB version of PSO. If the username and password are correct, this command converts your current character to BB format and saves it on the server in the given slot (1-4). Any character already in that slot is overwritten. (This command is similar to `$savechar`, except it overwrites a BB character slot, and can transfer characters across accounts.) Note that the character's chat data, quick menu config, and bank contents are not copied, since there is no way for the server to request those types of data.
-    * `$edit <stat> <value>`: Modifies your character data. If you are on V3 (GameCube/Xbox), this command does nothing. If you are on V1 or V2 (DC or PC, not BB), your changes will be undone if you join a game - to save your changes, disconnect from the lobby. If cheats are allowed on the server, `<stat>` can be any of `atp`, `mst`, `evp`, `hp`, `dfp`, `ata`, `lck`, `meseta`, `exp`, `level`, `namecolor`, `secid`, `name`, `npc`, or `tech`. If cheats are not allowed, only `namecolor`, `name`, and `npc` can be used.
+    * `$edit <stat> <value>`: Modifies your character data. If you are on V3 (GameCube/Xbox), this command does nothing. If you are on V1 or V2 (DC or PC, not BB), your changes will be undone if you join a game - to save your changes, disconnect from the lobby. If cheats are allowed on the server, `<stat>` can be any of `atp`, `mst`, `evp`, `hp`, `dfp`, `ata`, `lck`, `meseta`, `exp`, `level`, `namecolor`, `secid`, `name`, `language`, `npc`, or `tech`. If cheats are not allowed, only `namecolor`, `name`, `language`, and `npc` can be used. Changing your character's language is only useful on BB; to do so, use a single-character language code (e.g. to switch your character to English, use `$edit language E`; for Japanese, use `$edit language J`).
 
 * Blue Burst player commands (game server only)
     * `$bank [number]`: Switches your current bank, so you can access your other character's banks (if `number` is 1-4) or your shared account bank (if `number` is 0). If `number` is not given, switches back to your current character's bank.
@@ -413,12 +521,12 @@ Some commands only work on the game server and not on the proxy server. The chat
     * `$minlevel <level>`: Sets the minimum level for players to join the current game.
     * `$password <password>`: Sets the game's join password. To unlock the game, run `$password` with nothing after it.
     * `$dropmode [mode]`: Changes the way item drops behave in the current game. `mode` can be `none`, `client`, `shared`, `private`, or `duplicate`. If `mode` is not given, tells you the current drop mode without changing it. See the "Item tables and drop modes" section for more information.
-    * `$persist`: Enable or disable persistence for the current game. When persistence is on, the game will not be deleted when the last player leaves. The state of enemies and objects on the map will be reset when the last player leaves, but dropped items will not be deleted. If the game is empty for too long (15 minutes by default), it is then deleted.
+    * `$persist`: Enable or disable persistence for the current game. When persistence is on, the game will not be deleted when the last player leaves. The states of enemies, objects, and switches will be saved, and items left on the floor will not be deleted (except items only visible to the leaving player). If the game is empty for too long (15 minutes by default), it is then deleted. There is an edge case with persistence: if the player defeats a boss, leaves the room, joins again, and returns to the boss arena, neither the boss nor the exit warp will spawn, so they will be stuck there and have to use $warp or Quit Game to get out. For this reason, `$warp 0` is allowed in boss arenas once the boss is defeated, even if cheat mode is disabled.
 
 * Episode 3 commands (game server only)
     * `$spec`: Toggles the allow spectators flag for Episode 3 games. If any players are spectating when this flag is disabled, they will be sent back to the lobby.
     * `$inftime`: Toggles infinite-time mode. Must be used before starting a battle. If infinite-time mode is enabled, the overall and per-phase time limits will be disabled regardless of the values chosen during battle setup. After completing a battle, infinite-time mode is reset to the server's default value (which can be set in Episode3BehaviorFlags in config.json).
-    * `$defrange <min>-<max>`: Sets the DEF dice range for the next battle. If this is used, the dice range set during battle rules setup will apply only to ATK dice; DEF dice will use this range instead. Assist cards and other dice effects will still apply. Dice exchange also still applies if it is enabled.
+    * `$dicerange [d:L-H] [1:L-H] [a1:L-H] [d1:L-H]`: Sets override dice ranges for the next battle. The min and max dice values from the rules setup menu always apply to the ATK dice, but you can specify a different range for the DEF dice with `d:2-4` (for example). The `1:` override applies to the 1-player team in a 2v1 game (so you would set the 2-player team's desired dice range in the rules menu). You can also specify the 1-player team's ATK and DEF ranges separately with the `a1:` and `d1:` overrides. Note that these ranges will only be used if the chosen map or quest does not override them.
     * `$stat <what>`: Shows a statistic about your player or team in the current battle. `<what>` can be `duration`, `fcs-destroyed`, `cards-destroyed`, `damage-given`, `damage-taken`, `opp-cards-destroyed`, `own-cards-destroyed`, `move-distance`, `cards-set`, `fcs-set`, `attack-actions-set`, `techs-set`, `assists-set`, `defenses-self`, `defenses-ally`, `cards-drawn`, `max-attack-damage`, `max-combo`, `attacks-given`, `attacks-taken`, `sc-damage`, `damage-defended`, or `rank`.
     * `$surrender`: Causes your team to immediately lose the current battle.
     * `$saverec <name>`: Saves the recording of the last battle.
@@ -432,6 +540,7 @@ Some commands only work on the game server and not on the proxy server. The chat
     * `$next`: Warps yourself to the next floor.
     * `$item <desc>` (or `$i <desc>`): Create an item. `desc` may be a description of the item (e.g. "Hell Saber +5 0/10/25/0/10") or a string of hex data specifying the item code. Item codes are 16 hex bytes; at least 2 bytes must be specified, and all unspecified bytes are zeroes. If you are on the proxy server, you must not be using Blue Burst for this command to work. On the game server, this command works for all versions.
     * `$unset <index>` (game server only): In an Episode 3 battle, removes one of your set cards from the field. `<index>` is the index of the set card as it appears on your screen - 1 is the card next to your SC's icon, 2 is the card to the right of 1, etc. This does not cause a Hunters-side SC to lose HP, as they normally do when their items are destroyed.
+    * `$dropmode [mode]` (proxy server): Changes the way item drops behave in the current game, if you are not on BB. Unlike the game server version of this command, using this on the proxy server requires cheats to be enabled. This works by intercepting the drop requests sent to and from the leader. (So, if you are the leader and not using server drop mode on the remote server, it affects the entire game; otherwise, it affects only items generated by your actions.) `mode` can be `none` (no drops), `default` (normal drops), or `proxy` (use newserv's drop tables instead of the remote server's). If `mode` is not given, tells you the current drop mode without changing it.
 
 * Configuration commands
     * `$event <event>`: Sets the current holiday event in the current lobby. Holiday events are documented in the "Using $event" item in the information menu. If you're on the proxy server, this applies to all lobbies and games you join, but only you will see the new event - other players will not.
@@ -439,7 +548,8 @@ Some commands only work on the game server and not on the proxy server. The chat
     * `$song <song-id>` (Episode 3 only): Plays a specific song in the current lobby.
 
 * Administration commands (game server only)
-    * `$ann <message>`: Sends an announcement message. The message text is sent to all players in all games and lobbies.
+    * `$ann <message>`: Sends an announcement message. The message is sent as temporary on-screen text to all players in all games and lobbies.
+    * `$ann! <message>`: Sends an announcement message. The message is sent as a Simple Mail message to all players in all games and lobbies.
     * `$ax <message>`: Sends a message to the server's terminal. This cannot be used to run server shell commands; it only prints text to stderr.
     * `$silence <identifier>`: Silences a player (remove their ability to chat) or unsilences a player. The identifier may be the player's name or Guild Card number.
     * `$kick <identifier>`: Disconnects a player. The identifier may be the player's name or Guild Card number.

TODO.md

@@ -4,7 +4,8 @@
 - Implement decrypt/encrypt actions for VMS files
 - Make UI strings localizable (e.g. entries in menus, welcome message, etc.)
 - Add an idle connection timeout for proxy sessions
-- Clean up ItemParameterTable implementation (see comment ad the top of the class definition)
+- Clean up ItemParameterTable implementation (see comment at the top of the class definition)
+- Handle MeetUserExtensions properly in 41 and C4 commands on the proxy (rewrite the embedded 19 command and store a map of )
 
 ## PSO DC
 
@@ -13,16 +14,19 @@
 ## Episode 3
 
 - Enforce tournament deck restrictions (e.g. rank checks, No Assist option) when populating COMs at tournament start time
-- Make `reload licenses` not vulnerable to online players' licenses overwriting licenses on disk somehow
+- Make `reload accounts` not vulnerable to online players' accounts overwriting accounts on disk somehow
 - Implement ranks (based on total Meseta earned)
-- Support Trial Edition battles
+- Make an AR code that gets rid of the SAMPLE overlays on NTE
 
 ## PSO XBOX
 
 - Fix receiving Guild Cards from non-Xbox players
 - Research the F94D quest opcode
+- Finish porting the remaining GC patches
 
 ## PSOBB
 
 - Test all quest item subcommands
 - Figure out why Pouilly Slime EXP doesn't work
+- Make server-specified rare enemies work with maps loaded by the proxy
+- Implement serialization for various table types (ItemPMT, ItemPT, etc.)

notes/AITalk.bin-format.txt

@@ -1,16 +0,0 @@
-struct AITalkBin {
-  be_uint32_t num_scs;
-  be_uint32_t sc_offsets[num_scs];
-
-  struct SCDialogueEntry {
-    be_uint32_t num_entries;
-    be_uint32_t unknown_a1;
-    be_uint32_t size; // in bytes
-    struct WhenEntry {
-      be_uint32_t when;
-      be_uint32_t percent_chance; // 0-100
-      be_uint32_t count;
-      be_uint32_t string_ids[count];
-    } __attribute__((packed));
-  } __attribute__((packed));
-} __attribute__((packed));

notes/ar-codes.txt

@@ -1,29 +1,83 @@
-(Ep1&2 USA) Unlock all songs in BGM test
+Unlock all songs in BGM test
 (Note: sadly, there are no secret/unused ones)
-04368960 38600001
-04368964 4E800020
+Ep12-JP12 => 04367A68 38600001
+             04367A6C 4E800020
+Ep12-JP13 => 04368ED8 38600001
+             04368EDC 4E800020
+Ep12-JP14 => 0436A434 38600001
+             0436A438 4E800020
+Ep12-JP15 => 0436A1E8 38600001
+             0436A1EC 4E800020
+Ep12-US10 => 0436891C 38600001
+             04368920 4E800020
+Ep12-US11 => 04368960 38600001
+             04368964 4E800020
+Ep12-US12 => 0436A5B4 38600001
+             0436A5B8 4E800020
+Ep12-EU   => 043699A8 38600001
+             043699AC 4E800020
+Ep3-NTE   => 041EA948 38600001
+             041EA94C 4E800020
+Ep3-JP    => 041D8CF0 38600001
+             041D8CF4 4E800020
+Ep3-US    => 041D8D7C 38600001
+             041D8D80 4E800020
+Ep3-EU    => 041D93F0 38600001
+             041D93F4 4E800020
 
-(Ep1&2 USA v1.1) Play lobby (and event) music on Pioneer 2 also
-0417E0F0 60000000
+Play lobby (and event) music in Morgue also
+Ep12-JP12 => 0417DD34 60000000
+Ep12-JP13 => 0417E0E8 60000000
+Ep12-JP14 => 0417E24C 60000000
+Ep12-JP15 => 0417E1AC 60000000
+Ep12-US10 => 0417E0F0 60000000
+Ep12-US11 => 0417E0F0 60000000
+Ep12-US12 => 0417E210 60000000
+Ep12-EU   => 0417E6D4 60000000
+Ep3-NTE   => 040B8C7C 60000000
+Ep3-US    => 040B7028 60000000
+Ep3-JP    => 040B7044 60000000
+Ep3-EU    => 040B746C 60000000
 
-(Ep3 USA) Play lobby (and event) music in Morgue also
-040B7028 60000000
+Skip white logo screens during startup
+Ep12-JP12 => 0413EE54 38000007
+Ep12-JP13 => 0413F1DC 38000007
+Ep12-JP14 => 0413F338 38000007
+Ep12-JP15 => 0413F298 38000007
+Ep12-US10 => 0413F190 38000007
+Ep12-US11 => 0413F190 38000007
+Ep12-US12 => 0413F2A8 38000007
+Ep12-EU   => 0413F524 38000007
+Ep3-NTE   => 0409E10C 38000007
+Ep3-JP    => 0409D810 38000007
+Ep3-US    => 0409D774 38000007
+Ep3-EU    => 0409D9A4 38000007
 
-(Ep3 USA) Skip white logo screens during startup
-0409D774 38000007
-(Episodes 1&2 USA v1.1) Skip white logo screens during startup
-0413F190 38000007
+Skip agreement prompts before online game
+Ep12-JP12 => 0432737C 38000003
+Ep12-JP13 => 043283CC 38000003
+Ep12-JP14 => 043298E8 38000003
+Ep12-JP15 => 04329690 38000003
+Ep12-US10 => 04327D3C 38000003
+Ep12-US11 => 04327D80 38000003
+Ep12-US12 => 0432984C 38000003
+Ep12-EU   => 04328C58 38000003
+Ep3-NTE   => 041C67D0 38000003
+Ep3-JP    => 041B5234 38000003
+Ep3-US    => 041B50C8 38000003
+Ep3-EU    => 041B574C 38000003
 
-(Ep3 USA) Skip agreement prompts before online game
-041B50C8 38000003
-(Episodes 1&2 USA v1.1) Skip agreement prompt before online game
-04327D80 38000003
+Disable rate limit for pressing A during loading screens
+Ep3-NTE => 042E1030 38000000
+Ep3-JP  => 042F8BE4 38000000
+Ep3-US  => 042F9B30 38000000
+Ep3-EU  => 042FA734 38000000
 
-(Ep3 USA) Disable rate limit for pressing A during loading screens
-042F9B30 38000000
-
-(Ep3 USA) Auto-press A as fast as possible during loading screens
-042F9AC0 60000000
+Auto-press A as fast as possible during loading screens
+Ep3-EU  => 042FA6C4 60000000
+Ep3-US  => 042F9AC0 60000000
+Ep3-NTE => 040C2C48 60000000
+Ep3-JP  => 042F8B74 60000000
 
 (Ep3 USA) Replace loading screen A button sounds with random sounds
 042F9B18 4804BB19
@@ -32,6 +86,13 @@
 042F9B24 64630005
 042F9B28 38800000
 
+(Ep3 NTE) Replace loading screen A button sounds with random sounds
+042E1018 480309A9
+042E101C 5463063E
+042E1020 60631400
+042E1024 64630005
+042E1028 38800000
+
 (Ep3 USA) Change color of loading screens
 (Replace AA, RR, GG, BB appropriately)
 042FA704 3CC0AARR
@@ -43,11 +104,16 @@
 0400BD64 EC5D00B2
 0400BD68 4E800020
 
-(Ep3 USA) Disable darkening effect during battle details mode
-042F951C 4E800020
+Disable darkening effect during battle details mode
+Ep3-NTE => 042E09D8 4E800020
+Ep3-JP  => 042F85D0 4E800020
+Ep3-US  => 042F951C 4E800020
+Ep3-EU  => 042FA120 4E800020
 
-(Ep3 USA) Unlock all COM decks
-042CA908 38600001
+Unlock all COM decks
+Ep3-JP => 042C9B34 38600001
+Ep3-EU => 042CB414 38600001
+Ep3-US => 042CA908 38600001
 
 (Ep3 USA) Enable all lobby counter options in non-CARD lobbies
 04096A8C 480000C0
@@ -103,8 +169,11 @@
 040002BC 7C633050
 040002C0 4E800020
 
-(Ep3 USA) Unlock all offline free battle maps
-042CAA00 38600001
+Unlock all offline free battle maps
+Ep3-NTE => 042BE538 38600001
+Ep3-JP =>  042C9C2C 38600001
+Ep3-EU =>  042CB50C 38600001
+Ep3-US =>  042CAA00 38600001
 (This unlocks ALL maps, including a bunch of maps with garbage names that crash if you try to play them)
 
 (Ep3 USA) Talk to auction counter offline to get all cards
@@ -165,8 +234,10 @@
 025CB6AA 00000001
 TODO: Figure out more debug message conditionals (vars/functions) and add them here
 
-(Episode 3 USA) Able to find VIP cards offline (but they're still rare)
-042C0B20 4800000C
+Able to find VIP cards offline (but they're still rare)
+Ep3-EU => 042C15DC 4800000C
+Ep3-JP => 042BFE24 4800000C
+Ep3-US => 042C0B20 4800000C
 
 (Ep3 USA) Hold L when starting battle to enter debug menu
 042C5460 4BD3AF78
@@ -181,8 +252,11 @@ TODO: Figure out more debug message conditionals (vars/functions) and add them h
 040003F8 3800001A
 040003FC 482C5068
 
-(Ep3 USA) Dressing room always accessible
-041A16FC 38600001
+Dressing room always accessible
+Ep3-NTE => 041B2A2C 38600001
+Ep3-JP  => 041A1920 38600001
+Ep3-EU  => 041A1C84 38600001
+Ep3-US  => 041A16FC 38600001
 
 (Ep3 USA) Full dressing room v1
 Can't change your class, but you start with your existing appearance
@@ -201,8 +275,11 @@ Go online with this code on after using the dressing room to fully save changes
 (Ep3 USA) Replace Options menu with debug menu
 04149E70 38600019
 
-(Ep3 USA) Jukebox is free
-0430D1DC 48000024
+Jukebox is free
+Ep3-NTE => 042248C4 48000024 (useless because the jukebox isn't loaded in NTE, but apparently the code for it exists)
+Ep3-JP  => 0430C178 48000024
+Ep3-US  => 0430D1DC 48000024
+Ep3-EU  => 0430DE3C 48000024
 
 (Ep3 USA) Use own character in battle (online only)
 041FFAB0 4800001C
@@ -233,8 +310,11 @@ Go online with this code on after using the dressing room to fully save changes
 0412F8D4 7D0803A6
 0412F8D8 4BEDEBF4
 
-(Ep3 USA) Metal tiles don't appear in Simulator map
-04296904 4E800020
+Metal tiles don't appear in Simulator map
+Ep3-NTE => 0428FED8 4E800020
+Ep3-JP  => 04296054 4E800020
+Ep3-US  => 04296904 4E800020
+Ep3-EU  => 04297278 4E800020
 
 (Ep3 USA) Enable Boooo and Laughter soundchat sounds
 Note: Without a TextEnglish.pr2/pr3 patch, the menu items for these sounds will be blank (but they will still work)

notes/star-value-tables.txt

@@ -1,11 +0,0 @@
-star value tables
-
-psobb [B1-437]
-00010203 04090909 01020304 05090909 01020304 05090909 01020304 05090909 01020304 05090909 00010203 04090909 01020304 05090909 01020304 05090909 01020304 05090909 00010203 09090901 02030409 09090102 03040909 09090A0A 090A0A09 0A0A090C 0B0A0A0A 0A0A0A0B 0A090A0A 0A0A0A09 0A0A0A0A 0A0A0A0A 0A0A0B0A 0C0C0B0A 0A090A09 090A0A0A 0A0C090C 0B0A090A 090C0A0B 0A0A0A0A 0A0A0A0B 0B0A0A0A 09090A09 0C0A0A0A 0B0A0B09 0A0A090A 0A0B090B 0A0B0B0A 090A090A 0B090A0A 0A0A0A0A 0A0A0A09 090C0C0C 0C0C0C0C 0C0C0C0C 0C0C0C0C 0C0C0C0C 0C0C0C0C 0C0C0009 0A0A0A0B 090A0A09 0A0A0B0A 0A0A0A0A 0A0A0A0A 0A0A0A0A 0A0A0A0A 0A0A0A0A 0B0B0B0B 0B0B0B0B 0B0B0B0A 0C0A0C0B 0A0A0A0A 0A0B0A0B 0B0B0B0B 0A0A090A 0A0A090B 0B0B0B0C 0C0C0C0C 0A0A0C0A 090A0C09 0A0B0A0A 0A0A0C0A 0A0A0A09 0A0C0A09 0A0A0A0A 0A090C0B 09090909 09090909 09090909 09090909 0909090B 0A0C0A0B 0B0C0A0A 0A090A0A 0A0A0B0A 0A0A0A0A 0909090A 0A090C0A 0C0C0C0C 0C0C0C0C 0C0C0C0C 0C0C0C0C 0C0C0C0C 0C0C0C0C 0C0C0C0C 0C0C0102 03040102 03040203 04020304 01020304 01020304 01020304 01020304 01020304 01020304 03040000 00010102 02030304 04050506 06070707 07080808 08080809 09090A0A 0A0A0A0A 0B0C0A0A 0A0A0A0A 0B0B0B0A 0B0B0C0B 0B0B0B0B 0A0A0A0A 0A0A0C09 0909090A 0A0B0C09 0B0A0A0A 0A0A0A0A 0A0A0A0B 0A0A0A0A 0A0A0A00 00010203 03040405 05050606 07070808 08080808 0A0A0A0A 0A0A0909 090A0A0A 0A0A0A0A 0A0A0B0A 0A0B0A09 0909090A 0B0B0000 0B000000 00080808 08080808 09080808 09080808 09070707 07070909 090C0909 09090909 09090909 09090909 09090909 09090909 09090909 09090909 09090909 09090909 09090909 09090909 09090909 0A0A0B0A 0B0A0909 0B0B0B0C 0A0A0A09 0A0A0A0A 090A0A0A 0A0A0A0A 0A0A0A0A 0203050B 0203050B 0203050B 0203050B 0204060B 0204060B 0203050B 080B080A 0B020305 02030502 03050304 06030405 07080B04 06090406 09040609 06090B06 090B0909 09090909 0A0B0B0B 0B0B0B0B 0B0B0B0B 0B0B0B0B 0B0B0B0B 0A0B0B0B 0B0B0B0B
-
-psogc [94-2F7]
-00010203 04090909 01020304 05090909 01020304 05090909 01020304 05090909 01020304 05090909 00010203 04090909 01020304 05090909 01020304 05090909 01020304 05090909 00010203 09090901 02030409 09090102 03040909 09090A0A 090A0A09 0A0A090C 0B0A0A0A 0A0A0A0B 0A090A0A 0A0A0A09 0A0A0A0A 0A0A0A0A 0A0A0B0A 0C0C0B0A 0A090A09 090A0A0A 0A0C090C 0B0A090A 090C0A0B 0A0A0A0A 0A0A0A0B 0B0A0A0A 09090A09 0C0A0A0A 0B0A0B09 0A0A090A 0A0B090B 0A0B0B0A 090A090A 0B090A0A 0A0A0A0A 0A0A0A09 090C0C0C 0C0C0C0C 0C0C0C0C 0C0C0C0C 0C0C0C0C 0C0C0C0C 0C0C0009 0A0A0A0B 090A0A09 0A0A0B0A 0A0A0A0A 0A0A0A0A 0A0A0A0A 0A0A0A0A 0A0A0A0A 0B0B0B0B 0B0B0B0B 0B0B0B0A 0C0A0C0B 0A0A0A0A 0A0B0A0B 0B0B0B0B 0A0A090A 0A0A090B 0B0B0B0C 0C0C0C0C 01020304 01020304 02030402 03040102 03040102 03040102 03040102 03040102 03040102 03040304 00000001 01020203 03040405 05060607 07070708 08080808 08090909 0A0A0A0A 0A0A0B0C 0A0A0A0A 0A0A0B0B 0B0A0B0B 0C0B0B0B 0B0B0000 01020303 04040505 05060607 07080808 0808080A 0A0A0A0A 0A090909 0A0A0A0A 0A0A0A0A 0A0B0A0A 0B0A0909 09090A0B 0B000000 00000000 08080808 08080809 08080809 08080809 07070707 07090909 0C090909 09090909 09090909 09090909 09090909 09090909 09090909 09090909 09090909 09090909 09090909 09090909 09090902 03050B02 03050B02 03050B02 03050B02 04060B02 04060B02 03050B08 0B080A0B 02030502 03050203 05030406 03040507 080B0406 09040609 04060906 090B0609 0B090909 090909
-
-
-
-0203050B0203050B0203050B0203050B

src/AFSArchive.cc

@@ -7,6 +7,8 @@
 #include <phosg/Filesystem.hh>
 #include <phosg/Strings.hh>
 
+#include "Text.hh"
+
 using namespace std;
 
 AFSArchive::AFSArchive(shared_ptr<const string> data)
@@ -14,12 +16,12 @@ AFSArchive::AFSArchive(shared_ptr<const string> data)
   struct FileHeader {
     be_uint32_t magic;
     le_uint32_t num_files;
-  } __attribute__((packed));
+  } __packed_ws__(FileHeader, 8);
 
   struct FileEntry {
     le_uint32_t offset;
     le_uint32_t size;
-  } __attribute__((packed));
+  } __packed_ws__(FileEntry, 8);
 
   StringReader r(*this->data);
   const auto& header = r.get<FileHeader>();

src/ARCodeTranslator-Stub.hh

@@ -9,6 +9,10 @@ inline void run_ar_code_translator(const std::string&, const std::string&, const
   throw std::runtime_error("resource_file is not available; install it and rebuild newserv");
 }
 
+inline void run_xbe_patch_translator(const std::string&, const std::string&, const std::string&) {
+  throw std::runtime_error("resource_file is not available; install it and rebuild newserv");
+}
+
 inline std::vector<std::pair<uint32_t, std::string>> diff_dol_files(const std::string&, const std::string&) {
   throw std::runtime_error("resource_file is not available; install it and rebuild newserv");
 }

src/ARCodeTranslator.cc

@@ -5,6 +5,7 @@
 #include <phosg/Filesystem.hh>
 #include <phosg/Strings.hh>
 #include <resource_file/ExecutableFormats/DOLFile.hh>
+#include <resource_file/ExecutableFormats/XBEFile.hh>
 
 using namespace std;
 
@@ -63,11 +64,11 @@ public:
   }
   void set_source_file(const string& filename) {
     this->src_filename = filename;
-    this->src_file = files.at(this->src_filename);
+    this->src_file = this->files.at(this->src_filename);
   }
 
   void find_rtoc_global_regs() const {
-    for (const auto& it : files) {
+    for (const auto& it : this->files) {
       bool r2_high_found = false;
       bool r2_low_found = false;
       bool r13_high_found = false;
@@ -258,7 +259,7 @@ public:
     }
 
     unordered_map<string, uint32_t> results;
-    for (const auto& it : files) {
+    for (const auto& it : this->files) {
       if (it.second == this->src_file) {
         log.info("(%s) %08" PRIX32 " (from source)", it.first.c_str(), src_addr);
         results.emplace(it.first, src_addr);
@@ -349,6 +350,231 @@ private:
   shared_ptr<const DOLFile> src_file;
 };
 
+class XBEPatchTranslator {
+public:
+  enum class ExpandMethod {
+    FORWARD = 0,
+    BACKWARD,
+    BOTH,
+  };
+
+  static const char* name_for_expand_method(ExpandMethod method) {
+    switch (method) {
+      case ExpandMethod::FORWARD:
+        return "FORWARD";
+      case ExpandMethod::BACKWARD:
+        return "BACKWARD";
+      case ExpandMethod::BOTH:
+        return "BOTH";
+      default:
+        throw logic_error("invalid expand method");
+    }
+  }
+
+  XBEPatchTranslator(const string& directory)
+      : log("[xbe-trans] "),
+        directory(directory) {
+    while (ends_with(this->directory, "/")) {
+      this->directory.pop_back();
+    }
+    for (const auto& filename : list_directory(this->directory)) {
+      if (ends_with(filename, ".xbe")) {
+        string name = filename.substr(0, filename.size() - 4);
+        string path = directory + "/" + filename;
+        this->files.emplace(name, make_shared<XBEFile>(path.c_str()));
+        this->log.info("Loaded %s", name.c_str());
+      }
+    }
+  }
+  ~XBEPatchTranslator() = default;
+
+  const string& get_source_filename() const {
+    return this->src_filename;
+  }
+  void set_source_file(const string& filename) {
+    this->src_filename = filename;
+    this->src_file = this->files.at(this->src_filename);
+  }
+
+  uint32_t find_match(
+      shared_ptr<const XBEFile> dest_file,
+      uint32_t src_address,
+      uint32_t src_size,
+      ExpandMethod expand_method) const {
+    if (!this->src_file) {
+      throw runtime_error("no source file selected");
+    }
+
+    const XBEFile::Section* src_section = nullptr;
+    for (const auto& sec : this->src_file->sections) {
+      if (src_address >= sec.addr && src_address < sec.addr + sec.file_size) {
+        src_section = &sec;
+        break;
+      }
+    }
+    if (!src_section) {
+      throw runtime_error("source address not within any section");
+    }
+
+    const char* method_token = this->name_for_expand_method(expand_method);
+
+    size_t src_offset = src_address - src_section->addr;
+    size_t src_bytes_available_before = src_offset;
+    size_t src_bytes_available_after = src_section->file_size - src_offset - src_size;
+    this->log.info("(find_match/%s) Source offset = %08zX with %zX/%zX bytes available before/after",
+        method_token, src_offset, src_bytes_available_before, src_bytes_available_after);
+
+    size_t match_bytes_before = 0;
+    size_t match_bytes_after = 0;
+    while (match_bytes_before + match_bytes_after + src_size < 0x100) {
+      size_t num_matches = 0;
+      size_t last_match_address = 0;
+      size_t match_length = match_bytes_before + match_bytes_after + src_size;
+      auto src_r = this->src_file->read_from_addr(src_section->addr + src_offset - match_bytes_before, match_length);
+      for (const auto& dest_section : dest_file->sections) {
+        for (size_t dest_match_offset = 0; dest_match_offset + match_length <= dest_section.file_size; dest_match_offset++) {
+          src_r.go(0);
+          StringReader dest_r = dest_file->read_from_addr(dest_section.addr + dest_match_offset, match_length);
+          size_t z;
+          for (z = 0; z < match_length; z++) {
+            uint8_t src_data = src_r.get_u8();
+            uint8_t dest_data = dest_r.get_u8();
+            if (src_data != dest_data) {
+              break;
+            }
+          }
+          if (z == match_length) {
+            num_matches++;
+            last_match_address = dest_section.addr + dest_match_offset + match_bytes_before;
+          }
+        }
+      }
+      this->log.info("(find_match/%s) For match length %zX, %zu matches found", method_token, match_length, num_matches);
+      if (num_matches == 1) {
+        return last_match_address;
+      } else if (num_matches == 0) {
+        throw runtime_error("did not find exactly one match");
+      }
+      bool can_expand_backward = false;
+      bool can_expand_forward = false;
+      switch (expand_method) {
+        case ExpandMethod::BACKWARD:
+          can_expand_backward = (src_bytes_available_before > match_bytes_before);
+          break;
+        case ExpandMethod::FORWARD:
+          can_expand_forward = (src_bytes_available_after > match_bytes_after);
+          break;
+        case ExpandMethod::BOTH:
+          can_expand_backward = (src_bytes_available_before > match_bytes_before);
+          can_expand_forward = (src_bytes_available_after > match_bytes_after);
+          break;
+        default:
+          throw logic_error("invalid expand method");
+      }
+      if (!can_expand_backward && !can_expand_forward) {
+        throw runtime_error("no further expansion is allowed");
+      }
+      if (can_expand_backward) {
+        match_bytes_before++;
+      }
+      if (can_expand_forward) {
+        match_bytes_after++;
+      }
+    }
+    throw runtime_error("scan field too long; too many matches");
+  }
+
+  void find_all_matches(uint32_t src_addr, uint32_t src_size) const {
+    if (!this->src_file) {
+      throw runtime_error("no source file selected");
+    }
+
+    unordered_map<string, uint32_t> results;
+    for (const auto& it : files) {
+      if (it.second == this->src_file) {
+        log.info("(%s) %08" PRIX32 " (from source)", it.first.c_str(), src_addr);
+        results.emplace(it.first, src_addr);
+      } else {
+
+        array<future<uint32_t>, 3> futures;
+        static const array<ExpandMethod, 3> methods = {
+            ExpandMethod::FORWARD,
+            ExpandMethod::BACKWARD,
+            ExpandMethod::BOTH,
+        };
+        for (size_t z = 0; z < methods.size(); z++) {
+          futures[z] = async(&XBEPatchTranslator::find_match, this, it.second, src_addr, src_size, methods[z]);
+        }
+
+        unordered_set<uint32_t> match_addrs;
+        for (size_t z = 0; z < futures.size(); z++) {
+          const char* method_name = this->name_for_expand_method(methods[z]);
+          try {
+            uint32_t ret = futures[z].get();
+            log.info("(%s) (%s) %08" PRIX32, it.first.c_str(), method_name, ret);
+            match_addrs.emplace(ret);
+          } catch (const exception& e) {
+            log.error("(%s) (%s) failed: %s", it.first.c_str(), method_name, e.what());
+          }
+        }
+
+        if (match_addrs.empty()) {
+          log.error("(%s) no match found", it.first.c_str());
+        } else if (match_addrs.size() > 1) {
+          log.error("(%s) different matches found by different methods", it.first.c_str());
+        } else {
+          results.emplace(it.first, *match_addrs.begin());
+        }
+      }
+    }
+    for (const auto& it : results) {
+      fprintf(stdout, "%s => %08" PRIX32 "\n", it.first.c_str(), it.second);
+    }
+  }
+
+  void handle_command(const string& command) {
+    auto tokens = split(command, ' ');
+    if (tokens.empty()) {
+      throw runtime_error("no command given");
+    }
+    strip_trailing_whitespace(tokens[tokens.size() - 1]);
+
+    if (tokens[0] == "use") {
+      this->set_source_file(tokens.at(1));
+    } else if (tokens[0] == "match") {
+      this->find_all_matches(stoul(tokens.at(1), nullptr, 16), stoul(tokens.at(2), nullptr, 16));
+    } else if (!tokens[0].empty()) {
+      throw runtime_error("unknown command");
+    }
+  }
+
+  void run_shell() {
+    while (!feof(stdin)) {
+      if (!this->src_filename.empty()) {
+        fprintf(stdout, "ar-trans:%s/%s> ", this->directory.c_str(), this->src_filename.c_str());
+      } else {
+        fprintf(stdout, "ar-trans:%s> ", this->directory.c_str());
+      }
+      fflush(stdout);
+
+      string command = fgets(stdin);
+      try {
+        this->handle_command(command);
+      } catch (const exception& e) {
+        this->log.error("Failed: %s", e.what());
+      }
+    }
+    fputc('\n', stdout);
+  }
+
+private:
+  PrefixedLogger log;
+  string directory;
+  unordered_map<string, shared_ptr<const XBEFile>> files;
+  string src_filename;
+  shared_ptr<const XBEFile> src_file;
+};
+
 void run_ar_code_translator(const std::string& directory, const std::string& use_filename, const std::string& command) {
   ARCodeTranslator trans(directory);
   if (!use_filename.empty()) {
@@ -362,6 +588,19 @@ void run_ar_code_translator(const std::string& directory, const std::string& use
   }
 }
 
+void run_xbe_patch_translator(const std::string& directory, const std::string& use_filename, const std::string& command) {
+  XBEPatchTranslator trans(directory);
+  if (!use_filename.empty()) {
+    trans.set_source_file(use_filename);
+  }
+
+  if (!command.empty()) {
+    trans.handle_command(command);
+  } else {
+    trans.run_shell();
+  }
+}
+
 vector<pair<uint32_t, string>> diff_dol_files(const string& a_filename, const string& b_filename) {
   DOLFile a(a_filename.c_str());
   DOLFile b(b_filename.c_str());

src/ARCodeTranslator.hh

@@ -6,5 +6,6 @@
 #include <utility>
 #include <vector>
 
-void run_ar_code_translator(const std::string& initial_directory, const std::string& use_file, const std::string& command);
+void run_ar_code_translator(const std::string& directory, const std::string& use_filename, const std::string& command);
+void run_xbe_patch_translator(const std::string& directory, const std::string& use_filename, const std::string& command);
 std::vector<std::pair<uint32_t, std::string>> diff_dol_files(const std::string& a_filename, const std::string& b_filename);

src/Account.hh

@@ -0,0 +1,263 @@
+#pragma once
+
+#include <memory>
+#include <mutex>
+#include <phosg/JSON.hh>
+#include <shared_mutex>
+#include <string>
+#include <unordered_map>
+#include <vector>
+
+#include "Text.hh"
+
+class LicenseIndex;
+
+struct DCNTELicense {
+  std::string serial_number;
+  std::string access_key;
+
+  static std::shared_ptr<DCNTELicense> from_json(const JSON& json);
+  JSON json() const;
+};
+
+struct V1V2License {
+  uint32_t serial_number = 0;
+  std::string access_key;
+
+  static std::shared_ptr<V1V2License> from_json(const JSON& json);
+  JSON json() const;
+};
+
+struct GCLicense {
+  uint32_t serial_number = 0;
+  std::string access_key;
+  std::string password;
+
+  static std::shared_ptr<GCLicense> from_json(const JSON& json);
+  JSON json() const;
+};
+
+struct XBLicense {
+  std::string gamertag;
+  uint64_t user_id = 0;
+  uint64_t account_id = 0;
+
+  static std::shared_ptr<XBLicense> from_json(const JSON& json);
+  JSON json() const;
+};
+
+struct BBLicense {
+  std::string username;
+  std::string password;
+
+  static std::shared_ptr<BBLicense> from_json(const JSON& json);
+  JSON json() const;
+};
+
+struct Account {
+  enum class Flag : uint32_t {
+    // clang-format off
+    KICK_USER                   = 0x00000001,
+    BAN_USER                    = 0x00000002,
+    SILENCE_USER                = 0x00000004,
+    CHANGE_EVENT                = 0x00000010,
+    ANNOUNCE                    = 0x00000020,
+    FREE_JOIN_GAMES             = 0x00000040,
+    DEBUG                       = 0x01000000,
+    CHEAT_ANYWHERE              = 0x02000000,
+    DISABLE_QUEST_REQUIREMENTS  = 0x04000000,
+    ALWAYS_ENABLE_CHAT_COMMANDS = 0x08000000,
+    MODERATOR                   = 0x00000007,
+    ADMINISTRATOR               = 0x000000FF,
+    ROOT                        = 0x7FFFFFFF,
+    IS_SHARED_ACCOUNT           = 0x80000000,
+    // NOTE: When adding or changing license flags, don't forget to change the
+    // documentation in the shell's help text.
+    UNUSED_BITS                 = 0x70FFFF00,
+    // clang-format on
+  };
+
+  // account_id is also the account's guild card number
+  uint32_t account_id = 0;
+
+  uint32_t flags = 0;
+  uint64_t ban_end_time = 0; // 0 = not banned
+  std::string last_player_name;
+  std::string auto_reply_message;
+
+  uint32_t ep3_current_meseta = 0;
+  uint32_t ep3_total_meseta_earned = 0;
+
+  uint32_t bb_team_id = 0;
+  bool is_temporary = false; // If true, isn't saved to disk
+
+  std::unordered_set<std::string> auto_patches_enabled;
+
+  std::unordered_map<std::string, std::shared_ptr<DCNTELicense>> dc_nte_licenses;
+  std::unordered_map<uint32_t, std::shared_ptr<V1V2License>> dc_licenses;
+  std::unordered_map<uint32_t, std::shared_ptr<V1V2License>> pc_licenses;
+  std::unordered_map<uint32_t, std::shared_ptr<GCLicense>> gc_licenses;
+  std::unordered_map<std::string, std::shared_ptr<XBLicense>> xb_licenses;
+  std::unordered_map<std::string, std::shared_ptr<BBLicense>> bb_licenses;
+
+  Account() = default;
+  explicit Account(const JSON& json);
+  virtual ~Account() = default;
+
+  JSON json() const;
+  virtual void save() const;
+  virtual void delete_file() const;
+
+  [[nodiscard]] inline bool check_flag(Flag flag) const {
+    return !!(this->flags & static_cast<uint32_t>(flag));
+  }
+  inline void set_flag(Flag flag) {
+    this->flags |= static_cast<uint32_t>(flag);
+  }
+  inline void clear_flag(Flag flag) {
+    this->flags &= (~static_cast<uint32_t>(flag));
+  }
+  inline void toggle_flag(Flag flag) {
+    this->flags ^= static_cast<uint32_t>(flag);
+  }
+  inline void replace_all_flags(Flag mask) {
+    this->flags = static_cast<uint32_t>(mask);
+  }
+
+  void print(FILE* stream) const;
+};
+
+struct Login {
+  bool account_was_created = false;
+  // This field will never be null
+  std::shared_ptr<Account> account;
+  // Exactly one of the following will be non-null, representing the license
+  // that the client logged in with
+  std::shared_ptr<DCNTELicense> dc_nte_license;
+  std::shared_ptr<V1V2License> dc_license;
+  std::shared_ptr<V1V2License> pc_license;
+  std::shared_ptr<GCLicense> gc_license;
+  std::shared_ptr<XBLicense> xb_license;
+  std::shared_ptr<BBLicense> bb_license;
+};
+
+class AccountIndex {
+public:
+  class no_username : public std::invalid_argument {
+  public:
+    no_username() : invalid_argument("serial number is zero or username is missing") {}
+  };
+  class incorrect_password : public std::invalid_argument {
+  public:
+    incorrect_password() : invalid_argument("incorrect password") {}
+  };
+  class incorrect_access_key : public std::invalid_argument {
+  public:
+    incorrect_access_key() : invalid_argument("incorrect access key") {}
+  };
+  class missing_account : public std::invalid_argument {
+  public:
+    missing_account() : invalid_argument("missing account") {}
+  };
+
+  explicit AccountIndex(bool force_all_temporary);
+  virtual ~AccountIndex() = default;
+
+  std::shared_ptr<Account> create_account(bool is_temporary) const;
+
+  size_t count() const;
+  std::vector<std::shared_ptr<Account>> all() const;
+
+  void add(std::shared_ptr<Account> a);
+  void remove(uint32_t serial_number);
+
+  void add_dc_nte_license(std::shared_ptr<Account> account, std::shared_ptr<DCNTELicense> license);
+  void add_dc_license(std::shared_ptr<Account> account, std::shared_ptr<V1V2License> license);
+  void add_pc_license(std::shared_ptr<Account> account, std::shared_ptr<V1V2License> license);
+  void add_gc_license(std::shared_ptr<Account> account, std::shared_ptr<GCLicense> license);
+  void add_xb_license(std::shared_ptr<Account> account, std::shared_ptr<XBLicense> license);
+  void add_bb_license(std::shared_ptr<Account> account, std::shared_ptr<BBLicense> license);
+  void remove_dc_nte_license(std::shared_ptr<Account> account, const std::string& serial_number);
+  void remove_dc_license(std::shared_ptr<Account> account, uint32_t serial_number);
+  void remove_pc_license(std::shared_ptr<Account> account, uint32_t serial_number);
+  void remove_gc_license(std::shared_ptr<Account> account, uint32_t serial_number);
+  void remove_xb_license(std::shared_ptr<Account> account, const std::string& gamertag);
+  void remove_bb_license(std::shared_ptr<Account> account, const std::string& username);
+
+  std::shared_ptr<Account> from_account_id(uint32_t account_id) const;
+  std::shared_ptr<Login> from_dc_nte_credentials(
+      const std::string& serial_number,
+      const std::string& access_key,
+      bool allow_create);
+  std::shared_ptr<Login> from_dc_credentials(
+      uint32_t serial_number,
+      const std::string& access_key,
+      const std::string& character_name,
+      bool allow_create);
+  std::shared_ptr<Login> from_pc_nte_credentials(
+      uint32_t guild_card_number,
+      bool allow_create);
+  std::shared_ptr<Login> from_pc_credentials(
+      uint32_t serial_number,
+      const std::string& access_key,
+      const std::string& character_name,
+      bool allow_create);
+  std::shared_ptr<Login> from_gc_credentials(
+      uint32_t serial_number,
+      const std::string& access_key,
+      const std::string* password,
+      const std::string& character_name,
+      bool allow_create);
+  std::shared_ptr<Login> from_xb_credentials(
+      const std::string& gamertag,
+      uint64_t user_id,
+      uint64_t account_id,
+      bool allow_create);
+  std::shared_ptr<Login> from_bb_credentials(
+      const std::string& username,
+      const std::string* password,
+      bool allow_create);
+
+  std::shared_ptr<Account> create_temporary_account_for_shared_account(
+      std::shared_ptr<const Account> src_a, const std::string& variation_data) const;
+
+protected:
+  bool force_all_temporary;
+
+  // This class must be thread-safe because it's used by both the patch server
+  // and game server threads
+  mutable std::shared_mutex lock;
+  std::unordered_map<uint32_t, std::shared_ptr<Account>> by_account_id;
+  std::unordered_map<std::string, std::shared_ptr<Account>> by_dc_nte_serial_number;
+  std::unordered_map<uint32_t, std::shared_ptr<Account>> by_dc_serial_number;
+  std::unordered_map<uint32_t, std::shared_ptr<Account>> by_pc_serial_number;
+  std::unordered_map<uint32_t, std::shared_ptr<Account>> by_gc_serial_number;
+  std::unordered_map<std::string, std::shared_ptr<Account>> by_xb_gamertag;
+  std::unordered_map<std::string, std::shared_ptr<Account>> by_bb_username;
+
+  void add_locked(std::shared_ptr<Account> a);
+
+  std::shared_ptr<Login> from_dc_nte_credentials_locked(
+      const std::string& serial_number,
+      const std::string& access_key);
+  std::shared_ptr<Login> from_dc_credentials_locked(
+      uint32_t serial_number,
+      const std::string& access_key,
+      const std::string& character_name);
+  std::shared_ptr<Login> from_pc_credentials_locked(
+      uint32_t serial_number,
+      const std::string& access_key,
+      const std::string& character_name);
+  std::shared_ptr<Login> from_gc_credentials_locked(
+      uint32_t serial_number,
+      const std::string& access_key,
+      const std::string* password,
+      const std::string& character_name);
+  std::shared_ptr<Login> from_xb_credentials_locked(
+      const std::string& gamertag,
+      uint64_t user_id,
+      uint64_t account_id);
+  std::shared_ptr<Login> from_bb_credentials_locked(
+      const std::string& username,
+      const std::string* password);
+};

src/BMLArchive.cc

@@ -9,16 +9,21 @@
 using namespace std;
 
 template <bool IsBigEndian>
-struct BMLHeader {
+struct BMLHeaderT {
   using U32T = typename std::conditional<IsBigEndian, be_uint32_t, le_uint32_t>::type;
 
   parray<uint8_t, 0x04> unknown_a1;
   U32T num_entries;
   parray<uint8_t, 0x38> unknown_a2;
-} __attribute__((packed));
+} __packed__;
+
+using BMLHeader = BMLHeaderT<false>;
+using BMLHeaderBE = BMLHeaderT<true>;
+check_struct_size(BMLHeader, 0x40);
+check_struct_size(BMLHeaderBE, 0x40);
 
 template <bool IsBigEndian>
-struct BMLHeaderEntry {
+struct BMLHeaderEntryT {
   using U32T = typename std::conditional<IsBigEndian, be_uint32_t, le_uint32_t>::type;
 
   pstring<TextEncoding::ASCII, 0x20> filename;
@@ -28,17 +33,22 @@ struct BMLHeaderEntry {
   U32T compressed_gvm_size;
   U32T decompressed_gvm_size;
   parray<uint8_t, 0x0C> unknown_a2;
-} __attribute__((packed));
+} __packed__;
+
+using BMLHeaderEntry = BMLHeaderEntryT<false>;
+using BMLHeaderEntryBE = BMLHeaderEntryT<true>;
+check_struct_size(BMLHeaderEntry, 0x40);
+check_struct_size(BMLHeaderEntryBE, 0x40);
 
 template <bool IsBigEndian>
 void BMLArchive::load_t() {
   StringReader r(*this->data);
 
-  const auto& header = r.get<BMLHeader<IsBigEndian>>();
+  const auto& header = r.get<BMLHeaderT<IsBigEndian>>();
 
   size_t offset = 0x800;
   while (this->entries.size() < header.num_entries) {
-    const auto& entry = r.get<BMLHeaderEntry<IsBigEndian>>();
+    const auto& entry = r.get<BMLHeaderEntryT<IsBigEndian>>();
 
     if (offset + entry.compressed_size > this->data->size()) {
       throw runtime_error("BML data entry extends beyond end of data");

src/BattleParamsIndex.cc

@@ -20,7 +20,7 @@ void BattleParamsIndex::Table::print(FILE* stream) const {
         e.char_stats.dfp.load(),
         e.char_stats.ata.load(),
         e.char_stats.lck.load(),
-        e.unknown_a1.load(),
+        e.esp.load(),
         e.experience.load(),
         e.meseta.load());
   };

src/BattleParamsIndex.hh

@@ -24,7 +24,7 @@ public:
     /* 04 */ le_int16_t ata_bonus;
     /* 06 */ le_uint16_t unknown_a4;
     /* 08 */ le_float distance_x;
-    /* 0C */ le_float angle_x;
+    /* 0C */ le_uint32_t angle_x; // Out of 0x10000 (high 16 bits are unused)
     /* 10 */ le_float distance_y;
     /* 14 */ le_uint16_t unknown_a8;
     /* 16 */ le_uint16_t unknown_a9;
@@ -36,7 +36,7 @@ public:
     /* 28 */ le_uint32_t unknown_a15;
     /* 2C */ le_uint32_t unknown_a16;
     /* 30 */
-  } __attribute__((packed));
+  } __packed_ws__(AttackData, 0x30);
 
   struct ResistData {
     /* 00 */ le_int16_t evp_bonus;
@@ -51,7 +51,7 @@ public:
     /* 18 */ le_uint32_t unknown_a9;
     /* 1C */ le_int32_t dfp_bonus;
     /* 20 */
-  } __attribute__((packed));
+  } __packed_ws__(ResistData, 0x20);
 
   struct MovementData {
     /* 00 */ le_float idle_move_speed;
@@ -67,7 +67,7 @@ public:
     /* 28 */ le_uint32_t unknown_a7;
     /* 2C */ le_uint32_t unknown_a8;
     /* 30 */
-  } __attribute__((packed));
+  } __packed_ws__(MovementData, 0x30);
 
   struct Table {
     /* 0000 */ parray<parray<PlayerStats, 0x60>, 4> stats;
@@ -77,7 +77,7 @@ public:
     /* F600 */
 
     void print(FILE* stream) const;
-  } __attribute__((packed));
+  } __packed_ws__(Table, 0xF600);
 
   BattleParamsIndex(
       std::shared_ptr<const std::string> data_on_ep1, // BattleParamEntry_on.dat

src/CatSession.cc

@@ -31,21 +31,19 @@
 
 using namespace std;
 
+CatSession::exit_shell::exit_shell() : runtime_error("shell exited") {}
+
 CatSession::CatSession(
     shared_ptr<struct event_base> base,
     const struct sockaddr_storage& remote,
     Version version,
     shared_ptr<const PSOBBEncryption::KeyFile> bb_key_file)
-    : Shell(base),
-      log(string_printf("[CatSession:%s] ", name_for_enum(version)), proxy_server_log.min_level),
-      channel(
-          version,
-          1,
-          CatSession::dispatch_on_channel_input,
-          CatSession::dispatch_on_channel_error,
-          this,
-          "CatSession"),
+    : log(string_printf("[CatSession:%s] ", name_for_enum(version)), proxy_server_log.min_level),
+      base(base),
+      read_event(event_new(this->base.get(), 0, EV_READ | EV_PERSIST, CatSession::dispatch_read_stdin, this), event_free),
+      channel(version, 1, CatSession::dispatch_on_channel_input, CatSession::dispatch_on_channel_error, this, "CatSession"),
       bb_key_file(bb_key_file) {
+
   if (remote.ss_family != AF_INET) {
     throw runtime_error("remote is not AF_INET");
   }
@@ -58,12 +56,20 @@ CatSession::CatSession(
   if (!bev) {
     throw runtime_error(string_printf("failed to open socket (%d)", EVUTIL_SOCKET_ERROR()));
   }
-  this->channel.set_bufferevent(bev);
+  this->channel.set_bufferevent(bev, 0);
 
   if (bufferevent_socket_connect(this->channel.bev.get(),
           reinterpret_cast<const sockaddr*>(&remote), sizeof(struct sockaddr_in)) != 0) {
     throw runtime_error(string_printf("failed to connect (%d)", EVUTIL_SOCKET_ERROR()));
   }
+
+  event_add(this->read_event.get(), nullptr);
+  this->poll.add(0, POLLIN);
+}
+
+void CatSession::execute_command(const std::string& command) {
+  string full_cmd = parse_data_string(command, nullptr, ParseDataFlags::ALLOW_FILES);
+  send_command_with_header(this->channel, full_cmd.data(), full_cmd.size());
 }
 
 void CatSession::dispatch_on_channel_input(
@@ -129,9 +135,54 @@ void CatSession::on_channel_error(short events) {
   }
 }
 
-void CatSession::print_prompt() {}
-
-void CatSession::execute_command(const std::string& command) {
-  string full_cmd = parse_data_string(command, nullptr, ParseDataFlags::ALLOW_FILES);
-  send_command_with_header(this->channel, full_cmd.data(), full_cmd.size());
+void CatSession::dispatch_read_stdin(evutil_socket_t, short, void* ctx) {
+  reinterpret_cast<CatSession*>(ctx)->read_stdin();
+}
+
+void CatSession::read_stdin() {
+  bool any_command_read = false;
+  for (;;) {
+    auto poll_result = this->poll.poll();
+    short fd_events = 0;
+    try {
+      fd_events = poll_result.at(0);
+    } catch (const out_of_range&) {
+    }
+
+    if (!(fd_events & POLLIN)) {
+      break;
+    }
+
+    string command(2048, '\0');
+    if (!fgets(command.data(), command.size(), stdin)) {
+      if (!any_command_read) {
+        // ctrl+d probably; we should exit
+        fputc('\n', stderr);
+        event_base_loopexit(this->base.get(), nullptr);
+        return;
+      } else {
+        break; // probably not EOF; just no more commands for now
+      }
+    }
+
+    // trim the extra data off the string
+    size_t len = strlen(command.c_str());
+    if (len == 0) {
+      break;
+    }
+    if (command[len - 1] == '\n') {
+      len--;
+    }
+    command.resize(len);
+    any_command_read = true;
+
+    try {
+      execute_command(command);
+    } catch (const exit_shell&) {
+      event_base_loopexit(this->base.get(), nullptr);
+      return;
+    } catch (const exception& e) {
+      fprintf(stderr, "FAILED: %s\n", e.what());
+    }
+  }
 }

src/CatSession.hh

@@ -14,28 +14,41 @@
 #include "PSOEncryption.hh"
 #include "PSOProtocol.hh"
 #include "ServerState.hh"
-#include "Shell.hh"
 
-class CatSession : public Shell {
+class CatSession {
 public:
   CatSession(
       std::shared_ptr<struct event_base> base,
       const struct sockaddr_storage& remote,
       Version version,
       std::shared_ptr<const PSOBBEncryption::KeyFile> bb_key_file);
+  CatSession(const CatSession&) = delete;
+  CatSession(CatSession&&) = delete;
+  CatSession& operator=(const CatSession&) = delete;
+  CatSession& operator=(CatSession&&) = delete;
   virtual ~CatSession() = default;
 
 protected:
   PrefixedLogger log;
+  std::shared_ptr<struct event_base> base;
+  std::unique_ptr<struct event, void (*)(struct event*)> read_event;
+  Poll poll;
+
   Channel channel;
   std::shared_ptr<const PSOBBEncryption::KeyFile> bb_key_file;
 
-  virtual void print_prompt();
+  class exit_shell : public std::runtime_error {
+  public:
+    exit_shell();
+    ~exit_shell() = default;
+  };
+
   virtual void execute_command(const std::string& command);
 
-  static void dispatch_on_channel_input(
-      Channel& ch, uint16_t command, uint32_t flag, std::string& msg);
+  static void dispatch_read_stdin(evutil_socket_t fd, short events, void* ctx);
+  static void dispatch_on_channel_input(Channel& ch, uint16_t command, uint32_t flag, std::string& msg);
   static void dispatch_on_channel_error(Channel& ch, short events);
   void on_channel_input(uint16_t command, uint32_t flag, std::string& msg);
   void on_channel_error(short events);
+  void read_stdin();
 };

src/Channel.cc

@@ -32,6 +32,7 @@ Channel::Channel(
     TerminalFormat terminal_send_color,
     TerminalFormat terminal_recv_color)
     : bev(nullptr, flush_and_free_bufferevent),
+      virtual_network_id(0),
       version(version),
       language(language),
       name(name),
@@ -44,6 +45,7 @@ Channel::Channel(
 
 Channel::Channel(
     struct bufferevent* bev,
+    uint64_t virtual_network_id,
     Version version,
     uint8_t language,
     on_command_received_t on_command_received,
@@ -61,7 +63,7 @@ Channel::Channel(
       on_command_received(on_command_received),
       on_error(on_error),
       context_obj(context_obj) {
-  this->set_bufferevent(bev);
+  this->set_bufferevent(bev, virtual_network_id);
 }
 
 void Channel::replace_with(
@@ -70,10 +72,9 @@ void Channel::replace_with(
     on_error_t on_error,
     void* context_obj,
     const std::string& name) {
-  this->set_bufferevent(other.bev.release());
+  this->set_bufferevent(other.bev.release(), other.virtual_network_id);
   this->local_addr = other.local_addr;
   this->remote_addr = other.remote_addr;
-  this->is_virtual_connection = other.is_virtual_connection;
   this->version = other.version;
   this->language = other.language;
   this->crypt_in = other.crypt_in;
@@ -87,27 +88,23 @@ void Channel::replace_with(
   other.disconnect(); // Clears crypts, addrs, etc.
 }
 
-void Channel::set_bufferevent(struct bufferevent* bev) {
+void Channel::set_bufferevent(struct bufferevent* bev, uint64_t virtual_network_id) {
   this->bev.reset(bev);
+  this->virtual_network_id = virtual_network_id;
 
   if (this->bev.get()) {
     int fd = bufferevent_getfd(this->bev.get());
     if (fd < 0) {
-      this->is_virtual_connection = true;
       memset(&this->local_addr, 0, sizeof(this->local_addr));
       memset(&this->remote_addr, 0, sizeof(this->remote_addr));
     } else {
-      this->is_virtual_connection = false;
       get_socket_addresses(fd, &this->local_addr, &this->remote_addr);
     }
 
-    bufferevent_setcb(this->bev.get(),
-        &Channel::dispatch_on_input, nullptr,
-        &Channel::dispatch_on_error, this);
+    bufferevent_setcb(this->bev.get(), &Channel::dispatch_on_input, nullptr, &Channel::dispatch_on_error, this);
     bufferevent_enable(this->bev.get(), EV_READ | EV_WRITE);
 
   } else {
-    this->is_virtual_connection = false;
     memset(&this->local_addr, 0, sizeof(this->local_addr));
     memset(&this->remote_addr, 0, sizeof(this->remote_addr));
   }
@@ -149,7 +146,7 @@ void Channel::disconnect() {
 
   memset(&this->local_addr, 0, sizeof(this->local_addr));
   memset(&this->remote_addr, 0, sizeof(this->remote_addr));
-  this->is_virtual_connection = false;
+  this->virtual_network_id = false;
   this->crypt_in.reset();
   this->crypt_out.reset();
 }

src/Channel.hh

@@ -13,7 +13,7 @@ struct Channel {
   std::unique_ptr<struct bufferevent, void (*)(struct bufferevent*)> bev;
   struct sockaddr_storage local_addr;
   struct sockaddr_storage remote_addr;
-  bool is_virtual_connection;
+  uint64_t virtual_network_id; // 0 = normal TCP connection
 
   Version version;
   uint8_t language;
@@ -50,6 +50,7 @@ struct Channel {
   // Creates a connected channel
   Channel(
       struct bufferevent* bev,
+      uint64_t virtual_network_id,
       Version version,
       uint8_t language,
       on_command_received_t on_command_received,
@@ -70,7 +71,7 @@ struct Channel {
       void* context_obj,
       const std::string& name = "");
 
-  void set_bufferevent(struct bufferevent* bev);
+  void set_bufferevent(struct bufferevent* bev, uint64_t virtual_network_id);
 
   inline bool connected() const {
     return this->bev.get() != nullptr;

src/ChatCommands.cc

@@ -38,11 +38,11 @@ private:
   std::string user_msg;
 };
 
-static void check_license_flag(shared_ptr<Client> c, License::Flag flag) {
-  if (!c->license) {
+static void check_account_flag(shared_ptr<Client> c, Account::Flag flag) {
+  if (!c->login) {
     throw precondition_failed("$C6You are not\nlogged in.");
   }
-  if (!c->license->check_flag(flag)) {
+  if (!c->login->account->check_flag(flag)) {
     throw precondition_failed("$C6You do not have\npermission to\nrun this command.");
   }
 }
@@ -65,21 +65,29 @@ static void check_is_ep3(shared_ptr<Client> c, bool is_ep3) {
   }
 }
 
+static void check_debug_enabled(shared_ptr<Client> c) {
+  if (!c->config.check_flag(Client::Flag::DEBUG_ENABLED)) {
+    throw precondition_failed("$C6This command can only\nbe run in debug mode\n(run %sdebug first)");
+  }
+}
+
 static void check_cheats_enabled(shared_ptr<Lobby> l, shared_ptr<Client> c) {
-  if (!l->check_flag(Lobby::Flag::CHEATS_ENABLED) && !c->license->check_flag(License::Flag::CHEAT_ANYWHERE)) {
+  if (!l->check_flag(Lobby::Flag::CHEATS_ENABLED) &&
+      !c->login->account->check_flag(Account::Flag::CHEAT_ANYWHERE)) {
     throw precondition_failed("$C6This command can\nonly be used in\ncheat mode.");
   }
 }
 
 static void check_cheats_allowed(shared_ptr<ServerState> s, shared_ptr<Client> c) {
-  if ((s->cheat_mode_behavior == ServerState::BehaviorSwitch::OFF) && !c->license->check_flag(License::Flag::CHEAT_ANYWHERE)) {
+  if ((s->cheat_mode_behavior == ServerState::BehaviorSwitch::OFF) &&
+      !c->login->account->check_flag(Account::Flag::CHEAT_ANYWHERE)) {
     throw precondition_failed("$C6Cheats are disabled\non this server.");
   }
 }
 
 static void check_cheats_allowed(shared_ptr<ServerState> s, shared_ptr<ProxyServer::LinkedSession> ses) {
   if ((s->cheat_mode_behavior == ServerState::BehaviorSwitch::OFF) &&
-      (!ses->license || !ses->license->check_flag(License::Flag::CHEAT_ANYWHERE))) {
+      (!ses->login || !ses->login->account->check_flag(Account::Flag::CHEAT_ANYWHERE))) {
     throw precondition_failed("$C6Cheats are disabled\non this proxy.");
   }
 }
@@ -96,11 +104,8 @@ static void check_is_leader(shared_ptr<Lobby> l, shared_ptr<Client> c) {
 static void server_command_server_info(shared_ptr<Client> c, const std::string&) {
   auto s = c->require_server_state();
   string uptime_str = format_duration(now() - s->creation_time);
-  string build_date = format_time(BUILD_TIMESTAMP);
   send_text_message_printf(c,
-      "Revision: $C6%s$C7\n$C6%s$C7\nUptime: $C6%s$C7\nLobbies: $C6%zu$C7\nClients: $C6%zu$C7(g) $C6%zu$C7(p)",
-      GIT_REVISION_HASH,
-      build_date.c_str(),
+      "Uptime: $C6%s$C7\nLobbies: $C6%zu$C7\nClients: $C6%zu$C7(g) $C6%zu$C7(p)",
       uptime_str.c_str(),
       s->id_to_lobby.size(),
       s->channel_to_client.size(),
@@ -122,7 +127,7 @@ static void server_command_lobby_info(shared_ptr<Client> c, const std::string&)
         } else {
           lines.emplace_back(string_printf("$C6%08X$C7 L$C6%d-%d$C7", l->lobby_id, l->min_level + 1, l->max_level + 1));
         }
-        lines.emplace_back(string_printf("$C7Section ID: $C6%s$C7", name_for_section_id(l->section_id)));
+        lines.emplace_back(string_printf("$C7Section ID: $C6%s$C7", name_for_section_id(l->effective_section_id())));
 
         switch (l->drop_mode) {
           case Lobby::DropMode::DISABLED:
@@ -258,16 +263,22 @@ static void proxy_command_lobby_info(shared_ptr<ProxyServer::LinkedSession> ses,
 }
 
 static void server_command_ax(shared_ptr<Client> c, const std::string& args) {
-  check_license_flag(c, License::Flag::ANNOUNCE);
+  check_account_flag(c, Account::Flag::ANNOUNCE);
   ax_messages_log.info("%s", args.c_str());
 }
 
 static void server_command_announce(shared_ptr<Client> c, const std::string& args) {
   auto s = c->require_server_state();
-  check_license_flag(c, License::Flag::ANNOUNCE);
+  check_account_flag(c, Account::Flag::ANNOUNCE);
   send_text_message(s, args);
 }
 
+static void server_command_announce_mail(shared_ptr<Client> c, const std::string& args) {
+  auto s = c->require_server_state();
+  check_account_flag(c, Account::Flag::ANNOUNCE);
+  send_simple_mail(s, 0, s->name, args);
+}
+
 static void server_command_arrow(shared_ptr<Client> c, const std::string& args) {
   auto l = c->require_lobby();
   c->lobby_arrow_color = stoull(args, nullptr, 0);
@@ -281,16 +292,13 @@ static void proxy_command_arrow(shared_ptr<ProxyServer::LinkedSession> ses, cons
 }
 
 static void server_command_debug(shared_ptr<Client> c, const std::string&) {
-  check_license_flag(c, License::Flag::DEBUG);
+  check_account_flag(c, Account::Flag::DEBUG);
   c->config.toggle_flag(Client::Flag::DEBUG_ENABLED);
   send_text_message_printf(c, "Debug %s", (c->config.check_flag(Client::Flag::DEBUG_ENABLED) ? "enabled" : "disabled"));
 }
 
 static void server_command_quest(shared_ptr<Client> c, const std::string& args) {
-  if (!c->config.check_flag(Client::Flag::DEBUG_ENABLED)) {
-    send_text_message(c, "$C6This command can only\nbe run in debug mode\n(run %sdebug first)");
-    return;
-  }
+  check_debug_enabled(c);
 
   Version effective_version = is_ep3(c->version()) ? Version::GC_V3 : c->version();
 
@@ -308,17 +316,142 @@ static void server_command_qcheck(shared_ptr<Client> c, const std::string& args)
   auto l = c->require_lobby();
   uint16_t flag_num = stoul(args, nullptr, 0);
 
-  send_text_message_printf(c, "$C7Quest flag 0x%hX (%hu)\nis %s on %s",
-      flag_num, flag_num,
-      c->character()->quest_flags.get(l->difficulty, flag_num) ? "set" : "not set",
-      name_for_difficulty(l->difficulty));
+  if (l->is_game()) {
+    if (!l->quest_flags_known || l->quest_flags_known->get(l->difficulty, flag_num)) {
+      send_text_message_printf(c, "$C7Game: flag 0x%hX (%hu)\nis %s on %s",
+          flag_num, flag_num,
+          c->character()->quest_flags.get(l->difficulty, flag_num) ? "set" : "not set",
+          name_for_difficulty(l->difficulty));
+    } else {
+      send_text_message_printf(c, "$C7Game: flag 0x%hX (%hu)\nis unknown on %s",
+          flag_num, flag_num, name_for_difficulty(l->difficulty));
+    }
+  } else if (c->version() == Version::BB_V4) {
+    send_text_message_printf(c, "$C7Player: flag 0x%hX (%hu)\nis %s on %s",
+        flag_num, flag_num,
+        c->character()->quest_flags.get(l->difficulty, flag_num) ? "set" : "not set",
+        name_for_difficulty(l->difficulty));
+  }
+}
+
+static void server_command_swset_swclear(shared_ptr<Client> c, const std::string& args, bool should_set) {
+  check_debug_enabled(c);
+  auto l = c->require_lobby();
+  if (!l->is_game()) {
+    send_text_message(c, "$C6This command cannot\nbe used in the lobby");
+    return;
+  }
+
+  auto tokens = split(args, ' ');
+  uint8_t floor, flag_num;
+  if (tokens.size() == 1) {
+    floor = c->floor;
+    flag_num = stoul(tokens[0], nullptr, 0);
+  } else if (tokens.size() == 2) {
+    floor = stoul(tokens[0], nullptr, 0);
+    flag_num = stoul(tokens[1], nullptr, 0);
+  } else {
+    send_text_message(c, "$C4Incorrect parameters");
+    return;
+  }
+
+  if (should_set) {
+    l->switch_flags->set(floor, flag_num);
+  } else {
+    l->switch_flags->clear(floor, flag_num);
+  }
+
+  uint8_t cmd_flags = should_set ? 0x01 : 0x00;
+  G_SwitchStateChanged_6x05 cmd = {{0x05, 0x03, 0xFFFF}, 0, 0, flag_num, floor, cmd_flags};
+  send_command_t(l, 0x60, 0x00, cmd);
+}
+
+static void server_command_swset(shared_ptr<Client> c, const std::string& args) {
+  return server_command_swset_swclear(c, args, true);
+}
+
+static void server_command_swclear(shared_ptr<Client> c, const std::string& args) {
+  return server_command_swset_swclear(c, args, false);
+}
+
+static void proxy_command_swset_swclear(shared_ptr<ProxyServer::LinkedSession> ses, const std::string& args, bool should_set) {
+  if (!ses->is_in_game) {
+    send_text_message(ses->client_channel, "$C6This command cannot\nbe used in the lobby");
+    return;
+  }
+
+  auto tokens = split(args, ' ');
+  uint8_t floor, flag_num;
+  if (tokens.size() == 1) {
+    floor = ses->floor;
+    flag_num = stoul(tokens[0], nullptr, 0);
+  } else if (tokens.size() == 2) {
+    floor = stoul(tokens[0], nullptr, 0);
+    flag_num = stoul(tokens[1], nullptr, 0);
+  } else {
+    send_text_message(ses->client_channel, "$C4Incorrect parameters");
+    return;
+  }
+
+  uint8_t cmd_flags = should_set ? 0x01 : 0x00;
+  G_SwitchStateChanged_6x05 cmd = {{0x05, 0x03, 0xFFFF}, 0, 0, flag_num, floor, cmd_flags};
+  ses->client_channel.send(0x60, 0x00, &cmd, sizeof(cmd));
+  ses->server_channel.send(0x60, 0x00, &cmd, sizeof(cmd));
+}
+
+static void proxy_command_swset(shared_ptr<ProxyServer::LinkedSession> ses, const std::string& args) {
+  return proxy_command_swset_swclear(ses, args, true);
+}
+
+static void proxy_command_swclear(shared_ptr<ProxyServer::LinkedSession> ses, const std::string& args) {
+  return proxy_command_swset_swclear(ses, args, false);
+}
+
+static void server_command_swsetall(shared_ptr<Client> c, const std::string&) {
+  check_debug_enabled(c);
+  auto l = c->require_lobby();
+  if (!l->is_game()) {
+    send_text_message(c, "$C6This command cannot\nbe used in the lobby");
+    return;
+  }
+
+  l->switch_flags->data[c->floor].clear(0xFF);
+
+  parray<G_SwitchStateChanged_6x05, 0x100> cmds;
+  for (size_t z = 0; z < cmds.size(); z++) {
+    auto& cmd = cmds[z];
+    cmd.header.subcommand = 0x05;
+    cmd.header.size = 0x03;
+    cmd.header.object_id = 0xFFFF;
+    cmd.switch_flag_floor = c->floor;
+    cmd.switch_flag_num = z;
+    cmd.flags = 0x01;
+  }
+  send_command_t(l, 0x6C, 0x00, cmds);
+}
+
+static void proxy_command_swsetall(shared_ptr<ProxyServer::LinkedSession> ses, const std::string&) {
+  if (!ses->is_in_game) {
+    send_text_message(ses->client_channel, "$C6This command cannot\nbe used in the lobby");
+    return;
+  }
+
+  parray<G_SwitchStateChanged_6x05, 0x100> cmds;
+  for (size_t z = 0; z < cmds.size(); z++) {
+    auto& cmd = cmds[z];
+    cmd.header.subcommand = 0x05;
+    cmd.header.size = 0x03;
+    cmd.header.object_id = 0xFFFF;
+    cmd.switch_flag_floor = ses->floor;
+    cmd.switch_flag_num = z;
+    cmd.flags = 0x01;
+  }
+  ses->client_channel.send(0x6C, 0x00, &cmds, sizeof(cmds));
+  ses->server_channel.send(0x6C, 0x00, &cmds, sizeof(cmds));
 }
 
 static void server_command_qset_qclear(shared_ptr<Client> c, const std::string& args, bool should_set) {
-  if (!c->config.check_flag(Client::Flag::DEBUG_ENABLED)) {
-    send_text_message(c, "$C6This command can only\nbe run in debug mode\n(run %sdebug first)");
-    return;
-  }
+  check_debug_enabled(c);
   auto l = c->require_lobby();
   if (!l->is_game()) {
     send_text_message(c, "$C6This command cannot\nbe used in the lobby");
@@ -327,10 +460,22 @@ static void server_command_qset_qclear(shared_ptr<Client> c, const std::string&
 
   uint16_t flag_num = stoul(args, nullptr, 0);
 
-  if (should_set) {
-    c->character()->quest_flags.set(l->difficulty, flag_num);
-  } else {
-    c->character()->quest_flags.clear(l->difficulty, flag_num);
+  if (l->is_game()) {
+    if (l->quest_flags_known) {
+      l->quest_flags_known->set(l->difficulty, flag_num);
+    }
+    if (should_set) {
+      l->quest_flag_values->set(l->difficulty, flag_num);
+    } else {
+      l->quest_flag_values->clear(l->difficulty, flag_num);
+    }
+  } else if (c->version() == Version::BB_V4) {
+    auto p = c->character();
+    if (should_set) {
+      p->quest_flags.set(l->difficulty, flag_num);
+    } else {
+      p->quest_flags.clear(l->difficulty, flag_num);
+    }
   }
 
   if (is_v1_or_v2(c->version())) {
@@ -362,7 +507,7 @@ static void proxy_command_qset_qclear(shared_ptr<ProxyServer::LinkedSession> ses
     ses->client_channel.send(0x60, 0x00, &cmd, sizeof(cmd));
     ses->server_channel.send(0x60, 0x00, &cmd, sizeof(cmd));
   } else {
-    G_UpdateQuestFlag_V3_BB_6x75 cmd = {{{0x75, 0x03, 0x0000}, flag_num, should_set ? 0 : 1}, ses->difficulty, 0x0000};
+    G_UpdateQuestFlag_V3_BB_6x75 cmd = {{{0x75, 0x03, 0x0000}, flag_num, should_set ? 0 : 1}, ses->lobby_difficulty, 0x0000};
     ses->client_channel.send(0x60, 0x00, &cmd, sizeof(cmd));
     ses->server_channel.send(0x60, 0x00, &cmd, sizeof(cmd));
   }
@@ -392,10 +537,7 @@ static void server_command_qgwrite(shared_ptr<Client> c, const std::string& args
     send_text_message(c, "$C6This command can\nonly be used on BB");
     return;
   }
-  if (!c->config.check_flag(Client::Flag::DEBUG_ENABLED)) {
-    send_text_message(c, "$C6This command can only\nbe run in debug mode\n(run %sdebug first)");
-    return;
-  }
+  check_debug_enabled(c);
   auto l = c->require_lobby();
   if (!l->is_game()) {
     send_text_message(c, "$C6This command cannot\nbe used in the lobby");
@@ -422,10 +564,7 @@ static void server_command_qgwrite(shared_ptr<Client> c, const std::string& args
 }
 
 static void server_command_qsync_qsyncall(shared_ptr<Client> c, const std::string& args, bool send_to_lobby) {
-  if (!c->config.check_flag(Client::Flag::DEBUG_ENABLED)) {
-    send_text_message(c, "$C6This command can only\nbe run in debug mode\n(run %sdebug first)");
-    return;
-  }
+  check_debug_enabled(c);
   auto l = c->require_lobby();
   if (!l->is_game()) {
     send_text_message(c, "$C6This command cannot\nbe used in the lobby");
@@ -504,10 +643,7 @@ static void proxy_command_qsyncall(shared_ptr<ProxyServer::LinkedSession> ses, c
 }
 
 static void server_command_qcall(shared_ptr<Client> c, const std::string& args) {
-  if (!c->config.check_flag(Client::Flag::DEBUG_ENABLED)) {
-    send_text_message(c, "$C6This command can only\nbe run in debug mode\n(run %sdebug first)");
-    return;
-  }
+  check_debug_enabled(c);
 
   auto l = c->require_lobby();
   if (l->is_game() && l->quest) {
@@ -544,7 +680,7 @@ static void server_command_show_material_counts(shared_ptr<Client> c, const std:
 }
 
 static void server_command_auction(shared_ptr<Client> c, const std::string&) {
-  check_license_flag(c, License::Flag::DEBUG);
+  check_account_flag(c, Account::Flag::DEBUG);
   auto l = c->require_lobby();
   if (l->is_game() && l->is_ep3()) {
     G_InitiateCardAuction_Ep3_6xB5x42 cmd;
@@ -570,8 +706,7 @@ static void server_command_patch(shared_ptr<Client> c, const std::string& args)
       auto s = c->require_server_state();
       // Note: We can't look this up outside of the closure because
       // c->specific_version can change during prepare_client_for_patches
-      auto fn = s->function_code_index->name_and_specific_version_to_patch_function.at(
-          string_printf("%s-%08" PRIX32, args.c_str(), c->config.specific_version));
+      auto fn = s->function_code_index->get_patch(args, c->config.specific_version);
       send_function_call(c, fn);
       c->function_call_response_queue.emplace_back(empty_function_call_response_handler);
     } catch (const out_of_range&) {
@@ -590,8 +725,7 @@ static void proxy_command_patch(shared_ptr<ProxyServer::LinkedSession> ses, cons
         ses->log.info("Version detected as %08" PRIX32, ses->config.specific_version);
       }
       auto s = ses->require_server_state();
-      auto fn = s->function_code_index->name_and_specific_version_to_patch_function.at(
-          string_printf("%s-%08" PRIX32, args.c_str(), ses->config.specific_version));
+      auto fn = s->function_code_index->get_patch(args, ses->config.specific_version);
       send_function_call(ses->client_channel, ses->config, fn);
       // Don't forward the patch response to the server
       ses->function_call_return_handler_queue.emplace_back(empty_patch_return_handler);
@@ -657,6 +791,7 @@ static void server_command_exit(shared_ptr<Client> c, const std::string&) {
       G_UnusedHeader cmd = {0x73, 0x01, 0x0000};
       c->channel.send(0x60, 0x00, cmd);
       c->floor = 0;
+      c->recent_switch_flags.clear();
     } else if (is_ep3(c->version())) {
       c->channel.send(0xED, 0x00);
     } else {
@@ -716,11 +851,19 @@ static void proxy_command_get_player_card(shared_ptr<ProxyServer::LinkedSession>
 }
 
 static void server_command_send_client(shared_ptr<Client> c, const std::string& args) {
+  check_debug_enabled(c);
   string data = parse_data_string(args);
   data.resize((data.size() + 3) & (~3));
   c->channel.send(data);
 }
 
+static void server_command_send_server(shared_ptr<Client> c, const std::string& args) {
+  check_debug_enabled(c);
+  string data = parse_data_string(args);
+  data.resize((data.size() + 3) & (~3));
+  on_command_with_header(c, data);
+}
+
 static void proxy_command_send_client(shared_ptr<ProxyServer::LinkedSession> ses, const std::string& args) {
   string data = parse_data_string(args);
   data.resize((data.size() + 3) & (~3));
@@ -733,6 +876,16 @@ static void proxy_command_send_server(shared_ptr<ProxyServer::LinkedSession> ses
   ses->server_channel.send(data);
 }
 
+static void server_command_send_both(shared_ptr<Client> c, const std::string& args) {
+  server_command_send_client(c, args);
+  server_command_send_server(c, args);
+}
+
+static void proxy_command_send_both(shared_ptr<ProxyServer::LinkedSession> ses, const std::string& args) {
+  proxy_command_send_client(ses, args);
+  proxy_command_send_server(ses, args);
+}
+
 ////////////////////////////////////////////////////////////////////////////////
 // Lobby commands
 
@@ -747,7 +900,7 @@ static void server_command_cheat(shared_ptr<Client> c, const std::string&) {
     l->toggle_flag(Lobby::Flag::CHEATS_ENABLED);
     send_text_message_printf(l, "Cheat mode %s", l->check_flag(Lobby::Flag::CHEATS_ENABLED) ? "enabled" : "disabled");
 
-    if (!c->license->check_flag(License::Flag::CHEAT_ANYWHERE)) {
+    if (!c->login->account->check_flag(Account::Flag::CHEAT_ANYWHERE)) {
       size_t default_min_level = s->default_min_level_for_game(l->base_version, l->episode, l->difficulty);
       if (l->min_level < default_min_level) {
         l->min_level = default_min_level;
@@ -760,7 +913,7 @@ static void server_command_cheat(shared_ptr<Client> c, const std::string&) {
 static void server_command_lobby_event(shared_ptr<Client> c, const std::string& args) {
   auto l = c->require_lobby();
   check_is_game(l, false);
-  check_license_flag(c, License::Flag::CHANGE_EVENT);
+  check_account_flag(c, Account::Flag::CHANGE_EVENT);
 
   uint8_t new_event = event_for_name(args);
   if (new_event == 0xFF) {
@@ -789,7 +942,7 @@ static void proxy_command_lobby_event(shared_ptr<ProxyServer::LinkedSession> ses
 }
 
 static void server_command_lobby_event_all(shared_ptr<Client> c, const std::string& args) {
-  check_license_flag(c, License::Flag::CHANGE_EVENT);
+  check_account_flag(c, Account::Flag::CHANGE_EVENT);
 
   uint8_t new_event = event_for_name(args);
   if (new_event == 0xFF) {
@@ -841,13 +994,13 @@ static void proxy_command_lobby_type(shared_ptr<ProxyServer::LinkedSession> ses,
   ses->config.override_lobby_number = new_type;
 }
 
-static string file_path_for_recording(const std::string& args, uint32_t serial_number) {
+static string file_path_for_recording(const std::string& args, uint32_t account_id) {
   for (char ch : args) {
     if (ch <= 0x20 || ch > 0x7E || ch == '/') {
       throw runtime_error("invalid recording name");
     }
   }
-  return string_printf("system/ep3/battle-records/%010" PRIu32 "_%s.mzrd", serial_number, args.c_str());
+  return string_printf("system/ep3/battle-records/%010" PRIu32 "_%s.mzrd", account_id, args.c_str());
 }
 
 static void server_command_saverec(shared_ptr<Client> c, const std::string& args) {
@@ -855,7 +1008,7 @@ static void server_command_saverec(shared_ptr<Client> c, const std::string& args
     send_text_message(c, "$C4No finished\nrecording is\npresent");
     return;
   }
-  string file_path = file_path_for_recording(args, c->license->serial_number);
+  string file_path = file_path_for_recording(args, c->login->account->account_id);
   string data = c->ep3_prev_battle_record->serialize();
   save_file(file_path, data);
   send_text_message(c, "$C7Recording saved");
@@ -872,7 +1025,7 @@ static void server_command_playrec(shared_ptr<Client> c, const std::string& args
   if (l->is_game() && l->battle_player) {
     l->battle_player->start();
   } else if (!l->is_game()) {
-    string file_path = file_path_for_recording(args, c->license->serial_number);
+    string file_path = file_path_for_recording(args, c->login->account->account_id);
 
     auto s = c->require_server_state();
     string filename = args;
@@ -906,17 +1059,14 @@ static void server_command_playrec(shared_ptr<Client> c, const std::string& args
 
 static void server_command_meseta(shared_ptr<Client> c, const std::string& args) {
   check_is_ep3(c, true);
-  if (!c->config.check_flag(Client::Flag::DEBUG_ENABLED)) {
-    send_text_message(c, "$C6This command can only\nbe run in debug mode\n(run %sdebug first)");
-    return;
-  }
+  check_debug_enabled(c);
 
   uint32_t amount = stoul(args, nullptr, 0);
-  c->license->ep3_current_meseta += amount;
-  c->license->ep3_total_meseta_earned += amount;
-  c->license->save();
+  c->login->account->ep3_current_meseta += amount;
+  c->login->account->ep3_total_meseta_earned += amount;
+  c->login->account->save();
   send_ep3_rank_update(c);
-  send_text_message_printf(c, "You now have\n$C6%" PRIu32 "$C7 Meseta", c->license->ep3_current_meseta);
+  send_text_message_printf(c, "You now have\n$C6%" PRIu32 "$C7 Meseta", c->login->account->ep3_current_meseta);
 }
 
 ////////////////////////////////////////////////////////////////////////////////
@@ -924,21 +1074,28 @@ static void server_command_meseta(shared_ptr<Client> c, const std::string& args)
 
 static void server_command_secid(shared_ptr<Client> c, const std::string& args) {
   auto l = c->require_lobby();
-  check_is_game(l, false);
   check_cheats_allowed(c->require_server_state(), c);
 
+  uint8_t new_override_section_id;
+
   if (!args[0]) {
-    c->config.override_section_id = 0xFF;
+    new_override_section_id = 0xFF;
     send_text_message(c, "$C6Override section ID\nremoved");
   } else {
-    uint8_t new_secid = section_id_for_name(args);
-    if (new_secid == 0xFF) {
+    new_override_section_id = section_id_for_name(args);
+    if (new_override_section_id == 0xFF) {
       send_text_message(c, "$C6Invalid section ID");
+      return;
     } else {
-      c->config.override_section_id = new_secid;
-      send_text_message_printf(c, "$C6Override section ID\nset to %s", name_for_section_id(new_secid));
+      send_text_message_printf(c, "$C6Override section ID\nset to %s", name_for_section_id(new_override_section_id));
     }
   }
+
+  c->config.override_section_id = new_override_section_id;
+  if (l->is_game() && (l->leader_id == c->lobby_client_id)) {
+    l->override_section_id = new_override_section_id;
+    l->change_section_id();
+  }
 }
 
 static void proxy_command_secid(shared_ptr<ProxyServer::LinkedSession> ses, const std::string& args) {
@@ -957,6 +1114,22 @@ static void proxy_command_secid(shared_ptr<ProxyServer::LinkedSession> ses, cons
   }
 }
 
+static void server_command_variations(shared_ptr<Client> c, const std::string& args) {
+  // Note: This command is intentionally undocumented, since it's primarily used
+  // for testing. If we ever make it public, we should add some kind of user
+  // feedback (currently it sends no message when it runs).
+  auto s = c->require_server_state();
+  auto l = c->require_lobby();
+  check_is_game(l, false);
+  check_cheats_allowed(s, c);
+
+  c->override_variations = make_unique<parray<le_uint32_t, 0x20>>();
+  c->override_variations->clear(0);
+  for (size_t z = 0; z < min<size_t>(c->override_variations->size(), args.size()); z++) {
+    c->override_variations->at(z) = args[z] - '0';
+  }
+}
+
 static void server_command_rand(shared_ptr<Client> c, const std::string& args) {
   auto s = c->require_server_state();
   auto l = c->require_lobby();
@@ -1044,7 +1217,8 @@ static void server_command_min_level(shared_ptr<Client> c, const std::string& ar
   size_t new_min_level = stoull(args) - 1;
 
   auto s = c->require_server_state();
-  bool cheats_allowed = (l->check_flag(Lobby::Flag::CHEATS_ENABLED) || c->license->check_flag(License::Flag::CHEAT_ANYWHERE));
+  bool cheats_allowed = (l->check_flag(Lobby::Flag::CHEATS_ENABLED) ||
+      c->login->account->check_flag(Account::Flag::CHEAT_ANYWHERE));
   if (!cheats_allowed) {
     size_t default_min_level = s->default_min_level_for_game(l->base_version, l->episode, l->difficulty);
     if (new_min_level < default_min_level) {
@@ -1086,7 +1260,7 @@ static void server_command_edit(shared_ptr<Client> c, const std::string& args) {
   }
 
   bool cheats_allowed = ((s->cheat_mode_behavior != ServerState::BehaviorSwitch::OFF) ||
-      c->license->check_flag(License::Flag::CHEAT_ANYWHERE));
+      c->login->account->check_flag(Account::Flag::CHEAT_ANYWHERE));
 
   string encoded_args = tolower(args);
   vector<string> tokens = split(encoded_args, ' ');
@@ -1113,12 +1287,24 @@ static void server_command_edit(shared_ptr<Client> c, const std::string& args) {
       p->disp.stats.experience = stoul(tokens.at(1));
     } else if (tokens.at(0) == "level" && cheats_allowed) {
       uint32_t level = stoul(tokens.at(1)) - 1;
-      p->disp.stats.reset_to_base(p->disp.visual.char_class, s->level_table);
-      p->disp.stats.advance_to_level(p->disp.visual.char_class, level, s->level_table);
+      s->level_table->reset_to_base(p->disp.stats, p->disp.visual.char_class);
+      s->level_table->advance_to_level(p->disp.stats, level, p->disp.visual.char_class);
     } else if (tokens.at(0) == "namecolor") {
       uint32_t new_color;
       sscanf(tokens.at(1).c_str(), "%8X", &new_color);
       p->disp.visual.name_color = new_color;
+    } else if (tokens.at(0) == "language" || tokens.at(0) == "lang") {
+      if (tokens.at(1).size() != 1) {
+        throw runtime_error("invalid language");
+      }
+      uint8_t new_language = language_code_for_char(tokens.at(1).at(0));
+      c->channel.language = new_language;
+      p->inventory.language = new_language;
+      p->guild_card.language = new_language;
+      auto sys = c->system_file(false);
+      if (sys) {
+        sys->base.language = new_language;
+      }
     } else if (tokens.at(0) == "secid" && cheats_allowed) {
       uint8_t secid = section_id_for_name(tokens.at(1));
       if (secid == 0xFF) {
@@ -1129,9 +1315,7 @@ static void server_command_edit(shared_ptr<Client> c, const std::string& args) {
       }
     } else if (tokens.at(0) == "name") {
       vector<string> orig_tokens = split(args, ' ');
-      string name = ((p->inventory.language == 0) ? "\tE" : "\tJ") + orig_tokens.at(1);
-      p->disp.name.clear();
-      p->disp.name.encode(name, p->inventory.language);
+      p->disp.name.encode(orig_tokens.at(1), p->inventory.language);
     } else if (tokens.at(0) == "npc") {
       if (tokens.at(1) == "none") {
         p->disp.visual.extra_model = 0;
@@ -1220,7 +1404,6 @@ static void server_command_bbchar_savechar(shared_ptr<Client> c, const std::stri
   check_is_game(l, false);
 
   auto pending_export = make_unique<Client::PendingCharacterExport>();
-  pending_export->is_bb_conversion = is_bb_conversion;
 
   if (is_bb_conversion) {
     vector<string> tokens = split(args, ' ');
@@ -1237,7 +1420,9 @@ static void server_command_bbchar_savechar(shared_ptr<Client> c, const std::stri
     }
 
     try {
-      c->pending_character_export->license = s->license_index->verify_bb(tokens[0].c_str(), tokens[1].c_str());
+      auto dest_login = s->account_index->from_bb_credentials(tokens[0], &tokens[1], false);
+      pending_export->dest_account = dest_login->account;
+      pending_export->dest_bb_license = dest_login->bb_license;
     } catch (const exception& e) {
       send_text_message_printf(c, "$C6Login failed: %s", e.what());
       return;
@@ -1249,13 +1434,14 @@ static void server_command_bbchar_savechar(shared_ptr<Client> c, const std::stri
       send_text_message(c, "$C6Player index must\nbe in range 1-4");
       return;
     }
-    pending_export->license = c->license;
+    pending_export->dest_account = c->login->account;
   }
 
   c->pending_character_export = std::move(pending_export);
-  // Request the player data. The client will respond with a 61, and the handler
-  // for that command will execute the conversion
-  send_get_player_info(c);
+
+  // Request the player data. The client will respond with a 61 or 30, and the
+  // handler for either of those commands will execute the conversion
+  send_get_player_info(c, true);
 }
 
 static void server_command_bbchar(shared_ptr<Client> c, const std::string& args) {
@@ -1263,8 +1449,8 @@ static void server_command_bbchar(shared_ptr<Client> c, const std::string& args)
 }
 
 static void server_command_savechar(shared_ptr<Client> c, const std::string& args) {
-  if (c->license->check_flag(License::Flag::IS_SHARED_SERIAL)) {
-    send_text_message(c, "$C7This command cannot\nbe used on a shared\nserial number");
+  if (c->login->account->check_flag(Account::Flag::IS_SHARED_ACCOUNT)) {
+    send_text_message(c, "$C7This command cannot\nbe used on a shared\naccount");
     return;
   }
   server_command_bbchar_savechar(c, args, false);
@@ -1275,8 +1461,8 @@ static void server_command_loadchar(shared_ptr<Client> c, const std::string& arg
     send_text_message(c, "$C7This command can only\nbe used on v1 or v2");
     return;
   }
-  if (c->license->check_flag(License::Flag::IS_SHARED_SERIAL)) {
-    send_text_message(c, "$C7This command cannot\nbe used on a shared\nserial number");
+  if (c->login->account->check_flag(Account::Flag::IS_SHARED_ACCOUNT)) {
+    send_text_message(c, "$C7This command cannot\nbe used on a shared\naccount");
     return;
   }
   auto l = c->require_lobby();
@@ -1287,7 +1473,7 @@ static void server_command_loadchar(shared_ptr<Client> c, const std::string& arg
     send_text_message(c, "$C6Player index must\nbe in range 1-4");
     return;
   }
-  c->load_backup_character(c->license->serial_number, index);
+  c->load_backup_character(c->login->account->account_id, index);
 
   auto s = c->require_server_state();
   send_player_leave_notification(l, c->lobby_client_id);
@@ -1314,8 +1500,8 @@ static string name_for_client(shared_ptr<Client> c) {
     return escape_player_name(player->disp.name.decode(player->inventory.language));
   }
 
-  if (c->license.get()) {
-    return string_printf("SN:%" PRIu32, c->license->serial_number);
+  if (c->login) {
+    return string_printf("SN:%" PRIu32, c->login->account->account_id);
   }
 
   return "Player";
@@ -1324,16 +1510,16 @@ static string name_for_client(shared_ptr<Client> c) {
 static void server_command_silence(shared_ptr<Client> c, const std::string& args) {
   auto s = c->require_server_state();
   auto l = c->require_lobby();
-  check_license_flag(c, License::Flag::SILENCE_USER);
+  check_account_flag(c, Account::Flag::SILENCE_USER);
 
   auto target = s->find_client(&args);
-  if (!target->license) {
+  if (!target->login) {
     // this should be impossible, but I'll bet it's not actually
     send_text_message(c, "$C6Client not logged in");
     return;
   }
 
-  if (target->license->check_flag(License::Flag::SILENCE_USER)) {
+  if (target->login->account->check_flag(Account::Flag::SILENCE_USER)) {
     send_text_message(c, "$C6You do not have\nsufficient privileges.");
     return;
   }
@@ -1347,16 +1533,16 @@ static void server_command_silence(shared_ptr<Client> c, const std::string& args
 static void server_command_kick(shared_ptr<Client> c, const std::string& args) {
   auto s = c->require_server_state();
   auto l = c->require_lobby();
-  check_license_flag(c, License::Flag::KICK_USER);
+  check_account_flag(c, Account::Flag::KICK_USER);
 
   auto target = s->find_client(&args);
-  if (!target->license) {
+  if (!target->login) {
     // This should be impossible, but I'll bet it's not actually
     send_text_message(c, "$C6Client not logged in");
     return;
   }
 
-  if (target->license->check_flag(License::Flag::KICK_USER)) {
+  if (target->login->account->check_flag(Account::Flag::KICK_USER)) {
     send_text_message(c, "$C6You do not have\nsufficient privileges.");
     return;
   }
@@ -1370,7 +1556,7 @@ static void server_command_kick(shared_ptr<Client> c, const std::string& args) {
 static void server_command_ban(shared_ptr<Client> c, const std::string& args) {
   auto s = c->require_server_state();
   auto l = c->require_lobby();
-  check_license_flag(c, License::Flag::BAN_USER);
+  check_account_flag(c, Account::Flag::BAN_USER);
 
   size_t space_pos = args.find(' ');
   if (space_pos == string::npos) {
@@ -1380,13 +1566,13 @@ static void server_command_ban(shared_ptr<Client> c, const std::string& args) {
 
   string identifier = args.substr(space_pos + 1);
   auto target = s->find_client(&identifier);
-  if (!target->license) {
+  if (!target->login) {
     // This should be impossible, but I'll bet it's not actually
     send_text_message(c, "$C6Client not logged in");
     return;
   }
 
-  if (target->license->check_flag(License::Flag::BAN_USER)) {
+  if (target->login->account->check_flag(Account::Flag::BAN_USER)) {
     send_text_message(c, "$C6You do not have\nsufficient privileges.");
     return;
   }
@@ -1410,8 +1596,8 @@ static void server_command_ban(shared_ptr<Client> c, const std::string& args) {
     usecs *= 60 * 60 * 24 * 365;
   }
 
-  target->license->ban_end_time = now() + usecs;
-  target->license->save();
+  target->login->account->ban_end_time = now() + usecs;
+  target->login->account->save();
   send_message_box(target, "$C6You were banned by a moderator.");
   target->should_disconnect = true;
   string target_name = name_for_client(target);
@@ -1425,18 +1611,34 @@ static void server_command_warp(shared_ptr<Client> c, const std::string& args, b
   auto s = c->require_server_state();
   auto l = c->require_lobby();
   check_is_game(l, true);
-  check_cheats_enabled(l, c);
 
   uint32_t floor = stoul(args, nullptr, 0);
   if (c->floor == floor) {
     return;
   }
 
+  // Special case: $warp[me] 0 is allowed in boss arenas if the boss is already
+  // defeated, even if cheats are disabled. This is because if a player returns
+  // to a boss arena after a persistence gap in the game, the exit warp won't
+  // exist, so they need a way to get out.
+  bool should_check_cheats = is_warpall || (floor != 0) || !floor_is_boss_arena(l->episode, c->floor);
+  if (!should_check_cheats) {
+    for (const auto* event : l->map->get_events(c->floor)) {
+      if (!(event->flags & 0x18)) {
+        should_check_cheats = true;
+        break;
+      }
+    }
+  }
+  if (should_check_cheats) {
+    check_cheats_enabled(l, c);
+  }
+
   size_t limit = floor_limit_for_episode(l->episode);
   if (limit == 0) {
     return;
   } else if (floor > limit) {
-    send_text_message_printf(c, "$C6Area numbers must\nbe %zu or less.", limit);
+    send_text_message_printf(c, "$C6Area numbers must\nbe %zu or less", limit);
     return;
   }
 
@@ -1556,20 +1758,20 @@ static void proxy_command_song(shared_ptr<ProxyServer::LinkedSession> ses, const
 }
 
 static void command_item_notifs(Channel& ch, Client::Config& config, const std::string& args) {
-  if (args == "all" || args == "on") {
-    config.clear_flag(Client::Flag::RARE_DROP_NOTIFICATIONS_ENABLED);
-    config.set_flag(Client::Flag::ALL_DROP_NOTIFICATIONS_ENABLED);
+  if (args == "every" || args == "everything") {
+    config.set_drop_notification_mode(Client::ItemDropNotificationMode::ALL_ITEMS_INCLUDING_MESETA);
+    send_text_message_printf(ch, "$C6Notifications enabled\nfor all items and\nMeseta");
+  } else if (args == "all" || args == "on") {
+    config.set_drop_notification_mode(Client::ItemDropNotificationMode::ALL_ITEMS);
     send_text_message_printf(ch, "$C6Notifications enabled\nfor all items");
   } else if (args == "rare" || args == "rares") {
-    config.set_flag(Client::Flag::RARE_DROP_NOTIFICATIONS_ENABLED);
-    config.clear_flag(Client::Flag::ALL_DROP_NOTIFICATIONS_ENABLED);
+    config.set_drop_notification_mode(Client::ItemDropNotificationMode::RARES_ONLY);
     send_text_message_printf(ch, "$C6Notifications enabled\nfor rare items only");
   } else if (args == "none" || args == "off") {
-    config.clear_flag(Client::Flag::RARE_DROP_NOTIFICATIONS_ENABLED);
-    config.clear_flag(Client::Flag::ALL_DROP_NOTIFICATIONS_ENABLED);
+    config.set_drop_notification_mode(Client::ItemDropNotificationMode::NOTHING);
     send_text_message_printf(ch, "$C6Notifications disabled\nfor all items");
   } else {
-    send_text_message_printf(ch, "$C6You must specify\n$C6off$C7, $C6rare$C7, or $C6on$C7");
+    send_text_message_printf(ch, "$C6You must specify\n$C6off$C7, $C6rare$C7, $C6on$C7, or\n$C6everything$C7");
   }
 }
 
@@ -1590,7 +1792,7 @@ static void server_command_infinite_hp(shared_ptr<Client> c, const std::string&)
   c->config.toggle_flag(Client::Flag::INFINITE_HP_ENABLED);
   bool enabled = c->config.check_flag(Client::Flag::INFINITE_HP_ENABLED);
   send_text_message_printf(c, "$C6Infinite HP %s", enabled ? "enabled" : "disabled");
-  if (enabled && l->is_game() && is_v1_or_v2(c->version())) {
+  if (enabled && l->is_game()) {
     send_remove_conditions(c);
   }
 }
@@ -1601,7 +1803,7 @@ static void proxy_command_infinite_hp(shared_ptr<ProxyServer::LinkedSession> ses
   ses->config.toggle_flag(Client::Flag::INFINITE_HP_ENABLED);
   bool enabled = ses->config.check_flag(Client::Flag::INFINITE_HP_ENABLED);
   send_text_message_printf(ses->client_channel, "$C6Infinite HP %s", enabled ? "enabled" : "disabled");
-  if (enabled && ses->is_in_game && is_v1_or_v2(ses->version())) {
+  if (enabled && ses->is_in_game) {
     send_remove_conditions(ses->client_channel, ses->lobby_client_id);
     send_remove_conditions(ses->server_channel, ses->lobby_client_id);
   }
@@ -1708,6 +1910,51 @@ static void server_command_dropmode(shared_ptr<Client> c, const std::string& arg
   }
 }
 
+static void proxy_command_dropmode(shared_ptr<ProxyServer::LinkedSession> ses, const std::string& args) {
+  check_cheats_allowed(ses->require_server_state(), ses);
+
+  using DropMode = ProxyServer::LinkedSession::DropMode;
+  if (args.empty()) {
+    switch (ses->drop_mode) {
+      case DropMode::DISABLED:
+        send_text_message(ses->client_channel, "Drop mode: disabled");
+        break;
+      case DropMode::PASSTHROUGH:
+        send_text_message(ses->client_channel, "Drop mode: default");
+        break;
+      case DropMode::INTERCEPT:
+        send_text_message(ses->client_channel, "Drop mode: proxy");
+        break;
+    }
+
+  } else {
+    DropMode new_mode;
+    if ((args == "none") || (args == "disabled")) {
+      new_mode = DropMode::DISABLED;
+    } else if ((args == "default") || (args == "passthrough")) {
+      new_mode = DropMode::PASSTHROUGH;
+    } else if ((args == "proxy") || (args == "intercept")) {
+      new_mode = DropMode::INTERCEPT;
+    } else {
+      send_text_message(ses->client_channel, "Invalid drop mode");
+      return;
+    }
+
+    ses->set_drop_mode(new_mode);
+    switch (ses->drop_mode) {
+      case DropMode::DISABLED:
+        send_text_message(ses->client_channel, "Item drops disabled");
+        break;
+      case DropMode::PASSTHROUGH:
+        send_text_message(ses->client_channel, "Item drops changed\nto default mode");
+        break;
+      case DropMode::INTERCEPT:
+        send_text_message(ses->client_channel, "Item drops changed\nto proxy mode");
+        break;
+    }
+  }
+}
+
 static void server_command_item(shared_ptr<Client> c, const std::string& args) {
   auto s = c->require_server_state();
   auto l = c->require_lobby();
@@ -1766,10 +2013,7 @@ static void proxy_command_item(shared_ptr<ProxyServer::LinkedSession> ses, const
 }
 
 static void server_command_enable_ep3_battle_debug_menu(shared_ptr<Client> c, const std::string& args) {
-  if (!c->config.check_flag(Client::Flag::DEBUG_ENABLED)) {
-    send_text_message(c, "$C6This command can only\nbe run in debug mode\n(run %sdebug first)");
-    return;
-  }
+  check_debug_enabled(c);
 
   auto l = c->require_lobby();
   check_is_game(l, true);
@@ -1817,7 +2061,7 @@ static void server_command_ep3_infinite_time(shared_ptr<Client> c, const std::st
   send_text_message(l, infinite_time_enabled ? "$C6Infinite time enabled" : "$C6Infinite time disabled");
 }
 
-static void server_command_ep3_set_def_dice_range(shared_ptr<Client> c, const std::string& args) {
+static void server_command_ep3_set_dice_range(shared_ptr<Client> c, const std::string& args) {
   auto l = c->require_lobby();
   check_is_game(l, true);
   check_is_ep3(c, true);
@@ -1835,37 +2079,63 @@ static void server_command_ep3_set_def_dice_range(shared_ptr<Client> c, const st
     return;
   }
   if (l->tournament_match) {
-    send_text_message(c, "$C6Cannot override\nDEF range in a\ntournament");
+    send_text_message(c, "$C6Cannot override\ndice ranges in a\ntournament");
     return;
   }
 
-  if (args.empty()) {
-    l->ep3_server->map_and_rules->rules.def_dice_range = 0;
-    send_text_message_printf(l, "$C6DEF dice range\nset to default");
-  } else {
-    uint8_t min_dice, max_dice;
-    auto tokens = split(args, '-');
+  auto parse_dice_range = +[](const string& spec) -> uint8_t {
+    auto tokens = split(spec, '-');
     if (tokens.size() == 1) {
-      min_dice = stoul(tokens[0]);
-      max_dice = min_dice;
+      uint8_t v = stoull(spec);
+      return (v << 4) | (v & 0x0F);
     } else if (tokens.size() == 2) {
-      min_dice = stoul(tokens[0]);
-      max_dice = stoul(tokens[1]);
+      return (stoull(tokens[0]) << 4) | (stoull(tokens[1]) & 0x0F);
     } else {
-      send_text_message(c, "$C6Specify DEF dice\nrange as MIN-MAX");
+      throw runtime_error("invalid dice spec format");
+    }
+  };
+
+  uint8_t def_dice_range = 0;
+  uint8_t atk_dice_range_2v1 = 0;
+  uint8_t def_dice_range_2v1 = 0;
+  for (const auto& spec : split(args, ' ')) {
+    auto tokens = split(spec, ':');
+    if (tokens.size() != 2) {
+      send_text_message(c, "$C6Invalid dice spec\nformat");
       return;
     }
-    if (min_dice == 0 || min_dice > 9 || max_dice == 0 || max_dice > 9) {
-      send_text_message(c, "$C6DEF dice must be\nin range 1-9");
-      return;
+    if (tokens[0] == "d") {
+      def_dice_range = parse_dice_range(tokens[1]);
+    } else if (tokens[0] == "1") {
+      atk_dice_range_2v1 = parse_dice_range(tokens[1]);
+      def_dice_range_2v1 = atk_dice_range_2v1;
+    } else if (tokens[0] == "a1") {
+      atk_dice_range_2v1 = parse_dice_range(tokens[1]);
+    } else if (tokens[0] == "d1") {
+      def_dice_range_2v1 = parse_dice_range(tokens[1]);
     }
-    if (min_dice > max_dice) {
-      uint8_t t = min_dice;
-      min_dice = max_dice;
-      max_dice = t;
+  }
+
+  l->ep3_server->def_dice_value_range_override = def_dice_range;
+  l->ep3_server->atk_dice_value_range_2v1_override = atk_dice_range_2v1;
+  l->ep3_server->def_dice_value_range_2v1_override = def_dice_range_2v1;
+
+  if (!def_dice_range && !atk_dice_range_2v1 && !def_dice_range_2v1) {
+    send_text_message_printf(l, "$C7Dice ranges reset\nto defaults");
+  } else {
+    send_text_message_printf(l, "$C7Dice ranges changed:");
+    if (def_dice_range) {
+      send_text_message_printf(l, "$C7DEF: $C6%hhu-%hhu",
+          static_cast<uint8_t>(def_dice_range >> 4), static_cast<uint8_t>(def_dice_range & 0x0F));
+    }
+    if (atk_dice_range_2v1) {
+      send_text_message_printf(l, "$C7ATK (1p in 2v1): $C6%hhu-%hhu",
+          static_cast<uint8_t>(atk_dice_range_2v1 >> 4), static_cast<uint8_t>(atk_dice_range_2v1 & 0x0F));
+    }
+    if (def_dice_range_2v1) {
+      send_text_message_printf(l, "$C7DEF (1p in 2v1): $C6%hhu-%hhu",
+          static_cast<uint8_t>(def_dice_range_2v1 >> 4), static_cast<uint8_t>(def_dice_range_2v1 & 0x0F));
     }
-    l->ep3_server->map_and_rules->rules.def_dice_range = ((min_dice << 4) & 0xF0) | (max_dice & 0x0F);
-    send_text_message_printf(l, "$C6DEF dice range\nset to %hhu-%hhu", min_dice, max_dice);
   }
 }
 
@@ -2059,6 +2329,7 @@ struct ChatCommandDefinition {
 static const unordered_map<string, ChatCommandDefinition> chat_commands({
     {"$allevent", {server_command_lobby_event_all, nullptr}},
     {"$ann", {server_command_announce, nullptr}},
+    {"$ann!", {server_command_announce_mail, nullptr}},
     {"$arrow", {server_command_arrow, proxy_command_arrow}},
     {"$auction", {server_command_auction, proxy_command_auction}},
     {"$ax", {server_command_ax, nullptr}},
@@ -2067,8 +2338,8 @@ static const unordered_map<string, ChatCommandDefinition> chat_commands({
     {"$bbchar", {server_command_bbchar, nullptr}},
     {"$cheat", {server_command_cheat, nullptr}},
     {"$debug", {server_command_debug, nullptr}},
-    {"$defrange", {server_command_ep3_set_def_dice_range, nullptr}},
-    {"$dropmode", {server_command_dropmode, nullptr}},
+    {"$dicerange", {server_command_ep3_set_dice_range, nullptr}},
+    {"$dropmode", {server_command_dropmode, proxy_command_dropmode}},
     {"$edit", {server_command_edit, nullptr}},
     {"$ep3battledebug", {server_command_enable_ep3_battle_debug_menu, nullptr}},
     {"$event", {server_command_lobby_event, proxy_command_lobby_event}},
@@ -2107,6 +2378,7 @@ static const unordered_map<string, ChatCommandDefinition> chat_commands({
     {"$save", {server_command_save, nullptr}},
     {"$savechar", {server_command_savechar, nullptr}},
     {"$saverec", {server_command_saverec, nullptr}},
+    {"$sb", {server_command_send_both, proxy_command_send_both}},
     {"$sc", {server_command_send_client, proxy_command_send_client}},
     {"$secid", {server_command_secid, proxy_command_secid}},
     {"$setassist", {server_command_ep3_replace_assist_card, nullptr}},
@@ -2114,11 +2386,15 @@ static const unordered_map<string, ChatCommandDefinition> chat_commands({
     {"$silence", {server_command_silence, nullptr}},
     {"$song", {server_command_song, proxy_command_song}},
     {"$spec", {server_command_toggle_spectator_flag, nullptr}},
-    {"$ss", {nullptr, proxy_command_send_server}},
+    {"$ss", {server_command_send_server, proxy_command_send_server}},
     {"$stat", {server_command_get_ep3_battle_stat, nullptr}},
     {"$surrender", {server_command_surrender, nullptr}},
     {"$swa", {server_command_switch_assist, proxy_command_switch_assist}},
+    {"$swclear", {server_command_swclear, proxy_command_swclear}},
+    {"$swset", {server_command_swset, proxy_command_swset}},
+    {"$swsetall", {server_command_swsetall, proxy_command_swsetall}},
     {"$unset", {server_command_ep3_unset_field_character, nullptr}},
+    {"$variations", {server_command_variations, nullptr}},
     {"$warp", {server_command_warpme, proxy_command_warpme}},
     {"$warpme", {server_command_warpme, proxy_command_warpme}},
     {"$warpall", {server_command_warpall, proxy_command_warpall}},

src/ChoiceSearch.hh

@@ -10,12 +10,16 @@
 
 class Client;
 
-struct ChoiceSearchConfig {
-  le_uint32_t disabled = 1; // 0 = enabled, 1 = disabled. Unused in command C3
+template <bool IsBigEndian>
+struct ChoiceSearchConfigT {
+  using U16T = typename std::conditional<IsBigEndian, be_uint16_t, le_uint16_t>::type;
+  using U32T = typename std::conditional<IsBigEndian, be_uint32_t, le_uint32_t>::type;
+
+  U32T disabled = 1; // 0 = enabled, 1 = disabled. Unused in command C3
   struct Entry {
-    le_uint16_t parent_choice_id = 0;
-    le_uint16_t choice_id = 0;
-  } __attribute__((packed));
+    U16T parent_choice_id = 0;
+    U16T choice_id = 0;
+  } __packed_ws__(Entry, 4);
   parray<Entry, 5> entries;
 
   int32_t get_setting(uint16_t parent_choice_id) const {
@@ -26,7 +30,24 @@ struct ChoiceSearchConfig {
     }
     return -1;
   }
-} __attribute__((packed));
+
+  operator ChoiceSearchConfigT<!IsBigEndian>() const {
+    ChoiceSearchConfigT<!IsBigEndian> ret;
+    ret.disabled = this->disabled.load();
+    for (size_t z = 0; z < this->entries.size(); z++) {
+      auto& ret_e = ret.entries[z];
+      const auto& this_e = this->entries[z];
+      ret_e.parent_choice_id = this_e.parent_choice_id.load();
+      ret_e.choice_id = this_e.choice_id.load();
+    }
+    return ret;
+  }
+} __packed__;
+
+using ChoiceSearchConfig = ChoiceSearchConfigT<false>;
+using ChoiceSearchConfigBE = ChoiceSearchConfigT<true>;
+check_struct_size(ChoiceSearchConfig, 0x18);
+check_struct_size(ChoiceSearchConfigBE, 0x18);
 
 struct ChoiceSearchCategory {
   struct Choice {

src/Client.cc

@@ -11,6 +11,7 @@
 #include <phosg/Network.hh>
 #include <phosg/Time.hh>
 
+#include "IPStackSimulator.hh"
 #include "Loggers.hh"
 #include "Server.hh"
 #include "Version.hh"
@@ -44,6 +45,9 @@ void Client::Config::set_flags_for_version(Version version, int64_t sub_version)
         case Version::DC_NTE:
         case Version::DC_V1_11_2000_PROTOTYPE:
         case Version::DC_V1:
+          this->set_flag(Flag::NO_D6);
+          this->set_flag(Flag::NO_SEND_FUNCTION_CALL);
+          break;
         case Version::DC_V2:
           this->set_flag(Flag::NO_D6);
           this->set_flag(Flag::SEND_FUNCTION_CALL_NO_CACHE_PATCH);
@@ -73,16 +77,16 @@ void Client::Config::set_flags_for_version(Version version, int64_t sub_version)
     case 0x21: // DCv1 US
       this->set_flag(Flag::NO_D6);
       this->set_flag(Flag::NO_SEND_FUNCTION_CALL);
-      // In the case of DCNTE, the IS_DC_TRIAL_EDITION flag is already set when
-      // we get here
       break;
-    case 0x23: // DCv1 EU
+    case 0x22: // DCv1 EU 50Hz (presumably)
+    case 0x23: // DCv1 EU 60Hz (presumably)
       this->set_flag(Flag::NO_D6);
       this->set_flag(Flag::NO_SEND_FUNCTION_CALL);
       break;
     case 0x25: // DCv2 JP
     case 0x26: // DCv2 US
-    case 0x28: // DCv2 EU
+    case 0x27: // DCv2 EU 50Hz (presumably)
+    case 0x28: // DCv2 EU 60Hz (presumably)
       this->set_flag(Flag::NO_D6);
       this->set_flag(Flag::SEND_FUNCTION_CALL_NO_CACHE_PATCH);
       break;
@@ -93,13 +97,10 @@ void Client::Config::set_flags_for_version(Version version, int64_t sub_version)
       break;
     case 0x30: // GC Ep1&2 GameJam demo, GC Ep1&2 Trial Edition, GC Ep1&2 JP v1.2, at least one version of XB
     case 0x31: // GC Ep1&2 US v1.0, GC US v1.1, XB US
-    case 0x34: // GC Ep1&2 JP v1.3
-      // In the case of GC Trial Edition, the IS_GC_TRIAL_EDITION flag is
-      // already set when we get here (because the client has used V2 encryption
-      // instead of V3)
       break;
     case 0x32: // GC Ep1&2 EU 50Hz
     case 0x33: // GC Ep1&2 EU 60Hz
+    case 0x34: // GC Ep1&2 JP v1.3
       this->set_flag(Flag::NO_D6_AFTER_LOBBY);
       break;
     case 0x35: // GC Ep1&2 JP v1.4 (Plus)
@@ -117,8 +118,7 @@ void Client::Config::set_flags_for_version(Version version, int64_t sub_version)
       this->set_flag(Flag::ENCRYPTED_SEND_FUNCTION_CALL);
       this->set_flag(Flag::SEND_FUNCTION_CALL_NO_CACHE_PATCH);
       // sub_version can't be used to tell JP final and Trial Edition apart; we
-      // instead look at header.flag in the 61 command and set the
-      // IS_EP3_TRIAL_EDITION flag there.
+      // instead look at header.flag in the 61 command and set the version then.
       break;
     case 0x41: // GC Ep3 US (and BB)
       this->set_flag(Flag::NO_D6_AFTER_LOBBY);
@@ -135,6 +135,26 @@ void Client::Config::set_flags_for_version(Version version, int64_t sub_version)
   }
 }
 
+Client::ItemDropNotificationMode Client::Config::get_drop_notification_mode() const {
+  uint8_t mode_s = (this->check_flag(Flag::ITEM_DROP_NOTIFICATIONS_1) ? 1 : 0) |
+      (this->check_flag(Flag::ITEM_DROP_NOTIFICATIONS_2) ? 2 : 0);
+  return static_cast<Client::ItemDropNotificationMode>(mode_s);
+}
+
+void Client::Config::set_drop_notification_mode(ItemDropNotificationMode new_mode) {
+  uint8_t mode_s = static_cast<uint8_t>(new_mode);
+  if (mode_s & 1) {
+    this->set_flag(Client::Flag::ITEM_DROP_NOTIFICATIONS_1);
+  } else {
+    this->clear_flag(Client::Flag::ITEM_DROP_NOTIFICATIONS_1);
+  }
+  if (mode_s & 2) {
+    this->set_flag(Client::Flag::ITEM_DROP_NOTIFICATIONS_2);
+  } else {
+    this->clear_flag(Client::Flag::ITEM_DROP_NOTIFICATIONS_2);
+  }
+}
+
 bool Client::Config::should_update_vs(const Config& other) const {
   constexpr uint64_t mask = static_cast<uint64_t>(Flag::CLIENT_SIDE_MASK);
   return ((this->enabled_flags ^ other.enabled_flags) & mask) ||
@@ -150,12 +170,13 @@ bool Client::Config::should_update_vs(const Config& other) const {
 Client::Client(
     shared_ptr<Server> server,
     struct bufferevent* bev,
+    uint64_t virtual_network_id,
     Version version,
     ServerBehavior server_behavior)
     : server(server),
       id(next_id++),
       log(string_printf("[C-%" PRIX64 "] ", this->id), client_log.min_level),
-      channel(bev, version, 1, nullptr, nullptr, this, string_printf("C-%" PRIX64, this->id), TerminalFormat::FG_YELLOW, TerminalFormat::FG_GREEN),
+      channel(bev, virtual_network_id, version, 1, nullptr, nullptr, this, "", TerminalFormat::FG_YELLOW, TerminalFormat::FG_GREEN),
       server_behavior(server_behavior),
       should_disconnect(false),
       should_send_to_lobby_server(false),
@@ -187,6 +208,7 @@ Client::Client(
       card_battle_table_number(-1),
       card_battle_table_seat_number(0),
       card_battle_table_seat_state(0),
+      last_game_info_requested(0),
       should_update_play_time(false),
       bb_character_index(-1),
       next_exp_value(0),
@@ -194,14 +216,15 @@ Client::Client(
       dol_base_addr(0),
       external_bank_character_index(-1),
       last_play_time_update(0) {
+  this->update_channel_name();
+
   this->config.set_flags_for_version(version, -1);
   auto s = server->get_state();
   if (is_v1_or_v2(this->version()) ? s->default_rare_notifs_enabled_v1_v2 : s->default_rare_notifs_enabled_v3_v4) {
-    this->config.set_flag(Flag::RARE_DROP_NOTIFICATIONS_ENABLED);
+    this->config.set_drop_notification_mode(ItemDropNotificationMode::RARES_ONLY);
   }
   this->config.specific_version = default_specific_version_for_version(version, -1);
 
-  this->last_switch_enabled_command.header.subcommand = 0;
   memset(&this->next_connection_addr, 0, sizeof(this->next_connection_addr));
 
   this->reschedule_save_game_data_event();
@@ -233,6 +256,19 @@ Client::~Client() {
   this->log.info("Deleted");
 }
 
+void Client::update_channel_name() {
+  string ip_str = this->require_server_state()->format_address_for_channel_name(
+      this->channel.remote_addr, this->channel.virtual_network_id);
+
+  auto player = this->character(false, false);
+  if (player) {
+    string name_str = player->disp.name.decode(this->language());
+    this->channel.name = string_printf("C-%" PRIX64 " (%s) @ %s", this->id, name_str.c_str(), ip_str.c_str());
+  } else {
+    this->channel.name = string_printf("C-%" PRIX64 " @ %s", this->id, ip_str.c_str());
+  }
+}
+
 void Client::reschedule_save_game_data_event() {
   if (this->version() == Version::BB_V4) {
     struct timeval tv = usecs_to_timeval(60000000); // 1 minute
@@ -248,33 +284,16 @@ void Client::reschedule_ping_and_timeout_events() {
   event_add(this->idle_timeout_event.get(), &idle_tv);
 }
 
-void Client::set_license(shared_ptr<License> l) {
-  if (this->version() == Version::BB_V4) {
-    // Make sure bb_username is filename-safe
-    for (char ch : l->bb_username) {
-      if (!isalnum(ch) && (ch != '-') && (ch != '_')) {
-        throw runtime_error("invalid characters in username");
-      }
-    }
-  }
-  this->license = l;
-}
-
-void Client::convert_license_to_temporary_if_nte() {
-  // If the session is a prototype version and the license was created and we
-  // should use a temporary license instead, delete the permanent license and
-  // replace it with a temporary license.
+void Client::convert_account_to_temporary_if_nte() {
+  // If the session is a prototype version and the account was created and we
+  // should use a temporary account instead, delete the permanent account and
+  // replace it with a temporary account.
   auto s = this->require_server_state();
-  if (s->use_temp_licenses_for_prototypes &&
-      this->config.check_flag(Client::Flag::LICENSE_WAS_CREATED) &&
-      is_any_nte(this->version())) {
-    this->log.info("Client is a prototype version and the license was created during this session; converting permanent license to temporary license");
-    s->license_index->remove(this->license->serial_number);
-    auto new_l = s->license_index->create_temporary_license();
-    this->license->delete_file();
-    *new_l = std::move(*this->license);
-    this->set_license(new_l);
-    this->config.clear_flag(Client::Flag::LICENSE_WAS_CREATED);
+  if (s->use_temp_accounts_for_prototypes && this->login->account_was_created && is_any_nte(this->version())) {
+    this->log.info("Client is a prototype version and the account was created during this session; converting permanent account to temporary account");
+    this->login->account->is_temporary = true;
+    this->login->account->delete_file();
+    this->login->account_was_created = false;
   }
 }
 
@@ -295,29 +314,29 @@ shared_ptr<Lobby> Client::require_lobby() const {
 }
 
 shared_ptr<const TeamIndex::Team> Client::team() const {
-  if (!this->license) {
-    throw logic_error("Client::team called on client with no license");
+  if (!this->login) {
+    throw logic_error("Client::team called on client with no account");
   }
 
-  if (this->license->bb_team_id == 0) {
+  if (this->login->account->bb_team_id == 0) {
     return nullptr;
   }
 
   auto p = this->character(false);
   auto s = this->require_server_state();
-  auto team = s->team_index->get_by_id(this->license->bb_team_id);
+  auto team = s->team_index->get_by_id(this->login->account->bb_team_id);
   if (!team) {
-    this->log.info("License contains a team ID, but the team does not exist; clearing team ID from license");
-    this->license->bb_team_id = 0;
-    this->license->save();
+    this->log.info("Account contains a team ID, but the team does not exist; clearing team ID from account");
+    this->login->account->bb_team_id = 0;
+    this->login->account->save();
     return nullptr;
   }
 
-  auto member_it = team->members.find(this->license->serial_number);
+  auto member_it = team->members.find(this->login->account->account_id);
   if (member_it == team->members.end()) {
-    this->log.info("License contains a team ID, but the team does not contain this member; clearing team ID from license");
-    this->license->bb_team_id = 0;
-    this->license->save();
+    this->log.info("Account contains a team ID, but the team does not contain this member; clearing team ID from account");
+    this->login->account->bb_team_id = 0;
+    this->login->account->save();
     return nullptr;
   }
 
@@ -328,53 +347,74 @@ shared_ptr<const TeamIndex::Team> Client::team() const {
     string name = p->disp.name.decode(this->language());
     if (m.name != name) {
       this->log.info("Updating player name in team config");
-      s->team_index->update_member_name(this->license->serial_number, name);
+      s->team_index->update_member_name(this->login->account->account_id, name);
     }
   }
 
   return team;
 }
 
-bool Client::can_see_quest(shared_ptr<const Quest> q, uint8_t event, uint8_t difficulty, size_t num_players) const {
-  if (this->license && this->license->check_flag(License::Flag::DISABLE_QUEST_REQUIREMENTS)) {
+bool Client::evaluate_quest_availability_expression(
+    shared_ptr<const IntegralExpression> expr,
+    shared_ptr<const Lobby> game,
+    uint8_t event,
+    uint8_t difficulty,
+    size_t num_players,
+    bool v1_present) const {
+  if (this->login && this->login->account->check_flag(Account::Flag::DISABLE_QUEST_REQUIREMENTS)) {
     return true;
   }
-  if (!q->available_expression) {
+  if (!expr) {
     return true;
   }
+  if (game && !game->quest_flag_values) {
+    throw logic_error("quest flags are missing from game");
+  }
   auto p = this->character();
-  string expr = q->available_expression->str();
-  QuestAvailabilityExpression::Env env = {
+  IntegralExpression::Env env = {
       .flags = &p->quest_flags.data.at(difficulty),
       .challenge_records = &p->challenge_records,
       .team = this->team(),
       .num_players = num_players,
       .event = event,
+      .v1_present = v1_present,
   };
-  int64_t ret = q->available_expression->evaluate(env);
-  this->log.info("Evaluated quest availability expression %s => %s", expr.c_str(), ret ? "TRUE" : "FALSE");
+  int64_t ret = expr->evaluate(env);
+  if (this->log.should_log(LogLevel::INFO)) {
+    string expr_str = expr->str();
+    this->log.info("Evaluated quest availability expression %s => %s", expr_str.c_str(), ret ? "TRUE" : "FALSE");
+  }
   return ret;
 }
 
-bool Client::can_play_quest(shared_ptr<const Quest> q, uint8_t event, uint8_t difficulty, size_t num_players) const {
-  if (this->license && this->license->check_flag(License::Flag::DISABLE_QUEST_REQUIREMENTS)) {
+bool Client::can_see_quest(
+    shared_ptr<const Quest> q,
+    shared_ptr<const Lobby> game,
+    uint8_t event,
+    uint8_t difficulty,
+    size_t num_players,
+    bool v1_present) const {
+  return this->evaluate_quest_availability_expression(q->available_expression, game, event, difficulty, num_players, v1_present);
+}
+
+bool Client::can_play_quest(
+    shared_ptr<const Quest> q,
+    shared_ptr<const Lobby> game,
+    uint8_t event,
+    uint8_t difficulty,
+    size_t num_players,
+    bool v1_present) const {
+  return this->evaluate_quest_availability_expression(q->enabled_expression, game, event, difficulty, num_players, v1_present);
+}
+
+bool Client::can_use_chat_commands() const {
+  if (!this->login) {
+    return false;
+  }
+  if (this->login->account->check_flag(Account::Flag::ALWAYS_ENABLE_CHAT_COMMANDS)) {
     return true;
   }
-  if (!q->enabled_expression) {
-    return true;
-  }
-  auto p = this->character();
-  string expr = q->enabled_expression->str();
-  QuestAvailabilityExpression::Env env = {
-      .flags = &p->quest_flags.data.at(difficulty),
-      .challenge_records = &p->challenge_records,
-      .team = this->team(),
-      .num_players = num_players,
-      .event = event,
-  };
-  bool ret = q->enabled_expression->evaluate(env);
-  this->log.info("Evaluating quest enabled expression %s => %s", expr.c_str(), ret ? "TRUE" : "FALSE");
-  return ret;
+  return this->require_server_state()->enable_chat_commands;
 }
 
 void Client::dispatch_save_game_data(evutil_socket_t, short, void* ctx) {
@@ -451,10 +491,10 @@ void Client::create_battle_overlay(shared_ptr<const BattleRules> rules, shared_p
     uint8_t char_class = this->overlay_character_data->disp.visual.char_class;
     auto& stats = this->overlay_character_data->disp.stats;
 
-    stats.reset_to_base(char_class, level_table);
-    stats.advance_to_level(char_class, target_level, level_table);
+    level_table->reset_to_base(stats, char_class);
+    level_table->advance_to_level(stats, target_level, char_class);
 
-    stats.unknown_a1 = 40;
+    stats.esp = 40;
     stats.meseta = 300;
   }
   if (rules->tech_disk_mode == BattleRules::TechDiskMode::LIMIT_LEVEL) {
@@ -497,10 +537,10 @@ void Client::create_challenge_overlay(Version version, size_t template_index, sh
 
   overlay->inventory.items[13].extension_data2 = 1;
 
-  overlay->disp.stats.reset_to_base(overlay->disp.visual.char_class, level_table);
-  overlay->disp.stats.advance_to_level(overlay->disp.visual.char_class, tpl.level, level_table);
+  level_table->reset_to_base(overlay->disp.stats, overlay->disp.visual.char_class);
+  level_table->advance_to_level(overlay->disp.stats, tpl.level, overlay->disp.visual.char_class);
 
-  overlay->disp.stats.unknown_a1 = 40;
+  overlay->disp.stats.esp = 40;
   overlay->disp.stats.unknown_a3 = 10.0;
   overlay->disp.stats.experience = level_table->stats_delta_for_level(overlay->disp.visual.char_class, overlay->disp.stats.level).experience;
   overlay->disp.stats.meseta = 0;
@@ -587,10 +627,10 @@ string Client::system_filename() const {
   if (this->version() != Version::BB_V4) {
     throw logic_error("non-BB players do not have system data");
   }
-  if (!this->license) {
+  if (!this->login || !this->login->bb_license) {
     throw logic_error("client is not logged in");
   }
-  return string_printf("system/players/system_%s.psosys", this->license->bb_username.c_str());
+  return string_printf("system/players/system_%s.psosys", this->login->bb_license->username.c_str());
 }
 
 string Client::character_filename(const std::string& bb_username, int8_t index) {
@@ -603,55 +643,55 @@ string Client::character_filename(const std::string& bb_username, int8_t index)
   return string_printf("system/players/player_%s_%hhd.psochar", bb_username.c_str(), index);
 }
 
-string Client::backup_character_filename(uint32_t serial_number, size_t index) {
-  return string_printf("system/players/backup_player_%" PRIu32 "_%zu.psochar", serial_number, index);
+string Client::backup_character_filename(uint32_t account_id, size_t index) {
+  return string_printf("system/players/backup_player_%" PRIu32 "_%zu.psochar", account_id, index);
 }
 
 string Client::character_filename(int8_t index) const {
   if (this->version() != Version::BB_V4) {
     throw logic_error("non-BB players do not have character data");
   }
-  if (!this->license) {
+  if (!this->login || !this->login->bb_license) {
     throw logic_error("client is not logged in");
   }
-  return this->character_filename(this->license->bb_username, (index < 0) ? this->bb_character_index : index);
+  return this->character_filename(this->login->bb_license->username, (index < 0) ? this->bb_character_index : index);
 }
 
 string Client::guild_card_filename() const {
   if (this->version() != Version::BB_V4) {
     throw logic_error("non-BB players do not have character data");
   }
-  if (!this->license) {
+  if (!this->login || !this->login->bb_license) {
     throw logic_error("client is not logged in");
   }
-  return string_printf("system/players/guild_cards_%s.psocard", this->license->bb_username.c_str());
+  return string_printf("system/players/guild_cards_%s.psocard", this->login->bb_license->username.c_str());
 }
 
 string Client::shared_bank_filename() const {
   if (this->version() != Version::BB_V4) {
     throw logic_error("non-BB players do not have character data");
   }
-  if (!this->license) {
+  if (!this->login || !this->login->bb_license) {
     throw logic_error("client is not logged in");
   }
-  return string_printf("system/players/shared_bank_%s.psobank", this->license->bb_username.c_str());
+  return string_printf("system/players/shared_bank_%s.psobank", this->login->bb_license->username.c_str());
 }
 
 string Client::legacy_account_filename() const {
   if (this->version() != Version::BB_V4) {
     throw logic_error("non-BB players do not have character data");
   }
-  if (!this->license) {
+  if (!this->login || !this->login->bb_license) {
     throw logic_error("client is not logged in");
   }
-  return string_printf("system/players/account_%s.nsa", this->license->bb_username.c_str());
+  return string_printf("system/players/account_%s.nsa", this->login->bb_license->username.c_str());
 }
 
 string Client::legacy_player_filename() const {
   if (this->version() != Version::BB_V4) {
     throw logic_error("non-BB players do not have character data");
   }
-  if (!this->license) {
+  if (!this->login || !this->login->bb_license) {
     throw logic_error("client is not logged in");
   }
   if (this->bb_character_index < 0) {
@@ -659,7 +699,7 @@ string Client::legacy_player_filename() const {
   }
   return string_printf(
       "system/players/player_%s_%hhd.nsc",
-      this->license->bb_username.c_str(),
+      this->login->bb_license->username.c_str(),
       static_cast<int8_t>(this->bb_character_index + 1));
 }
 
@@ -679,7 +719,7 @@ void Client::load_all_files() {
     this->guild_card_data = make_shared<PSOBBGuildCardFile>();
     return;
   }
-  if (!this->license) {
+  if (!this->login || !this->login->bb_license) {
     throw logic_error("cannot load BB player data until client is logged in");
   }
 
@@ -718,6 +758,7 @@ void Client::load_all_files() {
       if (header.flag != 0x00000000) {
         throw runtime_error("incorrect flag in character file header");
       }
+      static_assert(sizeof(PSOBBCharacterFile) + sizeof(PSOBBFullSystemFile) == 0x3994, ".psochar size is incorrect");
       this->character_data = make_shared<PSOBBCharacterFile>(freadx<PSOBBCharacterFile>(f.get()));
       files_manager->set_character(char_filename, this->character_data);
       player_data_log.info("Loaded character data from %s", char_filename.c_str());
@@ -795,12 +836,11 @@ void Client::load_all_files() {
       files_manager->set_character(this->character_filename(), this->character_data);
       this->character_data->inventory = nsc_data.inventory;
       this->character_data->disp = nsc_data.disp;
-      this->character_data->play_time_seconds = nsc_data.disp.play_time;
-      this->character_data->unknown_a2 = nsc_data.unknown_a2;
+      this->character_data->play_time_seconds = 0;
       this->character_data->quest_flags = nsc_data.quest_flags;
       this->character_data->death_count = nsc_data.death_count;
       this->character_data->bank = nsc_data.bank;
-      this->character_data->guild_card.guild_card_number = this->license->serial_number;
+      this->character_data->guild_card.guild_card_number = this->login->account->account_id;
       this->character_data->guild_card.name = nsc_data.disp.name;
       this->character_data->guild_card.description = nsc_data.guild_card_description;
       this->character_data->guild_card.present = 1;
@@ -811,7 +851,7 @@ void Client::load_all_files() {
       this->character_data->info_board = nsc_data.info_board;
       this->character_data->battle_records = nsc_data.battle_records;
       this->character_data->challenge_records = nsc_data.challenge_records;
-      this->character_data->tech_menu_config = nsc_data.tech_menu_config;
+      this->character_data->tech_menu_shortcut_entries = nsc_data.tech_menu_shortcut_entries;
       this->character_data->quest_counters = nsc_data.quest_counters;
       if (nsa_data) {
         this->character_data->option_flags = nsa_data->option_flags;
@@ -824,11 +864,6 @@ void Client::load_all_files() {
     }
   }
 
-  if (this->character_data) {
-    this->license->auto_reply_message = this->character_data->auto_reply.decode();
-    this->license->save();
-  }
-
   this->blocked_senders.clear();
   for (size_t z = 0; z < this->guild_card_data->blocked.size(); z++) {
     if (this->guild_card_data->blocked[z].present) {
@@ -837,6 +872,10 @@ void Client::load_all_files() {
   }
 
   if (this->character_data) {
+    // Clear legacy play_time field
+    this->character_data->disp.name.clear_after_bytes(0x18);
+    this->login->account->auto_reply_message = this->character_data->auto_reply.decode();
+    this->login->account->save();
     this->last_play_time_update = now();
   }
 }
@@ -883,10 +922,10 @@ void Client::save_character_file(
   fwritex(f.get(), *character);
   fwritex(f.get(), *system);
   // TODO: Technically, we should write the actual team membership struct to the
-  // file here, but that would cause Client to depend on License, which
+  // file here, but that would cause Client to depend on Account, which
   // it currently does not. This data doesn't matter at all for correctness
   // within newserv, since it ignores this data entirely and instead generates
-  // the membership struct from the team ID in the License and the team's state.
+  // the membership struct from the team ID in the Account and the team's state.
   // So, writing correct data here would mostly be for compatibility with other
   // PSO servers. But if the other server is newserv, then this data would be
   // used anyway, and if it's not, then it would presumably have a different set
@@ -909,8 +948,7 @@ void Client::save_character_file() {
     // off each time we save. I'm lazy, so insert shrug emoji here.
     uint64_t t = now();
     uint64_t seconds = (t - this->last_play_time_update) / 1000000;
-    this->character_data->disp.play_time += seconds;
-    this->character_data->play_time_seconds = this->character_data->disp.play_time;
+    this->character_data->play_time_seconds += seconds;
     player_data_log.info("Added %" PRIu64 " seconds to play time", seconds);
     this->last_play_time_update = t;
   }
@@ -927,8 +965,8 @@ void Client::save_guild_card_file() const {
   player_data_log.info("Saved Guild Card file %s", filename.c_str());
 }
 
-void Client::load_backup_character(uint32_t serial_number, size_t index) {
-  string filename = this->backup_character_filename(serial_number, index);
+void Client::load_backup_character(uint32_t account_id, size_t index) {
+  string filename = this->backup_character_filename(account_id, index);
   auto f = fopen_unique(filename, "rb");
   auto header = freadx<PSOCommandHeaderBB>(f.get());
   if (header.size != 0x399C) {
@@ -1035,44 +1073,28 @@ void Client::use_character_bank(int8_t index) {
 }
 
 void Client::print_inventory(FILE* stream) const {
+  auto s = this->require_server_state();
   auto p = this->character();
-  shared_ptr<const ItemNameIndex> name_index;
-  try {
-    name_index = this->require_server_state()->item_name_index(this->version());
-  } catch (const runtime_error&) {
-  }
   fprintf(stream, "[PlayerInventory] Meseta: %" PRIu32 "\n", p->disp.stats.meseta.load());
   fprintf(stream, "[PlayerInventory] %hhu items\n", p->inventory.num_items);
   for (size_t x = 0; x < p->inventory.num_items; x++) {
     const auto& item = p->inventory.items[x];
     auto hex = item.data.hex();
-    if (name_index) {
-      auto name = name_index->describe_item(item.data);
-      fprintf(stream, "[PlayerInventory]   %2zu: [+%08" PRIX32 "] %s (%s)\n", x, item.flags.load(), hex.c_str(), name.c_str());
-    } else {
-      fprintf(stream, "[PlayerInventory]   %2zu: [+%08" PRIX32 "] %s\n", x, item.flags.load(), hex.c_str());
-    }
+    auto name = s->describe_item(this->version(), item.data, false);
+    fprintf(stream, "[PlayerInventory]   %2zu: [+%08" PRIX32 "] %s (%s)\n", x, item.flags.load(), hex.c_str(), name.c_str());
   }
 }
 
 void Client::print_bank(FILE* stream) const {
+  auto s = this->require_server_state();
   auto p = this->character();
-  shared_ptr<const ItemNameIndex> name_index;
-  try {
-    name_index = this->require_server_state()->item_name_index(this->version());
-  } catch (const runtime_error&) {
-  }
   fprintf(stream, "[PlayerBank] Meseta: %" PRIu32 "\n", p->bank.meseta.load());
   fprintf(stream, "[PlayerBank] %" PRIu32 " items\n", p->bank.num_items.load());
   for (size_t x = 0; x < p->bank.num_items; x++) {
     const auto& item = p->bank.items[x];
     const char* present_token = item.present ? "" : " (missing present flag)";
     auto hex = item.data.hex();
-    if (name_index) {
-      auto name = name_index->describe_item(item.data);
-      fprintf(stream, "[PlayerBank]   %3zu: %s (%s) (x%hu)%s\n", x, hex.c_str(), name.c_str(), item.amount.load(), present_token);
-    } else {
-      fprintf(stream, "[PlayerBank]   %3zu: %s (x%hu)%s\n", x, hex.c_str(), item.amount.load(), present_token);
-    }
+    auto name = s->describe_item(this->version(), item.data, false);
+    fprintf(stream, "[PlayerBank]   %3zu: %s (%s) (x%hu)%s\n", x, hex.c_str(), name.c_str(), item.amount.load(), present_token);
   }
 }

src/Client.hh

@@ -5,13 +5,13 @@
 #include <memory>
 #include <stdexcept>
 
+#include "Account.hh"
 #include "Channel.hh"
 #include "CommandFormats.hh"
 #include "Episode3/BattleRecord.hh"
 #include "Episode3/Tournament.hh"
 #include "FileContentsCache.hh"
 #include "FunctionCompiler.hh"
-#include "License.hh"
 #include "PSOEncryption.hh"
 #include "PSOProtocol.hh"
 #include "PatchFileIndex.hh"
@@ -34,62 +34,70 @@ public:
     // TODO: It'd be nice to use a pattern here (e.g. all server-side flags are
     // in the high bits) but that would require re-recording or manually
     // rewriting all the tests
-    CLIENT_SIDE_MASK                    = 0xFFFFFFFFFC0FFFFB,
+    CLIENT_SIDE_MASK                           = 0xFF3CFFFF7C0FFFFB,
 
     // Version-related flags
-    CHECKED_FOR_DC_V1_PROTOTYPE         = 0x0000000000000002,
-    LICENSE_WAS_CREATED                 = 0x0000000000000004, // Server-side only
-    NO_D6_AFTER_LOBBY                   = 0x0000000000000100,
-    NO_D6                               = 0x0000000000000200,
-    FORCE_ENGLISH_LANGUAGE_BB           = 0x0000000000000400,
+    CHECKED_FOR_DC_V1_PROTOTYPE                = 0x0000000000000002,
+    NO_D6_AFTER_LOBBY                          = 0x0000000000000100,
+    NO_D6                                      = 0x0000000000000200,
+    FORCE_ENGLISH_LANGUAGE_BB                  = 0x0000000000000400,
 
     // Flags describing the behavior for send_function_call
-    NO_SEND_FUNCTION_CALL               = 0x0000000000001000,
-    ENCRYPTED_SEND_FUNCTION_CALL        = 0x0000000000002000,
-    SEND_FUNCTION_CALL_CHECKSUM_ONLY    = 0x0000000000004000,
-    SEND_FUNCTION_CALL_NO_CACHE_PATCH   = 0x0000000000008000,
-    USE_OVERFLOW_FOR_SEND_FUNCTION_CALL = 0x0000000000010000,
+    NO_SEND_FUNCTION_CALL                      = 0x0000000000001000,
+    ENCRYPTED_SEND_FUNCTION_CALL               = 0x0000000000002000,
+    SEND_FUNCTION_CALL_CHECKSUM_ONLY           = 0x0000000000004000,
+    SEND_FUNCTION_CALL_NO_CACHE_PATCH          = 0x0000000000008000,
+    USE_OVERFLOW_FOR_SEND_FUNCTION_CALL        = 0x0000000000010000,
 
     // State flags
-    LOADING                             = 0x0000000000100000, // Server-side only
-    LOADING_QUEST                       = 0x0000000000200000, // Server-side only
-    LOADING_RUNNING_JOINABLE_QUEST      = 0x0000000000400000, // Server-side only
-    LOADING_TOURNAMENT                  = 0x0000000000800000, // Server-side only
-    IN_INFORMATION_MENU                 = 0x0000000001000000, // Server-side only
-    AT_WELCOME_MESSAGE                  = 0x0000000002000000, // Server-side only
-    SAVE_ENABLED                        = 0x0000000004000000,
-    HAS_EP3_CARD_DEFS                   = 0x0000000008000000,
-    HAS_EP3_MEDIA_UPDATES               = 0x0000000010000000,
-    USE_OVERRIDE_RANDOM_SEED            = 0x0000000020000000,
-    HAS_GUILD_CARD_NUMBER               = 0x0000000040000000,
-    AT_BANK_COUNTER                     = 0x0000000080000000, // Server-side only
-    SHOULD_SEND_ARTIFICIAL_ITEM_STATE   = 0x0001000000000000, // Server-side only
-    SHOULD_SEND_ARTIFICIAL_FLAG_STATE   = 0x0002000000000000, // Server-side only
-    SHOULD_SEND_ENABLE_SAVE             = 0x0004000000000000,
-    SWITCH_ASSIST_ENABLED               = 0x0000000100000000,
+    LOADING                                    = 0x0000000000100000, // Server-side only
+    LOADING_QUEST                              = 0x0000000000200000, // Server-side only
+    LOADING_RUNNING_JOINABLE_QUEST             = 0x0000000000400000, // Server-side only
+    LOADING_TOURNAMENT                         = 0x0000000000800000, // Server-side only
+    IN_INFORMATION_MENU                        = 0x0000000001000000, // Server-side only
+    AT_WELCOME_MESSAGE                         = 0x0000000002000000, // Server-side only
+    SAVE_ENABLED                               = 0x0000000004000000,
+    HAS_EP3_CARD_DEFS                          = 0x0000000008000000,
+    HAS_EP3_MEDIA_UPDATES                      = 0x0000000010000000,
+    USE_OVERRIDE_RANDOM_SEED                   = 0x0000000020000000,
+    HAS_GUILD_CARD_NUMBER                      = 0x0000000040000000,
+    HAS_AUTO_PATCHES                           = 0x0000004000000000,
+    AT_BANK_COUNTER                            = 0x0000000080000000, // Server-side only
+    SHOULD_SEND_ARTIFICIAL_ITEM_STATE          = 0x0001000000000000, // Server-side only
+    SHOULD_SEND_ARTIFICIAL_ENEMY_AND_SET_STATE = 0x0040000000000000, // Server-side only
+    SHOULD_SEND_ARTIFICIAL_OBJECT_STATE        = 0x0080000000000000, // Server-side only
+    SHOULD_SEND_ARTIFICIAL_FLAG_STATE          = 0x0002000000000000, // Server-side only
+    SHOULD_SEND_ENABLE_SAVE                    = 0x0004000000000000,
+    SWITCH_ASSIST_ENABLED                      = 0x0000000100000000,
 
     // Cheat mode and option flags
-    INFINITE_HP_ENABLED                 = 0x0000000200000000,
-    INFINITE_TP_ENABLED                 = 0x0000000400000000,
-    DEBUG_ENABLED                       = 0x0000000800000000,
-    RARE_DROP_NOTIFICATIONS_ENABLED     = 0x0010000000000000,
-    ALL_DROP_NOTIFICATIONS_ENABLED      = 0x0020000000000000,
+    INFINITE_HP_ENABLED                        = 0x0000000200000000,
+    INFINITE_TP_ENABLED                        = 0x0000000400000000,
+    DEBUG_ENABLED                              = 0x0000000800000000,
+    ITEM_DROP_NOTIFICATIONS_1                  = 0x0010000000000000,
+    ITEM_DROP_NOTIFICATIONS_2                  = 0x0020000000000000,
 
     // Proxy option flags
-    PROXY_SAVE_FILES                    = 0x0000001000000000,
-    PROXY_CHAT_COMMANDS_ENABLED         = 0x0000002000000000,
-    PROXY_PLAYER_NOTIFICATIONS_ENABLED  = 0x0000008000000000,
-    PROXY_SUPPRESS_CLIENT_PINGS         = 0x0000010000000000,
-    PROXY_SUPPRESS_REMOTE_LOGIN         = 0x0000020000000000,
-    PROXY_ZERO_REMOTE_GUILD_CARD        = 0x0000040000000000,
-    PROXY_EP3_INFINITE_MESETA_ENABLED   = 0x0000080000000000,
-    PROXY_EP3_INFINITE_TIME_ENABLED     = 0x0000100000000000,
-    PROXY_RED_NAME_ENABLED              = 0x0000200000000000,
-    PROXY_BLANK_NAME_ENABLED            = 0x0000400000000000,
-    PROXY_BLOCK_FUNCTION_CALLS          = 0x0000800000000000,
-    PROXY_EP3_UNMASK_WHISPERS           = 0x0008000000000000,
+    PROXY_SAVE_FILES                           = 0x0000001000000000,
+    PROXY_CHAT_COMMANDS_ENABLED                = 0x0000002000000000,
+    PROXY_PLAYER_NOTIFICATIONS_ENABLED         = 0x0000008000000000,
+    PROXY_SUPPRESS_CLIENT_PINGS                = 0x0000010000000000,
+    PROXY_SUPPRESS_REMOTE_LOGIN                = 0x0000020000000000,
+    PROXY_ZERO_REMOTE_GUILD_CARD               = 0x0000040000000000,
+    PROXY_EP3_INFINITE_MESETA_ENABLED          = 0x0000080000000000,
+    PROXY_EP3_INFINITE_TIME_ENABLED            = 0x0000100000000000,
+    PROXY_RED_NAME_ENABLED                     = 0x0000200000000000,
+    PROXY_BLANK_NAME_ENABLED                   = 0x0000400000000000,
+    PROXY_BLOCK_FUNCTION_CALLS                 = 0x0000800000000000,
+    PROXY_EP3_UNMASK_WHISPERS                  = 0x0008000000000000,
     // clang-format on
   };
+  enum class ItemDropNotificationMode {
+    NOTHING = 0,
+    RARES_ONLY = 1,
+    ALL_ITEMS = 2,
+    ALL_ITEMS_INCLUDING_MESETA = 3,
+  };
 
   static constexpr uint64_t DEFAULT_FLAGS = static_cast<uint64_t>(Flag::PROXY_CHAT_COMMANDS_ENABLED);
 
@@ -129,6 +137,9 @@ public:
 
     void set_flags_for_version(Version version, int64_t sub_version);
 
+    ItemDropNotificationMode get_drop_notification_mode() const;
+    void set_drop_notification_mode(ItemDropNotificationMode new_mode);
+
     template <size_t Bytes>
     void parse_from(const parray<uint8_t, Bytes>& data) {
       StringReader r(data.data(), data.size());
@@ -170,8 +181,7 @@ public:
   uint64_t id;
   PrefixedLogger log;
 
-  // License & account
-  std::shared_ptr<License> license;
+  std::shared_ptr<Login> login;
 
   // Network
   Channel channel;
@@ -186,12 +196,10 @@ public:
   uint8_t bb_connection_phase;
   uint64_t ping_start_time;
 
-  // Patch server
-  std::vector<PatchFileChecksumRequest> patch_file_checksum_requests;
-
   // Lobby/positioning
   Config config;
   Config synced_config;
+  std::unique_ptr<parray<le_uint32_t, 0x20>> override_variations;
   int32_t sub_version;
   float x;
   float z;
@@ -208,8 +216,9 @@ public:
   uint16_t card_battle_table_seat_number;
   uint16_t card_battle_table_seat_state;
   std::weak_ptr<Episode3::Tournament::Team> ep3_tournament_team;
-  std::shared_ptr<Episode3::BattleRecord> ep3_prev_battle_record;
+  std::shared_ptr<const Episode3::BattleRecord> ep3_prev_battle_record;
   std::shared_ptr<const Menu> last_menu_sent;
+  uint32_t last_game_info_requested;
   struct JoinCommand {
     uint16_t command;
     uint32_t flag;
@@ -241,12 +250,12 @@ public:
 
   // Miscellaneous (used by chat commands)
   uint32_t next_exp_value; // next EXP value to give
-  G_SwitchStateChanged_6x05 last_switch_enabled_command;
+  RecentSwitchFlags recent_switch_flags; // used for switch assist
   bool can_chat;
   struct PendingCharacterExport {
-    std::shared_ptr<const License> license;
+    std::shared_ptr<const Account> dest_account;
     ssize_t character_index = -1;
-    bool is_bb_conversion = false;
+    std::shared_ptr<const BBLicense> dest_bb_license; // Only used for $bbchar; null for $savechar
   };
   std::unique_ptr<PendingCharacterExport> pending_character_export;
   std::deque<std::function<void(uint32_t, uint32_t)>> function_call_response_queue;
@@ -259,10 +268,13 @@ public:
   Client(
       std::shared_ptr<Server> server,
       struct bufferevent* bev,
+      uint64_t virtual_network_id,
       Version version,
       ServerBehavior server_behavior);
   ~Client();
 
+  void update_channel_name();
+
   void reschedule_save_game_data_event();
   void reschedule_ping_and_timeout_events();
 
@@ -273,8 +285,7 @@ public:
     return this->channel.language;
   }
 
-  void set_license(std::shared_ptr<License> l);
-  void convert_license_to_temporary_if_nte();
+  void convert_account_to_temporary_if_nte();
 
   void sync_config();
 
@@ -283,8 +294,29 @@ public:
 
   std::shared_ptr<const TeamIndex::Team> team() const;
 
-  bool can_see_quest(std::shared_ptr<const Quest> q, uint8_t event, uint8_t difficulty, size_t num_players) const;
-  bool can_play_quest(std::shared_ptr<const Quest> q, uint8_t event, uint8_t difficulty, size_t num_players) const;
+  bool evaluate_quest_availability_expression(
+      std::shared_ptr<const IntegralExpression> expr,
+      std::shared_ptr<const Lobby> game,
+      uint8_t event,
+      uint8_t difficulty,
+      size_t num_players,
+      bool v1_present) const;
+  bool can_see_quest(
+      std::shared_ptr<const Quest> q,
+      std::shared_ptr<const Lobby> game,
+      uint8_t event,
+      uint8_t difficulty,
+      size_t num_players,
+      bool v1_present) const;
+  bool can_play_quest(
+      std::shared_ptr<const Quest> q,
+      std::shared_ptr<const Lobby> game,
+      uint8_t event,
+      uint8_t difficulty,
+      size_t num_players,
+      bool v1_present) const;
+
+  bool can_use_chat_commands() const;
 
   static void dispatch_save_game_data(evutil_socket_t, short, void* ctx);
   void save_game_data();
@@ -324,7 +356,7 @@ public:
 
   std::string system_filename() const;
   static std::string character_filename(const std::string& bb_username, int8_t index);
-  static std::string backup_character_filename(uint32_t serial_number, size_t index);
+  static std::string backup_character_filename(uint32_t account_id, size_t index);
   std::string character_filename(int8_t index = -1) const;
   std::string guild_card_filename() const;
   std::string shared_bank_filename() const;
@@ -342,7 +374,7 @@ public:
   void save_character_file();
   void save_guild_card_file() const;
 
-  void load_backup_character(uint32_t serial_number, size_t index);
+  void load_backup_character(uint32_t account_id, size_t index);
   void save_and_unload_character();
 
   PlayerBank& current_bank();

src/CommonItemSet.cc

@@ -1,11 +1,116 @@
 #include "CommonItemSet.hh"
 
 #include "AFSArchive.hh"
+#include "EnemyType.hh"
 #include "GSLArchive.hh"
 #include "StaticGameData.hh"
 
 using namespace std;
 
+template <typename IntT, size_t Count>
+JSON to_json(const parray<IntT, Count>& v) {
+  auto ret = JSON::list();
+  for (size_t z = 0; z < Count; z++) {
+    ret.emplace_back(v[z]);
+  }
+  return ret;
+}
+
+template <typename IntT, size_t Count>
+JSON to_json(const parray<CommonItemSet::Table::Range<IntT>, Count>& v) {
+  auto ret = JSON::list();
+  for (size_t z = 0; z < Count; z++) {
+    ret.emplace_back(to_json(v[z]));
+  }
+  return ret;
+}
+
+template <typename IntT>
+JSON to_json(const CommonItemSet::Table::Range<IntT>& v) {
+  if (v.min == v.max) {
+    return JSON(v.min);
+  } else {
+    return JSON::list({v.min, v.max});
+  }
+}
+
+template <typename IntT, size_t Count1, size_t Count2>
+JSON to_json(const parray<parray<IntT, Count2>, Count1>& v) {
+  auto ret = JSON::list();
+  for (size_t z = 0; z < Count1; z++) {
+    ret.emplace_back(to_json(v[z]));
+  }
+  return ret;
+}
+
+JSON CommonItemSet::Table::json() const {
+  JSON enemy_meseta_ranges_json = JSON::dict();
+  JSON enemy_type_drop_probs_json = JSON::dict();
+  JSON enemy_item_classes_json = JSON::dict();
+  for (size_t z = 0; z < 0x64; z++) {
+    static const array<Episode, 3> episodes = {Episode::EP1, Episode::EP2, Episode::EP4};
+    for (Episode episode : episodes) {
+      for (auto type : enemy_types_for_rare_table_index(episode, z)) {
+        string name = string_printf("%s:%s", abbreviation_for_episode(episode), name_for_enum(type));
+        enemy_meseta_ranges_json.emplace(name, to_json(this->enemy_meseta_ranges[z]));
+        enemy_type_drop_probs_json.emplace(name, this->enemy_type_drop_probs[z]);
+        enemy_item_classes_json.emplace(name, this->enemy_item_classes[z]);
+      }
+    }
+  }
+  return JSON::dict({
+      {"BaseWeaponTypeProbTable", to_json(this->base_weapon_type_prob_table)},
+      {"SubtypeBaseTable", to_json(this->subtype_base_table)},
+      {"SubtypeAreaLengthTable", to_json(this->subtype_area_length_table)},
+      {"GrindProbTable", to_json(this->grind_prob_table)},
+      {"ArmorShieldTypeIndexProbTable", to_json(this->armor_shield_type_index_prob_table)},
+      {"ArmorSlotCountProbTable", to_json(this->armor_slot_count_prob_table)},
+      {"EnemyMesetaRanges", std::move(enemy_meseta_ranges_json)},
+      {"EnemyTypeDropProbs", std::move(enemy_type_drop_probs_json)},
+      {"EnemyItemClasses", std::move(enemy_item_classes_json)},
+      {"BoxMesetaRanges", to_json(this->box_meseta_ranges)},
+      {"HasRareBonusValueProbTable", this->has_rare_bonus_value_prob_table},
+      {"BonusValueProbTable", to_json(this->bonus_value_prob_table)},
+      {"NonRareBonusProbSpec", to_json(this->nonrare_bonus_prob_spec)},
+      {"BonusTypeProbTable", to_json(this->bonus_type_prob_table)},
+      {"SpecialMult", to_json(this->special_mult)},
+      {"SpecialPercent", to_json(this->special_percent)},
+      {"ToolClassProbTable", to_json(this->tool_class_prob_table)},
+      {"TechniqueIndexProbTable", to_json(this->technique_index_prob_table)},
+      {"TechniqueLevelRanges", to_json(this->technique_level_ranges)},
+      {"ArmorOrShieldTypeBias", this->armor_or_shield_type_bias},
+      {"UnitMaxStarsTable", to_json(this->unit_max_stars_table)},
+      {"BoxItemClassProbTable", to_json(this->box_item_class_prob_table)},
+  });
+}
+
+JSON CommonItemSet::json() const {
+  auto modes_dict = JSON::dict();
+  static const array<GameMode, 4> modes = {GameMode::NORMAL, GameMode::BATTLE, GameMode::CHALLENGE, GameMode::SOLO};
+  for (const auto& mode : modes) {
+    auto episodes_dict = JSON::dict();
+    static const array<Episode, 3> episodes = {Episode::EP1, Episode::EP2, Episode::EP4};
+    for (const auto& episode : episodes) {
+      auto difficulty_dict = JSON::dict();
+      for (uint8_t difficulty = 0; difficulty < 4; difficulty++) {
+        auto section_id_dict = JSON::dict();
+        for (uint8_t section_id = 0; section_id < 10; section_id++) {
+          try {
+            auto table = this->get_table(episode, mode, difficulty, section_id);
+            section_id_dict.emplace(name_for_section_id(section_id), table->json());
+          } catch (const runtime_error&) {
+          }
+        }
+        difficulty_dict.emplace(token_name_for_difficulty(difficulty), std::move(section_id_dict));
+      }
+      episodes_dict.emplace(token_name_for_episode(episode), std::move(difficulty_dict));
+    }
+    modes_dict.emplace(name_for_mode(mode), std::move(episodes_dict));
+  }
+
+  return modes_dict;
+}
+
 CommonItemSet::Table::Table(const StringReader& r, bool is_big_endian, bool is_v3) {
   if (is_big_endian) {
     this->parse_itempt_t<true>(r, is_v3);
@@ -19,7 +124,7 @@ void CommonItemSet::Table::parse_itempt_t(const StringReader& r, bool is_v3) {
   using U16T = typename std::conditional<IsBigEndian, be_uint16_t, le_uint16_t>::type;
   using U32T = typename std::conditional<IsBigEndian, be_uint32_t, le_uint32_t>::type;
 
-  const auto& offsets = r.pget<Offsets<IsBigEndian>>(r.pget<U32T>(r.size() - 0x10));
+  const auto& offsets = r.pget<OffsetsT<IsBigEndian>>(r.pget<U32T>(r.size() - 0x10));
 
   this->base_weapon_type_prob_table = r.pget<parray<uint8_t, 0x0C>>(offsets.base_weapon_type_prob_table_offset);
   this->subtype_base_table = r.pget<parray<int8_t, 0x0C>>(offsets.subtype_base_table_offset);

src/CommonItemSet.hh

@@ -2,6 +2,7 @@
 
 #include <array>
 #include <phosg/Encoding.hh>
+#include <phosg/JSON.hh>
 
 #include "GSLArchive.hh"
 #include "PSOEncryption.hh"
@@ -19,7 +20,7 @@ public:
     struct Range {
       IntT min;
       IntT max;
-    } __attribute__((packed));
+    } __packed__;
 
     parray<uint8_t, 0x0C> base_weapon_type_prob_table;
     parray<int8_t, 0x0C> subtype_base_table;
@@ -45,13 +46,14 @@ public:
     parray<parray<uint8_t, 10>, 7> box_item_class_prob_table;
 
     void print_enemy_table(FILE* stream) const;
+    JSON json() const;
 
   private:
     template <bool IsBigEndian>
     void parse_itempt_t(const StringReader& r, bool is_v3);
 
     template <bool IsBigEndian>
-    struct Offsets {
+    struct OffsetsT {
       using U16T = typename std::conditional<IsBigEndian, be_uint16_t, le_uint16_t>::type;
       using U32T = typename std::conditional<IsBigEndian, be_uint32_t, le_uint32_t>::type;
 
@@ -252,10 +254,15 @@ public:
       /* 50 */ U32T box_item_class_prob_table_offset;
 
       // There are several unused fields here.
-    } __attribute__((packed));
+    } __packed__;
+    using Offsets = OffsetsT<false>;
+    using OffsetsBE = OffsetsT<true>;
+    check_struct_size(Offsets, 0x54);
+    check_struct_size(OffsetsBE, 0x54);
   };
 
   std::shared_ptr<const Table> get_table(Episode episode, GameMode mode, uint8_t difficulty, uint8_t secid) const;
+  JSON json() const;
 
 protected:
   CommonItemSet() = default;
@@ -298,22 +305,22 @@ struct ProbabilityTable {
     return this->items[--this->count];
   }
 
-  void shuffle(PSOLFGEncryption& random_crypt) {
+  void shuffle(std::shared_ptr<PSOLFGEncryption> opt_rand_crypt) {
     for (size_t z = 1; z < this->count; z++) {
-      size_t other_z = random_crypt.next() % (z + 1);
+      size_t other_z = random_from_optional_crypt(opt_rand_crypt) % (z + 1);
       ItemT t = this->items[z];
       this->items[z] = this->items[other_z];
       this->items[other_z] = t;
     }
   }
 
-  ItemT sample(PSOLFGEncryption& random_crypt) const {
+  ItemT sample(std::shared_ptr<PSOLFGEncryption> opt_rand_crypt) const {
     if (this->count == 0) {
       throw std::runtime_error("sample from empty probability table");
     } else if (this->count == 1) {
       return this->items[0];
     } else {
-      return this->items[random_crypt.next() % this->count];
+      return this->items[random_from_optional_crypt(opt_rand_crypt) % this->count];
     }
   }
 };
@@ -324,10 +331,12 @@ public:
   struct WeightTableEntry {
     ValueT value;
     WeightT weight;
-  } __attribute__((packed));
+  } __packed__;
 
   using WeightTableEntry8 = WeightTableEntry<uint8_t>;
   using WeightTableEntry32 = WeightTableEntry<be_uint32_t>;
+  check_struct_size(WeightTableEntry8, 2);
+  check_struct_size(WeightTableEntry32, 8);
 
 protected:
   std::shared_ptr<const std::string> data;
@@ -337,7 +346,7 @@ protected:
     be_uint32_t offset;
     uint8_t entries_per_table;
     parray<uint8_t, 3> unused;
-  } __attribute__((packed));
+  } __packed_ws__(TableSpec, 8);
 
   RELFileSet(std::shared_ptr<const std::string> data);
 
@@ -378,7 +387,7 @@ public:
     Mode mode;
     uint8_t player_level_divisor_or_min_level;
     uint8_t max_level;
-  } __attribute__((packed));
+  } __packed_ws__(TechDiskLevelEntry, 3);
 
   std::pair<const uint8_t*, size_t> get_common_recovery_table(size_t index) const;
   std::pair<const WeightTableEntry8*, size_t> get_rare_recovery_table(size_t index) const;
@@ -400,7 +409,7 @@ public:
   struct RangeTableEntry {
     be_uint32_t min;
     be_uint32_t max;
-  } __attribute__((packed));
+  } __packed_ws__(RangeTableEntry, 8);
 
   std::pair<const WeightTableEntry8*, size_t> get_weapon_type_table(size_t index) const;
   const parray<WeightTableEntry32, 6>* get_bonus_type_table(size_t which, size_t index) const;
@@ -419,7 +428,7 @@ private:
     be_uint32_t special_mode_table; // [[{u32 value, u32 weight}](3)](8)
     be_uint32_t standard_grind_range_table; // [{u32 min, u32 max}](6)
     be_uint32_t favored_grind_range_table; // [{u32 min, u32 max}](6)
-  } __attribute__((packed));
+  } __packed_ws__(Offsets, 0x20);
 
   const Offsets* offsets;
 };
@@ -452,11 +461,11 @@ private:
     uint8_t delta_index;
     uint8_t count_default;
     uint8_t count_favored;
-  } __attribute__((packed));
+  } __packed_ws__(DeltaProbabilityEntry, 3);
   struct LuckTableEntry {
     uint8_t delta_index;
     int8_t luck;
-  } __attribute__((packed));
+  } __packed_ws__(LuckTableEntry, 2);
 
   struct Offsets {
     // Each section ID's favored weapon class has different probabilities than
@@ -562,7 +571,7 @@ private:
     // In PSO V3, the bonus delta luck table is:
     //   +10 => +15, +5 => +8, 0 => 0, -5 => -8, -10 => -15
     be_uint32_t bonus_delta_luck_offset; // LuckTableEntry[...]; ending with FF FF
-  } __attribute__((packed));
+  } __packed_ws__(Offsets, 0x18);
 
   const Offsets* offsets;
 

src/Compression.cc

@@ -438,6 +438,45 @@ string prs_compress_optimal(const string& data, ProgressCallback progress_fn) {
   return prs_compress_optimal(data.data(), data.size(), progress_fn);
 }
 
+string prs_compress_pessimal(const void* vdata, size_t size) {
+  const uint8_t* in_data = reinterpret_cast<const uint8_t*>(vdata);
+
+  // The worst possible encoding we can do is a literal byte when no byte with
+  // the same value is within the window, or an extended copy if there is a byte
+  // with the same value in the window.
+  WindowIndex<0x1FFF, 1> window(in_data, size);
+  LZSSInterleavedWriter w;
+  for (size_t z = 0; z < size; z++) {
+    auto match = window.get_best_match();
+    if (match.second >= 1) {
+      // Write extended copy
+      int16_t offset = match.first - window.offset;
+      w.write_control(false);
+      w.flush_if_ready();
+      w.write_control(true);
+      uint16_t a = (offset << 3);
+      w.write_data(a & 0xFF);
+      w.write_data(a >> 8);
+      w.write_data(0);
+    } else {
+      // Write literal
+      w.write_control(true);
+      w.write_data(in_data[z]);
+    }
+    w.flush_if_ready();
+    window.advance();
+  }
+
+  // Write stop command
+  w.write_control(false);
+  w.flush_if_ready();
+  w.write_control(true);
+  w.write_data(0);
+  w.write_data(0);
+
+  return std::move(w.close());
+}
+
 PRSCompressor::PRSCompressor(
     ssize_t compression_level, ProgressCallback progress_fn)
     : compression_level(compression_level),
@@ -976,41 +1015,48 @@ void prs_disassemble(FILE* stream, const void* data, size_t size) {
   ControlStreamReader cr(r);
 
   while (!r.eof()) {
+    uint8_t buffered_bits = cr.buffered_bits();
     if (cr.read()) {
-      fprintf(stream, "[%zX] literal %02hhX\n", output_bytes, r.get_u8());
+      uint8_t literal_value = r.get_u8();
+      fprintf(stream, "[%zX] %hhu> 1    %02hhX        literal %02hhX\n",
+          output_bytes, buffered_bits, literal_value, literal_value);
       output_bytes++;
 
     } else {
-      ssize_t offset;
-      size_t count;
-      const char* copy_type;
-
+      size_t count, read_offset;
       if (cr.read()) {
-        uint16_t a = r.get_u8();
-        a |= (r.get_u8() << 8);
-        offset = (a >> 3) | (~0x1FFF);
+        uint8_t a_low = r.get_u8();
+        uint8_t a_high = r.get_u8();
+        uint16_t a = (a_high << 8) | a_low;
+        ssize_t offset = (a >> 3) | (~0x1FFF);
         if (offset == ~0x1FFF) {
           fprintf(stream, "[%zX] end\n", output_bytes);
           break;
         }
         if (a & 7) {
-          copy_type = "long";
           count = (a & 7) + 2;
+          read_offset = output_bytes + offset;
+          fprintf(stream, "[%zX] %hhu> 01   %02hhX %02hhX     long copy from %zd (offset=%zX) size=%zX\n",
+              output_bytes, buffered_bits, a_low, a_high, offset, read_offset, count);
         } else {
-          copy_type = "extended";
-          count = r.get_u8() + 1;
+          uint8_t count_u8 = r.get_u8();
+          count = count_u8 + 1;
+          read_offset = output_bytes + offset;
+          fprintf(stream, "[%zX] %hhu> 01   %02hhX %02hhX %02hhX  extended copy from %zd (offset=%zX) size=%zX\n",
+              output_bytes, buffered_bits, a_low, a_high, count_u8, offset, read_offset, count);
         }
 
       } else {
-        copy_type = "short";
-        count = cr.read() << 1;
-        count = (count | cr.read()) + 2;
-        offset = r.get_u8() | (~0xFF);
+        bool first_bit = cr.read();
+        bool second_bit = cr.read();
+        uint8_t offset_u8 = r.get_u8();
+        count = ((first_bit ? 2 : 0) | (second_bit ? 1 : 0)) + 2;
+        ssize_t offset = offset_u8 | (~0xFF);
+        read_offset = output_bytes + offset;
+        fprintf(stream, "[%zX] %hhu> 00%c%c %02hhX        short copy from %zd (offset=%zX) size=%zX\n",
+            output_bytes, buffered_bits, first_bit ? '1' : '0', second_bit ? '1' : '0', offset_u8, offset, read_offset, count);
       }
 
-      size_t read_offset = output_bytes + offset;
-      fprintf(stream, "[%zX] %s copy %zX\n", output_bytes, copy_type, count);
-
       if (read_offset >= output_bytes) {
         throw runtime_error("backreference offset beyond beginning of output");
       }

src/Compression.hh

@@ -177,6 +177,10 @@ std::string prs_compress_indexed(
 std::string prs_compress_optimal(const void* vdata, size_t size, ProgressCallback progress_fn = nullptr);
 std::string prs_compress_optimal(const std::string& data, ProgressCallback progress_fn = nullptr);
 
+// Compresses data using PRS to the LARGEST possible output size. There is no
+// practical use for this function except for amusement.
+std::string prs_compress_pessimal(const void* vdata, size_t size);
+
 // Decompresses PRS-compressed data.
 struct PRSDecompressResult {
   std::string data;

src/DCSerialNumbers.cc

@@ -1386,3 +1386,74 @@ void dc_serial_number_speed_test(uint64_t seed) {
   fprintf(stderr, "Fast vs. slow speedup: %zux\n", static_cast<size_t>(time_slow / time_fast));
   fprintf(stderr, "Disagreements: %zu\n", num_disagreements);
 }
+
+string decrypt_dp_address_jpn(
+    const string& executable,
+    const string& values,
+    const string& indexes) {
+  StringReader values_r(values);
+  StringReader indexes_r(indexes);
+
+  size_t fixup_values_offset = values_r.pget_u32l(0x3FFC) - 0x8C004000;
+  size_t fixup_steps_offset = indexes_r.pget_u32l(0x3BFC) - 0x8C008400;
+  StringReader fixup_values_r = values_r.sub(fixup_values_offset);
+  StringReader fixup_steps_r = indexes_r.sub(fixup_steps_offset);
+
+  auto decrypted = decrypt_pr2_data<false>(executable);
+  size_t fixup_offset = 0;
+  while (fixup_steps_r.get_u8(false)) {
+    fixup_offset += (fixup_steps_r.get_u8() << 2);
+    fixup_steps_r.skip(1);
+    if (fixup_offset + 4 > decrypted.compressed_data.size()) {
+      throw runtime_error("fixup beyond end of compressed data");
+    }
+    *reinterpret_cast<le_uint32_t*>(decrypted.compressed_data.data() + fixup_offset) = fixup_values_r.get_u32l();
+  }
+
+  return prs_decompress(decrypted.compressed_data);
+}
+
+EncryptedDCv2Executables encrypt_dp_address_jpn(const string& executable, const string& indexes) {
+  EncryptedDCv2Executables ret;
+
+  string compressed = prs_compress(executable);
+  ret.executable = encrypt_pr2_data<false>(compressed, executable.size(), random_object<uint32_t>() & 0x7FFFFF7F);
+
+  StringReader indexes_r(indexes);
+  size_t fixup_steps_offset = indexes_r.pget_u32l(0x3BFC) - 0x8C008400;
+  ret.indexes = indexes;
+  ret.indexes.at(fixup_steps_offset) = 0;
+  return ret;
+}
+
+std::string crypt_dp_address_jpn_simple(const std::string& data, int64_t mask_key) {
+  if (data.size() & 3) {
+    throw runtime_error("size is not a multiple of 4");
+  }
+
+  StringReader r(data);
+  if (mask_key < 0) {
+    unordered_map<uint32_t, size_t> key_freq;
+    while (!r.eof()) {
+      key_freq[r.get_u32l()] += 1;
+    }
+    size_t max_v = 0;
+    for (const auto& it : key_freq) {
+      if (it.second > max_v) {
+        max_v = it.second;
+        mask_key = it.first;
+      }
+    }
+    if (mask_key < 0) {
+      throw runtime_error("cannot determine mask key");
+    }
+    log_info("Determined %08" PRIX64 " to be the most likely mask key", mask_key);
+    r.go(0);
+  }
+
+  StringWriter w;
+  while (!r.eof()) {
+    w.put_u32l(r.get_u32l() ^ mask_key);
+  }
+  return std::move(w.str());
+}

src/DCSerialNumbers.hh

@@ -21,3 +21,16 @@ std::string generate_dc_serial_number(uint8_t domain, uint8_t subdomain = 0xFF);
 std::unordered_map<uint32_t, std::string> generate_all_dc_serial_numbers(uint8_t domain = 0xFF, uint8_t subdomain = 0xFF);
 
 void dc_serial_number_speed_test(uint64_t seed = 0xFFFFFFFFFFFFFFFF);
+
+struct EncryptedDCv2Executables {
+  std::string executable;
+  std::string indexes;
+};
+
+std::string decrypt_dp_address_jpn(
+    const std::string& dp_address_jpn_data,
+    const std::string& iwashi_sea_data,
+    const std::string& katsuo_sea_data);
+EncryptedDCv2Executables encrypt_dp_address_jpn(const std::string& executable, const std::string& indexes);
+
+std::string crypt_dp_address_jpn_simple(const std::string& data, int64_t seed = -1);

src/DNSServer.cc

@@ -19,10 +19,13 @@ using namespace std;
 
 DNSServer::DNSServer(
     shared_ptr<struct event_base> base,
-    uint32_t local_connect_address, uint32_t external_connect_address)
+    uint32_t local_connect_address,
+    uint32_t external_connect_address,
+    shared_ptr<const IPV4RangeSet> banned_ipv4_ranges)
     : base(base),
       local_connect_address(local_connect_address),
-      external_connect_address(external_connect_address) {}
+      external_connect_address(external_connect_address),
+      banned_ipv4_ranges(banned_ipv4_ranges) {}
 
 DNSServer::~DNSServer() {
   for (const auto& it : this->fd_to_receive_event) {
@@ -55,8 +58,7 @@ void DNSServer::dispatch_on_receive_message(evutil_socket_t fd,
   reinterpret_cast<DNSServer*>(ctx)->on_receive_message(fd, events);
 }
 
-string DNSServer::response_for_query(
-    const void* vdata, size_t size, uint32_t resolved_address) {
+string DNSServer::response_for_query(const void* vdata, size_t size, uint32_t resolved_address) {
   if (size < 0x0C) {
     throw invalid_argument("query too small");
   }
@@ -82,7 +84,7 @@ string DNSServer::response_for_query(
 
 void DNSServer::on_receive_message(int fd, short) {
   for (;;) {
-    sockaddr_in remote;
+    struct sockaddr_storage remote;
     socklen_t remote_size = sizeof(sockaddr_in);
     memset(&remote, 0, remote_size);
 
@@ -104,9 +106,10 @@ void DNSServer::on_receive_message(int fd, short) {
       dns_server_log.warning("input query too small");
       print_data(stderr, input.data(), bytes);
 
-    } else {
+    } else if (!this->banned_ipv4_ranges->check(remote)) {
       input.resize(bytes);
-      uint32_t remote_address = ntohl(remote.sin_addr.s_addr);
+      const sockaddr_in* remote_sin = reinterpret_cast<const sockaddr_in*>(&remote);
+      uint32_t remote_address = ntohl(remote_sin->sin_addr.s_addr);
       uint32_t connect_address = is_local_address(remote_address)
           ? this->local_connect_address
           : this->external_connect_address;

src/DNSServer.hh

@@ -7,29 +7,37 @@
 #include <string>
 #include <unordered_map>
 
+#include "IPV4RangeSet.hh"
+
 class DNSServer {
 public:
-  DNSServer(std::shared_ptr<struct event_base> base,
-      uint32_t local_connect_address, uint32_t external_connect_address);
+  DNSServer(
+      std::shared_ptr<struct event_base> base,
+      uint32_t local_connect_address,
+      uint32_t external_connect_address,
+      std::shared_ptr<const IPV4RangeSet> banned_ipv4_ranges);
   DNSServer(const DNSServer&) = delete;
   DNSServer(DNSServer&&) = delete;
   virtual ~DNSServer();
 
+  inline void set_banned_ipv4_ranges(std::shared_ptr<const IPV4RangeSet> banned_ipv4_ranges) {
+    this->banned_ipv4_ranges = banned_ipv4_ranges;
+  }
+
   void listen(const std::string& socket_path);
   void listen(const std::string& addr, int port);
   void listen(int port);
   void add_socket(int fd);
 
-  static std::string response_for_query(
-      const void* vdata, size_t size, uint32_t resolved_address);
-  static std::string response_for_query(
-      const std::string& query, uint32_t resolved_address);
+  static std::string response_for_query(const void* vdata, size_t size, uint32_t resolved_address);
+  static std::string response_for_query(const std::string& query, uint32_t resolved_address);
 
 private:
   std::shared_ptr<struct event_base> base;
   std::unordered_map<int, std::unique_ptr<struct event, void (*)(struct event*)>> fd_to_receive_event;
   uint32_t local_connect_address;
   uint32_t external_connect_address;
+  std::shared_ptr<const IPV4RangeSet> banned_ipv4_ranges;
 
   static void dispatch_on_receive_message(evutil_socket_t fd, short events, void* ctx);
   void on_receive_message(int fd, short event);

src/EnemyType.cc

@@ -38,6 +38,8 @@ const char* name_for_enum<EnemyType>(EnemyType type) {
       return "BOOTA";
     case EnemyType::BULCLAW:
       return "BULCLAW";
+    case EnemyType::BULK:
+      return "BULK";
     case EnemyType::CANADINE:
       return "CANADINE";
     case EnemyType::CANADINE_GROUP:
@@ -287,6 +289,7 @@ EnemyType enum_for_name<EnemyType>(const char* name) {
       {"BOOMA", EnemyType::BOOMA},
       {"BOOTA", EnemyType::BOOTA},
       {"BULCLAW", EnemyType::BULCLAW},
+      {"BULK", EnemyType::BULK},
       {"CANADINE", EnemyType::CANADINE},
       {"CANADINE_GROUP", EnemyType::CANADINE_GROUP},
       {"CANANE", EnemyType::CANANE},
@@ -409,150 +412,152 @@ bool enemy_type_valid_for_episode(Episode episode, EnemyType enemy_type) {
   switch (episode) {
     case Episode::EP1:
       switch (enemy_type) {
-        case EnemyType::MOTHMANT:
-        case EnemyType::MONEST:
-        case EnemyType::SAVAGE_WOLF:
-        case EnemyType::BARBAROUS_WOLF:
-        case EnemyType::POISON_LILY:
-        case EnemyType::NAR_LILY:
-        case EnemyType::SINOW_BEAT:
-        case EnemyType::CANADINE:
-        case EnemyType::CANADINE_GROUP:
-        case EnemyType::CANANE:
-        case EnemyType::CHAOS_SORCERER:
-        case EnemyType::CHAOS_BRINGER:
-        case EnemyType::DARK_BELRA:
-        case EnemyType::DE_ROL_LE:
-        case EnemyType::DRAGON:
-        case EnemyType::SINOW_GOLD:
-        case EnemyType::RAG_RAPPY:
         case EnemyType::AL_RAPPY:
-        case EnemyType::NANO_DRAGON:
-        case EnemyType::DUBCHIC:
-        case EnemyType::GILLCHIC:
-        case EnemyType::GARANZ:
-        case EnemyType::DARK_GUNNER:
+        case EnemyType::BARBAROUS_WOLF:
+        case EnemyType::BOOMA:
         case EnemyType::BULCLAW:
+        case EnemyType::BULK:
+        case EnemyType::CANADINE_GROUP:
+        case EnemyType::CANADINE:
+        case EnemyType::CANANE:
+        case EnemyType::CHAOS_BRINGER:
+        case EnemyType::CHAOS_SORCERER:
         case EnemyType::CLAW:
-        case EnemyType::VOL_OPT_2:
-        case EnemyType::POUILLY_SLIME:
-        case EnemyType::POFUILLY_SLIME:
-        case EnemyType::PAN_ARMS:
-        case EnemyType::HIDOOM:
-        case EnemyType::MIGIUM:
-        case EnemyType::DARVANT:
-        case EnemyType::DARVANT_ULTIMATE:
+        case EnemyType::DARK_BELRA:
         case EnemyType::DARK_FALZ_1:
         case EnemyType::DARK_FALZ_2:
         case EnemyType::DARK_FALZ_3:
-        case EnemyType::HILDEBEAR:
-        case EnemyType::HILDEBLUE:
-        case EnemyType::BOOMA:
-        case EnemyType::GOBOOMA:
-        case EnemyType::GIGOBOOMA:
-        case EnemyType::GRASS_ASSASSIN:
-        case EnemyType::EVIL_SHARK:
-        case EnemyType::PAL_SHARK:
-        case EnemyType::GUIL_SHARK:
+        case EnemyType::DARK_GUNNER:
+        case EnemyType::DARVANT_ULTIMATE:
+        case EnemyType::DARVANT:
+        case EnemyType::DE_ROL_LE:
+        case EnemyType::DEATH_GUNNER:
         case EnemyType::DELSABER:
         case EnemyType::DIMENIAN:
+        case EnemyType::DRAGON:
+        case EnemyType::DUBCHIC:
+        case EnemyType::EVIL_SHARK:
+        case EnemyType::GARANZ:
+        case EnemyType::GIGOBOOMA:
+        case EnemyType::GILLCHIC:
+        case EnemyType::GOBOOMA:
+        case EnemyType::GRASS_ASSASSIN:
+        case EnemyType::GUIL_SHARK:
+        case EnemyType::HIDOOM:
+        case EnemyType::HILDEBEAR:
+        case EnemyType::HILDEBLUE:
         case EnemyType::LA_DIMENIAN:
+        case EnemyType::MIGIUM:
+        case EnemyType::MONEST:
+        case EnemyType::MOTHMANT:
+        case EnemyType::NANO_DRAGON:
+        case EnemyType::NAR_LILY:
+        case EnemyType::PAL_SHARK:
+        case EnemyType::PAN_ARMS:
+        case EnemyType::POFUILLY_SLIME:
+        case EnemyType::POISON_LILY:
+        case EnemyType::POUILLY_SLIME:
+        case EnemyType::RAG_RAPPY:
+        case EnemyType::SAVAGE_WOLF:
+        case EnemyType::SINOW_BEAT:
+        case EnemyType::SINOW_GOLD:
         case EnemyType::SO_DIMENIAN:
+        case EnemyType::VOL_OPT_2:
           return true;
         default:
           return false;
       }
     case Episode::EP2:
       switch (enemy_type) {
-        case EnemyType::MOTHMANT:
-        case EnemyType::MONEST:
-        case EnemyType::SAVAGE_WOLF:
-        case EnemyType::BARBAROUS_WOLF:
-        case EnemyType::POISON_LILY:
-        case EnemyType::NAR_LILY:
-        case EnemyType::SINOW_BERILL:
-        case EnemyType::GEE:
-        case EnemyType::CHAOS_SORCERER:
-        case EnemyType::DELBITER:
-        case EnemyType::DARK_BELRA:
         case EnemyType::BARBA_RAY:
-        case EnemyType::GOL_DRAGON:
-        case EnemyType::SINOW_SPIGELL:
-        case EnemyType::RAG_RAPPY:
-        case EnemyType::LOVE_RAPPY:
-        case EnemyType::SAINT_RAPPY:
-        case EnemyType::EGG_RAPPY:
-        case EnemyType::HALLO_RAPPY:
-        case EnemyType::GI_GUE:
-        case EnemyType::DUBCHIC:
-        case EnemyType::GILLCHIC:
-        case EnemyType::GARANZ:
-        case EnemyType::GAL_GRYPHON:
-        case EnemyType::EPSILON:
+        case EnemyType::BARBAROUS_WOLF:
+        case EnemyType::CHAOS_SORCERER:
+        case EnemyType::DARK_BELRA:
         case EnemyType::DEL_LILY:
-        case EnemyType::ILL_GILL:
-        case EnemyType::OLGA_FLOW_1:
-        case EnemyType::OLGA_FLOW_2:
-        case EnemyType::GAEL:
+        case EnemyType::DELBITER:
         case EnemyType::DELDEPTH:
-        case EnemyType::PAN_ARMS:
-        case EnemyType::HIDOOM:
-        case EnemyType::MIGIUM:
-        case EnemyType::MERICAROL:
-        case EnemyType::UL_GIBBON:
-        case EnemyType::ZOL_GIBBON:
-        case EnemyType::GIBBLES:
-        case EnemyType::MORFOS:
-        case EnemyType::RECOBOX:
-        case EnemyType::RECON:
-        case EnemyType::SINOW_ZOA:
-        case EnemyType::SINOW_ZELE:
-        case EnemyType::MERIKLE:
-        case EnemyType::MERICUS:
-        case EnemyType::HILDEBEAR:
-        case EnemyType::HILDEBLUE:
-        case EnemyType::MERILLIA:
-        case EnemyType::MERILTAS:
-        case EnemyType::GRASS_ASSASSIN:
-        case EnemyType::DOLMOLM:
-        case EnemyType::DOLMDARL:
         case EnemyType::DELSABER:
         case EnemyType::DIMENIAN:
+        case EnemyType::DOLMDARL:
+        case EnemyType::DOLMOLM:
+        case EnemyType::DUBCHIC:
+        case EnemyType::EGG_RAPPY:
+        case EnemyType::EPSILON:
+        case EnemyType::GAEL:
+        case EnemyType::GAL_GRYPHON:
+        case EnemyType::GARANZ:
+        case EnemyType::GEE:
+        case EnemyType::GI_GUE:
+        case EnemyType::GIBBLES:
+        case EnemyType::GILLCHIC:
+        case EnemyType::GOL_DRAGON:
+        case EnemyType::GRASS_ASSASSIN:
+        case EnemyType::HALLO_RAPPY:
+        case EnemyType::HIDOOM:
+        case EnemyType::HILDEBEAR:
+        case EnemyType::HILDEBLUE:
+        case EnemyType::ILL_GILL:
         case EnemyType::LA_DIMENIAN:
+        case EnemyType::LOVE_RAPPY:
+        case EnemyType::MERICAROL:
+        case EnemyType::MERICUS:
+        case EnemyType::MERIKLE:
+        case EnemyType::MERILLIA:
+        case EnemyType::MERILTAS:
+        case EnemyType::MIGIUM:
+        case EnemyType::MONEST:
+        case EnemyType::MORFOS:
+        case EnemyType::MOTHMANT:
+        case EnemyType::NAR_LILY:
+        case EnemyType::OLGA_FLOW_1:
+        case EnemyType::OLGA_FLOW_2:
+        case EnemyType::PAN_ARMS:
+        case EnemyType::POISON_LILY:
+        case EnemyType::RAG_RAPPY:
+        case EnemyType::RECOBOX:
+        case EnemyType::RECON:
+        case EnemyType::SAINT_RAPPY:
+        case EnemyType::SAVAGE_WOLF:
+        case EnemyType::SINOW_BERILL:
+        case EnemyType::SINOW_SPIGELL:
+        case EnemyType::SINOW_ZELE:
+        case EnemyType::SINOW_ZOA:
         case EnemyType::SO_DIMENIAN:
+        case EnemyType::UL_GIBBON:
+        case EnemyType::ZOL_GIBBON:
           return true;
         default:
           return false;
       }
     case Episode::EP4:
       switch (enemy_type) {
-        case EnemyType::BOOTA:
-        case EnemyType::ZE_BOOTA:
-        case EnemyType::BA_BOOTA:
-        case EnemyType::SAND_RAPPY:
-        case EnemyType::DEL_RAPPY:
-        case EnemyType::ZU:
-        case EnemyType::PAZUZU:
         case EnemyType::ASTARK:
-        case EnemyType::SATELLITE_LIZARD:
-        case EnemyType::YOWIE:
-        case EnemyType::DORPHON:
-        case EnemyType::DORPHON_ECLAIR:
-        case EnemyType::GORAN:
-        case EnemyType::PYRO_GORAN:
-        case EnemyType::GORAN_DETONATOR:
-        case EnemyType::SAND_RAPPY_ALT:
+        case EnemyType::BA_BOOTA:
+        case EnemyType::BOOTA:
         case EnemyType::DEL_RAPPY_ALT:
+        case EnemyType::DEL_RAPPY:
+        case EnemyType::DORPHON_ECLAIR:
+        case EnemyType::DORPHON:
+        case EnemyType::GIRTABLULU:
+        case EnemyType::GORAN_DETONATOR:
+        case EnemyType::GORAN:
+        case EnemyType::KONDRIEU:
         case EnemyType::MERISSA_A:
         case EnemyType::MERISSA_AA:
-        case EnemyType::ZU_ALT:
         case EnemyType::PAZUZU_ALT:
-        case EnemyType::SATELLITE_LIZARD_ALT:
-        case EnemyType::YOWIE_ALT:
-        case EnemyType::GIRTABLULU:
+        case EnemyType::PAZUZU:
+        case EnemyType::PYRO_GORAN:
         case EnemyType::SAINT_MILLION:
+        case EnemyType::SAND_RAPPY_ALT:
+        case EnemyType::SAND_RAPPY:
+        case EnemyType::SATELLITE_LIZARD_ALT:
+        case EnemyType::SATELLITE_LIZARD:
         case EnemyType::SHAMBERTIN:
-        case EnemyType::KONDRIEU:
+        case EnemyType::YOWIE_ALT:
+        case EnemyType::YOWIE:
+        case EnemyType::ZE_BOOTA:
+        case EnemyType::ZU_ALT:
+        case EnemyType::ZU:
           return true;
         default:
           return false;
@@ -611,8 +616,10 @@ uint8_t battle_param_index_for_enemy_type(Episode episode, EnemyType enemy_type)
         case EnemyType::GARANZ:
           return 0x1D;
         case EnemyType::DARK_GUNNER:
+        case EnemyType::DEATH_GUNNER:
           return 0x1E;
         case EnemyType::BULCLAW:
+        case EnemyType::BULK:
           return 0x1F;
         case EnemyType::CLAW:
           return 0x20;
@@ -834,9 +841,11 @@ uint8_t battle_param_index_for_enemy_type(Episode episode, EnemyType enemy_type)
         case EnemyType::GIRTABLULU:
           return 0x1F;
         case EnemyType::SAINT_MILLION:
-        case EnemyType::SHAMBERTIN:
-        case EnemyType::KONDRIEU:
           return 0x22;
+        case EnemyType::SHAMBERTIN:
+          return 0x26;
+        case EnemyType::KONDRIEU:
+          return 0x2A;
         default:
           throw runtime_error(string_printf("%s does not have battle parameters in Episode 4", name_for_enum(enemy_type)));
       }
@@ -863,6 +872,8 @@ uint8_t rare_table_index_for_enemy_type(EnemyType enemy_type) {
       return 0x09;
     case EnemyType::BOOTA:
       return 0x4D;
+    case EnemyType::BULK:
+      return 0x27;
     case EnemyType::BULCLAW:
       return 0x28;
     case EnemyType::CANADINE:
@@ -884,6 +895,8 @@ uint8_t rare_table_index_for_enemy_type(EnemyType enemy_type) {
       return 0x2F;
     case EnemyType::DARK_GUNNER:
       return 0x22;
+    case EnemyType::DEATH_GUNNER:
+      return 0x23;
     case EnemyType::DE_ROL_LE:
       return 0x2D;
     case EnemyType::DEL_LILY:

src/EnemyType.hh

@@ -20,6 +20,7 @@ enum class EnemyType {
   BOOMA,
   BOOTA,
   BULCLAW,
+  BULK,
   CANADINE,
   CANADINE_GROUP,
   CANANE,

src/Episode3/AssistServer.hh

@@ -39,17 +39,17 @@ private:
 
 public:
   parray<AssistEffect, 4> assist_effects;
-  std::shared_ptr<const CardIndex::CardEntry> assist_card_defs[4];
+  bcarray<std::shared_ptr<const CardIndex::CardEntry>, 4> assist_card_defs;
   uint32_t num_assist_cards_set;
   parray<uint8_t, 4> client_ids_with_assists;
   parray<AssistEffect, 4> active_assist_effects;
-  std::shared_ptr<const CardIndex::CardEntry> active_assist_card_defs[4];
+  bcarray<std::shared_ptr<const CardIndex::CardEntry>, 4> active_assist_card_defs;
   uint32_t num_active_assists;
-  std::shared_ptr<HandAndEquipState> hand_and_equip_states[4];
-  std::shared_ptr<parray<CardShortStatus, 0x10>> card_short_statuses[4];
-  std::shared_ptr<DeckEntry> deck_entries[4];
-  std::shared_ptr<parray<ActionChainWithConds, 9>> set_card_action_chains[4];
-  std::shared_ptr<parray<ActionMetadata, 9>> set_card_action_metadatas[4];
+  bcarray<std::shared_ptr<HandAndEquipState>, 4> hand_and_equip_states;
+  bcarray<std::shared_ptr<parray<CardShortStatus, 0x10>>, 4> card_short_statuses;
+  bcarray<std::shared_ptr<DeckEntry>, 4> deck_entries;
+  bcarray<std::shared_ptr<parray<ActionChainWithConds, 9>>, 4> set_card_action_chains;
+  bcarray<std::shared_ptr<parray<ActionMetadata, 9>>, 4> set_card_action_metadatas;
 };
 
 } // namespace Episode3

src/Episode3/BattleRecord.cc

@@ -9,6 +9,12 @@ using namespace std;
 
 namespace Episode3 {
 
+void BattleRecord::PlayerEntry::print(FILE* stream) const {
+  // TODO: Format this nicely somehow. Maybe factor out the functions in
+  // QuestScript that format some of these structures
+  print_data(stream, this, sizeof(this));
+}
+
 BattleRecord::Event::Event(StringReader& r) {
   this->type = r.get<Event::Type>();
   this->timestamp = r.get_u64l();
@@ -32,6 +38,7 @@ BattleRecord::Event::Event(StringReader& r) {
     case Event::Type::GAME_COMMAND:
     case Event::Type::BATTLE_COMMAND:
     case Event::Type::EP3_GAME_COMMAND:
+    case Event::Type::SERVER_DATA_COMMAND:
       this->data = r.read(r.get_u16l());
       break;
     default:
@@ -64,6 +71,7 @@ void BattleRecord::Event::serialize(StringWriter& w) const {
     case Event::Type::GAME_COMMAND:
     case Event::Type::BATTLE_COMMAND:
     case Event::Type::EP3_GAME_COMMAND:
+    case Event::Type::SERVER_DATA_COMMAND:
       w.put_u16l(this->data.size());
       w.write(this->data);
       break;
@@ -72,6 +80,51 @@ void BattleRecord::Event::serialize(StringWriter& w) const {
   }
 }
 
+void BattleRecord::Event::print(FILE* stream) const {
+  string time_str = format_time(this->timestamp);
+  fprintf(stream, "Event @%016" PRIX64 " (%s) ", this->timestamp, time_str.c_str());
+  switch (this->type) {
+    case Type::PLAYER_JOIN:
+      fprintf(stream, "PLAYER_JOIN %02" PRIX32 "\n", this->players[0].lobby_data.client_id.load());
+      this->players[0].print(stream);
+      break;
+    case Type::PLAYER_LEAVE:
+      fprintf(stream, "PLAYER_LEAVE %02hhu\n", this->leaving_client_id);
+      break;
+    case Type::SET_INITIAL_PLAYERS:
+      fprintf(stream, "SET_INITIAL_PLAYERS");
+      for (const auto& player : this->players) {
+        fprintf(stream, " %02" PRIX32, player.lobby_data.client_id.load());
+      }
+      for (const auto& player : this->players) {
+        player.print(stream);
+      }
+      break;
+    case Type::BATTLE_COMMAND:
+      fprintf(stream, "BATTLE_COMMAND\n");
+      print_data(stream, this->data);
+      break;
+    case Type::GAME_COMMAND:
+      fprintf(stream, "GAME_COMMAND\n");
+      print_data(stream, this->data);
+      break;
+    case Type::EP3_GAME_COMMAND:
+      fprintf(stream, "EP3_GAME_COMMAND\n");
+      print_data(stream, this->data);
+      break;
+    case Type::CHAT_MESSAGE:
+      fprintf(stream, "CHAT_MESSAGE %08" PRIX32 "\n", this->guild_card_number);
+      print_data(stream, this->data);
+      break;
+    case Type::SERVER_DATA_COMMAND:
+      fprintf(stream, "SERVER_DATA_COMMAND\n");
+      print_data(stream, this->data);
+      break;
+    default:
+      throw runtime_error("unknown event type in battle record");
+  }
+}
+
 BattleRecord::BattleRecord(uint32_t behavior_flags)
     : is_writable(true),
       behavior_flags(behavior_flags),
@@ -84,14 +137,23 @@ BattleRecord::BattleRecord(const string& data)
       battle_start_timestamp(0),
       battle_end_timestamp(0) {
   StringReader r(data);
+
   uint64_t signature = r.get_u64l();
-  if (signature != this->SIGNATURE) {
+  bool has_random_stream;
+  if (signature == this->SIGNATURE_V1) {
+    has_random_stream = false;
+  } else if (signature == this->SIGNATURE_V2) {
+    has_random_stream = true;
+  } else {
     throw runtime_error("incorrect battle record signature");
   }
 
   this->battle_start_timestamp = r.get_u64l();
   this->battle_end_timestamp = r.get_u64l();
   this->behavior_flags = r.get_u32l();
+  if (has_random_stream) {
+    this->random_stream = r.read(r.get_u32l());
+  }
   while (!r.eof()) {
     this->events.emplace_back(r);
   }
@@ -99,10 +161,12 @@ BattleRecord::BattleRecord(const string& data)
 
 string BattleRecord::serialize() const {
   StringWriter w;
-  w.put_u64l(this->SIGNATURE);
+  w.put_u64l(this->SIGNATURE_V2);
   w.put_u64l(this->battle_start_timestamp);
   w.put_u64l(this->battle_end_timestamp);
   w.put_u32l(this->behavior_flags);
+  w.put_u32l(this->random_stream.size());
+  w.write(this->random_stream);
   for (const auto& ev : this->events) {
     ev.serialize(w);
   }
@@ -187,6 +251,10 @@ void BattleRecord::add_chat_message(
   ev.data = std::move(data);
 }
 
+void BattleRecord::add_random_data(const void* data, size_t size) {
+  this->random_stream.append(reinterpret_cast<const char*>(data), size);
+}
+
 bool BattleRecord::is_map_definition_event(const Event& ev) {
   if (ev.type == Event::Type::BATTLE_COMMAND) {
     auto& header = check_size_t<G_CardBattleCommandHeader>(ev.data, 0xFFFF);
@@ -268,12 +336,30 @@ void BattleRecord::set_battle_start_timestamp() {
     }
   }
   this->events = std::move(new_events);
+
+  // Clear any existing random data (there shouldn't be any)
+  this->random_stream.clear();
 }
 
 void BattleRecord::set_battle_end_timestamp() {
   this->battle_end_timestamp = now();
 }
 
+void BattleRecord::print(FILE* stream) const {
+  string start_str = format_time(this->battle_start_timestamp);
+  string end_str = format_time(this->battle_end_timestamp);
+  fprintf(stream, "BattleRecord %s behavior_flags=%08" PRIX32 " start=%016" PRIX64 " (%s) end=%016" PRIX64 " (%s); %zu events\n",
+      this->is_writable ? "writable" : "read-only",
+      this->behavior_flags,
+      this->battle_start_timestamp,
+      start_str.c_str(),
+      this->battle_end_timestamp,
+      end_str.c_str(), this->events.size());
+  for (const auto& event : this->events) {
+    event.print(stream);
+  }
+}
+
 BattleRecordPlayer::BattleRecordPlayer(
     shared_ptr<const BattleRecord> rec,
     shared_ptr<struct event_base> base)
@@ -356,6 +442,10 @@ void BattleRecordPlayer::schedule_events() {
           case BattleRecord::Event::Type::CHAT_MESSAGE:
             send_prepared_chat_message(l, ev.guild_card_number, ev.data);
             break;
+          case BattleRecord::Event::Type::SERVER_DATA_COMMAND:
+            // These are not replayed, since the battle record also contains
+            // the results of these commands.
+            break;
         }
         this->event_it++;
 

src/Episode3/BattleRecord.hh

@@ -24,7 +24,9 @@ public:
     PlayerInventory inventory;
     PlayerDispDataDCPCV3 disp;
     le_uint32_t level;
-  } __attribute__((packed));
+
+    void print(FILE* stream) const;
+  } __packed_ws__(PlayerEntry, 0x440);
 
   struct Event {
     enum class Type : uint8_t {
@@ -35,6 +37,7 @@ public:
       GAME_COMMAND = 4,
       EP3_GAME_COMMAND = 5,
       CHAT_MESSAGE = 6,
+      SERVER_DATA_COMMAND = 7,
     };
 
     // Fields used for all events
@@ -52,6 +55,7 @@ public:
     Event() = default;
     explicit Event(StringReader& r);
     void serialize(StringWriter& w) const;
+    void print(FILE* stream) const;
   };
 
   explicit BattleRecord(uint32_t behavior_flags);
@@ -72,6 +76,7 @@ public:
   void add_command(Event::Type type, const void* data, size_t size);
   void add_command(Event::Type type, std::string&& data);
   void add_chat_message(uint32_t guild_card_number, std::string&& data);
+  void add_random_data(const void* data, size_t size);
   // This function collapses all the existing player join/leave events into a
   // single SET_INITIAL_PLAYERS event, and deletes all events before the latest
   // BATTLE_COMMAND command that specifies the battle map. This should provide a
@@ -79,8 +84,11 @@ public:
   void set_battle_start_timestamp();
   void set_battle_end_timestamp();
 
+  void print(FILE* stream) const;
+
 private:
-  static constexpr uint64_t SIGNATURE = 0x14C946D56D1DAC50;
+  static constexpr uint64_t SIGNATURE_V1 = 0x14C946D56D1DAC50;
+  static constexpr uint64_t SIGNATURE_V2 = 0xD01E5EC12853C377;
 
   static bool is_map_definition_event(const Event& ev);
 
@@ -90,15 +98,14 @@ private:
   uint64_t battle_start_timestamp;
   uint64_t battle_end_timestamp;
   std::deque<Event> events;
+  std::string random_stream;
 
   friend class BattleRecordPlayer;
 };
 
 class BattleRecordPlayer {
 public:
-  BattleRecordPlayer(
-      std::shared_ptr<const BattleRecord> rec,
-      std::shared_ptr<struct event_base> base);
+  BattleRecordPlayer(std::shared_ptr<const BattleRecord> rec, std::shared_ptr<struct event_base> base);
   ~BattleRecordPlayer() = default;
 
   std::shared_ptr<const BattleRecord> get_record() const;
@@ -116,6 +123,7 @@ private:
   std::shared_ptr<struct event_base> base;
   std::weak_ptr<Lobby> lobby;
   std::shared_ptr<struct event> next_command_ev;
+  StringReader random_r;
 };
 
 } // namespace Episode3

src/Episode3/Card.cc

@@ -204,7 +204,7 @@ ssize_t Card::apply_abnormal_condition(
     }
   }
 
-  string cond_str = cond.str();
+  string cond_str = cond.str(s);
   log.debug("wrote condition %zd => %s", cond_index, cond_str.c_str());
 
   if (!is_nte) {
@@ -213,7 +213,7 @@ ssize_t Card::apply_abnormal_condition(
       if (this->action_chain.conditions[z].type == ConditionType::NONE) {
         continue;
       }
-      string cond_str = cond.str();
+      string cond_str = cond.str(s);
       log.debug("sorted conditions: [%zu] => %s", z, cond_str.c_str());
     }
   }
@@ -914,7 +914,7 @@ void Card::compute_action_chain_results(bool apply_action_conditions, bool ignor
   this->action_chain.chain.tp_effect_bonus = 0;
 
   log.debug("(initial) medium=%s, strike_count=%hhu, ap_effect_bonus=%hhd, tp_effect_bonus=%hhd",
-      name_for_attack_medium(this->action_chain.chain.attack_medium),
+      name_for_enum(this->action_chain.chain.attack_medium),
       this->action_chain.chain.strike_count,
       this->action_chain.chain.ap_effect_bonus,
       this->action_chain.chain.tp_effect_bonus);
@@ -1361,11 +1361,12 @@ bool Card::is_guard_item() const {
 }
 
 bool Card::unknown_80236554(shared_ptr<Card> other_card, const ActionState* as) {
-  auto log = this->server()->log_stack(other_card
+  auto s = this->server();
+  auto log = s->log_stack(other_card
           ? string_printf("unknown_80236554(@%04hX #%04hX, @%04hX #%04hX): ", this->get_card_ref(), this->get_card_id(), other_card->get_card_ref(), other_card->get_card_id())
           : string_printf("unknown_80236554(@%04hX #%04hX, null): ", this->get_card_ref(), this->get_card_id()));
   if (as) {
-    string as_str = as->str();
+    string as_str = as->str(s);
     log.debug("as = %s", as_str.c_str());
   } else {
     log.debug("as = null");
@@ -1403,8 +1404,8 @@ bool Card::unknown_80236554(shared_ptr<Card> other_card, const ActionState* as)
   log.debug("last attack damage stats cleared");
 
   if (other_card) {
-    this->server()->card_special->apply_action_conditions(0x03, other_card, this->shared_from_this(), 0x20, as);
-    this->server()->card_special->apply_action_conditions(0x17, other_card, this->shared_from_this(), 0x40, as);
+    s->card_special->apply_action_conditions(0x03, other_card, this->shared_from_this(), 0x20, as);
+    s->card_special->apply_action_conditions(0x17, other_card, this->shared_from_this(), 0x40, as);
     if (other_card->action_chain.check_flag(0x20000)) {
       this->action_metadata.attack_bonus = 0;
       return ret;
@@ -1417,8 +1418,9 @@ bool Card::unknown_80236554(shared_ptr<Card> other_card, const ActionState* as)
 }
 
 void Card::unknown_802362D8(shared_ptr<Card> other_card) {
+  auto s = this->server();
   for (size_t client_id = 0; client_id < 4; client_id++) {
-    auto ps = this->server()->player_states[client_id];
+    auto ps = s->player_states[client_id];
     if (ps) {
       shared_ptr<Card> card = ps->get_sc_card();
       if (card) {
@@ -1493,7 +1495,7 @@ void Card::apply_attack_result() {
         } else {
           auto target_sc = target_ps->get_sc_card();
           if (!(target_sc->card_flags & 2)) {
-            temp_chain.chain.target_card_refs[temp_chain.chain.target_card_ref_count] = candidate_card->get_card_ref();
+            temp_chain.chain.target_card_refs[temp_chain.chain.target_card_ref_count] = target_sc->get_card_ref();
             temp_chain.chain.target_card_ref_count++;
           }
         }

src/Episode3/Card.hh

@@ -15,11 +15,7 @@ class PlayerState;
 
 class Card : public std::enable_shared_from_this<Card> {
 public:
-  Card(
-      uint16_t card_id,
-      uint16_t card_ref,
-      uint16_t client_id,
-      std::shared_ptr<Server> server);
+  Card(uint16_t card_id, uint16_t card_ref, uint16_t client_id, std::shared_ptr<Server> server);
   void init();
   std::shared_ptr<Server> server();
   std::shared_ptr<const Server> server() const;
@@ -47,8 +43,7 @@ public:
       G_ApplyConditionEffect_Ep3_6xB4x06* cmd,
       size_t strike_number,
       int16_t* out_effective_damage);
-  int16_t compute_defense_power_for_attacker_card(
-      std::shared_ptr<const Card> attacker_card);
+  int16_t compute_defense_power_for_attacker_card(std::shared_ptr<const Card> attacker_card);
   void destroy_set_card(std::shared_ptr<Card> attacker_card);
   int32_t error_code_for_move_to_location(const Location& loc) const;
   void execute_attack(std::shared_ptr<Card> attacker_card);
@@ -73,12 +68,10 @@ public:
   void send_6xB4x4E_4C_4D_if_needed(bool always_send = false);
   void send_6xB4x4E_if_needed(bool always_send = false);
   void set_current_and_max_hp(int16_t hp);
-  void set_current_hp(
-      uint32_t new_hp, bool propagate_shared_hp = true, bool enforce_max_hp = true);
+  void set_current_hp(uint32_t new_hp, bool propagate_shared_hp = true, bool enforce_max_hp = true);
   void update_stats_on_destruction();
   void clear_action_chain_and_metadata_and_most_flags();
-  void compute_action_chain_results(
-      bool apply_action_conditions, bool ignore_this_card_ap_tp);
+  void compute_action_chain_results(bool apply_action_conditions, bool ignore_this_card_ap_tp);
   void unknown_802380C0();
   void unknown_80237F98(bool require_condition_20_or_21);
   void unknown_80237F88();

src/Episode3/CardSpecial.cc

@@ -393,9 +393,7 @@ bool CardSpecial::apply_defense_condition(
 
   string expr = orig_eff->expr.decode();
   int16_t expr_value = this->evaluate_effect_expr(astats, expr.c_str(), dice_roll);
-  this->execute_effect(
-      *defender_cond, defender_card, expr_value, defender_cond->value,
-      orig_eff->type, flags, attacker_card_ref);
+  this->execute_effect(*defender_cond, defender_card, expr_value, defender_cond->value, orig_eff->type, flags, attacker_card_ref);
   if (flags & 4) {
     if (is_nte || !(defender_card->card_flags & 2)) {
       defender_card->compute_action_chain_results(true, false);
@@ -460,7 +458,7 @@ bool CardSpecial::apply_stat_deltas_to_card_from_condition_and_clear_cond(Condit
   auto log = s->log_stack(string_printf("apply_stat_deltas_to_card_from_condition_and_clear_cond(@%04hX #%04hX): ", card->get_card_ref(), card->get_card_id()));
   bool is_nte = s->options.is_nte();
 
-  string cond_str = cond.str();
+  string cond_str = cond.str(s);
   log.debug("cond: %s", cond_str.c_str());
 
   ConditionType cond_type = cond.type;
@@ -589,7 +587,7 @@ bool CardSpecial::apply_stat_deltas_to_card_from_condition_and_clear_cond(Condit
       break;
     trial_unimplemented:
     default:
-      log.debug("%s: no adjustments for condition type", name_for_condition_type(cond_type));
+      log.debug("%s: no adjustments for condition type", name_for_enum(cond_type));
       break;
   }
 
@@ -706,7 +704,7 @@ CardSpecial::AttackEnvStats CardSpecial::compute_attack_env_stats(
   auto log = s->log_stack("compute_attack_env_stats: ");
   bool is_nte = s->options.is_nte();
 
-  string pa_str = pa.str();
+  string pa_str = pa.str(s);
   log.debug("pa=%s, card=@%04hX #%04hX, dice_roll=%hhu, target=@%04hX, condition_giver=@%04hX", pa_str.c_str(), card->get_card_ref(), card->get_card_id(), dice_roll.value, target_card_ref, condition_giver_card_ref);
 
   auto attacker_card = s->card_for_set_card_ref(pa.attacker_card_ref);
@@ -834,16 +832,16 @@ CardSpecial::AttackEnvStats CardSpecial::compute_attack_env_stats(
   size_t z = 0;
 
   uint16_t z_ref = pa.attacker_card_ref;
-  // Note: The (z < 9) conditions in these two loops are not present in the
+  // Note: The (z < 8) conditions in these two loops are not present in the
   // original code.
   for (z = 0;
-       ((target_card_ref != z_ref) && (z < 9) && ((z_ref = pa.action_card_refs[z]) != 0xFFFF));
+       ((target_card_ref != z_ref) && (z < 8) && ((z_ref = pa.action_card_refs[z]) != 0xFFFF));
        z++) {
   }
 
   ast.action_cards_ap = 0;
   ast.action_cards_tp = 0;
-  for (; (z < 9) && (pa.action_card_refs[z] != 0xFFFF); z++) {
+  for (; (z < 8) && (pa.action_card_refs[z] != 0xFFFF); z++) {
     auto ce = s->definition_for_card_ref(pa.action_card_refs[z]);
     if (ce) {
       if (ce->def.ap.type != CardDefinition::Stat::Type::MINUS_STAT) {
@@ -1192,7 +1190,8 @@ shared_ptr<Card> CardSpecial::compute_replaced_target_based_on_conditions(
 }
 
 StatSwapType CardSpecial::compute_stat_swap_type(shared_ptr<const Card> card) const {
-  auto log = this->server()->log_stack(string_printf("compute_stat_swap_type(@%04hX #%04hX): ", card->get_card_ref(), card->get_card_id()));
+  auto s = this->server();
+  auto log = s->log_stack(string_printf("compute_stat_swap_type(@%04hX #%04hX): ", card->get_card_ref(), card->get_card_id()));
   if (!card) {
     log.debug("card is missing");
     return StatSwapType::NONE;
@@ -1203,7 +1202,7 @@ StatSwapType CardSpecial::compute_stat_swap_type(shared_ptr<const Card> card) co
     auto& cond = card->action_chain.conditions[cond_index];
     if (cond.type != ConditionType::NONE) {
       auto cond_log = log.sub(string_printf("(%zu) ", cond_index));
-      string cond_str = cond.str();
+      string cond_str = cond.str(s);
       cond_log.debug("%s", cond_str.c_str());
       if (!this->card_ref_has_ability_trap(cond)) {
         if (cond.type == ConditionType::UNKNOWN_75) {
@@ -1423,15 +1422,14 @@ bool CardSpecial::evaluate_effect_arg2_condition(
     attacker_card_ref = as.original_attacker_card_ref;
   }
 
+  bool is_nte = s->options.is_nte();
   auto set_card = s->card_for_set_card_ref(set_card_ref);
   bool set_card_has_ability_trap =
-      (!s->options.is_nte() &&
-          set_card &&
-          this->card_has_condition_with_ref(set_card, ConditionType::ABILITY_TRAP, 0xFFFF, 0xFFFF));
+      (!is_nte && set_card && this->card_has_condition_with_ref(set_card, ConditionType::ABILITY_TRAP, 0xFFFF, 0xFFFF));
 
   switch (arg2_text[0]) {
     case 'C':
-      if (s->options.is_nte()) {
+      if (is_nte) {
         return false;
       }
       card = s->card_for_set_card_ref(set_card_ref);
@@ -1533,16 +1531,16 @@ bool CardSpecial::evaluate_effect_arg2_condition(
             uint16_t action_card_ref = as.action_card_refs[z];
             if (action_card_ref != 0xFFFF) {
               auto ce = s->definition_for_card_ref(action_card_ref);
-              if (card_class_is_tech_like(ce->def.card_class(), s->options.is_nte())) {
+              if (card_class_is_tech_like(ce->def.card_class(), is_nte)) {
                 return true;
               }
             }
           }
           return false;
         case 0x04: // n04
-          return card->action_chain.check_flag(s->options.is_nte() ? 0x00000080 : 0x0001E000);
+          return card->action_chain.check_flag(is_nte ? 0x00000080 : 0x0001E000);
         case 0x05: // n05
-          return card->action_chain.check_flag(s->options.is_nte() ? 0x00000002 : 0x00001E00);
+          return card->action_chain.check_flag(is_nte ? 0x00000002 : 0x00001E00);
         case 0x06: // n06
           return (card->get_definition()->def.card_class() == CardClass::NATIVE_CREATURE);
         case 0x07: // n07
@@ -1560,9 +1558,8 @@ bool CardSpecial::evaluate_effect_arg2_condition(
         case 0x0D: { // n13
           auto ce = card->get_definition();
           return ((ce->def.card_class() == CardClass::GUARD_ITEM) ||
-              (!s->options.is_nte() && (ce->def.card_class() == CardClass::MAG_ITEM)) ||
-              s->ruler_server->find_condition_on_card_ref(
-                  card->get_card_ref(), ConditionType::GUARD_CREATURE, 0, 0, 0));
+              (!is_nte && (ce->def.card_class() == CardClass::MAG_ITEM)) ||
+              s->ruler_server->find_condition_on_card_ref(card->get_card_ref(), ConditionType::GUARD_CREATURE, 0, 0, 0));
         }
         case 0x0E: // n14
           return card->get_definition()->def.is_sc();
@@ -1600,7 +1597,7 @@ bool CardSpecial::evaluate_effect_arg2_condition(
           return s->ruler_server->find_condition_on_card_ref(
               card->get_card_ref(), ConditionType::FREEZE, 0, 0, 0);
         case 0x15: { // n21
-          if (!s->options.is_nte()) {
+          if (!is_nte) {
             uint8_t client_id = client_id_for_card_ref(sc_card_ref);
             if (client_id != 0xFF) {
               return card->action_chain.check_flag(0x00002000 << client_id);
@@ -1609,7 +1606,7 @@ bool CardSpecial::evaluate_effect_arg2_condition(
           return false;
         }
         case 0x16: { // n22
-          if (!s->options.is_nte()) {
+          if (!is_nte) {
             uint8_t client_id = client_id_for_card_ref(sc_card_ref);
             if (client_id != 0xFF) {
               return card->action_chain.check_flag(0x00000200 << client_id);
@@ -1637,7 +1634,7 @@ bool CardSpecial::evaluate_effect_arg2_condition(
                   card, ConditionType::ANY, set_card_ref, ((v % 10) == 0) ? 0xFF : (v % 10)) != nullptr);
     }
     case 'r':
-      return (!set_card_has_ability_trap || s->options.is_nte()) && (random_percent < atoi(arg2_text + 1));
+      return (!set_card_has_ability_trap || is_nte) && (random_percent < atoi(arg2_text + 1));
     case 's': {
       auto ce = card->get_definition();
       return ((ce->def.self_cost >= arg2_text[1] - '0') &&
@@ -1651,7 +1648,7 @@ bool CardSpecial::evaluate_effect_arg2_condition(
       uint8_t v = atoi(arg2_text + 1);
       // TODO: Figure out what this logic actually does and rename the variables
       // or comment it appropriately.
-      if (s->options.is_nte()) {
+      if (is_nte) {
         return (v < set_card->unknown_a9);
       } else if (when == 4) {
         uint32_t y = set_card->unknown_a9 & 0xFFFFFFFE;
@@ -1775,8 +1772,8 @@ bool CardSpecial::execute_effect(
   auto s = this->server();
   auto log = s->log_stack(string_printf("execute_effect(@%04hX #%04hX): ", card->get_card_ref(), card->get_card_id()));
   {
-    string cond_str = cond.str();
-    log.debug("cond=%s, card=@%04hX, expr_value=%hd, unknown_p5=%hd, cond_type=%s, unknown_p7=%" PRIu32 ", attacker_card_ref=@%04hX", cond_str.c_str(), ref_for_card(card), expr_value, unknown_p5, name_for_condition_type(cond_type), unknown_p7, attacker_card_ref);
+    string cond_str = cond.str(s);
+    log.debug("cond=%s, card=@%04hX, expr_value=%hd, unknown_p5=%hd, cond_type=%s, unknown_p7=%" PRIu32 ", attacker_card_ref=@%04hX", cond_str.c_str(), ref_for_card(card), expr_value, unknown_p5, name_for_enum(cond_type), unknown_p7, attacker_card_ref);
   }
   bool is_nte = s->options.is_nte();
 
@@ -2460,13 +2457,13 @@ bool CardSpecial::execute_effect(
       [[fallthrough]];
     case ConditionType::SLAYERS_ASSASSINS:
       if (is_nte) {
-        auto card = s->card_for_set_card_ref(attacker_card_ref);
+        auto set_card = s->card_for_set_card_ref(attacker_card_ref);
         bool card_found = false;
-        if (!card) {
+        if (!set_card) {
           card_found = false;
         } else {
-          for (size_t z = 0; z < card->action_chain.chain.target_card_ref_count; z++) {
-            if (card->action_chain.chain.target_card_refs[z] == card->get_card_ref()) {
+          for (size_t z = 0; z < set_card->action_chain.chain.target_card_ref_count; z++) {
+            if (set_card->action_chain.chain.target_card_refs[z] == card->get_card_ref()) {
               card_found = true;
               break;
             }
@@ -2484,7 +2481,7 @@ bool CardSpecial::execute_effect(
 
       } else if (unknown_p7 & 0x20) {
         card->action_metadata.attack_bonus = clamp<int16_t>(
-            positive_expr_value + card->action_metadata.attack_bonus, -99, 99);
+            card->action_metadata.attack_bonus + positive_expr_value, -99, 99);
       }
       return true;
 
@@ -2840,7 +2837,7 @@ vector<shared_ptr<const Card>> CardSpecial::get_targeted_cards_for_condition(
   AttackMedium attack_medium = card2
       ? card2->action_chain.chain.attack_medium
       : AttackMedium::UNKNOWN;
-  log.debug("attack_medium=%s", name_for_attack_medium(attack_medium));
+  log.debug("attack_medium=%s", name_for_enum(attack_medium));
 
   auto add_card_refs = [&](const vector<uint16_t>& result_card_refs) -> void {
     for (uint16_t result_card_ref : result_card_refs) {
@@ -2899,9 +2896,9 @@ vector<shared_ptr<const Card>> CardSpecial::get_targeted_cards_for_condition(
       break;
     case 0x04: // p04
       size_t z;
-      for (z = 0; (z < 9) && (as.action_card_refs[z] != 0xFFFF) && (as.action_card_refs[z] != card_ref); z++) {
+      for (z = 0; (z < 8) && (as.action_card_refs[z] != 0xFFFF) && (as.action_card_refs[z] != card_ref); z++) {
       }
-      for (; (z < 9) && (as.action_card_refs[z] != 0xFFFF); z++) {
+      for (; (z < 8) && (as.action_card_refs[z] != 0xFFFF); z++) {
         auto result_card = s->card_for_set_card_ref(as.action_card_refs[z]);
         if (result_card) {
           ret.emplace_back(result_card);
@@ -3153,20 +3150,35 @@ vector<shared_ptr<const Card>> CardSpecial::get_targeted_cards_for_condition(
       }
       break;
     case 0x24: { // p36
+      auto log36 = log.sub("(p36) ");
       // On NTE, this includes SCs and items; on other versions, it's SCs only
       static const auto should_include = +[](shared_ptr<const CardIndex::CardEntry> ce, bool is_nte) -> bool {
         return (ce && (ce->def.is_sc() || (is_nte ? (ce->def.type == CardType::ITEM) : false)));
       };
       bool is_nte = s->options.is_nte();
       if (as.original_attacker_card_ref == 0xFFFF) {
+        log36.debug("original_attacker_card_ref missing");
+        // debug_str_for_card_ref
         for (size_t z = 0; (z < 4 * 9) && (as.target_card_refs[z] != 0xFFFF); z++) {
+          string debug_ref_str = s->debug_str_for_card_ref(as.target_card_refs[z]);
+          log36.debug("examining %s", debug_ref_str.c_str());
           auto result_card = s->card_for_set_card_ref(as.target_card_refs[z]);
           if (result_card && should_include(result_card->get_definition(), is_nte)) {
+            log36.debug("adding %s", debug_ref_str.c_str());
             ret.emplace_back(result_card);
+          } else {
+            log36.debug("skipping %s", debug_ref_str.c_str());
           }
         }
       } else if (card2 && should_include(card2->get_definition(), is_nte)) {
+        string debug_ref_str = s->debug_str_for_card_ref(card2->get_card_ref());
+        log36.debug("original_attacker_card_ref present; adding card2 = %s", debug_ref_str.c_str());
         ret.emplace_back(card2);
+      } else if (card2) {
+        string debug_ref_str = s->debug_str_for_card_ref(card2->get_card_ref());
+        log36.debug("original_attacker_card_ref present and card2 (%s) not eligible", debug_ref_str.c_str());
+      } else {
+        log36.debug("original_attacker_card_ref present and card2 missing");
       }
       break;
     }
@@ -3510,8 +3522,7 @@ void CardSpecial::on_card_set(shared_ptr<PlayerState> ps, uint16_t card_ref) {
   this->evaluate_and_apply_effects(0x01, card_ref, as, sc_card_ref);
 }
 
-const CardDefinition::Effect* CardSpecial::original_definition_for_condition(
-    const Condition& cond) const {
+const CardDefinition::Effect* CardSpecial::original_definition_for_condition(const Condition& cond) const {
   auto ce = this->server()->definition_for_card_ref(cond.card_ref);
   if (!ce) {
     return nullptr;
@@ -3525,8 +3536,7 @@ bool CardSpecial::card_ref_has_ability_trap(const Condition& cond) const {
   if (!card) {
     return false;
   } else {
-    return this->card_has_condition_with_ref(
-        card, ConditionType::ABILITY_TRAP, 0xFFFF, 0xFFFF);
+    return this->card_has_condition_with_ref(card, ConditionType::ABILITY_TRAP, 0xFFFF, 0xFFFF);
   }
 }
 
@@ -3566,8 +3576,7 @@ void CardSpecial::send_6xB4x06_for_card_destroyed(
   this->server()->send(cmd);
 }
 
-uint16_t CardSpecial::send_6xB4x06_if_card_ref_invalid(
-    uint16_t card_ref, int16_t value) const {
+uint16_t CardSpecial::send_6xB4x06_if_card_ref_invalid(uint16_t card_ref, int16_t value) const {
   auto s = this->server();
   if (!s->options.is_nte() && !s->card_ref_is_empty_or_has_valid_card_id(card_ref)) {
     if (value != 0) {
@@ -3850,8 +3859,10 @@ void CardSpecial::check_for_defense_interference(
     shared_ptr<const Card> attacker_card,
     shared_ptr<Card> target_card,
     int16_t* inout_unknown_p4) {
+  auto s = this->server();
+
   // Note: This check is not part of the original implementation.
-  if (this->server()->options.behavior_flags & BehaviorFlag::DISABLE_INTERFERENCE) {
+  if (s->options.behavior_flags & BehaviorFlag::DISABLE_INTERFERENCE) {
     return;
   }
 
@@ -3862,13 +3873,13 @@ void CardSpecial::check_for_defense_interference(
     return;
   }
 
-  uint16_t ally_sc_card_ref = this->server()->ruler_server->get_ally_sc_card_ref(
+  uint16_t ally_sc_card_ref = s->ruler_server->get_ally_sc_card_ref(
       target_card->get_card_ref());
   if (ally_sc_card_ref == 0xFFFF) {
     return;
   }
 
-  auto ally_sc = this->server()->card_for_set_card_ref(ally_sc_card_ref);
+  auto ally_sc = s->card_for_set_card_ref(ally_sc_card_ref);
   if (!ally_sc || (ally_sc->card_flags & 2)) {
     return;
   }
@@ -3883,17 +3894,17 @@ void CardSpecial::check_for_defense_interference(
     return;
   }
 
-  auto ally_hes = this->server()->ruler_server->get_hand_and_equip_state_for_client_id(target_ally_client_id);
-  if (!ally_hes || (!(this->server()->options.behavior_flags & BehaviorFlag::ALLOW_NON_COM_INTERFERENCE) && !ally_hes->is_cpu_player)) {
+  auto ally_hes = s->ruler_server->get_hand_and_equip_state_for_client_id(target_ally_client_id);
+  if (!ally_hes || (!(s->options.behavior_flags & BehaviorFlag::ALLOW_NON_COM_INTERFERENCE) && !ally_hes->is_cpu_player)) {
     return;
   }
 
-  uint16_t target_card_id = this->server()->card_id_for_card_ref(target_card->get_card_ref());
+  uint16_t target_card_id = s->card_id_for_card_ref(target_card->get_card_ref());
   if (target_card_id == 0xFFFF) {
     return;
   }
 
-  uint16_t ally_sc_card_id = this->server()->card_id_for_card_ref(ally_sc_card_ref);
+  uint16_t ally_sc_card_id = s->card_id_for_card_ref(ally_sc_card_ref);
   if (ally_sc_card_id == 0xFFFF) {
     return;
   }
@@ -3907,7 +3918,7 @@ void CardSpecial::check_for_defense_interference(
   }
   auto entry = get_interference_probability_entry(
       target_card_id, ally_sc_card_id, false);
-  if (!entry || (this->server()->get_random(99) >= entry->defense_probability)) {
+  if (!entry || (s->get_random(99) >= entry->defense_probability)) {
     return;
   }
 
@@ -3919,7 +3930,7 @@ void CardSpecial::check_for_defense_interference(
   cmd.effect.target_card_ref = target_card->get_card_ref();
   cmd.effect.value = 0;
   cmd.effect.operation = 0x7D;
-  this->server()->send(cmd);
+  s->send(cmd);
   if (inout_unknown_p4) {
     *inout_unknown_p4 = 0;
     target_card->action_metadata.set_flags(0x10);
@@ -3938,7 +3949,7 @@ void CardSpecial::evaluate_and_apply_effects(
   bool is_nte = s->options.is_nte();
 
   {
-    string as_str = as.str();
+    string as_str = as.str(s);
     log.debug("when=%02hhX, set_card_ref=@%04hX, as=%s, sc_card_ref=@%04hX, apply_defense_condition_to_all_cards=%s, apply_defense_condition_to_card_ref=@%04hX",
         when, set_card_ref, as_str.c_str(), sc_card_ref, apply_defense_condition_to_all_cards ? "true" : "false", apply_defense_condition_to_card_ref);
   }
@@ -4114,7 +4125,7 @@ void CardSpecial::evaluate_and_apply_effects(
           // bug probably does nothing in any reasonable scenario, since the
           // target card refs array immediately precedes the conditions array,
           // and the target card refs array is excessively long, so OR'ing a
-          // value that is amost certainly already 0xFFFF with 1 would do
+          // value that is almost certainly already 0xFFFF with 1 would do
           // nothing. In our implementation, however, we bounds-check
           // everything, so we've moved this check inside the relevant if block.
           if (dice_roll.value_used_in_expr) {
@@ -4139,8 +4150,7 @@ void CardSpecial::evaluate_and_apply_effects(
 
   if (any_expr_used_dice_roll) {
     dice_cmd.effect.flags = 0x08;
-    dice_cmd.effect.attacker_card_ref = this->send_6xB4x06_if_card_ref_invalid(
-        as_attacker_card_ref, 0x15);
+    dice_cmd.effect.attacker_card_ref = this->send_6xB4x06_if_card_ref_invalid(as_attacker_card_ref, 0x15);
     dice_cmd.effect.dice_roll_value = dice_roll.value;
     s->send(dice_cmd);
   }
@@ -4664,13 +4674,13 @@ vector<shared_ptr<const Card>> CardSpecial::filter_cards_by_range(
 }
 
 void CardSpecial::unknown_8024AAB8(const ActionState& as) {
-  auto log = this->server()->log_stack("unknown_8024AAB8: ");
-  string as_str = as.str();
+  auto s = this->server();
+  auto log = s->log_stack("unknown_8024AAB8: ");
+  string as_str = as.str(s);
   log.debug("as=%s", as_str.c_str());
 
   for (size_t z = 0; (z < 8) && (as.action_card_refs[z] != 0xFFFF); z++) {
-    uint16_t card_ref = this->send_6xB4x06_if_card_ref_invalid(
-        as.action_card_refs[z], 0x1E);
+    uint16_t card_ref = this->send_6xB4x06_if_card_ref_invalid(as.action_card_refs[z], 0x1E);
     if (card_ref == 0xFFFF) {
       break;
     }
@@ -4697,8 +4707,7 @@ void CardSpecial::unknown_8024AAB8(const ActionState& as) {
     card_ref2 = this->send_6xB4x06_if_card_ref_invalid(as.attacker_card_ref, 0x26);
     this->evaluate_and_apply_effects(0x34, card_ref2, as, card_ref1);
     for (size_t z = 0; (z < 4 * 9) && (as.target_card_refs[z] != 0xFFFF); z++) {
-      uint16_t card_ref = this->send_6xB4x06_if_card_ref_invalid(
-          as.action_card_refs[z], 0x27);
+      uint16_t card_ref = this->send_6xB4x06_if_card_ref_invalid(as.action_card_refs[z], 0x27);
       if (card_ref == 0xFFFF) {
         break;
       }
@@ -4720,9 +4729,10 @@ void CardSpecial::move_phase_before_for_card(shared_ptr<Card> card) {
 
 void CardSpecial::dice_phase_before_for_card(shared_ptr<Card> card) {
   auto s = this->server();
+  bool is_nte = s->options.is_nte();
 
   auto ps = card->player_state();
-  if (s->options.is_nte() && (!ps || !ps->is_team_turn())) {
+  if (is_nte && (!ps || !ps->is_team_turn())) {
     return;
   }
 
@@ -4739,7 +4749,7 @@ void CardSpecial::dice_phase_before_for_card(shared_ptr<Card> card) {
     }
   }
 
-  if (!s->options.is_nte()) {
+  if (!is_nte) {
     this->apply_defense_conditions(as, 0x46, card, 0x04);
     this->evaluate_and_apply_effects(0x46, card->get_card_ref(), as, sc_card_ref);
   }
@@ -4940,8 +4950,7 @@ void CardSpecial::check_for_attack_interference(shared_ptr<Card> unknown_p2) {
 
   G_ApplyConditionEffect_Ep3_6xB4x06 cmd;
   cmd.effect.flags = 0x04;
-  cmd.effect.attacker_card_ref = this->send_6xB4x06_if_card_ref_invalid(
-      unknown_p2->get_card_ref(), 0x11);
+  cmd.effect.attacker_card_ref = this->send_6xB4x06_if_card_ref_invalid(unknown_p2->get_card_ref(), 0x11);
   cmd.effect.target_card_ref = unknown_p2->get_card_ref();
   cmd.effect.value = 0;
   cmd.effect.operation = 0x7D;

src/Episode3/CardSpecial.hh

@@ -6,6 +6,7 @@
 
 #include "../Text.hh"
 #include "DataIndexes.hh"
+#include "Server.hh"
 
 namespace Episode3 {
 
@@ -94,7 +95,7 @@ public:
     void print(FILE* stream) const;
 
     uint32_t at(size_t index) const;
-  } __attribute__((packed));
+  } __packed_ws__(AttackEnvStats, 0x9C);
 
   CardSpecial(std::shared_ptr<Server> server);
   std::shared_ptr<Server> server();

src/Episode3/DataIndexes.hh

@@ -37,6 +37,7 @@ enum BehaviorFlag : uint32_t {
   DISABLE_INTERFERENCE = 0x00000100,
   ALLOW_NON_COM_INTERFERENCE = 0x00000200,
   IS_TRIAL_EDITION = 0x00000400,
+  LOG_COMMANDS_IF_LOBBY_MISSING = 0x00000800,
 };
 
 enum class StatSwapType : uint8_t {
@@ -59,8 +60,6 @@ enum class AttackMedium : uint8_t {
   INVALID_FF = 0xFF,
 };
 
-const char* name_for_attack_medium(AttackMedium medium);
-
 enum class CriterionCode : uint8_t {
   NONE = 0x00,
   HU_CLASS_SC = 0x01,
@@ -99,8 +98,6 @@ enum class CriterionCode : uint8_t {
   NON_PHYSICAL_NON_TECH_NON_UNKNOWN_ATTACK_MEDIUM_NON_SC = 0x22,
 };
 
-const char* name_for_criterion_code(CriterionCode code);
-
 enum class CardRank : uint8_t {
   N1 = 0x01,
   R1 = 0x02,
@@ -138,8 +135,6 @@ enum class CardType : uint8_t {
   END_CARD_LIST = 0xFF,
 };
 
-const char* name_for_card_type(CardType type);
-
 enum class CardClass : uint16_t {
   HU_SC = 0x0000,
   RA_SC = 0x0001,
@@ -163,7 +158,6 @@ enum class CardClass : uint16_t {
   ASSIST = 0x0028,
 };
 
-const char* name_for_card_class(CardClass cc);
 bool card_class_is_tech_like(CardClass cc, bool is_nte);
 
 enum class TargetMode : uint8_t {
@@ -310,8 +304,6 @@ enum class ConditionType : uint8_t {
   ANY_FF = 0xFF, // Used as a wildcard in some search functions
 };
 
-const char* name_for_condition_type(ConditionType cond_type);
-
 enum class AssistEffect : uint16_t {
   NONE = 0x0000,
   DICE_HALF = 0x0001,
@@ -408,8 +400,6 @@ enum class ActionSubphase : uint8_t {
   INVALID_FF = 0xFF,
 };
 
-const char* name_for_action_subphase(ActionSubphase subphase);
-
 enum class SetupPhase : uint8_t {
   REGISTRATION = 0,
   STARTER_ROLLS = 1,
@@ -439,7 +429,6 @@ enum class Direction : uint8_t {
 Direction turn_left(Direction d);
 Direction turn_right(Direction d);
 Direction turn_around(Direction d);
-const char* name_for_direction(Direction d);
 
 struct Location {
   /* 00 */ uint8_t x;
@@ -458,7 +447,7 @@ struct Location {
 
   void clear();
   void clear_FF();
-} __attribute__((packed));
+} __packed_ws__(Location, 4);
 
 struct CardDefinition {
   struct Stat {
@@ -480,7 +469,8 @@ struct CardDefinition {
 
     void decode_code();
     std::string str() const;
-  } __attribute__((packed));
+    JSON json() const;
+  } __packed_ws__(Stat, 4);
 
   struct Effect {
     // effect_num is the 1-based index of this effect within the card definition
@@ -516,10 +506,11 @@ struct CardDefinition {
     bool is_empty() const;
     static std::string str_for_arg(const std::string& arg);
     std::string str(const char* separator = ", ", const TextSet* text_archive = nullptr) const;
-  } __attribute__((packed));
+    JSON json() const;
+  } __packed_ws__(Effect, 0x20);
 
   /* 0000 */ be_uint32_t card_id;
-  /* 0004 */ parray<uint8_t, 0x40> jp_name;
+  /* 0004 */ pstring<TextEncoding::SJIS, 0x40> jp_name;
 
   // The list of card definitions ends with a "sentinel" definition that isn't a
   // real card, but instead has a negative number in the type field here.
@@ -765,9 +756,9 @@ struct CardDefinition {
   // enormous comment? That's what this array stores.
   /* 009C */ parray<be_uint16_t, 2> drop_rates;
 
-  /* 00A0 */ pstring<TextEncoding::SJIS, 0x14> en_name;
+  /* 00A0 */ pstring<TextEncoding::ISO8859, 0x14> en_name;
   /* 00B4 */ pstring<TextEncoding::SJIS, 0x0B> jp_short_name;
-  /* 00BF */ pstring<TextEncoding::SJIS, 0x08> en_short_name;
+  /* 00BF */ pstring<TextEncoding::ISO8859, 0x08> en_short_name;
   // These effects modify the card's behavior in various situations. Only
   // effects for which effect_num is not zero are used.
   /* 00C7 */ parray<Effect, 3> effects;
@@ -782,7 +773,8 @@ struct CardDefinition {
 
   void decode_range();
   std::string str(bool single_line = true, const TextSet* text_archive = nullptr) const;
-} __attribute__((packed)); // 0x128 bytes in total
+  JSON json() const;
+} __packed_ws__(CardDefinition, 0x128);
 
 struct CardDefinitionsFooter {
   // Technically the card definitions file is a REL file, so the last 0x20 bytes
@@ -798,10 +790,10 @@ struct CardDefinitionsFooter {
   /* 48 */ be_uint32_t footer_offset;
   /* 4C */ parray<be_uint32_t, 3> unused2;
   /* 58 */
-} __attribute__((packed));
+} __packed_ws__(CardDefinitionsFooter, 0x58);
 
 struct DeckDefinition {
-  /* 00 */ pstring<TextEncoding::SJIS, 0x10> name;
+  /* 00 */ pstring<TextEncoding::MARKED, 0x10> name;
   /* 10 */ be_uint32_t client_id; // 0-3
   // List of card IDs. The card count is the number of nonzero entries here
   // before a zero entry (or 50 if no entries are nonzero). The first card ID is
@@ -818,13 +810,13 @@ struct DeckDefinition {
   /* 82 */ uint8_t second;
   /* 83 */ uint8_t unknown_a2;
   /* 84 */
-} __attribute__((packed));
+} __packed_ws__(DeckDefinition, 0x84);
 
 struct PlayerConfig {
   // The game splits this internally into two structures. The first column of
   // offsets is relative to the start of the first structure; the second column
   // is relative to the start of the second structure.
-  /* 0000:---- */ pstring<TextEncoding::SJIS, 12> rank_text; // From B7 command
+  /* 0000:---- */ pstring<TextEncoding::MARKED, 12> rank_text; // From B7 command
   /* 000C:---- */ parray<uint8_t, 0x1C> unknown_a1;
   /* 0028:---- */ parray<be_uint16_t, 20> tech_menu_shortcut_entries;
   /* 0050:---- */ parray<be_uint32_t, 10> choice_search_config;
@@ -836,7 +828,7 @@ struct PlayerConfig {
   // earlier version, this was the offline records structure, but they later
   // decided to just count online and offline records together in the main
   // records structure and didn't remove the codepath that reads from this.
-  /* 0138:---- */ PlayerRecords_Battle<true> unused_offline_records;
+  /* 0138:---- */ PlayerRecordsBattleBE unused_offline_records;
   /* 0150:---- */ parray<uint8_t, 4> unknown_a4;
   // The PlayerDataSegment structure begins here. In newserv, we combine this
   // structure into PlayerConfig since the two are always used together.
@@ -886,8 +878,8 @@ struct PlayerConfig {
   /* 2124:1FD0 */ be_uint32_t online_clv_exp; // CLvOn = this / 100
   struct PlayerReference {
     /* 00 */ be_uint32_t guild_card_number;
-    /* 04 */ pstring<TextEncoding::SJIS, 0x18> player_name;
-  } __attribute__((packed));
+    /* 04 */ pstring<TextEncoding::MARKED, 0x18> name;
+  } __packed_ws__(PlayerReference, 0x1C);
   // This array is updated when a battle is started (via a 6xB4x05 command). The
   // client adds the opposing players' info to ths first two entries here if the
   // opponents are human. (The existing entries are always moved back by two
@@ -910,7 +902,7 @@ struct PlayerConfig {
 
   void decrypt();
   void encrypt(uint8_t basis);
-} __attribute__((packed));
+} __packed_ws__(PlayerConfig, 0x2350);
 
 enum class HPType : uint8_t {
   DEFEAT_PLAYER = 0,
@@ -940,8 +932,8 @@ struct Rules {
   /* 00 */ uint8_t overall_time_limit = 0;
   /* 01 */ uint8_t phase_time_limit = 0; // In seconds; 0 = unlimited
   /* 02 */ AllowedCards allowed_cards = AllowedCards::ALL;
-  /* 03 */ uint8_t min_dice = 1; // 0 = default (1)
-  /* 04 */ uint8_t max_dice = 6; // 0 = default (6)
+  /* 03 */ uint8_t min_dice_value = 1; // 0 = default (1)
+  /* 04 */ uint8_t max_dice_value = 6; // 0 = default (6)
   /* 05 */ uint8_t disable_deck_shuffle = 0; // 0 = shuffle on, 1 = off
   /* 06 */ uint8_t disable_deck_loop = 0; // 0 = loop on, 1 = off
   /* 07 */ uint8_t char_hp = 15;
@@ -952,8 +944,11 @@ struct Rules {
   /* 0C */ uint8_t disable_dice_boost = 0; // 0 = dice boost on, 1 = off
   // NOTE: The following fields are unused in PSO's implementation, but newserv
   // uses them to implement extended rules.
-  /* 0D */ uint8_t def_dice_range = 0; // High 4 bits = min, low 4 = max
-  /* 0E */ parray<uint8_t, 6> unused;
+  /* 0D */ uint8_t def_dice_value_range = 0; // High 4 bits = min, low 4 = max
+  // These fields specify override dice ranges for the 1-player team in 2v1
+  /* 0E */ uint8_t atk_dice_value_range_2v1 = 0; // High 4 bits = min, low 4 = max
+  /* 0F */ uint8_t def_dice_value_range_2v1 = 0; // High 4 bits = min, low 4 = max
+  /* 10 */ parray<uint8_t, 4> unused;
   /* 14 */
 
   // Annoyingly, this structure is a different size in Episode 3 Trial Edition.
@@ -973,11 +968,11 @@ struct Rules {
   bool check_invalid_fields() const;
   bool check_and_reset_invalid_fields();
 
-  uint8_t min_def_dice() const;
-  uint8_t max_def_dice() const;
+  std::pair<uint8_t, uint8_t> atk_dice_range(bool is_1p_2v1) const;
+  std::pair<uint8_t, uint8_t> def_dice_range(bool is_1p_2v1) const;
 
   std::string str() const;
-} __attribute__((packed));
+} __packed_ws__(Rules, 0x14);
 
 struct RulesTrial {
   // Most fields here have the same meanings as in the final version.
@@ -1001,7 +996,7 @@ struct RulesTrial {
   RulesTrial() = default;
   RulesTrial(const Rules&);
   operator Rules() const;
-} __attribute__((packed));
+} __packed_ws__(RulesTrial, 0x0C);
 
 struct StateFlags {
   /* 00 */ le_uint16_t turn_num;
@@ -1023,7 +1018,7 @@ struct StateFlags {
   bool operator!=(const StateFlags& other) const;
   void clear();
   void clear_FF();
-} __attribute__((packed));
+} __packed_ws__(StateFlags, 0x18);
 
 struct MapList {
   be_uint32_t num_maps;
@@ -1051,18 +1046,18 @@ struct MapList {
     /* 021C */ uint8_t map_category;
     /* 021D */ parray<uint8_t, 3> unused;
     /* 0220 */
-  } __attribute__((packed));
+  } __packed_ws__(Entry, 0x220);
 
   // Variable-length fields:
   // Entry entries[num_maps];
   // char strings[...EOF]; // Null-terminated strings, pointed to by offsets in Entry structs
-} __attribute__((packed));
+} __packed_ws__(MapList, 0x10);
 
 struct CompressedMapHeader { // .mnm file format
   le_uint32_t map_number;
   le_uint32_t compressed_data_size;
   // Compressed data immediately follows (which decompresses to a MapDefinition)
-} __attribute__((packed));
+} __packed_ws__(CompressedMapHeader, 8);
 
 struct MapDefinition { // .mnmd format; also the format of (decompressed) quests
   // If tag is not 0x00000100, the game considers the map to be corrupt in
@@ -1159,7 +1154,8 @@ struct MapDefinition { // .mnmd format; also the format of (decompressed) quests
     /* 48 */
 
     std::string str() const;
-  } __attribute__((packed));
+    JSON json() const;
+  } __packed_ws__(CameraSpec, 0x48);
 
   // This array specifies the camera zone maps. A camera zone map is a subset of
   // the main map (specified in map_tiles). Tiles that are part of each camera
@@ -1202,10 +1198,10 @@ struct MapDefinition { // .mnmd format; also the format of (decompressed) quests
   /* 1D68 */ parray<uint8_t, 0x74> unknown_a5;
   /* 1DDC */ Rules default_rules;
 
-  /* 1DF0 */ pstring<TextEncoding::SJIS, 0x14> name;
-  /* 1E04 */ pstring<TextEncoding::SJIS, 0x14> location_name;
-  /* 1E18 */ pstring<TextEncoding::SJIS, 0x3C> quest_name; // == location_name if not a quest
-  /* 1E54 */ pstring<TextEncoding::SJIS, 0x190> description;
+  /* 1DF0 */ pstring<TextEncoding::MARKED, 0x14> name;
+  /* 1E04 */ pstring<TextEncoding::MARKED, 0x14> location_name;
+  /* 1E18 */ pstring<TextEncoding::MARKED, 0x3C> quest_name; // == location_name if not a quest
+  /* 1E54 */ pstring<TextEncoding::MARKED, 0x190> description;
 
   // These fields describe where the map cursor on the preview screen should
   // scroll to
@@ -1213,10 +1209,11 @@ struct MapDefinition { // .mnmd format; also the format of (decompressed) quests
   /* 1FE6 */ be_uint16_t map_y;
 
   struct NPCDeck {
-    /* 00 */ pstring<TextEncoding::SJIS, 0x18> name;
+    /* 00 */ pstring<TextEncoding::MARKED, 0x18> deck_name;
     /* 18 */ parray<be_uint16_t, 0x20> card_ids; // Last one appears to always be FFFF
     /* 58 */
-  } __attribute__((packed));
+    JSON json(uint8_t language) const;
+  } __packed_ws__(NPCDeck, 0x58);
   /* 1FE8 */ parray<NPCDeck, 3> npc_decks; // Unused if name[0] == 0
 
   // These are almost (but not quite) the same format as the entries in
@@ -1227,11 +1224,12 @@ struct MapDefinition { // .mnmd format; also the format of (decompressed) quests
     /* 0000 */ parray<be_uint16_t, 2> unknown_a1;
     /* 0004 */ uint8_t is_arkz;
     /* 0005 */ parray<uint8_t, 3> unknown_a2;
-    /* 0008 */ pstring<TextEncoding::SJIS, 0x10> name;
+    /* 0008 */ pstring<TextEncoding::MARKED, 0x10> ai_name;
     // TODO: Figure out exactly how these are used and document here.
     /* 0018 */ parray<be_uint16_t, 0x7E> params;
     /* 0114 */
-  } __attribute__((packed));
+    JSON json(uint8_t language) const;
+  } __packed_ws__(AIParams, 0x114);
   /* 20F0 */ parray<AIParams, 3> npc_ai_params; // Unused if name[0] == 0
 
   /* 242C */ parray<uint8_t, 8> unknown_a7;
@@ -1261,9 +1259,9 @@ struct MapDefinition { // .mnmd format; also the format of (decompressed) quests
   // appears after the battle if it's not blank. dispatch_message appears right
   // before the player chooses a deck if it's not blank; usually it says
   // something like "You can only dispatch <character>".
-  /* 2440 */ pstring<TextEncoding::SJIS, 0x190> before_message;
-  /* 25D0 */ pstring<TextEncoding::SJIS, 0x190> after_message;
-  /* 2760 */ pstring<TextEncoding::SJIS, 0x190> dispatch_message;
+  /* 2440 */ pstring<TextEncoding::MARKED, 0x190> before_message;
+  /* 25D0 */ pstring<TextEncoding::MARKED, 0x190> after_message;
+  /* 2760 */ pstring<TextEncoding::MARKED, 0x190> dispatch_message;
 
   struct DialogueSet {
     // Dialogue sets specify lines that COMs can say at certain points during
@@ -1281,9 +1279,10 @@ struct MapDefinition { // .mnmd format; also the format of (decompressed) quests
     /* 0002 */ be_uint16_t percent_chance; // 0-100, or FFFF if unused
     // If the dialogue set activates, the game randomly chooses one of these
     // strings, excluding any that are empty or begin with the character '^'.
-    /* 0004 */ parray<pstring<TextEncoding::SJIS, 0x40>, 4> strings;
+    /* 0004 */ parray<pstring<TextEncoding::MARKED, 0x40>, 4> strings;
     /* 0104 */
-  } __attribute__((packed));
+    JSON json(uint8_t language) const;
+  } __packed_ws__(DialogueSet, 0x104);
 
   // There are up to 0x10 of these per valid NPC, but only the first 13 of them
   // are used, since each one must have a unique value for .when and the values
@@ -1365,7 +1364,8 @@ struct MapDefinition { // .mnmd format; also the format of (decompressed) quests
 
     bool operator==(const EntryState& other) const = default;
     bool operator!=(const EntryState& other) const = default;
-  } __attribute__((packed));
+    JSON json() const;
+  } __packed_ws__(EntryState, 2);
   /* 5A10 */ parray<EntryState, 4> entry_states;
   /* 5A18 */
 
@@ -1380,7 +1380,8 @@ struct MapDefinition { // .mnmd format; also the format of (decompressed) quests
   void assert_semantically_equivalent(const MapDefinition& other) const;
 
   std::string str(const CardIndex* card_index, uint8_t language) const;
-} __attribute__((packed));
+  JSON json(uint8_t language) const;
+} __packed_ws__(MapDefinition, 0x5A18);
 
 struct MapDefinitionTrial {
   // This is the format of Episode 3 Trial Edition maps. See the comments in
@@ -1400,19 +1401,19 @@ struct MapDefinitionTrial {
   /* 1C68 */ parray<parray<uint8_t, 0x10>, 0x10> modification_tiles;
   /* 1D68 */ parray<uint8_t, 0x74> unknown_a5;
   /* 1DD4 */ RulesTrial default_rules;
-  /* 1DE8 */ pstring<TextEncoding::SJIS, 0x14> name;
-  /* 1DFC */ pstring<TextEncoding::SJIS, 0x14> location_name;
-  /* 1E10 */ pstring<TextEncoding::SJIS, 0x3C> quest_name;
-  /* 1E4C */ pstring<TextEncoding::SJIS, 0x190> description;
+  /* 1DE8 */ pstring<TextEncoding::MARKED, 0x14> name;
+  /* 1DFC */ pstring<TextEncoding::MARKED, 0x14> location_name;
+  /* 1E10 */ pstring<TextEncoding::MARKED, 0x3C> quest_name;
+  /* 1E4C */ pstring<TextEncoding::MARKED, 0x190> description;
   /* 1FDC */ be_uint16_t map_x;
   /* 1FDE */ be_uint16_t map_y;
   /* 1FE0 */ parray<MapDefinition::NPCDeck, 3> npc_decks;
   /* 20E8 */ parray<MapDefinition::AIParams, 3> npc_ai_params;
   /* 2424 */ parray<uint8_t, 8> unknown_a7;
   /* 242C */ parray<be_int32_t, 3> npc_ai_params_entry_index;
-  /* 2438 */ pstring<TextEncoding::SJIS, 0x190> before_message;
-  /* 25C8 */ pstring<TextEncoding::SJIS, 0x190> after_message;
-  /* 2758 */ pstring<TextEncoding::SJIS, 0x190> dispatch_message;
+  /* 2438 */ pstring<TextEncoding::MARKED, 0x190> before_message;
+  /* 25C8 */ pstring<TextEncoding::MARKED, 0x190> after_message;
+  /* 2758 */ pstring<TextEncoding::MARKED, 0x190> dispatch_message;
   /* 28E8 */ parray<parray<MapDefinition::DialogueSet, 8>, 3> dialogue_sets;
   /* 4148 */ parray<be_uint16_t, 0x10> reward_card_ids;
   /* 4168 */ be_int32_t win_level_override;
@@ -1429,7 +1430,7 @@ struct MapDefinitionTrial {
 
   MapDefinitionTrial(const MapDefinition& map);
   operator MapDefinition() const;
-} __attribute__((packed));
+} __packed_ws__(MapDefinitionTrial, 0x41A0);
 
 struct COMDeckDefinition {
   size_t index;
@@ -1462,6 +1463,7 @@ public:
   std::shared_ptr<const CardEntry> definition_for_name_normalized(const std::string& name) const;
   std::set<uint32_t> all_ids() const;
   uint64_t definitions_mtime() const;
+  JSON definitions_json() const;
 
 private:
   static std::string normalize_card_name(const std::string& name);
@@ -1553,3 +1555,24 @@ template <>
 Episode3::AllowedCards enum_for_name<Episode3::AllowedCards>(const char* name);
 template <>
 const char* name_for_enum<Episode3::AllowedCards>(Episode3::AllowedCards allowed_cards);
+
+template <>
+const char* name_for_enum<Episode3::BattlePhase>(Episode3::BattlePhase phase);
+template <>
+const char* name_for_enum<Episode3::SetupPhase>(Episode3::SetupPhase phase);
+template <>
+const char* name_for_enum<Episode3::RegistrationPhase>(Episode3::RegistrationPhase phase);
+template <>
+const char* name_for_enum<Episode3::ActionSubphase>(Episode3::ActionSubphase phase);
+template <>
+const char* name_for_enum<Episode3::AttackMedium>(Episode3::AttackMedium medium);
+template <>
+const char* name_for_enum<Episode3::CriterionCode>(Episode3::CriterionCode code);
+template <>
+const char* name_for_enum<Episode3::CardType>(Episode3::CardType type);
+template <>
+const char* name_for_enum<Episode3::CardClass>(Episode3::CardClass cc);
+template <>
+const char* name_for_enum<Episode3::ConditionType>(Episode3::ConditionType cond_type);
+template <>
+const char* name_for_enum<Episode3::Direction>(Episode3::Direction d);

src/Episode3/DeckState.cc

@@ -1,5 +1,7 @@
 #include "DeckState.hh"
 
+#include "Server.hh"
+
 using namespace std;
 
 namespace Episode3 {
@@ -84,43 +86,45 @@ bool DeckState::draw_card_by_ref(uint16_t card_ref) {
   }
 
   uint8_t index = index_for_card_ref(card_ref);
-  if (index > this->entries.size()) {
+  if (index >= this->entries.size()) {
     return false;
   }
 
-  if (this->entries[index].state == CardState::DISCARDED) {
+  auto& entry = this->entries[index];
+  if (entry.state == CardState::DISCARDED) {
     // If the card is discarded, then it should be before the draw index, and we
     // can just change its state.
-    this->entries[index].state = CardState::IN_HAND;
+    entry.state = CardState::IN_HAND;
     return true;
+  }
 
-  } else if (this->entries[index].state == CardState::DRAWABLE) {
-    // If the card is still drawable, we need to move it so it's just in front
-    // of the draw index, then immediately draw it. Ep3 NTE does not handle this
-    // case, but we do even when playing NTE.
-    ssize_t ref_index;
-    for (ref_index = this->card_refs.size(); ref_index >= 0; ref_index--) {
-      if (this->card_refs[ref_index] == card_ref) {
-        break;
-      }
-    }
-    if (ref_index < 0) {
-      return false;
-    }
-
-    size_t ref_uindex = ref_index;
-    for (; ref_uindex > this->draw_index; ref_uindex--) {
-      // Note: draw_index is also unsigned, so ref_uindex cannot be zero here
-      this->card_refs[ref_uindex] = this->card_refs[ref_uindex - 1];
-    }
-    this->card_refs[this->draw_index] = card_ref;
-    this->entries[index].state = CardState::IN_HAND;
-    this->draw_index++;
-    return true;
-
-  } else {
+  if (entry.state != CardState::DRAWABLE) {
     return false;
   }
+
+  // If the card is still drawable, we need to move it so it's just in front of
+  // the draw index, then immediately draw it. Ep3 NTE does not handle this
+  // case, but we do even when playing NTE.
+  size_t ref_index;
+  for (ref_index = 0; ref_index < this->card_refs.size(); ref_index++) {
+    if (this->card_refs[ref_index] == card_ref) {
+      break;
+    }
+  }
+  if (ref_index >= this->card_refs.size()) {
+    return false;
+  }
+
+  for (; ref_index > this->draw_index; ref_index--) {
+    // this->draw_index is also unsigned, so ref_index cannot be zero here
+    this->card_refs[ref_index] = this->card_refs[ref_index - 1];
+  }
+  this->card_refs[this->draw_index] = card_ref;
+
+  // Draw the card
+  entry.state = CardState::IN_HAND;
+  this->draw_index++;
+  return true;
 }
 
 uint16_t DeckState::card_id_for_card_ref(uint16_t card_ref) const {
@@ -201,12 +205,17 @@ void DeckState::do_mulligan(bool is_nte) {
       this->card_refs[index + 5] = temp_ref;
     }
 
+    auto s = this->server.lock();
+    if (!s) {
+      throw runtime_error("server is missing");
+    }
+
     // Shuffle the deck, except the first 5 cards (which are about to be drawn).
     size_t max = this->num_drawable_cards() - 5;
     uint8_t base_index = this->draw_index + 5;
     for (size_t z = 0; z < this->card_refs.size(); z++) {
-      uint8_t index1 = this->random_crypt->next() % max;
-      uint8_t index2 = this->random_crypt->next() % max;
+      uint8_t index1 = s->get_random(max);
+      uint8_t index2 = s->get_random(max);
       uint16_t temp_ref = this->card_refs[base_index + index1];
       this->card_refs[base_index + index1] = this->card_refs[base_index + index2];
       this->card_refs[base_index + index2] = temp_ref;
@@ -258,6 +267,11 @@ void DeckState::set_card_discarded(uint16_t card_ref) {
 
 void DeckState::shuffle() {
   if (this->shuffle_enabled) {
+    auto s = this->server.lock();
+    if (!s) {
+      throw runtime_error("server is missing");
+    }
+
     size_t max = this->num_drawable_cards();
     for (size_t z = 0; z < this->card_refs.size(); z++) {
       // Note: This is the way Sega originally implemented shuffling - they just
@@ -265,8 +279,8 @@ void DeckState::shuffle() {
       // instead swap each item with another random item (possibly itself) that
       // doesn't appear earlier than it in the array, but this is not what Sega
       // did.
-      uint8_t index1 = this->draw_index + this->random_crypt->next() % max;
-      uint8_t index2 = this->draw_index + this->random_crypt->next() % max;
+      uint8_t index1 = this->draw_index + s->get_random(max);
+      uint8_t index2 = this->draw_index + s->get_random(max);
       uint16_t temp_ref = this->card_refs[index1];
       this->card_refs[index1] = this->card_refs[index2];
       this->card_refs[index2] = temp_ref;

src/Episode3/DeckState.hh

@@ -10,8 +10,10 @@
 
 namespace Episode3 {
 
+class Server;
+
 struct NameEntry {
-  /* 00 */ parray<char, 0x10> name;
+  /* 00 */ pstring<TextEncoding::MARKED, 0x10> name;
   /* 10 */ uint8_t client_id;
   /* 11 */ uint8_t present;
   /* 12 */ uint8_t is_cpu_player;
@@ -20,10 +22,10 @@ struct NameEntry {
 
   NameEntry();
   void clear();
-} __attribute__((packed));
+} __packed_ws__(NameEntry, 0x14);
 
 struct DeckEntry {
-  /* 00 */ pstring<TextEncoding::SJIS, 0x10> name;
+  /* 00 */ pstring<TextEncoding::MARKED, 0x10> name;
   /* 10 */ le_uint32_t team_id;
   /* 14 */ parray<le_uint16_t, 31> card_ids;
   // If the following flag is not set to 3, then the God Whim assist effect can
@@ -37,7 +39,7 @@ struct DeckEntry {
 
   DeckEntry();
   void clear();
-} __attribute__((packed));
+} __packed_ws__(DeckEntry, 0x58);
 
 uint8_t index_for_card_ref(uint16_t card_ref);
 uint8_t client_id_for_card_ref(uint16_t card_ref);
@@ -57,13 +59,13 @@ public:
   DeckState(
       uint8_t client_id,
       const parray<CardIDT, 0x1F>& card_ids,
-      std::shared_ptr<PSOLFGEncryption> random_crypt)
-      : client_id(client_id),
+      std::shared_ptr<Server> server)
+      : server(server),
+        client_id(client_id),
         draw_index(1),
         card_ref_base(this->client_id << 8),
         shuffle_enabled(true),
-        loop_enabled(true),
-        random_crypt(random_crypt) {
+        loop_enabled(true) {
     for (size_t z = 0; z < card_ids.size(); z++) {
       auto& e = this->entries[z];
       e.card_id = card_ids[z];
@@ -99,6 +101,8 @@ public:
   void print(FILE* stream, std::shared_ptr<const CardIndex> card_index = nullptr) const;
 
 private:
+  std::weak_ptr<Server> server;
+
   struct CardEntry {
     uint16_t card_id;
     uint8_t deck_index;
@@ -111,8 +115,6 @@ private:
   bool loop_enabled;
   parray<CardEntry, 31> entries;
   parray<uint16_t, 31> card_refs;
-
-  std::shared_ptr<PSOLFGEncryption> random_crypt;
 };
 
 } // namespace Episode3

src/Episode3/MapState.hh

@@ -20,7 +20,7 @@ struct MapState {
   void clear();
 
   void print(FILE* stream) const;
-} __attribute__((packed));
+} __packed_ws__(MapState, 0x110);
 
 struct MapAndRulesState {
   /* 0000 */ MapState map;
@@ -45,7 +45,7 @@ struct MapAndRulesState {
 
   void set_occupied_bit_for_tile(uint8_t x, uint8_t y);
   void clear_occupied_bit_for_tile(uint8_t x, uint8_t y);
-} __attribute__((packed));
+} __packed_ws__(MapAndRulesState, 0x138);
 
 struct MapAndRulesStateTrial {
   /* 0000 */ MapState map;
@@ -65,7 +65,7 @@ struct MapAndRulesStateTrial {
   MapAndRulesStateTrial() = default;
   MapAndRulesStateTrial(const MapAndRulesState& state);
   operator MapAndRulesState() const;
-} __attribute__((packed));
+} __packed_ws__(MapAndRulesStateTrial, 0x130);
 
 struct OverlayState {
   parray<parray<uint8_t, 0x10>, 0x10> tiles;
@@ -75,6 +75,6 @@ struct OverlayState {
 
   OverlayState();
   void clear();
-} __attribute__((packed));
+} __packed_ws__(OverlayState, 0x174);
 
 } // namespace Episode3

src/Episode3/PlayerState.cc

@@ -41,6 +41,7 @@ PlayerState::PlayerState(uint8_t client_id, shared_ptr<Server> server)
 
 void PlayerState::init() {
   auto s = this->server();
+  auto log = s->log_stack("PlayerState::init: ");
 
   if (s->player_states.at(this->client_id).get() != this) {
     // Note: The original code handles this, but we don't. This appears not to
@@ -48,7 +49,7 @@ void PlayerState::init() {
     throw logic_error("replacing a player state object is not permitted");
   }
 
-  this->deck_state = make_shared<DeckState>(this->client_id, s->deck_entries[client_id]->card_ids, s->options.random_crypt);
+  this->deck_state = make_shared<DeckState>(this->client_id, s->deck_entries[client_id]->card_ids, s);
   if (s->map_and_rules->rules.disable_deck_shuffle) {
     this->deck_state->disable_shuffle();
   }
@@ -223,8 +224,7 @@ void PlayerState::apply_assist_card_effect_on_set(
 
       size_t log_index;
       for (log_index = 0; log_index < 0x10; log_index++) {
-        auto ce = s->definition_for_card_ref(
-            this->discard_log_card_refs[log_index]);
+        auto ce = s->definition_for_card_ref(this->discard_log_card_refs[log_index]);
         if (ce && ((ce->def.type == CardType::ITEM || ce->def.type == CardType::CREATURE))) {
           break;
         }
@@ -832,7 +832,7 @@ vector<uint16_t> PlayerState::get_all_cards_within_range(
     uint8_t target_team_id) const {
   auto s = this->server();
 
-  auto log = this->server()->log_stack("get_all_cards_within_range: ");
+  auto log = s->log_stack("get_all_cards_within_range: ");
   string loc_str = loc.str();
   log.debug("loc=%s, target_team_id=%02hhX", loc_str.c_str(), target_team_id);
 
@@ -1415,7 +1415,7 @@ bool PlayerState::set_card_from_hand(
   s->send_6xB4x05();
 
   if (!is_nte) {
-    G_Unknown_Ep3_6xB4x4A cmd;
+    G_AddToSetCardlog_Ep3_6xB4x4A cmd;
     cmd.card_refs.clear(0xFFFF);
     cmd.card_refs[0] = card_ref;
     cmd.client_id = this->client_id;
@@ -1754,30 +1754,45 @@ int16_t PlayerState::get_assist_turns_remaining() {
 
 bool PlayerState::set_action_cards_for_action_state(const ActionState& pa) {
   auto s = this->server();
+  auto log = s->log_stack("set_action_cards_for_action_state: ");
   bool is_nte = s->options.is_nte();
 
   auto attacker_card = s->card_for_set_card_ref(pa.attacker_card_ref);
   if (attacker_card) {
+    log.debug("attacker card present");
     attacker_card->card_flags |= 0x100;
   }
 
   auto action_type = s->ruler_server->get_pending_action_type(pa);
+  if (action_type == ActionType::DEFENSE) {
+    log.debug("action type is DEFENSE");
+  } else if (action_type == ActionType::ATTACK) {
+    log.debug("action type is ATTACK");
+  } else {
+    log.debug("action type is UNKNOWN");
+  }
   if (!is_nte) {
-    this->subtract_or_check_atk_or_def_points_for_action(pa, 1);
+    log.debug("(non-nte) subtracting action points");
+    this->subtract_or_check_atk_or_def_points_for_action(pa, true);
   }
 
   if (action_type == ActionType::ATTACK) {
     auto card = s->card_for_set_card_ref(pa.attacker_card_ref);
     if (card) {
       card->loc.direction = pa.facing_direction;
+      log.debug("set facing direction to %s", name_for_enum(card->loc.direction));
 
-      G_Unknown_Ep3_6xB4x4A cmd;
+      G_AddToSetCardlog_Ep3_6xB4x4A cmd;
       cmd.card_refs.clear(0xFFFF);
       cmd.client_id = this->client_id;
       cmd.round_num = s->get_round_num();
       cmd.entry_count = 0;
       size_t z = 0;
       do {
+        if (log.should_log(LogLevel::DEBUG)) {
+          string ref_str = s->debug_str_for_card_ref(pa.action_card_refs[z]);
+          log.debug("on action card ref %s", ref_str.c_str());
+        }
         card->unknown_80237A90(pa, pa.action_card_refs[z]);
         card->unknown_802379BC(pa.action_card_refs[z]);
         if (!is_nte) {
@@ -1811,6 +1826,10 @@ bool PlayerState::set_action_cards_for_action_state(const ActionState& pa) {
     for (size_t z = 0; (z < 4 * 9) && (pa.target_card_refs[z] != 0xFFFF); z++) {
       auto target_card = s->card_for_set_card_ref(pa.target_card_refs[z]);
       if (target_card) {
+        if (log.should_log(LogLevel::DEBUG)) {
+          string ref_str = s->debug_str_for_card_ref(pa.target_card_refs[z]);
+          log.debug("on target card ref %s", ref_str.c_str());
+        }
         target_card->unknown_802379DC(pa);
         if (!is_nte) {
           if (this->client_id == target_card->get_client_id()) {
@@ -1823,7 +1842,7 @@ bool PlayerState::set_action_cards_for_action_state(const ActionState& pa) {
       }
     }
     if (!is_nte) {
-      G_Unknown_Ep3_6xB4x4A cmd;
+      G_AddToSetCardlog_Ep3_6xB4x4A cmd;
       cmd.card_refs.clear(0xFFFF);
       cmd.client_id = this->client_id;
       cmd.round_num = s->get_round_num();
@@ -1833,9 +1852,14 @@ bool PlayerState::set_action_cards_for_action_state(const ActionState& pa) {
     }
   }
   if (is_nte) {
+    log.debug("(nte) subtracting action points");
     this->subtract_or_check_atk_or_def_points_for_action(pa, 1);
   }
   for (size_t z = 0; (z < pa.action_card_refs.size()) && (pa.action_card_refs[z] != 0xFFFF); z++) {
+    if (log.should_log(LogLevel::DEBUG)) {
+      string ref_str = s->debug_str_for_card_ref(pa.action_card_refs[z]);
+      log.debug("discarding %s from hand", ref_str.c_str());
+    }
     this->discard_ref_from_hand(pa.action_card_refs[z]);
   }
   this->update_hand_and_equip_state_and_send_6xB4x02_if_needed();
@@ -1960,44 +1984,23 @@ void PlayerState::roll_main_dice_or_apply_after_effects() {
   // the non-NTE logic and assign the dice ranges at battle start time to yield
   // the NTE behavior. (See RulesTrial in DataIndexes.cc for how this is done.)
 
-  uint8_t min_atk_dice = rules.min_dice;
-  uint8_t max_atk_dice = rules.max_dice;
-  if (min_atk_dice == 0) {
-    min_atk_dice = 1;
-  }
-  if (max_atk_dice == 0) {
-    max_atk_dice = 6;
-  }
-  if (max_atk_dice < min_atk_dice) {
-    uint8_t t = max_atk_dice;
-    max_atk_dice = min_atk_dice;
-    min_atk_dice = t;
-  }
-  uint8_t atk_dice_range_width = (max_atk_dice - min_atk_dice) + 1;
+  bool is_1p_2v1 = (s->team_client_count.at(this->get_team_id()) < s->team_client_count[this->get_team_id() ^ 1]);
+
+  auto atk_range = rules.atk_dice_range(is_1p_2v1);
+  auto def_range = rules.def_dice_range(is_1p_2v1);
+
+  uint8_t atk_dice_range_width = (atk_range.second - atk_range.first) + 1;
   if (atk_dice_range_width < 2) {
-    this->dice_results[0] = min_atk_dice;
+    this->dice_results[0] = atk_range.first;
   } else {
-    this->dice_results[0] = min_atk_dice + s->get_random(atk_dice_range_width);
+    this->dice_results[0] = atk_range.first + s->get_random(atk_dice_range_width);
   }
 
-  uint8_t min_def_dice = rules.min_def_dice() ? rules.min_def_dice() : rules.min_dice;
-  uint8_t max_def_dice = rules.max_def_dice() ? rules.max_def_dice() : rules.max_dice;
-  if (min_def_dice == 0) {
-    min_def_dice = 1;
-  }
-  if (max_def_dice == 0) {
-    max_def_dice = 6;
-  }
-  if (max_def_dice < min_def_dice) {
-    uint8_t t = max_def_dice;
-    max_def_dice = min_def_dice;
-    min_def_dice = t;
-  }
-  uint8_t def_dice_range_width = (max_def_dice - min_def_dice) + 1;
+  uint8_t def_dice_range_width = (def_range.second - def_range.first) + 1;
   if (def_dice_range_width < 2) {
-    this->dice_results[1] = min_def_dice;
+    this->dice_results[1] = def_range.first;
   } else {
-    this->dice_results[1] = min_def_dice + s->get_random(def_dice_range_width);
+    this->dice_results[1] = def_range.first + s->get_random(def_dice_range_width);
   }
 
   bool should_exchange = false;

src/Episode3/PlayerState.hh

@@ -148,7 +148,7 @@ private:
 
 public:
   std::shared_ptr<Card> sc_card;
-  std::shared_ptr<Card> set_cards[8];
+  bcarray<std::shared_ptr<Card>, 8> set_cards;
   uint8_t client_id;
   uint16_t num_mulligans_allowed;
   CardType sc_card_type;

src/Episode3/PlayerStateSubordinates.cc

@@ -6,22 +6,6 @@ using namespace std;
 
 namespace Episode3 {
 
-template <size_t Count>
-std::string string_for_refs(const parray<le_uint16_t, Count>& card_refs) {
-  string ret = "[";
-  for (size_t z = 0; z < Count; z++) {
-    if (card_refs[z] != 0xFFFF) {
-      ret += string_printf("%zu:@$%04X ", z, card_refs[z].load());
-    }
-  }
-  if (!ret.empty()) {
-    ret.back() = ']'; // Replace the ' ' from the last added item
-  } else {
-    ret.push_back(']');
-  }
-  return ret;
-}
-
 Condition::Condition() {
   this->clear();
 }
@@ -77,20 +61,22 @@ void Condition::clear_FF() {
   this->unknown_a8 = 0xFF;
 }
 
-std::string Condition::str() const {
+std::string Condition::str(shared_ptr<const Server> s) const {
+  auto card_ref_str = s->debug_str_for_card_ref(this->card_ref);
+  auto giver_ref_str = s->debug_str_for_card_ref(this->condition_giver_card_ref);
   return string_printf(
       "Condition[type=%s, turns=%hhu, a_arg=%hhd, dice=%hhu, flags=%02hhX, "
-      "def_eff_index=%hhu, ref=@%04hX, value=%hd, giver_ref=@%04hX "
+      "def_eff_index=%hhu, ref=%s, value=%hd, giver_ref=%s "
       "percent=%hhu value8=%hd order=%hu a8=%hu]",
-      name_for_condition_type(this->type),
+      name_for_enum(this->type),
       this->remaining_turns,
       this->a_arg_value,
       this->dice_roll_value,
       this->flags,
       this->card_definition_effect_index,
-      this->card_ref.load(),
+      card_ref_str.c_str(),
       this->value.load(),
-      this->condition_giver_card_ref.load(),
+      giver_ref_str.c_str(),
       this->random_percent,
       this->value8,
       this->order,
@@ -114,13 +100,15 @@ void EffectResult::clear() {
   this->dice_roll_value = 0;
 }
 
-std::string EffectResult::str() const {
+std::string EffectResult::str(shared_ptr<const Server> s) const {
+  string attacker_ref_str = s->debug_str_for_card_ref(this->attacker_card_ref);
+  string target_ref_str = s->debug_str_for_card_ref(this->target_card_ref);
   return string_printf(
-      "EffectResult[att_ref=@%04hX, target_ref=@%04hX, value=%hhd, "
+      "EffectResult[att_ref=%s, target_ref=%s, value=%hhd, "
       "cur_hp=%hhd, ap=%hhd, tp=%hhd, flags=%02hhX, op=%hhd, "
       "cond_index=%hhu, dice=%hhu]",
-      this->attacker_card_ref.load(),
-      this->target_card_ref.load(),
+      attacker_ref_str.c_str(),
+      target_ref_str.c_str(),
       this->value,
       this->current_hp,
       this->ap,
@@ -148,12 +136,13 @@ bool CardShortStatus::operator!=(const CardShortStatus& other) const {
   return !this->operator==(other);
 }
 
-std::string CardShortStatus::str() const {
+std::string CardShortStatus::str(shared_ptr<const Server> s) const {
   string loc_s = this->loc.str();
+  string ref_str = s->debug_str_for_card_ref(this->card_ref);
   return string_printf(
-      "CardShortStatus[ref=@%04hX, cur_hp=%hd, flags=%08" PRIX32 ", loc=%s, "
+      "CardShortStatus[ref=%s, cur_hp=%hd, flags=%08" PRIX32 ", loc=%s, "
       "u1=%04hX, max_hp=%hhd, u2=%hhu]",
-      this->card_ref.load(),
+      ref_str.c_str(),
       this->current_hp.load(),
       this->card_flags.load(),
       loc_s.c_str(),
@@ -195,23 +184,27 @@ void ActionState::clear() {
   this->original_attacker_card_ref = 0xFFFF;
   this->target_card_refs.clear(0xFFFF);
   this->action_card_refs.clear(0xFFFF);
+  this->unused2 = 0xFFFF;
 }
 
-std::string ActionState::str() const {
-  string target_refs_s = string_for_refs(this->target_card_refs);
-  string action_refs_s = string_for_refs(this->action_card_refs);
+std::string ActionState::str(shared_ptr<const Server> s) const {
+  string attacker_ref_s = s->debug_str_for_card_ref(this->attacker_card_ref);
+  string defense_ref_s = s->debug_str_for_card_ref(this->defense_card_ref);
+  string original_attacker_ref_s = s->debug_str_for_card_ref(this->original_attacker_card_ref);
+  string target_refs_s = s->debug_str_for_card_refs(this->target_card_refs);
+  string action_refs_s = s->debug_str_for_card_refs(this->action_card_refs);
   return string_printf(
-      "ActionState[client=%hu, u=%hhu, facing=%s, attacker_ref=@%04hX, "
-      "def_ref=@%04hX, target_refs=%s, action_refs=%s, "
-      "orig_attacker_ref=@%04hX]",
+      "ActionState[client=%hu, u=%hhu, facing=%s, attacker_ref=%s, "
+      "def_ref=%s, target_refs=%s, action_refs=%s, "
+      "orig_attacker_ref=%s]",
       this->client_id.load(),
       this->unused,
-      name_for_direction(this->facing_direction),
-      this->attacker_card_ref.load(),
-      this->defense_card_ref.load(),
+      name_for_enum(this->facing_direction),
+      attacker_ref_s.c_str(),
+      defense_ref_s.c_str(),
       target_refs_s.c_str(),
       action_refs_s.c_str(),
-      this->original_attacker_card_ref.load());
+      original_attacker_ref_s.c_str());
 }
 
 ActionChain::ActionChain() {
@@ -245,28 +238,29 @@ bool ActionChain::operator!=(const ActionChain& other) const {
   return !this->operator==(other);
 }
 
-std::string ActionChain::str() const {
-  string attack_action_card_refs_s = string_for_refs(this->attack_action_card_refs);
-  string target_card_refs_s = string_for_refs(this->target_card_refs);
+std::string ActionChain::str(shared_ptr<const Server> s) const {
+  string acting_card_ref_s = s->debug_str_for_card_ref(this->acting_card_ref);
+  string unknown_card_ref_a3_s = s->debug_str_for_card_ref(this->unknown_card_ref_a3);
+  string attack_action_card_refs_s = s->debug_str_for_card_refs(this->attack_action_card_refs);
+  string target_card_refs_s = s->debug_str_for_card_refs(this->target_card_refs);
   return string_printf(
       "ActionChain[eff_ap=%hhd, eff_tp=%hhd, ap_bonus=%hhd, damage=%hhd, "
-      "acting_ref=@%04hX, unknown_ref_a3=@%04hX, "
-      "attack_action_refs=%s, attack_action_ref_count=%hhu, "
-      "medium=%s, target_ref_count=%hhu, subphase=%s, "
-      "strikes=%hhu, damage_mult=%hhd, attack_num=%hhu, "
+      "acting_ref=%s, unknown_ref_a3=%s, attack_action_refs=%s, "
+      "attack_action_ref_count=%hhu, medium=%s, target_ref_count=%hhu, "
+      "subphase=%s, strikes=%hhu, damage_mult=%hhd, attack_num=%hhu, "
       "tp_bonus=%hhd, phys_bonus_nte=%hhu, tech_bonus_nte=%hhu, card_ap=%hhd, "
       "card_tp=%hhd, flags=%08" PRIX32 ", target_refs=%s]",
       this->effective_ap,
       this->effective_tp,
       this->ap_effect_bonus,
       this->damage,
-      this->acting_card_ref.load(),
-      this->unknown_card_ref_a3.load(),
+      acting_card_ref_s.c_str(),
+      unknown_card_ref_a3_s.c_str(),
       attack_action_card_refs_s.c_str(),
       this->attack_action_card_ref_count,
-      name_for_attack_medium(this->attack_medium),
+      name_for_enum(this->attack_medium),
       this->target_card_ref_count,
-      name_for_action_subphase(this->action_subphase),
+      name_for_enum(this->action_subphase),
       this->strike_count,
       this->damage_multiplier,
       this->attack_number,
@@ -338,17 +332,17 @@ bool ActionChainWithConds::operator!=(const ActionChainWithConds& other) const {
   return !this->operator==(other);
 }
 
-std::string ActionChainWithConds::str() const {
+std::string ActionChainWithConds::str(shared_ptr<const Server> s) const {
   string ret = "ActionChainWithConds[chain=";
-  ret += this->chain.str();
+  ret += this->chain.str(s);
   ret += ", conds=[";
   for (size_t z = 0; z < this->conditions.size(); z++) {
     if (this->conditions[z].type != ConditionType::NONE) {
-      if (ret.back() != '=') {
+      if (ret.back() != '[') {
         ret += ", ";
       }
       ret += string_printf("%zu:", z);
-      ret += this->conditions[z].str();
+      ret += this->conditions[z].str(s);
     }
   }
   ret += "]]";
@@ -581,19 +575,20 @@ bool ActionMetadata::operator!=(const ActionMetadata& other) const {
   return !this->operator==(other);
 }
 
-std::string ActionMetadata::str() const {
-  string target_card_refs_s = string_for_refs(this->target_card_refs);
-  string defense_card_refs_s = string_for_refs(this->defense_card_refs);
-  string original_attacker_card_refs_s = string_for_refs(this->original_attacker_card_refs);
+std::string ActionMetadata::str(shared_ptr<const Server> s) const {
+  string card_ref_s = s->debug_str_for_card_ref(this->card_ref);
+  string target_card_refs_s = s->debug_str_for_card_refs(this->target_card_refs);
+  string defense_card_refs_s = s->debug_str_for_card_refs(this->defense_card_refs);
+  string original_attacker_card_refs_s = s->debug_str_for_card_refs(this->original_attacker_card_refs);
   return string_printf(
-      "ActionMetadata[ref=@%04hX, target_ref_count=%hhu, def_ref_count=%hhu, "
+      "ActionMetadata[ref=%s, target_ref_count=%hhu, def_ref_count=%hhu, "
       "subphase=%s, def_power=%hhd, def_bonus=%hhd, "
       "att_bonus=%hhd, flags=%08" PRIX32 ", target_refs=%s, "
       "defense_refs=%s, original_attacker_refs=%s]",
-      this->card_ref.load(),
+      card_ref_s.c_str(),
       this->target_card_ref_count,
       this->defense_card_ref_count,
-      name_for_action_subphase(this->action_subphase),
+      name_for_enum(this->action_subphase),
       this->defense_power,
       this->defense_bonus,
       this->attack_bonus,
@@ -679,20 +674,22 @@ HandAndEquipState::HandAndEquipState() {
   this->clear();
 }
 
-std::string HandAndEquipState::str() const {
-  string hand_card_refs_s = string_for_refs(this->hand_card_refs);
-  string set_card_refs_s = string_for_refs(this->set_card_refs);
-  string hand_card_refs2_s = string_for_refs(this->hand_card_refs2);
-  string set_card_refs2_s = string_for_refs(this->set_card_refs2);
+std::string HandAndEquipState::str(shared_ptr<const Server> s) const {
+  string assist_card_ref_s = s->debug_str_for_card_ref(this->assist_card_ref);
+  string assist_card_ref2_s = s->debug_str_for_card_ref(this->assist_card_ref2);
+  string assist_card_id_s = s->debug_str_for_card_id(this->assist_card_id);
+  string sc_card_ref_s = s->debug_str_for_card_ref(this->sc_card_ref);
+  string hand_card_refs_s = s->debug_str_for_card_refs(this->hand_card_refs);
+  string set_card_refs_s = s->debug_str_for_card_refs(this->set_card_refs);
+  string hand_card_refs2_s = s->debug_str_for_card_refs(this->hand_card_refs2);
+  string set_card_refs2_s = s->debug_str_for_card_refs(this->set_card_refs2);
   return string_printf(
       "HandAndEquipState[dice=[%hhu, %hhu], atk=%hhu, def=%hhu, atk2=%hhu, "
-      "a1=%hhu, total_set_cost=%hhu, is_cpu=%hhu, "
-      "assist_flags=%08" PRIX32 ", hand_refs=%s, "
-      "assist_ref=@%04hX, set_refs=%s, sc_ref=@%04hX, "
-      "hand_refs2=%s, set_refs2=%s, assist_ref2=@%04hX, "
-      "assist_set_num=%hu, assist_card_id=#%04hX, "
-      "assist_turns=%hhu, assit_dely=%hhu, atk_bonus=%hhu, "
-      "def_bonus=%hhu, u2=[%hhu, %hhu]]",
+      "a1=%hhu, total_set_cost=%hhu, is_cpu=%hhu, assist_flags=%08" PRIX32 ", "
+      "hand_refs=%s, assist_ref=%s, set_refs=%s, sc_ref=%s, hand_refs2=%s, "
+      "set_refs2=%s, assist_ref2=%s, assist_set_num=%hu, assist_card_id=%s, "
+      "assist_turns=%hhu, assist_delay=%hhu, atk_bonus=%hhu, def_bonus=%hhu, "
+      "u2=[%hhu, %hhu]]",
       this->dice_results[0],
       this->dice_results[1],
       this->atk_points,
@@ -703,14 +700,14 @@ std::string HandAndEquipState::str() const {
       this->is_cpu_player,
       this->assist_flags.load(),
       hand_card_refs_s.c_str(),
-      this->assist_card_ref.load(),
+      assist_card_ref_s.c_str(),
       set_card_refs_s.c_str(),
-      this->sc_card_ref.load(),
+      sc_card_ref_s.c_str(),
       hand_card_refs2_s.c_str(),
       set_card_refs2_s.c_str(),
-      this->assist_card_ref2.load(),
+      assist_card_ref2_s.c_str(),
       this->assist_card_set_number.load(),
-      this->assist_card_id.load(),
+      assist_card_id_s.c_str(),
       this->assist_remaining_turns,
       this->assist_delay_turns,
       this->atk_bonuses,

src/Episode3/PlayerStateSubordinates.hh

@@ -35,8 +35,8 @@ struct Condition {
   void clear();
   void clear_FF();
 
-  std::string str() const;
-} __attribute__((packed));
+  std::string str(std::shared_ptr<const Server> s) const;
+} __packed_ws__(Condition, 0x10);
 
 struct EffectResult {
   /* 00 */ le_uint16_t attacker_card_ref;
@@ -55,10 +55,10 @@ struct EffectResult {
   bool operator==(const EffectResult& other) const;
   bool operator!=(const EffectResult& other) const;
 
-  std::string str() const;
-
   void clear();
-} __attribute__((packed));
+
+  std::string str(std::shared_ptr<const Server> s) const;
+} __packed_ws__(EffectResult, 0x0C);
 
 struct CardShortStatus {
   /* 00 */ le_uint16_t card_ref;
@@ -77,8 +77,8 @@ struct CardShortStatus {
   void clear();
   void clear_FF();
 
-  std::string str() const;
-} __attribute__((packed));
+  std::string str(std::shared_ptr<const Server> s) const;
+} __packed_ws__(CardShortStatus, 0x10);
 
 struct ActionState {
   /* 00 */ le_uint16_t client_id;
@@ -87,7 +87,8 @@ struct ActionState {
   /* 04 */ le_uint16_t attacker_card_ref;
   /* 06 */ le_uint16_t defense_card_ref;
   /* 08 */ parray<le_uint16_t, 4 * 9> target_card_refs;
-  /* 50 */ parray<le_uint16_t, 9> action_card_refs;
+  /* 50 */ parray<le_uint16_t, 8> action_card_refs;
+  /* 60 */ le_uint16_t unused2;
   /* 62 */ le_uint16_t original_attacker_card_ref;
   /* 64 */
 
@@ -97,8 +98,8 @@ struct ActionState {
 
   void clear();
 
-  std::string str() const;
-} __attribute__((packed));
+  std::string str(std::shared_ptr<const Server> s) const;
+} __packed_ws__(ActionState, 0x64);
 
 struct ActionChain {
   // Note: Episode 3 Trial Edition has a different format for this structure.
@@ -133,8 +134,8 @@ struct ActionChain {
   void clear();
   void clear_FF();
 
-  std::string str() const;
-} __attribute__((packed));
+  std::string str(std::shared_ptr<const Server> s) const;
+} __packed_ws__(ActionChain, 0x70);
 
 struct ActionChainWithConds {
   /* 0000 */ ActionChain chain;
@@ -171,8 +172,8 @@ struct ActionChainWithConds {
 
   uint8_t get_adjusted_move_ability_nte(uint8_t ability) const;
 
-  std::string str() const;
-} __attribute__((packed));
+  std::string str(std::shared_ptr<const Server> s) const;
+} __packed_ws__(ActionChainWithConds, 0x100);
 
 struct ActionChainWithCondsTrial {
   /* 0000 */ int8_t effective_ap;
@@ -204,7 +205,7 @@ struct ActionChainWithCondsTrial {
   ActionChainWithCondsTrial() = default;
   ActionChainWithCondsTrial(const ActionChainWithConds& src);
   operator ActionChainWithConds() const;
-} __attribute__((packed));
+} __packed_ws__(ActionChainWithCondsTrial, 0x100);
 
 struct ActionMetadata {
   /* 00 */ le_uint16_t card_ref;
@@ -224,8 +225,6 @@ struct ActionMetadata {
   bool operator==(const ActionMetadata& other) const;
   bool operator!=(const ActionMetadata& other) const;
 
-  std::string str() const;
-
   void clear();
   void clear_FF();
 
@@ -240,7 +239,9 @@ struct ActionMetadata {
       uint16_t defense_card_ref,
       std::shared_ptr<Card> card,
       uint16_t original_attacker_card_ref);
-} __attribute__((packed));
+
+  std::string str(std::shared_ptr<const Server> s) const;
+} __packed_ws__(ActionMetadata, 0x74);
 
 struct HandAndEquipState {
   /* 00 */ parray<uint8_t, 2> dice_results;
@@ -274,8 +275,8 @@ struct HandAndEquipState {
   void clear();
   void clear_FF();
 
-  std::string str() const;
-} __attribute__((packed));
+  std::string str(std::shared_ptr<const Server> s) const;
+} __packed_ws__(HandAndEquipState, 0x54);
 
 struct PlayerBattleStats {
   /* 00 */ le_uint16_t damage_given;
@@ -309,7 +310,7 @@ struct PlayerBattleStats {
 
   static uint8_t rank_for_score(float score);
   static const char* name_for_rank(uint8_t rank);
-} __attribute__((packed));
+} __packed_ws__(PlayerBattleStats, 0x28);
 
 struct PlayerBattleStatsTrial {
   /* 00 */ le_uint32_t damage_given = 0;
@@ -322,7 +323,7 @@ struct PlayerBattleStatsTrial {
   PlayerBattleStatsTrial() = default;
   PlayerBattleStatsTrial(const PlayerBattleStats& data);
   operator PlayerBattleStats() const;
-} __attribute__((packed));
+} __packed_ws__(PlayerBattleStatsTrial, 0x14);
 
 std::vector<uint16_t> get_card_refs_within_range(
     const parray<uint8_t, 9 * 9>& range,

src/Episode3/RulerServer.cc

@@ -939,7 +939,8 @@ bool RulerServer::check_usability_or_condition_apply(
     bool is_item_usability_check,
     AttackMedium attack_medium) const {
   auto s = this->server();
-  auto log = s->log_stack(string_printf("check_usability_or_condition_apply(%02hhX, #%04hX, %02hhX, #%04hX, #%04hX, %02hhX, %s, %s): ", client_id1, card_id1, client_id2, card_id2, card_id3, def_effect_index, is_item_usability_check ? "true" : "false", name_for_attack_medium(attack_medium)));
+  bool is_nte = s->options.is_nte();
+  auto log = s->log_stack(string_printf("check_usability_or_condition_apply(%02hhX, #%04hX, %02hhX, #%04hX, #%04hX, %02hhX, %s, %s): ", client_id1, card_id1, client_id2, card_id2, card_id3, def_effect_index, is_item_usability_check ? "true" : "false", name_for_enum(attack_medium)));
 
   if (static_cast<uint8_t>(attack_medium) & 0x80) {
     attack_medium = AttackMedium::UNKNOWN;
@@ -952,7 +953,7 @@ bool RulerServer::check_usability_or_condition_apply(
     log.debug("ce1 missing");
     return false;
   }
-  if (!s->options.is_nte() && (ce1->def.type == CardType::ITEM) && this->card_id_is_boss_sc(card_id2)) {
+  if (!is_nte && (ce1->def.type == CardType::ITEM) && this->card_id_is_boss_sc(card_id2)) {
     log.debug("ce1 is item and card_id2 is boss sc");
     return false;
   }
@@ -967,7 +968,7 @@ bool RulerServer::check_usability_or_condition_apply(
     }
     criterion_code = ce1->def.effects[def_effect_index].apply_criterion;
   }
-  log.debug("criterion_code=%s", name_for_criterion_code(criterion_code));
+  log.debug("criterion_code=%s", name_for_enum(criterion_code));
 
   // For item usability checks, prevent criteria that depend on player
   // positioning/team setup
@@ -989,7 +990,7 @@ bool RulerServer::check_usability_or_condition_apply(
   // second should not be given, so we'd return true if the criterion passes. If
   // neither of these cases apply, we should return false as a failsafe even if
   // the criterion passes. NTE did not have such a check.
-  bool ret = s->options.is_nte() || (!(def_effect_index & 0x80) || (client_id1 == client_id2)) || (client_id2 == 0xFF);
+  bool ret = is_nte || (!(def_effect_index & 0x80) || (client_id1 == client_id2)) || (client_id2 == 0xFF);
   switch (criterion_code) {
     case CriterionCode::NONE:
       return ret;
@@ -1413,7 +1414,7 @@ uint16_t RulerServer::compute_attack_or_defense_costs(
         return 99;
       }
       total_cost += (ce->def.self_cost + cost_bias);
-      if (card_class_is_tech_like(ce->def.card_class(), s->options.is_nte())) {
+      if (card_class_is_tech_like(ce->def.card_class(), is_nte)) {
         total_cost += tech_cost_bias;
       }
       total_ally_cost += ce->def.ally_cost;
@@ -1441,8 +1442,7 @@ uint16_t RulerServer::compute_attack_or_defense_costs(
       assist_cost_bias++;
     } else if (is_nte && (assist_effect == AssistEffect::DEFLATION)) {
       assist_cost_bias--;
-    } else if ((assist_effect == AssistEffect::BATTLE_ROYALE) &&
-        (pa.action_card_refs[0] == 0xFFFF)) {
+    } else if ((assist_effect == AssistEffect::BATTLE_ROYALE) && (pa.action_card_refs[0] == 0xFFFF)) {
       total_cost = 0;
       final_cost = 0;
     }
@@ -1470,9 +1470,9 @@ bool RulerServer::compute_effective_range_and_target_mode_for_attack(
     TargetMode* out_effective_target_mode,
     uint16_t* out_orig_card_ref) const {
   size_t z;
-  for (z = 0; (z < 9) && (pa.action_card_refs[z] != 0xFFFF); z++) {
+  for (z = 0; (z < 8) && (pa.action_card_refs[z] != 0xFFFF); z++) {
   }
-  if (z >= 9) {
+  if (z >= 8) {
     return false;
   }
   uint16_t card_ref = (z == 0) ? pa.attacker_card_ref : pa.action_card_refs[z - 1];
@@ -1630,8 +1630,7 @@ bool RulerServer::defense_card_can_apply_to_attack(
   return true;
 }
 
-bool RulerServer::defense_card_matches_any_attack_card_top_color(
-    const ActionState& pa) const {
+bool RulerServer::defense_card_matches_any_attack_card_top_color(const ActionState& pa) const {
   auto ce = this->definition_for_card_ref(pa.action_card_refs[0]);
   if (!ce) {
     throw runtime_error("defense card definition is missing");
@@ -2274,7 +2273,7 @@ bool RulerServer::is_attack_valid(const ActionState& pa) {
 
   size_t conditional_card_count = 0;
   size_t z;
-  for (z = 0; z < 9; z++) {
+  for (z = 0; z < 8; z++) {
     uint16_t right_card_ref = pa.action_card_refs[z];
     if (right_card_ref == 0xFFFF) {
       break;

src/Episode3/RulerServer.hh

@@ -198,11 +198,11 @@ private:
   std::weak_ptr<Server> w_server;
 
 public:
-  std::shared_ptr<HandAndEquipState> hand_and_equip_states[4];
-  std::shared_ptr<parray<CardShortStatus, 0x10>> short_statuses[4];
-  std::shared_ptr<DeckEntry> deck_entries[4];
-  std::shared_ptr<parray<ActionChainWithConds, 9>> set_card_action_chains[4];
-  std::shared_ptr<parray<ActionMetadata, 9>> set_card_action_metadatas[4];
+  bcarray<std::shared_ptr<HandAndEquipState>, 4> hand_and_equip_states;
+  bcarray<std::shared_ptr<parray<CardShortStatus, 0x10>>, 4> short_statuses;
+  bcarray<std::shared_ptr<DeckEntry>, 4> deck_entries;
+  bcarray<std::shared_ptr<parray<ActionChainWithConds, 9>>, 4> set_card_action_chains;
+  bcarray<std::shared_ptr<parray<ActionMetadata, 9>>, 4> set_card_action_metadatas;
   std::shared_ptr<MapAndRulesState> map_and_rules;
   std::shared_ptr<StateFlags> state_flags;
   std::shared_ptr<AssistServer> assist_server;

src/Episode3/Server.cc

@@ -4,6 +4,7 @@
 #include <phosg/Time.hh>
 
 #include "../Loggers.hh"
+#include "../Revision.hh"
 #include "../SendCommands.hh"
 
 using namespace std;
@@ -29,11 +30,15 @@ void Server::PresenceEntry::clear() {
 
 Server::Server(shared_ptr<Lobby> lobby, Options&& options)
     : lobby(lobby),
+      battle_record(lobby->battle_record),
       has_lobby(lobby != nullptr),
       options(std::move(options)),
       last_chosen_map(this->options.tournament ? this->options.tournament->get_map() : nullptr),
       tournament_match_result_sent(false),
       override_environment_number(0xFF),
+      def_dice_value_range_override(0xFF),
+      atk_dice_value_range_2v1_override(0xFF),
+      def_dice_value_range_2v1_override(0xFF),
       battle_finished(false),
       battle_in_progress(false),
       round_num(1),
@@ -160,6 +165,32 @@ const Server::StackLogger& Server::log() const {
   return *this->logger_stack.back();
 }
 
+std::string Server::debug_str_for_card_ref(uint16_t card_ref) const {
+  if (card_ref == 0xFFFF) {
+    return "@FFFF";
+  }
+  auto ce = this->definition_for_card_ref(card_ref);
+  if (ce) {
+    string name = ce->def.en_name.decode();
+    return string_printf("@%04hX (#%04" PRIX32 " %s)", card_ref, ce->def.card_id.load(), name.c_str());
+  } else {
+    return string_printf("@%04hX (missing)", card_ref);
+  }
+}
+
+std::string Server::debug_str_for_card_id(uint16_t card_id) const {
+  if (card_id == 0xFFFF) {
+    return "#FFFF";
+  }
+  auto ce = this->definition_for_card_id(card_id);
+  if (ce) {
+    string name = ce->def.en_name.decode();
+    return string_printf("#%04hX (%s)", card_id, name.c_str());
+  } else {
+    return string_printf("#%04hX (missing)", card_id);
+  }
+}
+
 int8_t Server::get_winner_team_id() const {
   // Note: This function is not part of the original implementation.
 
@@ -223,11 +254,12 @@ void Server::send(const void* data, size_t size, uint8_t command, bool enable_ma
     for (auto watcher_l : l->watcher_lobbies) {
       send_command_if_not_loading(watcher_l, command, 0x00, data, size);
     }
-    if (l->battle_record && l->battle_record->writable()) {
-      l->battle_record->add_command(BattleRecord::Event::Type::BATTLE_COMMAND, data, size);
+    if (this->battle_record && this->battle_record->writable()) {
+      this->battle_record->add_command(BattleRecord::Event::Type::BATTLE_COMMAND, data, size);
     }
 
-  } else if (this->log().info("Generated command")) {
+  } else if ((this->options.behavior_flags & BehaviorFlag::LOG_COMMANDS_IF_LOBBY_MISSING) &&
+      this->log().info("Generated command")) {
     print_data(stderr, data, size);
   }
 }
@@ -235,25 +267,15 @@ void Server::send(const void* data, size_t size, uint8_t command, bool enable_ma
 void Server::send_6xB4x46() const {
   // Note: This function is not part of the original implementation; it was
   // factored out from its callsites in this file and the strings were changed.
-  if (this->options.is_nte()) {
-    G_ServerVersionStrings_Ep3NTE_6xB4x46 cmd;
-    cmd.version_signature.encode(VERSION_SIGNATURE_NTE, 1);
-    cmd.date_str1.encode(format_time(this->options.card_index->definitions_mtime() * 1000000), 1);
-    this->send(cmd);
-  } else {
-    G_ServerVersionStrings_Ep3_6xB4x46 cmd;
-    cmd.version_signature.encode(VERSION_SIGNATURE, 1);
-    cmd.date_str1.encode(format_time(this->options.card_index->definitions_mtime() * 1000000), 1);
-    string date_str2 = string_printf(
-        "Random:%08" PRIX32 "+%08" PRIX32,
-        this->options.random_crypt->seed(),
-        this->options.random_crypt->absolute_offset());
-    if (this->last_chosen_map) {
-      date_str2 += string_printf(" Map:%08" PRIX32, this->last_chosen_map->map_number);
-    }
-    cmd.date_str2.encode(date_str2, 1);
-    this->send(cmd);
-  }
+
+  // NTE doesn't have the date_str2 field, but we send it anyway to make
+  // debugging easier.
+  G_ServerVersionStrings_Ep3_6xB4x46 cmd;
+  cmd.version_signature.encode(this->options.is_nte() ? VERSION_SIGNATURE_NTE : VERSION_SIGNATURE, 1);
+  cmd.date_str1.encode(format_time(this->options.card_index->definitions_mtime() * 1000000), 1);
+  string build_date = format_time(BUILD_TIMESTAMP);
+  cmd.date_str2.encode(string_printf("newserv %s compiled at %s", GIT_REVISION_HASH, build_date.c_str()), 1);
+  this->send(cmd);
 }
 
 string Server::prepare_6xB6x41_map_definition(shared_ptr<const MapIndex::Map> map, uint8_t language, bool is_nte) {
@@ -752,6 +774,9 @@ void Server::destroy_cards_with_zero_hp() {
 void Server::determine_first_team_turn() {
   this->team_client_count[0] = this->map_and_rules->num_team0_players;
   this->team_client_count[1] = this->map_and_rules->num_players - this->team_client_count[0];
+  if (this->team_client_count[0] == 0 || this->team_client_count[1] == 0) {
+    throw runtime_error("one or both teams have no players");
+  }
   this->first_team_turn = 0xFF;
   while (this->first_team_turn == 0xFF) {
     uint8_t results[2] = {0, 0};
@@ -946,44 +971,63 @@ void Server::end_action_phase() {
 }
 
 bool Server::enqueue_attack_or_defense(uint8_t client_id, ActionState* pa) {
+  auto log = this->log_stack("enqueue_attack_or_defense: ");
+  if (log.should_log(LogLevel::DEBUG)) {
+    string s = pa->str(this->shared_from_this());
+    log.debug("input: %s", s.c_str());
+  }
+
   if (client_id >= 4) {
     this->ruler_server->error_code3 = -0x78;
+    log.debug("failed: invalid client ID");
     return false;
   }
 
   auto ps = this->player_states[client_id];
   if (!ps) {
     this->ruler_server->error_code3 = -0x72;
+    log.debug("failed: player not present");
     return false;
   }
 
   if (pa->action_card_refs[0] == 0xFFFF) {
     if (pa->defense_card_ref != 0xFFFF) {
       pa->action_card_refs[0] = pa->defense_card_ref;
+      log.debug("moved defense card ref to action card ref 0");
     }
   } else {
     pa->defense_card_ref = pa->action_card_refs[0];
+    log.debug("moved action card ref 0 to defense card ref");
   }
 
   if (!this->ruler_server->is_attack_or_defense_valid(*pa)) {
+    log.debug("failed: attack or defense not valid");
     return false;
   }
 
   int16_t ally_atk_result = this->send_6xB4x33_remove_ally_atk_if_needed(*pa);
   if (ally_atk_result == 1) {
+    log.debug("pending: need ally approval");
     return true;
   } else if (ally_atk_result == -1) {
+    log.debug("failed: ally declined");
     return false;
   }
 
   if (this->num_pending_attacks >= 0x20) {
     this->ruler_server->error_code3 = -0x71;
+    log.debug("failed: too many pending attacks");
     return false;
   }
 
   size_t attack_index = this->num_pending_attacks++;
   this->pending_attacks[attack_index] = *pa;
+  if (log.should_log(LogLevel::DEBUG)) {
+    string pa_str = this->pending_attacks[attack_index].str(this->shared_from_this());
+    log.debug("set pending attack %zu: %s", attack_index, pa_str.c_str());
+  }
   ps->set_action_cards_for_action_state(*pa);
+  log.debug("set action cards");
   auto card = this->card_for_set_card_ref(this->send_6xB4x06_if_card_ref_invalid(pa->attacker_card_ref, 1));
   if (card) {
     card->card_flags |= 0x400;
@@ -1030,16 +1074,24 @@ shared_ptr<const PlayerState> Server::get_player_state(uint8_t client_id) const
   return this->player_states[client_id];
 }
 
+uint32_t Server::get_random_raw() {
+  le_uint32_t ret = random_from_optional_crypt(this->options.opt_rand_crypt);
+  if (this->battle_record && this->battle_record->writable()) {
+    this->battle_record->add_random_data(&ret, sizeof(ret));
+  }
+  return ret;
+}
+
 uint32_t Server::get_random(uint32_t max) {
   // The original implementation was essentially:
-  // return (static_cast<double>(this->random_crypt->next() >> 16) / 65536.0) * max
-  // This is unnecessarily complicated, so we instead just do this:
-  return this->options.random_crypt->next() % max;
+  // return (static_cast<double>(this->random_source->next() >> 16) / 65536.0) * max
+  // This is unnecessarily complicated and imprecise, so we instead just do:
+  return this->get_random_raw() % max;
 }
 
 float Server::get_random_float_0_1() {
   // This lacks some precision, but matches the original implementation.
-  return (static_cast<double>(this->options.random_crypt->next() >> 16) / 65536.0);
+  return (static_cast<double>(this->get_random_raw() >> 16) / 65536.0);
 }
 
 uint32_t Server::get_round_num() const {
@@ -1318,6 +1370,13 @@ void Server::set_battle_started() {
   this->send_6xB4x05();
 }
 
+bool Server::player_can_receive_dice_boost(uint8_t client_id) const {
+  auto ps = this->player_states[client_id];
+  bool is_1p_2v1 = (this->team_client_count.at(ps->get_team_id()) < this->team_client_count[ps->get_team_id() ^ 1]);
+  const auto& rules = this->map_and_rules->rules;
+  return (rules.atk_dice_range(is_1p_2v1).second >= 3) || (rules.def_dice_range(is_1p_2v1).second >= 3);
+}
+
 void Server::set_client_id_ready_to_advance_phase(uint8_t client_id, BattlePhase battle_phase) {
   if (client_id >= 4) {
     return;
@@ -1332,7 +1391,10 @@ void Server::set_client_id_ready_to_advance_phase(uint8_t client_id, BattlePhase
     ps->assist_flags |= AssistFlag::READY_TO_END_PHASE;
     ps->update_hand_and_equip_state_and_send_6xB4x02_if_needed();
     if (this->battle_phase == BattlePhase::DICE) {
-      if (is_nte || !(ps->assist_flags & AssistFlag::ELIGIBLE_FOR_DICE_BOOST) || this->map_and_rules->rules.disable_dice_boost) {
+      if (is_nte ||
+          !(ps->assist_flags & AssistFlag::ELIGIBLE_FOR_DICE_BOOST) ||
+          this->map_and_rules->rules.disable_dice_boost ||
+          !this->player_can_receive_dice_boost(client_id)) {
         ps->assist_flags &= (~AssistFlag::ELIGIBLE_FOR_DICE_BOOST);
         ps->roll_main_dice_or_apply_after_effects();
         if (!is_nte && (ps->get_atk_points() < 3) && (ps->get_def_points() < 3)) {
@@ -1486,8 +1548,8 @@ void Server::setup_and_start_battle() {
 
   this->setup_phase = SetupPhase::STARTER_ROLLS;
 
-  // Note: This is where original implementation re-seeds random_crypt (it uses
-  // time() as the seed value).
+  // Note: This is where original implementation re-seeds its random generator
+  // (it uses time() as the seed value).
 
   for (size_t z = 0; z < 4; z++) {
     if (!this->check_presence_entry(z)) {
@@ -1761,8 +1823,10 @@ const unordered_map<uint8_t, Server::handler_t> Server::subcommand_handlers({
 
 void Server::on_server_data_input(shared_ptr<Client> sender_c, const string& data) {
   auto header = check_size_t<G_CardBattleCommandHeader>(data, 0xFFFF);
-  if (header.size * 4 < data.size()) {
-    throw runtime_error("command is incomplete");
+  size_t expected_size = header.size * 4;
+  if (expected_size < data.size()) {
+    print_data(stderr, data);
+    throw runtime_error(string_printf("command is incomplete: expected %zX bytes, received %zX bytes", expected_size, data.size()));
   }
   if (header.subcommand != 0xB3) {
     throw runtime_error("server data command is not 6xB3");
@@ -1775,7 +1839,11 @@ void Server::on_server_data_input(shared_ptr<Client> sender_c, const string& dat
     throw runtime_error("unknown CAx subsubcommand");
   }
 
-  if (this->options.is_nte() || !header.mask_key) {
+  if (this->battle_record && this->battle_record->writable()) {
+    this->battle_record->add_command(BattleRecord::Event::Type::SERVER_DATA_COMMAND, data.data(), data.size());
+  }
+
+  if ((sender_c && (sender_c->version() == Version::GC_EP3_NTE)) || !header.mask_key) {
     (this->*handler)(sender_c, data);
   } else {
     string unmasked_data = data;
@@ -2091,22 +2159,42 @@ void Server::handle_CAx12_end_attack_list(shared_ptr<Client>, const string& data
 }
 
 template <typename CmdT>
-void Server::handle_CAx13_update_map_during_setup_t(shared_ptr<Client>, const string& data) {
+void Server::handle_CAx13_update_map_during_setup_t(shared_ptr<Client> c, const string& data) {
   const auto& in_cmd = check_size_t<CmdT>(data);
-  this->send_debug_command_received_message(
-      in_cmd.header.subsubcommand, "UPDATE MAP");
+  this->send_debug_command_received_message(in_cmd.header.subsubcommand, "UPDATE MAP");
 
   if (!this->battle_in_progress &&
       (this->setup_phase == SetupPhase::REGISTRATION) &&
       (this->map_and_rules->num_players == 0) &&
       (this->registration_phase != RegistrationPhase::REGISTERED) &&
       (this->registration_phase != RegistrationPhase::BATTLE_STARTED)) {
-    // newserv's extended rules are stored in unused parts of the Rules struct,
-    // and clients will probably overwrite them with zeroes if we allow them to.
-    // So, we preserve the extended rules manually here.
-    uint8_t def_dice_range = this->map_and_rules->rules.def_dice_range;
     *this->map_and_rules = in_cmd.map_and_rules_state;
-    this->map_and_rules->rules.def_dice_range = def_dice_range;
+    // The client will likely send incorrect values for the extended rules (or
+    // in the case of NTE, no values at all, since the Rules structure is
+    // smaller). So, use the values from the last chosen map if applicable, or
+    // the values from the $dicerange command if available.
+    uint8_t language = c ? c->language() : 1;
+    const Rules* map_rules = this->last_chosen_map ? &this->last_chosen_map->version(language)->map->default_rules : nullptr;
+    auto& server_rules = this->map_and_rules->rules;
+    // NTE can specify the DEF dice value range in its Rules struct, so we use
+    // that unless the map or $dicerange overrides it.
+    server_rules.def_dice_value_range = (map_rules && (map_rules->def_dice_value_range != 0xFF))
+        ? map_rules->def_dice_value_range
+        : (this->def_dice_value_range_override != 0xFF)
+        ? this->def_dice_value_range_override
+        : this->options.is_nte()
+        ? server_rules.def_dice_value_range
+        : 0;
+    server_rules.atk_dice_value_range_2v1 = (map_rules && (map_rules->atk_dice_value_range_2v1 != 0xFF))
+        ? map_rules->atk_dice_value_range_2v1
+        : (this->atk_dice_value_range_2v1_override != 0xFF)
+        ? this->atk_dice_value_range_2v1_override
+        : 0;
+    server_rules.def_dice_value_range_2v1 = (map_rules && (map_rules->def_dice_value_range_2v1 != 0xFF))
+        ? map_rules->def_dice_value_range_2v1
+        : (this->def_dice_value_range_2v1_override != 0xFF)
+        ? this->def_dice_value_range_2v1_override
+        : 0;
 
     // If this match is part of a tournament, ignore the rules sent by the
     // client and use the tournament rules instead.
@@ -2266,11 +2354,12 @@ void Server::handle_CAx1D_start_battle(shared_ptr<Client>, const string& data) {
     }
 
     if (should_start) {
+      if (this->battle_record && this->battle_record->writable()) {
+        this->battle_record->set_battle_start_timestamp();
+      }
+
       auto l = this->lobby.lock();
       if (l) {
-        if (l->battle_record) {
-          l->battle_record->set_battle_start_timestamp();
-        }
         // Note: Sega's implementation doesn't set EX results values here; they
         // did it at game join time instead. We do it here for code simplicity.
         if ((l->base_version != Version::GC_EP3_NTE) && l->ep3_ex_result_values) {
@@ -2413,8 +2502,7 @@ void Server::handle_CAx34_subtract_ally_atk_points(shared_ptr<Client>, const str
               attacker_card->card_flags |= 0x400;
               attacker_card->player_state()->send_6xB4x04_if_needed();
             }
-            uint16_t card_ref = this->send_6xB4x06_if_card_ref_invalid(
-                pa.original_attacker_card_ref, 9);
+            uint16_t card_ref = this->send_6xB4x06_if_card_ref_invalid(pa.original_attacker_card_ref, 9);
             auto orig_attacker_card = this->card_for_set_card_ref(card_ref);
             auto target_card = this->card_for_set_card_ref(pa.target_card_refs[0]);
             if (orig_attacker_card && target_card) {
@@ -2474,8 +2562,7 @@ void Server::handle_CAx3A_time_limit_expired(shared_ptr<Client>, const string& d
 
 void Server::handle_CAx40_map_list_request(shared_ptr<Client> sender_c, const string& data) {
   const auto& in_cmd = check_size_t<G_MapListRequest_Ep3_CAx40>(data);
-  this->send_debug_command_received_message(
-      in_cmd.header.subsubcommand, "MAP LIST");
+  this->send_debug_command_received_message(in_cmd.header.subsubcommand, "MAP LIST");
 
   auto l = this->lobby.lock();
   if (!l) {
@@ -2499,6 +2586,10 @@ void Server::handle_CAx40_map_list_request(shared_ptr<Client> sender_c, const st
 }
 
 void Server::send_6xB6x41_to_all_clients() const {
+  if (!this->last_chosen_map) {
+    throw logic_error("cannot send 6xB4x41 without a map chosen");
+  }
+
   auto l = this->lobby.lock();
   if (l) {
     vector<string> map_commands_by_language;
@@ -2525,13 +2616,13 @@ void Server::send_6xB6x41_to_all_clients() const {
       }
     }
 
-    if (l->battle_record && l->battle_record->writable()) {
+    if (this->battle_record && this->battle_record->writable()) {
       // TODO: It's not great that we just pick the first one; ideally we'd put
       // all of them in the recording and send the appropriate one to the client
       // in the playback lobby
       for (string& data : map_commands_by_language) {
         if (!data.empty()) {
-          l->battle_record->add_command(
+          this->battle_record->add_command(
               BattleRecord::Event::Type::BATTLE_COMMAND, std::move(data));
           break;
         }
@@ -2546,9 +2637,7 @@ void Server::send_6xB6x41_to_all_clients() const {
 
 void Server::handle_CAx41_map_request(shared_ptr<Client>, const string& data) {
   const auto& cmd = check_size_t<G_MapDataRequest_Ep3_CAx41>(data);
-  this->send_debug_command_received_message(
-      cmd.header.subsubcommand, "MAP DATA");
-
+  this->send_debug_command_received_message(cmd.header.subsubcommand, "MAP DATA");
   this->last_chosen_map = this->options.map_index->for_number(cmd.map_number);
   this->send_6xB6x41_to_all_clients();
 }
@@ -2713,11 +2802,9 @@ uint32_t Server::get_team_exp(uint8_t team_id) const {
   return this->team_exp[team_id];
 }
 
-uint32_t Server::send_6xB4x06_if_card_ref_invalid(
-    uint16_t card_ref, int16_t negative_value) {
+uint32_t Server::send_6xB4x06_if_card_ref_invalid(uint16_t card_ref, int16_t negative_value) {
   if (this->card_special) {
-    return this->card_special->send_6xB4x06_if_card_ref_invalid(
-        card_ref, -negative_value);
+    return this->card_special->send_6xB4x06_if_card_ref_invalid(card_ref, -negative_value);
   }
   return card_ref;
 }
@@ -2735,16 +2822,20 @@ void Server::unknown_8023EEF4() {
     auto card = this->attack_cards[this->unknown_a14];
     if (this->get_current_team_turn() == card->get_team_id()) {
       ActionState as = this->pending_attacks_with_cards[this->unknown_a14];
-      log.debug("card @%04hX #%04hX can attack", card->get_card_ref(), card->get_card_id());
-      string as_str = as.str();
-      log.debug("as: %s", as_str.c_str());
+      if (log.should_log(LogLevel::DEBUG)) {
+        log.debug("card @%04hX #%04hX can attack", card->get_card_ref(), card->get_card_id());
+        string as_str = as.str(this->shared_from_this());
+        log.debug("as: %s", as_str.c_str());
+      }
       if (is_nte) {
         this->replace_targets_due_to_destruction_nte(&as);
       } else {
         this->replace_targets_due_to_destruction_or_conditions(&as);
       }
-      as_str = as.str();
-      log.debug("as after target replacement: %s", as_str.c_str());
+      if (log.should_log(LogLevel::DEBUG)) {
+        string as_str = as.str(this->shared_from_this());
+        log.debug("as after target replacement: %s", as_str.c_str());
+      }
       if (this->any_target_exists_for_attack(as)) {
         log.debug("as is valid");
         break;
@@ -2856,7 +2947,7 @@ void Server::replace_targets_due_to_destruction_nte(ActionState* as) {
     if (!target_card) {
       break;
     }
-    if ((target_card->card_flags & 2) ||
+    if (!(target_card->card_flags & 2) ||
         (target_card->get_definition()->def.type != CardType::ITEM) ||
         attacker_card->action_chain.check_flag(0x02)) {
       continue;

src/Episode3/Server.hh

@@ -9,6 +9,7 @@
 #include "../CommandFormats.hh"
 #include "../Text.hh"
 #include "AssistServer.hh"
+#include "BattleRecord.hh"
 #include "CardSpecial.hh"
 #include "MapState.hh"
 #include "PlayerState.hh"
@@ -71,7 +72,7 @@ public:
     std::shared_ptr<const CardIndex> card_index;
     std::shared_ptr<const MapIndex> map_index;
     uint32_t behavior_flags;
-    std::shared_ptr<PSOLFGEncryption> random_crypt;
+    std::shared_ptr<PSOLFGEncryption> opt_rand_crypt;
     std::shared_ptr<const Tournament> tournament;
     std::array<std::vector<uint16_t>, 5> trap_card_ids;
 
@@ -99,6 +100,33 @@ public:
   StackLogger log_stack(const std::string& prefix) const;
   const StackLogger& log() const;
 
+  std::string debug_str_for_card_ref(uint16_t card_ref) const;
+  std::string debug_str_for_card_id(uint16_t card_id) const;
+  template <typename U16T>
+  std::string debug_str_for_card_refs(const U16T* refs, size_t count) const {
+    std::string ret = "[";
+    for (size_t z = 0; z < count; z++) {
+      if (refs[z] != 0xFFFF) {
+        std::string ref_str = this->debug_str_for_card_ref(refs[z]);
+        ret += string_printf("%zu:%s ", z, ref_str.c_str());
+      }
+    }
+    if (ret.size() > 1) {
+      ret.back() = ']'; // Replace the ' ' from the last added item
+    } else {
+      ret.push_back(']');
+    }
+    return ret;
+  }
+  template <typename U16T>
+  std::string debug_str_for_card_refs(const std::vector<U16T>& refs) const {
+    return this->debug_str_for_card_refs(refs.data(), refs.size());
+  }
+  template <typename U16T, size_t Count>
+  std::string debug_str_for_card_refs(const parray<U16T, Count>& refs) const {
+    return this->debug_str_for_card_refs(refs.data(), refs.size());
+  }
+
   int8_t get_winner_team_id() const;
 
   template <typename T>
@@ -164,6 +192,7 @@ public:
   uint8_t get_current_team_turn() const;
   std::shared_ptr<PlayerState> get_player_state(uint8_t client_id);
   std::shared_ptr<const PlayerState> get_player_state(uint8_t client_id) const;
+  uint32_t get_random_raw();
   uint32_t get_random(uint32_t max);
   float get_random_float_0_1();
   uint32_t get_round_num() const;
@@ -178,6 +207,7 @@ public:
   void send_set_card_updates_and_6xB4x04_if_needed();
   void set_battle_ended();
   void set_battle_started();
+  bool player_can_receive_dice_boost(uint8_t client_id) const;
   void set_client_id_ready_to_advance_phase(uint8_t client_id, BattlePhase battle_phase);
   void set_phase_after();
   void move_phase_before();
@@ -245,11 +275,15 @@ private:
 public:
   // These fields are not part of the original implementation
   std::weak_ptr<Lobby> lobby;
+  std::shared_ptr<BattleRecord> battle_record;
   bool has_lobby;
   Options options;
   std::shared_ptr<const MapIndex::Map> last_chosen_map;
   bool tournament_match_result_sent;
   uint8_t override_environment_number;
+  uint8_t def_dice_value_range_override;
+  uint8_t atk_dice_value_range_2v1_override;
+  uint8_t def_dice_value_range_2v1_override;
   mutable std::deque<StackLogger*> logger_stack;
 
   // These fields were originally contained in the TCardServerBase object
@@ -259,9 +293,10 @@ public:
     uint8_t is_cpu_player;
     PresenceEntry();
     void clear();
-  } __attribute__((packed));
+  } __packed_ws__(PresenceEntry, 3);
+
   std::shared_ptr<MapAndRulesState> map_and_rules;
-  std::shared_ptr<DeckEntry> deck_entries[4];
+  bcarray<std::shared_ptr<DeckEntry>, 4> deck_entries;
   parray<PresenceEntry, 4> presence_entries;
   uint8_t num_clients_present;
   parray<NameEntry, 4> name_entries;
@@ -280,7 +315,7 @@ public:
   RegistrationPhase registration_phase;
   ActionSubphase action_subphase;
   uint8_t current_team_turn2;
-  ActionState pending_attacks[0x20];
+  bcarray<ActionState, 0x20> pending_attacks;
   uint32_t num_pending_attacks;
   parray<uint8_t, 4> client_done_enqueuing_attacks;
   parray<uint8_t, 4> player_ready_to_end_phase;
@@ -296,8 +331,8 @@ public:
   std::array<std::shared_ptr<PlayerState>, 4> player_states;
   parray<uint32_t, 4> clients_done_in_mulligan_phase;
   uint32_t num_pending_attacks_with_cards;
-  std::shared_ptr<Card> attack_cards[0x20];
-  ActionState pending_attacks_with_cards[0x20];
+  bcarray<std::shared_ptr<Card>, 0x20> attack_cards;
+  bcarray<ActionState, 0x20> pending_attacks_with_cards;
   uint32_t unknown_a14;
   uint32_t unknown_a15;
   parray<uint32_t, 4> defense_list_ended_for_client;
@@ -315,7 +350,7 @@ public:
   parray<parray<parray<uint8_t, 2>, 8>, 5> trap_tile_locs;
   parray<parray<uint8_t, 2>, 0x10> trap_tile_locs_nte;
   size_t num_trap_tiles_nte;
-  ActionState pb_action_states[4];
+  bcarray<ActionState, 4> pb_action_states;
   parray<uint8_t, 4> has_done_pb;
   parray<parray<uint8_t, 4>, 4> has_done_pb_with_client;
   mutable uint32_t num_6xB4x06_commands_sent;

src/Episode3/Tournament.cc

@@ -9,18 +9,18 @@ using namespace std;
 
 namespace Episode3 {
 
-Tournament::PlayerEntry::PlayerEntry(uint32_t serial_number, const string& player_name)
-    : serial_number(serial_number),
+Tournament::PlayerEntry::PlayerEntry(uint32_t account_id, const string& player_name)
+    : account_id(account_id),
       player_name(player_name) {}
 
 Tournament::PlayerEntry::PlayerEntry(shared_ptr<Client> c)
-    : serial_number(c->license->serial_number),
+    : account_id(c->login->account->account_id),
       client(c),
       player_name(c->character()->disp.name.decode(c->language())) {}
 
 Tournament::PlayerEntry::PlayerEntry(
     shared_ptr<const COMDeckDefinition> com_deck)
-    : serial_number(0),
+    : account_id(0),
       com_deck(com_deck) {}
 
 bool Tournament::PlayerEntry::is_com() const {
@@ -28,7 +28,7 @@ bool Tournament::PlayerEntry::is_com() const {
 }
 
 bool Tournament::PlayerEntry::is_human() const {
-  return (this->serial_number != 0);
+  return (this->account_id != 0);
 }
 
 Tournament::Team::Team(
@@ -56,9 +56,9 @@ string Tournament::Team::str() const {
   for (const auto& player : this->players) {
     if (player.is_human()) {
       if (player.player_name.empty()) {
-        ret += string_printf(" %08" PRIX32, player.serial_number);
+        ret += string_printf(" %08" PRIX32, player.account_id);
       } else {
-        ret += string_printf(" %08" PRIX32 " (%s)", player.serial_number, player.player_name.c_str());
+        ret += string_printf(" %08" PRIX32 " (%s)", player.account_id, player.player_name.c_str());
       }
     }
   }
@@ -81,12 +81,12 @@ void Tournament::Team::register_player(
   if (!tournament) {
     throw runtime_error("tournament has been deleted");
   }
-  if (!tournament->all_player_serial_numbers.emplace(c->license->serial_number).second) {
+  if (!tournament->all_player_account_ids.emplace(c->login->account->account_id).second) {
     throw runtime_error("player already registered in same tournament");
   }
 
   for (const auto& player : this->players) {
-    if (player.is_human() && (player.serial_number == c->license->serial_number)) {
+    if (player.is_human() && (player.account_id == c->login->account->account_id)) {
       throw logic_error("player already registered in team but not in tournament");
     }
   }
@@ -99,11 +99,11 @@ void Tournament::Team::register_player(
   }
 }
 
-bool Tournament::Team::unregister_player(uint32_t serial_number) {
+bool Tournament::Team::unregister_player(uint32_t account_id) {
   size_t index;
   for (index = 0; index < this->players.size(); index++) {
     if (this->players[index].is_human() &&
-        (this->players[index].serial_number == serial_number)) {
+        (this->players[index].account_id == account_id)) {
       break;
     }
   }
@@ -143,7 +143,7 @@ bool Tournament::Team::unregister_player(uint32_t serial_number) {
       // If the tournament has not started yet, just remove the player from the
       // team
     } else {
-      if (!tournament->all_player_serial_numbers.erase(serial_number)) {
+      if (!tournament->all_player_account_ids.erase(account_id)) {
         throw logic_error("player removed from team but not from tournament");
       }
     }
@@ -371,13 +371,13 @@ void Tournament::init() {
       team_index_to_rounds_cleared.emplace_back(team_json->get_int("num_rounds_cleared"));
       for (const auto& player_json : team_json->get_list("player_specs")) {
         if (player_json->is_list()) {
-          uint32_t serial_number = player_json->at(0).as_int();
-          team->players.emplace_back(serial_number, player_json->at(1).as_string());
-          this->all_player_serial_numbers.emplace(serial_number);
+          uint32_t account_id = player_json->at(0).as_int();
+          team->players.emplace_back(account_id, player_json->at(1).as_string());
+          this->all_player_account_ids.emplace(account_id);
         } else if (player_json->is_int()) {
-          uint32_t serial_number = player_json->as_int();
-          team->players.emplace_back(serial_number);
-          this->all_player_serial_numbers.emplace(serial_number);
+          uint32_t account_id = player_json->as_int();
+          team->players.emplace_back(account_id);
+          this->all_player_account_ids.emplace(account_id);
         } else if (player_json->is_string()) {
           team->players.emplace_back(this->com_deck_index->deck_for_name(player_json->as_string()));
         } else {
@@ -511,9 +511,9 @@ JSON Tournament::json() const {
     for (const auto& player : team->players) {
       if (player.is_human()) {
         if (!player.player_name.empty()) {
-          players_list.emplace_back(JSON::list({player.serial_number, player.player_name}));
+          players_list.emplace_back(JSON::list({player.account_id, player.player_name}));
         } else {
-          players_list.emplace_back(player.serial_number);
+          players_list.emplace_back(player.account_id);
         }
       } else {
         players_list.emplace_back(player.com_deck->deck_name);
@@ -571,25 +571,25 @@ shared_ptr<Tournament::Match> Tournament::get_final_match() const {
   return this->final_match;
 }
 
-shared_ptr<Tournament::Team> Tournament::team_for_serial_number(
-    uint32_t serial_number) const {
-  if (!this->all_player_serial_numbers.count(serial_number)) {
+shared_ptr<Tournament::Team> Tournament::team_for_account_id(
+    uint32_t account_id) const {
+  if (!this->all_player_account_ids.count(account_id)) {
     return nullptr;
   }
 
   for (auto team : this->teams) {
     for (const auto& player : team->players) {
-      if (player.serial_number == serial_number) {
+      if (player.account_id == account_id) {
         return team->is_active ? team : nullptr;
       }
     }
   }
 
-  throw logic_error("serial number registered in tournament but not in any team");
+  throw logic_error("account ID registered in tournament but not in any team");
 }
 
-const set<uint32_t>& Tournament::get_all_player_serial_numbers() const {
-  return this->all_player_serial_numbers;
+const set<uint32_t>& Tournament::get_all_player_account_ids() const {
+  return this->all_player_account_ids;
 }
 
 void Tournament::start() {
@@ -896,10 +896,10 @@ bool TournamentIndex::delete_tournament(const string& name) {
   return true;
 }
 
-shared_ptr<Tournament::Team> TournamentIndex::team_for_serial_number(uint32_t serial_number) const {
+shared_ptr<Tournament::Team> TournamentIndex::team_for_account_id(uint32_t account_id) const {
   for (const auto& it : this->name_to_tournament) {
     const auto& tourn = it.second;
-    auto team = tourn->team_for_serial_number(serial_number);
+    auto team = tourn->team_for_account_id(account_id);
     if (team) {
       return team;
     }
@@ -912,11 +912,11 @@ void TournamentIndex::link_client(shared_ptr<Client> c) {
     return;
   }
 
-  auto team = this->team_for_serial_number(c->license->serial_number);
+  auto team = this->team_for_account_id(c->login->account->account_id);
   auto tourn = team ? team->tournament.lock() : nullptr;
   if (team && team->is_active && tourn) {
     for (auto& player : team->players) {
-      if (player.serial_number == c->license->serial_number) {
+      if (player.account_id == c->login->account->account_id) {
         c->ep3_tournament_team = team;
         player.client = c;
         if (c->version() == Version::GC_EP3) {

src/Episode3/Tournament.hh

@@ -33,16 +33,16 @@ public:
   };
 
   struct PlayerEntry {
-    // Invariant: (serial_number == 0) != (com_deck == nullptr)
+    // Invariant: (account_id == 0) != (com_deck == nullptr)
     // (that is, exactly one of the following must be valid)
-    uint32_t serial_number;
+    uint32_t account_id;
     std::shared_ptr<const COMDeckDefinition> com_deck;
 
-    // client is valid if serial_number is nonzero and the client is connected
+    // client is valid if account_id is nonzero and the client is connected
     std::weak_ptr<Client> client;
     std::string player_name; // Not used for COM decks
 
-    explicit PlayerEntry(uint32_t serial_number, const std::string& player_name = "");
+    explicit PlayerEntry(uint32_t account_id, const std::string& player_name = "");
     explicit PlayerEntry(std::shared_ptr<Client> c);
     explicit PlayerEntry(std::shared_ptr<const COMDeckDefinition> com_deck);
 
@@ -73,7 +73,7 @@ public:
         std::shared_ptr<Client> c,
         const std::string& team_name,
         const std::string& password);
-    bool unregister_player(uint32_t serial_number);
+    bool unregister_player(uint32_t account_id);
 
     bool has_any_human_players() const;
     size_t num_human_players() const;
@@ -152,8 +152,8 @@ public:
   std::shared_ptr<Team> get_winner_team() const;
   std::shared_ptr<Match> next_match_for_team(std::shared_ptr<Team> team) const;
   std::shared_ptr<Match> get_final_match() const;
-  std::shared_ptr<Team> team_for_serial_number(uint32_t serial_number) const;
-  const std::set<uint32_t>& get_all_player_serial_numbers() const;
+  std::shared_ptr<Team> team_for_account_id(uint32_t account_id) const;
+  const std::set<uint32_t>& get_all_player_account_ids() const;
 
   void start();
 
@@ -178,7 +178,7 @@ private:
   State current_state;
   uint32_t menu_item_id;
 
-  std::set<uint32_t> all_player_serial_numbers;
+  std::set<uint32_t> all_player_account_ids;
   std::unordered_set<std::shared_ptr<Match>> pending_matches;
 
   // This vector contains all teams in the original starting order of the
@@ -231,7 +231,7 @@ public:
       uint8_t flags);
   bool delete_tournament(const std::string& name);
 
-  std::shared_ptr<Tournament::Team> team_for_serial_number(uint32_t serial_number) const;
+  std::shared_ptr<Tournament::Team> team_for_account_id(uint32_t account_id) const;
 
   void link_client(std::shared_ptr<Client> c);
   void link_all_clients(std::shared_ptr<ServerState> s);

src/EventUtils.cc

@@ -0,0 +1,43 @@
+#include "EventUtils.hh"
+
+#include <event2/event.h>
+
+#include <deque>
+#include <functional>
+#include <memory>
+#include <stdexcept>
+
+static void dispatch_forward_to_event_thread(evutil_socket_t, short, void* ctx) {
+  auto* fn = reinterpret_cast<std::function<void()>*>(ctx);
+  (*fn)();
+  delete fn;
+}
+
+void forward_to_event_thread(std::shared_ptr<struct event_base> base, std::function<void()>&& fn) {
+  struct timeval tv = {0, 0};
+  std::function<void()>* new_fn = new std::function<void()>(std::move(fn));
+  event_base_once(base.get(), -1, EV_TIMEOUT, dispatch_forward_to_event_thread, new_fn, &tv);
+}
+
+template <>
+void call_on_event_thread<void>(std::shared_ptr<struct event_base> base, std::function<void()>&& compute) {
+  bool succeeded = false;
+  std::string exc_what;
+  std::mutex ret_lock;
+  std::condition_variable ret_cv;
+  std::unique_lock<std::mutex> g(ret_lock);
+  forward_to_event_thread(base, [&]() -> void {
+    std::lock_guard<std::mutex> g(ret_lock);
+    try {
+      compute();
+      succeeded = true;
+    } catch (const std::exception& e) {
+      exc_what = e.what();
+    }
+    ret_cv.notify_one();
+  });
+  ret_cv.wait(g);
+  if (!succeeded) {
+    throw std::runtime_error(exc_what);
+  }
+}

src/EventUtils.hh

@@ -0,0 +1,38 @@
+#pragma once
+
+#include <event2/event.h>
+
+#include <condition_variable>
+#include <functional>
+#include <memory>
+#include <mutex>
+#include <optional>
+#include <stdexcept>
+
+void forward_to_event_thread(std::shared_ptr<struct event_base> base, std::function<void()>&& fn);
+
+template <typename T>
+T call_on_event_thread(std::shared_ptr<struct event_base> base, std::function<T()>&& compute) {
+  std::optional<T> ret;
+  std::string exc_what;
+  std::mutex ret_lock;
+  std::condition_variable ret_cv;
+  std::unique_lock<std::mutex> g(ret_lock);
+  forward_to_event_thread(base, [&]() -> void {
+    std::lock_guard<std::mutex> g(ret_lock);
+    try {
+      ret = compute();
+    } catch (const std::exception& e) {
+      exc_what = e.what();
+    }
+    ret_cv.notify_one();
+  });
+  ret_cv.wait(g);
+  if (!ret.has_value()) {
+    throw std::runtime_error(exc_what);
+  }
+  return ret.value();
+}
+
+template <>
+void call_on_event_thread<void>(std::shared_ptr<struct event_base> base, std::function<void()>&& compute);

src/FunctionCompiler.cc

@@ -10,6 +10,7 @@
 
 #ifdef HAVE_RESOURCE_FILE
 #include <resource_file/Emulators/PPC32Emulator.hh>
+#include <resource_file/Emulators/SH4Emulator.hh>
 #include <resource_file/Emulators/X86Emulator.hh>
 #endif
 
@@ -39,6 +40,8 @@ const char* name_for_architecture(CompiledFunctionCode::Architecture arch) {
       return "PowerPC";
     case CompiledFunctionCode::Architecture::X86:
       return "x86";
+    case CompiledFunctionCode::Architecture::SH4:
+      return "SH-4";
     default:
       throw logic_error("invalid architecture");
   }
@@ -117,12 +120,14 @@ bool CompiledFunctionCode::is_big_endian() const {
 
 shared_ptr<CompiledFunctionCode> compile_function_code(
     CompiledFunctionCode::Architecture arch,
-    const string& directory,
+    const string& function_directory,
+    const string& system_directory,
     const string& name,
     const string& text) {
 #ifndef HAVE_RESOURCE_FILE
   (void)arch;
-  (void)directory;
+  (void)function_directory;
+  (void)system_directory;
   (void)name;
   (void)text;
   throw runtime_error("function compiler is not available");
@@ -151,7 +156,11 @@ shared_ptr<CompiledFunctionCode> compile_function_code(
         throw runtime_error("unknown architecture");
     }
 
-    string asm_filename = string_printf("%s/%s.%s.inc.s", directory.c_str(), name.c_str(), arch_name_token);
+    // Look in the function directory first, then the system directory
+    string asm_filename = string_printf("%s/%s.%s.inc.s", function_directory.c_str(), name.c_str(), arch_name_token);
+    if (!isfile(asm_filename)) {
+      asm_filename = string_printf("%s/%s.%s.inc.s", system_directory.c_str(), name.c_str(), arch_name_token);
+    }
     if (isfile(asm_filename)) {
       if (!get_include_stack.emplace(name).second) {
         throw runtime_error("mutual recursion between includes: " + name);
@@ -165,14 +174,20 @@ shared_ptr<CompiledFunctionCode> compile_function_code(
           ret = X86Emulator::assemble(load_file(asm_filename), get_include);
           break;
         case CompiledFunctionCode::Architecture::SH4:
-          throw runtime_error("cannot compuile SH-4 assembly");
+          ret = SH4Emulator::assemble(load_file(asm_filename), get_include);
+          break;
         default:
           throw runtime_error("unknown architecture");
       }
       get_include_stack.erase(name);
       return ret.code;
     }
-    string bin_filename = directory + "/" + name + ".inc.bin";
+
+    string bin_filename = function_directory + "/" + name + ".inc.bin";
+    if (isfile(bin_filename)) {
+      return load_file(bin_filename);
+    }
+    bin_filename = system_directory + "/" + name + ".inc.bin";
     if (isfile(bin_filename)) {
       return load_file(bin_filename);
     }
@@ -184,6 +199,10 @@ shared_ptr<CompiledFunctionCode> compile_function_code(
     assembled = PPC32Emulator::assemble(text, get_include);
   } else if (arch == CompiledFunctionCode::Architecture::X86) {
     assembled = X86Emulator::assemble(text, get_include);
+  } else if (arch == CompiledFunctionCode::Architecture::SH4) {
+    assembled = SH4Emulator::assemble(text, get_include);
+  } else {
+    throw runtime_error("invalid architecture");
   }
   ret->code = std::move(assembled.code);
   ret->label_offsets = std::move(assembled.label_offsets);
@@ -237,83 +256,95 @@ FunctionCodeIndex::FunctionCodeIndex(const string& directory) {
     return;
   }
 
-  uint32_t next_menu_item_id = 0;
-  for (const auto& filename : list_directory_sorted(directory)) {
-    try {
-      if (!ends_with(filename, ".s")) {
-        continue;
-      }
+  string system_dir_path = ends_with(directory, "/") ? (directory + "System") : (directory + "/System");
 
-      string name = filename.substr(0, filename.size() - 2);
-      if (ends_with(name, ".inc")) {
-        continue;
-      }
+  uint32_t next_menu_item_id = 1;
+  for (const auto& subdir_name : list_directory_sorted(directory)) {
+    string subdir_path = ends_with(directory, "/") ? (directory + subdir_name) : (directory + "/" + subdir_name);
+    if (!isdir(subdir_path)) {
+      function_compiler_log.warning("Skipping %s (not a directory)", subdir_name.c_str());
+      continue;
+    }
 
-      bool is_patch = ends_with(name, ".patch");
-      if (is_patch) {
-        name.resize(name.size() - 6);
-      }
+    for (const auto& filename : list_directory_sorted(subdir_path)) {
+      try {
+        if (!ends_with(filename, ".s")) {
+          continue;
+        }
 
-      // Figure out the version or specific_version
-      CompiledFunctionCode::Architecture arch = CompiledFunctionCode::Architecture::UNKNOWN;
-      uint32_t specific_version = 0;
-      string short_name = name;
-      if (ends_with(name, ".ppc")) {
-        arch = CompiledFunctionCode::Architecture::POWERPC;
-        name.resize(name.size() - 4);
-        short_name = name;
-      } else if (ends_with(name, ".x86")) {
-        arch = CompiledFunctionCode::Architecture::X86;
-        name.resize(name.size() - 4);
-        short_name = name;
-      } else if (ends_with(name, ".sh4")) {
-        arch = CompiledFunctionCode::Architecture::SH4;
-        name.resize(name.size() - 4);
-        short_name = name;
-      } else if (is_patch && (name.size() >= 5) && (name[name.size() - 5] == '.')) {
-        specific_version = (name[name.size() - 4] << 24) | (name[name.size() - 3] << 16) | (name[name.size() - 2] << 8) | name[name.size() - 1];
-        if (specific_version_is_gc(specific_version)) {
+        string name = filename.substr(0, filename.size() - 2);
+        if (ends_with(name, ".inc")) {
+          continue;
+        }
+
+        bool is_patch = ends_with(name, ".patch");
+        if (is_patch) {
+          name.resize(name.size() - 6);
+        }
+
+        // Figure out the version or specific_version
+        CompiledFunctionCode::Architecture arch = CompiledFunctionCode::Architecture::UNKNOWN;
+        uint32_t specific_version = 0;
+        string short_name = name;
+        if (ends_with(name, ".ppc")) {
           arch = CompiledFunctionCode::Architecture::POWERPC;
-        } else if (specific_version_is_xb(specific_version) || specific_version_is_bb(specific_version)) {
+          name.resize(name.size() - 4);
+          short_name = name;
+        } else if (ends_with(name, ".x86")) {
           arch = CompiledFunctionCode::Architecture::X86;
-        } else {
-          throw runtime_error("unable to determine architecture from specific_version");
+          name.resize(name.size() - 4);
+          short_name = name;
+        } else if (ends_with(name, ".sh4")) {
+          arch = CompiledFunctionCode::Architecture::SH4;
+          name.resize(name.size() - 4);
+          short_name = name;
+        } else if (is_patch && (name.size() >= 5) && (name[name.size() - 5] == '.')) {
+          specific_version = (name[name.size() - 4] << 24) | (name[name.size() - 3] << 16) | (name[name.size() - 2] << 8) | name[name.size() - 1];
+          if (specific_version_is_dc(specific_version)) {
+            arch = CompiledFunctionCode::Architecture::SH4;
+          } else if (specific_version_is_gc(specific_version)) {
+            arch = CompiledFunctionCode::Architecture::POWERPC;
+          } else if (specific_version_is_xb(specific_version) || specific_version_is_bb(specific_version)) {
+            arch = CompiledFunctionCode::Architecture::X86;
+          } else {
+            throw runtime_error("unable to determine architecture from specific_version");
+          }
+          short_name = name.substr(0, name.size() - 5);
         }
-        short_name = name.substr(0, name.size() - 5);
-      }
 
-      if (arch == CompiledFunctionCode::Architecture::UNKNOWN) {
-        throw runtime_error("unable to determine architecture");
-      }
-
-      string path = directory + "/" + filename;
-      string text = load_file(path);
-      auto code = compile_function_code(arch, directory, name, text);
-      if (code->index != 0) {
-        if (!this->index_to_function.emplace(code->index, code).second) {
-          throw runtime_error(string_printf(
-              "duplicate function index: %08" PRIX32, code->index));
+        if (arch == CompiledFunctionCode::Architecture::UNKNOWN) {
+          throw runtime_error("unable to determine architecture");
         }
-      }
-      code->specific_version = specific_version;
-      code->source_path = path;
-      code->short_name = short_name;
-      this->name_to_function.emplace(name, code);
-      if (is_patch) {
-        code->menu_item_id = next_menu_item_id++;
-        this->menu_item_id_and_specific_version_to_patch_function.emplace(
-            static_cast<uint64_t>(code->menu_item_id) << 32 | specific_version, code);
-        this->name_and_specific_version_to_patch_function.emplace(
-            string_printf("%s-%08" PRIX32, short_name.c_str(), specific_version), code);
-      }
 
-      string index_prefix = code->index ? string_printf("%02X => ", code->index) : "";
-      string patch_prefix = is_patch ? string_printf("[%08" PRIX32 "/%08" PRIX32 "] ", code->menu_item_id, code->specific_version) : "";
-      function_compiler_log.info("Compiled function %s%s%s (%s)",
-          index_prefix.c_str(), patch_prefix.c_str(), name.c_str(), name_for_architecture(code->arch));
+        string path = subdir_path + "/" + filename;
+        string text = load_file(path);
+        auto code = compile_function_code(arch, subdir_path, system_dir_path, name, text);
+        if (code->index != 0) {
+          if (!this->index_to_function.emplace(code->index, code).second) {
+            throw runtime_error(string_printf(
+                "duplicate function index: %08" PRIX32, code->index));
+          }
+        }
+        code->specific_version = specific_version;
+        code->source_path = path;
+        code->short_name = short_name;
+        this->name_to_function.emplace(name, code);
+        if (is_patch) {
+          code->menu_item_id = next_menu_item_id++;
+          this->menu_item_id_and_specific_version_to_patch_function.emplace(
+              static_cast<uint64_t>(code->menu_item_id) << 32 | specific_version, code);
+          this->name_and_specific_version_to_patch_function.emplace(
+              string_printf("%s-%08" PRIX32, short_name.c_str(), specific_version), code);
+        }
 
-    } catch (const exception& e) {
-      function_compiler_log.warning("Failed to compile function %s: %s", filename.c_str(), e.what());
+        string index_prefix = code->index ? string_printf("%02X => ", code->index) : "";
+        string patch_prefix = is_patch ? string_printf("[%08" PRIX32 "/%08" PRIX32 "] ", code->menu_item_id, code->specific_version) : "";
+        function_compiler_log.info("Compiled function %s%s%s (%s)",
+            index_prefix.c_str(), patch_prefix.c_str(), name.c_str(), name_for_architecture(code->arch));
+
+      } catch (const exception& e) {
+        function_compiler_log.warning("Failed to compile function %s: %s", filename.c_str(), e.what());
+      }
     }
   }
 }
@@ -325,13 +356,33 @@ shared_ptr<const Menu> FunctionCodeIndex::patch_menu(uint32_t specific_version)
   ret->items.emplace_back(PatchesMenuItemID::GO_BACK, "Go back", "Return to the\nmain menu", 0);
   for (const auto& it : this->name_and_specific_version_to_patch_function) {
     const auto& fn = it.second;
-    if (!fn->hide_from_patches_menu && ends_with(it.first, suffix)) {
-      ret->items.emplace_back(
-          fn->menu_item_id,
-          fn->long_name.empty() ? fn->short_name : fn->long_name,
-          fn->description,
-          MenuItem::Flag::REQUIRES_SEND_FUNCTION_CALL);
+    if (fn->hide_from_patches_menu || !ends_with(it.first, suffix)) {
+      continue;
     }
+    ret->items.emplace_back(
+        fn->menu_item_id,
+        fn->long_name.empty() ? fn->short_name : fn->long_name,
+        fn->description,
+        MenuItem::Flag::REQUIRES_SEND_FUNCTION_CALL);
+  }
+  return ret;
+}
+
+shared_ptr<const Menu> FunctionCodeIndex::patch_switches_menu(
+    uint32_t specific_version, const std::unordered_set<std::string>& auto_patches_enabled) const {
+  auto suffix = string_printf("-%08" PRIX32, specific_version);
+
+  auto ret = make_shared<Menu>(MenuID::PATCH_SWITCHES, "Patch switches");
+  ret->items.emplace_back(PatchesMenuItemID::GO_BACK, "Go back", "Return to the\nmain menu", 0);
+  for (const auto& it : this->name_and_specific_version_to_patch_function) {
+    const auto& fn = it.second;
+    if (fn->hide_from_patches_menu || !ends_with(it.first, suffix)) {
+      continue;
+    }
+    string name;
+    name.push_back(auto_patches_enabled.count(fn->short_name) ? '*' : '-');
+    name += fn->long_name.empty() ? fn->short_name : fn->long_name;
+    ret->items.emplace_back(fn->menu_item_id, name, fn->description, MenuItem::Flag::REQUIRES_SEND_FUNCTION_CALL);
   }
   return ret;
 }
@@ -345,6 +396,12 @@ bool FunctionCodeIndex::patch_menu_empty(uint32_t specific_version) const {
   return true;
 }
 
+std::shared_ptr<const CompiledFunctionCode> FunctionCodeIndex::get_patch(
+    const std::string& name, uint32_t specific_version) const {
+  return this->name_and_specific_version_to_patch_function.at(
+      string_printf("%s-%08" PRIX32, name.c_str(), specific_version));
+}
+
 DOLFileIndex::DOLFileIndex(const string& directory) {
   if (!function_compiler_available()) {
     function_compiler_log.info("Function compiler is not available");
@@ -434,7 +491,7 @@ uint32_t specific_version_for_gc_header_checksum(uint32_t header_checksum) {
       char developer_code2 = 'P';
       uint8_t disc_number = 0;
       uint8_t version_code;
-    } __attribute__((packed)) data;
+    } __packed__ data;
     for (const char* game_code2 = "OS"; *game_code2; game_code2++) {
       data.game_code2 = *game_code2;
       for (const char* region_code = "JEP"; *region_code; region_code++) {

src/FunctionCompiler.hh

@@ -6,6 +6,7 @@
 #include <memory>
 #include <string>
 #include <unordered_map>
+#include <unordered_set>
 #include <vector>
 
 #include "Menu.hh"
@@ -69,7 +70,10 @@ struct FunctionCodeIndex {
   std::map<std::string, std::shared_ptr<CompiledFunctionCode>> name_and_specific_version_to_patch_function;
 
   std::shared_ptr<const Menu> patch_menu(uint32_t specific_version) const;
+  std::shared_ptr<const Menu> patch_switches_menu(uint32_t specific_version, const std::unordered_set<std::string>& auto_patches_enabled) const;
   bool patch_menu_empty(uint32_t specific_version) const;
+
+  std::shared_ptr<const CompiledFunctionCode> get_patch(const std::string& name, uint32_t specific_version) const;
 };
 
 struct DOLFileIndex {

src/GSLArchive.cc

@@ -9,21 +9,26 @@
 using namespace std;
 
 template <bool IsBigEndian>
-struct GSLHeaderEntry {
+struct GSLHeaderEntryT {
   using U32T = typename std::conditional<IsBigEndian, be_uint32_t, le_uint32_t>::type;
 
   pstring<TextEncoding::ASCII, 0x20> filename;
   U32T offset; // In pages, so actual offset is this * 0x800
   U32T size;
   uint64_t unused;
-} __attribute__((packed));
+} __packed__;
+
+using GSLHeaderEntry = GSLHeaderEntryT<false>;
+using GSLHeaderEntryBE = GSLHeaderEntryT<true>;
+check_struct_size(GSLHeaderEntry, 0x30);
+check_struct_size(GSLHeaderEntryBE, 0x30);
 
 template <bool IsBigEndian>
 void GSLArchive::load_t() {
   StringReader r(*this->data);
   uint64_t min_data_offset = 0xFFFFFFFFFFFFFFFF;
   while (r.where() < min_data_offset) {
-    const auto& entry = r.get<GSLHeaderEntry<IsBigEndian>>();
+    const auto& entry = r.get<GSLHeaderEntryT<IsBigEndian>>();
     if (entry.filename.empty()) {
       break;
     }
@@ -85,10 +90,10 @@ string GSLArchive::generate_t(const unordered_map<string, string>& files) {
 
   // Make sure there's enough space for a blank header entry before any file's
   // data pages begin
-  uint32_t data_start_offset = ((sizeof(GSLHeaderEntry<IsBigEndian>) * (files.size() + 1)) + 0x7FF) & (~0x7FF);
+  uint32_t data_start_offset = ((sizeof(GSLHeaderEntryT<IsBigEndian>) * (files.size() + 1)) + 0x7FF) & (~0x7FF);
   uint32_t data_offset = data_start_offset;
   for (const auto& file : files) {
-    GSLHeaderEntry<IsBigEndian> entry;
+    GSLHeaderEntryT<IsBigEndian> entry;
     entry.filename.encode(file.first);
     entry.offset = data_offset >> 11;
     entry.size = file.second.size();

src/GVMEncoder.cc

@@ -11,15 +11,18 @@ using namespace std;
 struct GVMFileEntry {
   be_uint16_t file_num;
   pstring<TextEncoding::ASCII, 0x1C> name;
-  parray<be_uint32_t, 2> unknown_a1;
-} __attribute__((packed));
+  uint8_t format_flags; // Same as in GVRHeader
+  GVRDataFormat data_format; // Same as in GVRHeader
+  be_uint16_t dimensions; // As powers of two in low nybbles (so e.g. 128x128 = 0x0055)
+  be_uint32_t global_index;
+} __packed_ws__(GVMFileEntry, 0x26);
 
 struct GVMFileHeader {
-  be_uint32_t magic; // 'GVMH'
+  be_uint32_t signature; // 'GVMH'
   le_uint32_t header_size;
-  be_uint16_t flags;
+  be_uint16_t flags; // Specifies which fields are present in GVMFileEntries; we always use 0xF (all fields present)
   be_uint16_t num_files;
-} __attribute__((packed));
+} __packed_ws__(GVMFileHeader, 0x0C);
 
 struct GVRHeader {
   be_uint32_t magic; // 'GVRT'
@@ -29,21 +32,26 @@ struct GVRHeader {
   GVRDataFormat data_format;
   be_uint16_t width;
   be_uint16_t height;
-} __attribute__((packed));
+} __packed_ws__(GVRHeader, 0x10);
 
-string encode_gvm(const Image& img, GVRDataFormat data_format) {
-  if (img.get_width() > 0xFFFF) {
-    throw runtime_error("image is too wide to be encoded as a GVR texture");
-  }
-  if (img.get_height() > 0xFFFF) {
-    throw runtime_error("image is too tall to be encoded as a GVR texture");
-  }
-  if (img.get_width() & 3) {
-    throw runtime_error("image width is not a multiple of 4");
-  }
-  if (img.get_height() & 3) {
-    throw runtime_error("image height is not a multiple of 4");
+string encode_gvm(const Image& img, GVRDataFormat data_format, const std::string& internal_name, uint32_t global_index) {
+  int8_t dimensions_field = -2;
+  {
+    size_t h = img.get_height();
+    size_t w = img.get_width();
+    if ((h != w) || (w & (w - 1)) || (h & (h - 1))) {
+      throw runtime_error("image must be square and dimensions must be powers of 2");
+    }
+    for (w >>= 1; w; w >>= 1, dimensions_field++) {
+    }
+    if (dimensions_field < 1) {
+      throw runtime_error("image is too small");
+    }
+    if (dimensions_field > 0xF) {
+      throw runtime_error("image is too large");
+    }
   }
+
   size_t pixel_count = img.get_width() * img.get_height();
   size_t pixel_bytes = 0;
   switch (data_format) {
@@ -59,11 +67,14 @@ string encode_gvm(const Image& img, GVRDataFormat data_format) {
   }
 
   StringWriter w;
-  w.put<GVMFileHeader>({.magic = 0x47564D48, .header_size = 0x48, .flags = 0x010F, .num_files = 1});
+  w.put<GVMFileHeader>({.signature = 0x47564D48, .header_size = 0x48, .flags = 0x000F, .num_files = 1});
   GVMFileEntry file_entry;
   file_entry.file_num = 0;
-  file_entry.name.encode("img", 1);
-  file_entry.unknown_a1.clear(0);
+  file_entry.name.encode(internal_name, 1);
+  file_entry.data_format = data_format;
+  file_entry.format_flags = 0;
+  file_entry.dimensions = (dimensions_field << 4) | dimensions_field;
+  file_entry.global_index = global_index;
   w.put(file_entry);
   w.extend_to(0x50, 0x00);
   w.put<GVRHeader>({.magic = 0x47565254,

src/GVMEncoder.hh

@@ -19,7 +19,7 @@ enum class GVRDataFormat : uint8_t {
   DXT1 = 0x0E,
 };
 
-std::string encode_gvm(const Image& img, GVRDataFormat data_format);
+std::string encode_gvm(const Image& img, GVRDataFormat data_format, const std::string& internal_name, uint32_t global_index);
 
 constexpr uint16_t encode_rgb565(uint8_t r, uint8_t g, uint8_t b) {
   return ((r << 8) & 0xF800) | ((g << 3) & 0x07E0) | ((b >> 3) & 0x001F);

src/HTTPServer.hh

@@ -0,0 +1,76 @@
+#pragma once
+
+#include <event2/buffer.h>
+#include <event2/event.h>
+#include <event2/http.h>
+#include <stdlib.h>
+
+#include <memory>
+#include <string>
+
+#include "ProxyServer.hh"
+#include "ServerState.hh"
+
+class HTTPServer {
+public:
+  HTTPServer(std::shared_ptr<ServerState> state);
+  HTTPServer(const HTTPServer&) = delete;
+  HTTPServer(HTTPServer&&) = delete;
+  HTTPServer& operator=(const HTTPServer&) = delete;
+  HTTPServer& operator=(HTTPServer&&) = delete;
+  virtual ~HTTPServer() = default;
+
+  void listen(const std::string& socket_path);
+  void listen(const std::string& addr, int port);
+  void listen(int port);
+  void add_socket(int fd);
+
+  void schedule_stop();
+  void wait_for_stop();
+
+protected:
+  class http_error : public std::runtime_error {
+  public:
+    http_error(int code, const std::string& what);
+    int code;
+  };
+
+  std::shared_ptr<ServerState> state;
+  std::shared_ptr<struct event_base> base;
+  std::shared_ptr<struct evhttp> http;
+  std::thread th;
+
+  void thread_fn();
+
+  static void dispatch_handle_request(struct evhttp_request* req, void* ctx);
+  void handle_request(struct evhttp_request* req);
+
+  static const std::unordered_map<int, const char*> explanation_for_response_code;
+  static void send_response(struct evhttp_request* req, int code, const char* content_type, struct evbuffer* b);
+  static void send_response(struct evhttp_request* req, int code, const char* content_type, const char* fmt, ...);
+
+  static std::unordered_multimap<std::string, std::string> parse_url_params(const std::string& query);
+  static std::unordered_map<std::string, std::string> parse_url_params_unique(const std::string& query);
+  static const std::string& get_url_param(
+      const std::unordered_multimap<std::string, std::string>& params,
+      const std::string& key,
+      const std::string* _default = nullptr);
+
+  static JSON generate_quest_json_st(std::shared_ptr<const Quest> q);
+  static JSON generate_client_config_json_st(const Client::Config& config);
+  static JSON generate_account_json_st(std::shared_ptr<const Account> a);
+  static JSON generate_game_client_json_st(std::shared_ptr<const Client> c, std::shared_ptr<const ItemNameIndex> item_name_index);
+  static JSON generate_proxy_client_json_st(std::shared_ptr<const ProxyServer::LinkedSession> ses);
+  static JSON generate_lobby_json_st(std::shared_ptr<const Lobby> l, std::shared_ptr<const ItemNameIndex> item_name_index);
+  JSON generate_game_server_clients_json() const;
+  JSON generate_proxy_server_clients_json() const;
+  JSON generate_server_info_json() const;
+  JSON generate_lobbies_json() const;
+  JSON generate_summary_json() const;
+  JSON generate_all_json() const;
+
+  JSON generate_ep3_cards_json(bool trial) const;
+  JSON generate_common_tables_json() const;
+  JSON generate_rare_tables_json() const;
+  JSON generate_rare_table_json(const std::string& table_name) const;
+};

src/IPFrameInfo.hh

@@ -12,31 +12,31 @@ struct HDLCHeader {
   uint8_t address; // 0xFF usually
   uint8_t control; // 0x03 for PPP
   be_uint16_t protocol;
-} __attribute__((packed));
+} __packed_ws__(HDLCHeader, 5);
 
 struct LCPHeader {
   uint8_t command;
   uint8_t request_id;
   be_uint16_t size;
-} __attribute__((packed));
+} __packed_ws__(LCPHeader, 4);
 
 struct PAPHeader {
   uint8_t command;
   uint8_t request_id;
   be_uint16_t size;
-} __attribute__((packed));
+} __packed_ws__(PAPHeader, 4);
 
 struct IPCPHeader {
   uint8_t command;
   uint8_t request_id;
   be_uint16_t size;
-} __attribute__((packed));
+} __packed_ws__(IPCPHeader, 4);
 
 struct EthernetHeader {
   parray<uint8_t, 6> dest_mac;
   parray<uint8_t, 6> src_mac;
   be_uint16_t protocol;
-} __attribute__((packed));
+} __packed_ws__(EthernetHeader, 0x0E);
 
 struct ARPHeader {
   be_uint16_t hardware_type;
@@ -44,7 +44,7 @@ struct ARPHeader {
   uint8_t hwaddr_len;
   uint8_t paddr_len;
   be_uint16_t operation;
-} __attribute__((packed));
+} __packed_ws__(ARPHeader, 8);
 
 struct IPv4Header {
   uint8_t version_ihl;
@@ -57,14 +57,14 @@ struct IPv4Header {
   be_uint16_t checksum;
   be_uint32_t src_addr;
   be_uint32_t dest_addr;
-} __attribute__((packed));
+} __packed_ws__(IPv4Header, 0x14);
 
 struct UDPHeader {
   be_uint16_t src_port;
   be_uint16_t dest_port;
   be_uint16_t size;
   be_uint16_t checksum;
-} __attribute__((packed));
+} __packed_ws__(UDPHeader, 8);
 
 struct TCPHeader {
   enum Flag {
@@ -87,7 +87,7 @@ struct TCPHeader {
   be_uint16_t window;
   be_uint16_t checksum;
   be_uint16_t urgent_ptr;
-} __attribute__((packed));
+} __packed_ws__(TCPHeader, 0x14);
 
 struct DHCPHeader {
   uint8_t opcode = 0;
@@ -105,7 +105,7 @@ struct DHCPHeader {
   parray<uint8_t, 0xC0> unused_bootp_legacy;
   be_uint32_t magic = 0x63825363;
   // Options follow here, terminated with FF
-} __attribute__((packed));
+} __packed_ws__(DHCPHeader, 0xF0);
 
 struct FrameInfo {
   enum class LinkType {

src/IPStackSimulator.cc

@@ -124,6 +124,7 @@ IPStackSimulator::IPStackSimulator(
     shared_ptr<ServerState> state)
     : base(base),
       state(state),
+      next_network_id(1),
       pcap_text_log_file(state->ip_stack_debug ? fopen("IPStackSimulator-Log.txt", "wt") : nullptr) {
   this->host_mac_address_bytes.clear(0x90);
   this->broadcast_mac_address_bytes.clear(0xFF);
@@ -135,28 +136,28 @@ IPStackSimulator::~IPStackSimulator() {
   }
 }
 
-void IPStackSimulator::listen(const string& name, const string& socket_path, FrameInfo::LinkType link_type) {
+void IPStackSimulator::listen(const string& name, const string& socket_path, Protocol proto) {
   int fd = ::listen(socket_path, 0, SOMAXCONN);
   ip_stack_simulator_log.info("Listening on Unix socket %s on fd %d as %s", socket_path.c_str(), fd, name.c_str());
-  this->add_socket(name, fd, link_type);
+  this->add_socket(name, fd, proto);
 }
 
-void IPStackSimulator::listen(const string& name, const string& addr, int port, FrameInfo::LinkType link_type) {
+void IPStackSimulator::listen(const string& name, const string& addr, int port, Protocol proto) {
   if (port == 0) {
-    this->listen(name, addr, link_type);
+    this->listen(name, addr, proto);
   } else {
     int fd = ::listen(addr, port, SOMAXCONN);
     string netloc_str = render_netloc(addr, port);
     ip_stack_simulator_log.info("Listening on TCP interface %s on fd %d as %s", netloc_str.c_str(), fd, name.c_str());
-    this->add_socket(name, fd, link_type);
+    this->add_socket(name, fd, proto);
   }
 }
 
-void IPStackSimulator::listen(const string& name, int port, FrameInfo::LinkType link_type) {
-  this->listen(name, "", port, link_type);
+void IPStackSimulator::listen(const string& name, int port, Protocol proto) {
+  this->listen(name, "", port, proto);
 }
 
-void IPStackSimulator::add_socket(const string& name, int fd, FrameInfo::LinkType link_type) {
+void IPStackSimulator::add_socket(const string& name, int fd, Protocol proto) {
   unique_listener l(
       evconnlistener_new(
           this->base.get(),
@@ -166,7 +167,11 @@ void IPStackSimulator::add_socket(const string& name, int fd, FrameInfo::LinkTyp
           0,
           fd),
       evconnlistener_free);
-  this->listening_sockets.emplace(piecewise_construct, forward_as_tuple(fd), forward_as_tuple(name, link_type, std::move(l)));
+  this->listening_sockets.emplace(piecewise_construct, forward_as_tuple(fd), forward_as_tuple(name, proto, std::move(l)));
+}
+
+shared_ptr<IPStackSimulator::IPClient> IPStackSimulator::get_network(uint64_t network_id) const {
+  return this->network_id_to_client.at(network_id);
 }
 
 uint32_t IPStackSimulator::connect_address_for_remote_address(uint32_t remote_addr) {
@@ -180,10 +185,12 @@ uint32_t IPStackSimulator::connect_address_for_remote_address(uint32_t remote_ad
   }
 }
 
-IPStackSimulator::IPClient::IPClient(shared_ptr<IPStackSimulator> sim, FrameInfo::LinkType link_type, struct bufferevent* bev)
+IPStackSimulator::IPClient::IPClient(
+    shared_ptr<IPStackSimulator> sim, uint64_t network_id, Protocol protocol, struct bufferevent* bev)
     : sim(sim),
+      network_id(network_id),
       bev(bev, bufferevent_free),
-      link_type(link_type),
+      protocol(protocol),
       mac_addr(0),
       ipv4_addr(0),
       idle_timeout_event(event_new(sim->base.get(), -1, EV_TIMEOUT, &IPStackSimulator::IPClient::dispatch_on_idle_timeout, this), event_free) {
@@ -200,7 +207,7 @@ void IPStackSimulator::IPClient::on_idle_timeout() {
   auto sim = this->sim.lock();
   if (sim) {
     ip_stack_simulator_log.info("Idle timeout expired on virtual network %d", bufferevent_getfd(this->bev.get()));
-    sim->disconnect_client(this->bev.get());
+    sim->disconnect_client(this->network_id);
   } else {
     ip_stack_simulator_log.info("Idle timeout expired on virtual network %d, but simulator is missing", bufferevent_getfd(this->bev.get()));
   }
@@ -227,40 +234,45 @@ IPStackSimulator::IPClient::TCPConnection::TCPConnection()
       bytes_received(0),
       bytes_sent(0) {}
 
-void IPStackSimulator::disconnect_client(struct bufferevent* bev) {
-  ip_stack_simulator_log.info("Virtual network %d disconnected", bufferevent_getfd(bev));
-  this->bev_to_client.erase(bev);
+void IPStackSimulator::disconnect_client(uint64_t network_id) {
+  ip_stack_simulator_log.info("Virtual network N-%" PRIu64 " disconnected", network_id);
+  this->network_id_to_client.erase(network_id);
 }
 
 void IPStackSimulator::dispatch_on_listen_accept(
     struct evconnlistener* listener, evutil_socket_t fd,
     struct sockaddr* address, int socklen, void* ctx) {
-  reinterpret_cast<IPStackSimulator*>(ctx)->on_listen_accept(
-      listener, fd, address, socklen);
+  reinterpret_cast<IPStackSimulator*>(ctx)->on_listen_accept(listener, fd, address, socklen);
 }
 
-void IPStackSimulator::on_listen_accept(struct evconnlistener* listener,
-    evutil_socket_t fd, struct sockaddr*, int) {
+void IPStackSimulator::on_listen_accept(struct evconnlistener* listener, evutil_socket_t fd, struct sockaddr*, int) {
+  struct sockaddr_storage remote_addr;
+  get_socket_addresses(fd, nullptr, &remote_addr);
+  if (this->state->banned_ipv4_ranges->check(remote_addr)) {
+    close(fd);
+    return;
+  }
+
   int listen_fd = evconnlistener_get_fd(listener);
 
   const ListeningSocket* listening_socket;
   try {
     listening_socket = &this->listening_sockets.at(listen_fd);
   } catch (const out_of_range&) {
-    ip_stack_simulator_log.info("Virtual network %d connected via unknown listener %d; disconnecting", fd, listen_fd);
+    ip_stack_simulator_log.info("Virtual network fd %d connected via unknown listener %d; disconnecting", fd, listen_fd);
     close(fd);
     return;
   }
 
-  ip_stack_simulator_log.info("Virtual network %d connected via %s", fd, listening_socket->name.c_str());
+  uint64_t network_id = this->next_network_id++;
+  ip_stack_simulator_log.info("Virtual network N-%" PRIu64 " connected via %s", network_id, listening_socket->name.c_str());
 
-  struct bufferevent* bev = bufferevent_socket_new(this->base.get(), fd,
-      BEV_OPT_CLOSE_ON_FREE | BEV_OPT_DEFER_CALLBACKS);
-  auto c = make_shared<IPClient>(this->shared_from_this(), listening_socket->link_type, bev);
-  this->bev_to_client.emplace(make_pair(bev, c));
+  struct bufferevent* bev = bufferevent_socket_new(this->base.get(), fd, BEV_OPT_CLOSE_ON_FREE | BEV_OPT_DEFER_CALLBACKS);
+  auto c = make_shared<IPClient>(this->shared_from_this(), network_id, listening_socket->protocol, bev);
+  this->network_id_to_client.emplace(c->network_id, c);
 
-  bufferevent_setcb(bev, &IPStackSimulator::dispatch_on_client_input, nullptr,
-      &IPStackSimulator::dispatch_on_client_error, this);
+  bufferevent_setcb(bev, &IPStackSimulator::IPClient::dispatch_on_client_input, nullptr,
+      &IPStackSimulator::IPClient::dispatch_on_client_error, c.get());
   bufferevent_enable(bev, EV_READ | EV_WRITE);
 }
 
@@ -276,63 +288,99 @@ void IPStackSimulator::on_listen_error(struct evconnlistener* listener) {
   event_base_loopexit(this->base.get(), nullptr);
 }
 
-void IPStackSimulator::dispatch_on_client_input(
-    struct bufferevent* bev, void* ctx) {
-  reinterpret_cast<IPStackSimulator*>(ctx)->on_client_input(bev);
+void IPStackSimulator::IPClient::dispatch_on_client_input(struct bufferevent* bev, void* ctx) {
+  reinterpret_cast<IPClient*>(ctx)->on_client_input(bev);
 }
 
-void IPStackSimulator::on_client_input(struct bufferevent* bev) {
+void IPStackSimulator::IPClient::on_client_input(struct bufferevent* bev) {
   struct evbuffer* buf = bufferevent_get_input(bev);
 
-  shared_ptr<IPClient> c;
-  try {
-    c = this->bev_to_client.at(bev);
-  } catch (const out_of_range&) {
+  auto sim = this->sim.lock();
+  if (!sim) {
     size_t bytes = evbuffer_get_length(buf);
-    ip_stack_simulator_log.warning("Ignoring data received from unregistered virtual network (0x%zX bytes)",
-        bytes);
+    ip_stack_simulator_log.warning("Ignoring data from unregistered virtual network (0x%zX bytes)", bytes);
     evbuffer_drain(buf, bytes);
     return;
   }
 
-  auto sim = c->sim.lock();
   uint64_t idle_timeout_usecs = sim ? sim->state->client_idle_timeout_usecs : 60000000;
   struct timeval tv = usecs_to_timeval(idle_timeout_usecs);
-  event_add(c->idle_timeout_event.get(), &tv);
+  event_add(this->idle_timeout_event.get(), &tv);
 
-  while (evbuffer_get_length(buf) >= 2) {
-    uint16_t frame_size;
-    evbuffer_copyout(buf, &frame_size, 2);
-    if (evbuffer_get_length(buf) < static_cast<size_t>(frame_size + 2)) {
-      break; // No complete frame available; done for now
-    }
+  switch (this->protocol) {
+    case Protocol::ETHERNET_TAPSERVER:
+    case Protocol::HDLC_TAPSERVER:
+      while (evbuffer_get_length(buf) >= 2) {
+        uint16_t frame_size;
+        evbuffer_copyout(buf, &frame_size, 2);
+        if (evbuffer_get_length(buf) < static_cast<size_t>(frame_size + 2)) {
+          break; // No complete frame available; done for now
+        }
 
-    evbuffer_drain(buf, 2);
-    string frame(frame_size, '\0');
-    evbuffer_remove(buf, frame.data(), frame.size());
+        evbuffer_drain(buf, 2);
+        string frame(frame_size, '\0');
+        evbuffer_remove(buf, frame.data(), frame.size());
 
-    try {
-      this->on_client_frame(c, frame);
-    } catch (const exception& e) {
-      if (ip_stack_simulator_log.warning("Failed to process frame: %s", e.what())) {
-        print_data(stderr, frame);
+        try {
+          sim->on_client_frame(this->shared_from_this(), frame);
+        } catch (const exception& e) {
+          if (ip_stack_simulator_log.warning("Failed to process frame: %s", e.what())) {
+            print_data(stderr, frame);
+          }
+        }
       }
-    }
+      break;
+    case Protocol::HDLC_RAW:
+      while (evbuffer_get_length(buf) >= 2) {
+        struct evbuffer_ptr res = evbuffer_search(buf, "\x7E", 1, nullptr);
+        if (res.pos < 0) {
+          break;
+        }
+        size_t start_offset = res.pos;
+
+        if (evbuffer_ptr_set(buf, &res, 1, EVBUFFER_PTR_ADD)) {
+          ip_stack_simulator_log.warning("Cannot advance search for end of frame");
+          break;
+        }
+
+        struct evbuffer_ptr end_res = evbuffer_search(buf, "\x7E", 1, &res);
+        if (end_res.pos < 0) {
+          break;
+        }
+        size_t frame_size = end_res.pos + 1 - start_offset;
+
+        if (start_offset) {
+          evbuffer_drain(buf, start_offset);
+        }
+
+        string frame(frame_size, '\0');
+        evbuffer_remove(buf, frame.data(), frame.size());
+
+        try {
+          sim->on_client_frame(this->shared_from_this(), frame);
+        } catch (const exception& e) {
+          if (ip_stack_simulator_log.warning("Failed to process frame: %s", e.what())) {
+            print_data(stderr, frame);
+          }
+        }
+      }
+      break;
   }
 }
 
-void IPStackSimulator::dispatch_on_client_error(
-    struct bufferevent* bev, short events, void* ctx) {
-  reinterpret_cast<IPStackSimulator*>(ctx)->on_client_error(bev, events);
+void IPStackSimulator::IPClient::dispatch_on_client_error(struct bufferevent* bev, short events, void* ctx) {
+  reinterpret_cast<IPClient*>(ctx)->on_client_error(bev, events);
 }
-void IPStackSimulator::on_client_error(struct bufferevent* bev, short events) {
+void IPStackSimulator::IPClient::on_client_error(struct bufferevent*, short events) {
   if (events & BEV_EVENT_ERROR) {
     int err = EVUTIL_SOCKET_ERROR();
-    ip_stack_simulator_log.warning("Virtual network caused error %d (%s)", err,
-        evutil_socket_error_to_string(err));
+    ip_stack_simulator_log.warning("Virtual network caused error %d (%s)", err, evutil_socket_error_to_string(err));
   }
   if (events & (BEV_EVENT_EOF | BEV_EVENT_ERROR)) {
-    this->disconnect_client(bev);
+    auto sim = this->sim.lock();
+    if (sim) {
+      sim->disconnect_client(this->network_id);
+    }
   }
 }
 
@@ -343,8 +391,8 @@ void IPStackSimulator::send_layer3_frame(shared_ptr<IPClient> c, FrameInfo::Prot
 void IPStackSimulator::send_layer3_frame(shared_ptr<IPClient> c, FrameInfo::Protocol proto, const void* data, size_t size) const {
   struct evbuffer* out_buf = bufferevent_get_output(c->bev.get());
 
-  switch (c->link_type) {
-    case FrameInfo::LinkType::ETHERNET: {
+  switch (c->protocol) {
+    case Protocol::ETHERNET_TAPSERVER: {
       EthernetHeader ether;
       ether.dest_mac = c->mac_addr;
       ether.src_mac = this->host_mac_address_bytes;
@@ -376,7 +424,8 @@ void IPStackSimulator::send_layer3_frame(shared_ptr<IPClient> c, FrameInfo::Prot
       break;
     }
 
-    case FrameInfo::LinkType::HDLC: {
+    case Protocol::HDLC_TAPSERVER:
+    case Protocol::HDLC_RAW: {
       HDLCHeader hdlc;
       hdlc.start_sentinel1 = 0x7E;
       hdlc.address = 0xFF;
@@ -413,8 +462,10 @@ void IPStackSimulator::send_layer3_frame(shared_ptr<IPClient> c, FrameInfo::Prot
         print_data(stderr, w.str());
       }
 
-      le_uint16_t frame_size = escaped.size();
-      evbuffer_add(out_buf, &frame_size, 2);
+      if (c->protocol == Protocol::HDLC_TAPSERVER) {
+        le_uint16_t frame_size = escaped.size();
+        evbuffer_add(out_buf, &frame_size, 2);
+      }
       evbuffer_add(out_buf, escaped.data(), escaped.size());
       if (this->pcap_text_log_file) {
         this->log_frame(escaped);
@@ -428,9 +479,13 @@ void IPStackSimulator::send_layer3_frame(shared_ptr<IPClient> c, FrameInfo::Prot
 }
 
 void IPStackSimulator::on_client_frame(shared_ptr<IPClient> c, const string& frame) {
+  FrameInfo::LinkType link_type = (c->protocol == Protocol::ETHERNET_TAPSERVER)
+      ? FrameInfo::LinkType::ETHERNET
+      : FrameInfo::LinkType::HDLC;
+
   const string* effective_data = &frame;
   string hdlc_unescaped_data;
-  if (c->link_type == FrameInfo::LinkType::HDLC) {
+  if (link_type == FrameInfo::LinkType::HDLC) {
     hdlc_unescaped_data = unescape_hdlc_frame(frame);
     effective_data = &hdlc_unescaped_data;
   }
@@ -439,7 +494,7 @@ void IPStackSimulator::on_client_frame(shared_ptr<IPClient> c, const string& fra
   }
   this->log_frame(*effective_data);
 
-  FrameInfo fi(c->link_type, *effective_data);
+  FrameInfo fi(link_type, *effective_data);
   if (ip_stack_simulator_log.should_log(LogLevel::DEBUG)) {
     string fi_header = fi.header_str();
     ip_stack_simulator_log.debug("Frame header: %s", fi_header.c_str());
@@ -1119,6 +1174,7 @@ void IPStackSimulator::on_client_tcp_frame(
       throw runtime_error("non-SYN frame does not correspond to any open TCP connection");
     }
     bool conn_valid = true;
+    bool acked_seq_changed = false;
 
     if (fi.tcp->flags & TCPHeader::Flag::ACK) {
       ip_stack_simulator_log.debug("Client sent ACK %08" PRIX32, fi.tcp->ack_num.load());
@@ -1142,6 +1198,7 @@ void IPStackSimulator::on_client_tcp_frame(
           conn->acked_server_seq += ack_delta;
           conn->resend_push_usecs = DEFAULT_RESEND_PUSH_USECS;
           conn->next_push_max_frame_size = conn->max_frame_size;
+          acked_seq_changed = true;
 
           ip_stack_simulator_log.debug("Removed %08" PRIX32 " bytes from pending buffer and advanced acked_server_seq to %08" PRIX32,
               ack_delta, conn->acked_server_seq);
@@ -1251,9 +1308,9 @@ void IPStackSimulator::on_client_tcp_frame(
           conn_str.c_str(), conn->acked_server_seq, conn->next_client_seq, conn->bytes_received);
     }
 
-    if (conn_valid) {
+    if (conn_valid && acked_seq_changed) {
       // Try to send some more data if the client is waiting on it
-      this->send_pending_push_frame(c, *conn);
+      this->send_pending_push_frame(c, *conn, true);
     }
   }
 }
@@ -1286,40 +1343,45 @@ void IPStackSimulator::open_server_connection(shared_ptr<IPClient> c, IPClient::
   string conn_str = this->str_for_tcp_connection(c, conn);
   if (port_config->behavior == ServerBehavior::PROXY_SERVER) {
     if (!this->state->proxy_server.get()) {
-      ip_stack_simulator_log.error("TCP connection %s is to non-running proxy server",
-          conn_str.c_str());
+      ip_stack_simulator_log.error("TCP connection %s is to non-running proxy server", conn_str.c_str());
       flush_and_free_bufferevent(bevs[1]);
     } else {
-      this->state->proxy_server->connect_client(bevs[1], conn.server_port);
-      ip_stack_simulator_log.info("Connected TCP connection %s to proxy server",
-          conn_str.c_str());
+      this->state->proxy_server->connect_virtual_client(bevs[1], c->network_id, conn.server_port);
+      ip_stack_simulator_log.info("Connected TCP connection %s to proxy server", conn_str.c_str());
     }
   } else if (this->state->game_server.get()) {
-    this->state->game_server->connect_client(bevs[1], c->ipv4_addr,
-        conn.client_port, conn.server_port, port_config->version,
-        port_config->behavior);
-    ip_stack_simulator_log.info("Connected TCP connection %s to game server",
-        conn_str.c_str());
+    this->state->game_server->connect_virtual_client(
+        bevs[1], c->network_id, c->ipv4_addr, conn.client_port,
+        conn.server_port, port_config->version, port_config->behavior);
+    ip_stack_simulator_log.info("Connected TCP connection %s to game server", conn_str.c_str());
   } else {
-    ip_stack_simulator_log.error("No server available for TCP connection %s",
-        conn_str.c_str());
+    ip_stack_simulator_log.error("No server available for TCP connection %s", conn_str.c_str());
     flush_and_free_bufferevent(bevs[1]);
   }
 }
 
-void IPStackSimulator::send_pending_push_frame(shared_ptr<IPClient> c, IPClient::TCPConnection& conn) {
+void IPStackSimulator::send_pending_push_frame(
+    shared_ptr<IPClient> c, IPClient::TCPConnection& conn, bool always_send) {
   size_t pending_bytes = evbuffer_get_length(conn.pending_data.get());
   if (!pending_bytes) {
+    event_del(conn.resend_push_event.get());
+    return;
+  }
+
+  // If we're waiting to receive an ACK from the client, don't send another PSH
+  // until we get the ACK (unless this is a resend of a previous PSH due to a
+  // timeout)
+  if (!always_send && event_pending(conn.resend_push_event.get(), EV_TIMEOUT, nullptr)) {
     return;
   }
 
   size_t bytes_to_send = min<size_t>(pending_bytes, conn.next_push_max_frame_size);
-  if ((c->link_type == FrameInfo::LinkType::HDLC) && (bytes_to_send > 200)) {
+  if (c->protocol == Protocol::HDLC_TAPSERVER) {
     // There is a bug in Dolphin's modem implementation (which I wrote, so it's
     // my fault) that causes commands to be dropped when too much data is sent
     // at once. To work around this, we only send up to 200 bytes in each push
     // frame.
-    bytes_to_send = 200;
+    bytes_to_send = min<size_t>(bytes_to_send, 200);
   }
 
   ip_stack_simulator_log.debug("Sending PSH frame with seq_num %08" PRIX32 ", 0x%zX/0x%zX data bytes",
@@ -1338,8 +1400,7 @@ void IPStackSimulator::send_pending_push_frame(shared_ptr<IPClient> c, IPClient:
   if (conn.resend_push_usecs > 5000000) {
     conn.resend_push_usecs = 5000000;
   }
-  conn.next_push_max_frame_size = max<size_t>(
-      0x100, conn.next_push_max_frame_size - 0x100);
+  conn.next_push_max_frame_size = max<size_t>(0x100, conn.next_push_max_frame_size - 0x100);
 }
 
 void IPStackSimulator::send_tcp_frame(
@@ -1400,15 +1461,11 @@ void IPStackSimulator::dispatch_on_resend_push(evutil_socket_t, short, void* ctx
     if (!sim) {
       ip_stack_simulator_log.warning("Resend push event triggered for client on deleted simulator; ignoring");
     } else {
-      sim->on_resend_push(c, *conn);
+      sim->send_pending_push_frame(c, *conn, true);
     }
   }
 }
 
-void IPStackSimulator::on_resend_push(shared_ptr<IPClient> c, IPClient::TCPConnection& conn) {
-  this->send_pending_push_frame(c, conn);
-}
-
 void IPStackSimulator::dispatch_on_server_input(struct bufferevent*, void* ctx) {
   auto* conn = reinterpret_cast<IPClient::TCPConnection*>(ctx);
   auto c = conn->client.lock();
@@ -1435,7 +1492,7 @@ void IPStackSimulator::on_server_input(shared_ptr<IPClient> c, IPClient::TCPConn
   event_add(c->idle_timeout_event.get(), &tv);
 
   evbuffer_add_buffer(conn.pending_data.get(), buf);
-  this->send_pending_push_frame(c, conn);
+  this->send_pending_push_frame(c, conn, false);
 }
 
 void IPStackSimulator::dispatch_on_server_error(

src/IPStackSimulator.hh

@@ -16,32 +16,23 @@
 
 class IPStackSimulator : public std::enable_shared_from_this<IPStackSimulator> {
 public:
-  IPStackSimulator(
-      std::shared_ptr<struct event_base> base,
-      std::shared_ptr<ServerState> state);
-  ~IPStackSimulator();
-
-  void listen(const std::string& name, const std::string& socket_path, FrameInfo::LinkType link_type);
-  void listen(const std::string& name, const std::string& addr, int port, FrameInfo::LinkType link_type);
-  void listen(const std::string& name, int port, FrameInfo::LinkType link_type);
-  void add_socket(const std::string& name, int fd, FrameInfo::LinkType link_type);
-
-  static uint32_t connect_address_for_remote_address(uint32_t remote_addr);
-
-private:
-  std::shared_ptr<struct event_base> base;
-  std::shared_ptr<ServerState> state;
+  enum class Protocol {
+    ETHERNET_TAPSERVER = 0,
+    HDLC_TAPSERVER,
+    HDLC_RAW,
+  };
 
   using unique_listener = std::unique_ptr<struct evconnlistener, void (*)(struct evconnlistener*)>;
   using unique_bufferevent = std::unique_ptr<struct bufferevent, void (*)(struct bufferevent*)>;
   using unique_evbuffer = std::unique_ptr<struct evbuffer, void (*)(struct evbuffer*)>;
   using unique_event = std::unique_ptr<struct event, void (*)(struct event*)>;
 
-  struct IPClient {
+  struct IPClient : std::enable_shared_from_this<IPClient> {
     std::weak_ptr<IPStackSimulator> sim;
+    uint64_t network_id;
 
     unique_bufferevent bev;
-    FrameInfo::LinkType link_type;
+    Protocol protocol;
     uint32_t hdlc_escape_control_character_flags = 0xFFFFFFFF;
     uint32_t hdlc_remote_magic_number = 0;
     parray<uint8_t, 6> mac_addr; // Only used for LinkType::ETHERNET
@@ -80,53 +71,73 @@ private:
 
     unique_event idle_timeout_event;
 
-    IPClient(std::shared_ptr<IPStackSimulator> sim, FrameInfo::LinkType link_type, struct bufferevent* bev);
+    IPClient(std::shared_ptr<IPStackSimulator> sim, uint64_t network_id, Protocol protocol, struct bufferevent* bev);
+
+    static void dispatch_on_client_input(struct bufferevent* bev, void* ctx);
+    void on_client_input(struct bufferevent* bev);
+    static void dispatch_on_client_error(struct bufferevent* bev, short events, void* ctx);
+    void on_client_error(struct bufferevent* bev, short events);
 
     static void dispatch_on_idle_timeout(evutil_socket_t fd, short events, void* ctx);
     void on_idle_timeout();
   };
 
+  IPStackSimulator(
+      std::shared_ptr<struct event_base> base,
+      std::shared_ptr<ServerState> state);
+  ~IPStackSimulator();
+
+  void listen(const std::string& name, const std::string& socket_path, Protocol protocol);
+  void listen(const std::string& name, const std::string& addr, int port, Protocol protocol);
+  void listen(const std::string& name, int port, Protocol protocol);
+  void add_socket(const std::string& name, int fd, Protocol protocol);
+
+  static uint32_t connect_address_for_remote_address(uint32_t remote_addr);
+
+  std::shared_ptr<IPClient> get_network(uint64_t network_id) const;
+  inline const std::unordered_map<uint64_t, std::shared_ptr<IPClient>>& all_networks() const {
+    return this->network_id_to_client;
+  }
+
+  void disconnect_client(uint64_t network_id);
+
+private:
+  std::shared_ptr<struct event_base> base;
+  std::shared_ptr<ServerState> state;
+  uint64_t next_network_id;
+
   struct ListeningSocket {
     std::string name;
-    FrameInfo::LinkType link_type;
+    Protocol protocol;
     unique_listener listener;
 
-    ListeningSocket(const std::string& name, FrameInfo::LinkType link_type, unique_listener&& l)
+    ListeningSocket(const std::string& name, Protocol protocol, unique_listener&& l)
         : name(name),
-          link_type(link_type),
+          protocol(protocol),
           listener(std::move(l)) {}
   };
 
   std::unordered_map<int, ListeningSocket> listening_sockets;
-  std::unordered_map<struct bufferevent*, std::shared_ptr<IPClient>> bev_to_client;
+  std::unordered_map<uint64_t, std::shared_ptr<IPClient>> network_id_to_client;
 
   parray<uint8_t, 6> host_mac_address_bytes;
   parray<uint8_t, 6> broadcast_mac_address_bytes;
 
   FILE* pcap_text_log_file;
 
-  void disconnect_client(struct bufferevent* bev);
-
   static uint64_t tcp_conn_key_for_connection(const IPClient::TCPConnection& conn);
   static uint64_t tcp_conn_key_for_client_frame(const IPv4Header& ipv4, const TCPHeader& tcp);
   static uint64_t tcp_conn_key_for_client_frame(const FrameInfo& fi);
 
   static std::string str_for_ipv4_netloc(uint32_t addr, uint16_t port);
-  static std::string str_for_tcp_connection(std::shared_ptr<const IPClient> c,
-      const IPClient::TCPConnection& conn);
+  static std::string str_for_tcp_connection(std::shared_ptr<const IPClient> c, const IPClient::TCPConnection& conn);
 
   static void dispatch_on_listen_accept(struct evconnlistener* listener,
       evutil_socket_t fd, struct sockaddr* address, int socklen, void* ctx);
-  void on_listen_accept(struct evconnlistener* listener, evutil_socket_t fd,
-      struct sockaddr* address, int socklen);
+  void on_listen_accept(struct evconnlistener* listener, evutil_socket_t fd, struct sockaddr* address, int socklen);
   static void dispatch_on_listen_error(struct evconnlistener* listener, void* ctx);
   void on_listen_error(struct evconnlistener* listener);
 
-  static void dispatch_on_client_input(struct bufferevent* bev, void* ctx);
-  void on_client_input(struct bufferevent* bev);
-  static void dispatch_on_client_error(struct bufferevent* bev, short events, void* ctx);
-  void on_client_error(struct bufferevent* bev, short events);
-
   void send_layer3_frame(std::shared_ptr<IPClient> c, FrameInfo::Protocol proto, const std::string& data) const;
   void send_layer3_frame(std::shared_ptr<IPClient> c, FrameInfo::Protocol proto, const void* data, size_t size) const;
 
@@ -138,15 +149,13 @@ private:
   void on_client_udp_frame(std::shared_ptr<IPClient> c, const FrameInfo& fi);
   void on_client_tcp_frame(std::shared_ptr<IPClient> c, const FrameInfo& fi);
 
-  static void dispatch_on_resend_push(evutil_socket_t fd, short events, void* ctx);
-  void on_resend_push(std::shared_ptr<IPClient> c, IPClient::TCPConnection& conn);
-
   static void dispatch_on_server_input(struct bufferevent* bev, void* ctx);
   void on_server_input(std::shared_ptr<IPClient> c, IPClient::TCPConnection& conn);
   static void dispatch_on_server_error(struct bufferevent* bev, short events, void* ctx);
   void on_server_error(std::shared_ptr<IPClient> c, IPClient::TCPConnection& conn, short events);
 
-  void send_pending_push_frame(std::shared_ptr<IPClient> c, IPClient::TCPConnection& conn);
+  static void dispatch_on_resend_push(evutil_socket_t, short, void* ctx);
+  void send_pending_push_frame(std::shared_ptr<IPClient> c, IPClient::TCPConnection& conn, bool always_send);
   void send_tcp_frame(
       std::shared_ptr<IPClient> c,
       IPClient::TCPConnection& conn,
@@ -154,8 +163,7 @@ private:
       struct evbuffer* src_buf = nullptr,
       size_t src_bytes = 0);
 
-  void open_server_connection(
-      std::shared_ptr<IPClient> c, IPClient::TCPConnection& conn);
+  void open_server_connection(std::shared_ptr<IPClient> c, IPClient::TCPConnection& conn);
 
   void log_frame(const std::string& data) const;
 };

src/IPV4RangeSet.cc

@@ -0,0 +1,73 @@
+#include "IPV4RangeSet.hh"
+
+#include <arpa/inet.h>
+
+using namespace std;
+
+IPV4RangeSet::IPV4RangeSet(const JSON& json) {
+  for (const auto& it : json.as_list()) {
+    // String should be of the form a.b.c.d or a.b.c.d/e
+    auto tokens = split(it->as_string(), '/');
+
+    size_t mask_bits;
+    if (tokens.size() == 1) {
+      mask_bits = 32;
+    } else if (tokens.size() == 2) {
+      mask_bits = stoul(tokens[1], nullptr, 10);
+      if (mask_bits > 32) {
+        throw runtime_error("invalid IPv4 address range");
+      }
+    } else {
+      throw runtime_error("invalid IPv4 address range");
+    }
+
+    auto addr_tokens = split(tokens[0], '.');
+    if (addr_tokens.size() != 4) {
+      throw runtime_error("invalid IPv4 address");
+    }
+    uint32_t addr = 0;
+    for (size_t z = 0; z < 4; z++) {
+      size_t end_pos = 0;
+      size_t new_byte = stoul(addr_tokens[z], &end_pos, 10);
+      if (end_pos != addr_tokens[z].size() || new_byte > 0xFF) {
+        throw runtime_error("invalid IPv4 address");
+      }
+      addr = (addr << 8) | new_byte;
+    }
+    addr &= (0xFFFFFFFF << (32 - mask_bits));
+
+    this->ranges.emplace(addr, mask_bits);
+  }
+}
+
+JSON IPV4RangeSet::json() const {
+  auto ret = JSON::list();
+  for (const auto& it : this->ranges) {
+    uint32_t addr = it.first;
+    uint8_t mask_bits = it.second;
+    ret.emplace_back(string_printf("%hhu.%hhu.%hhu.%hhu/%hhu",
+        static_cast<uint8_t>((addr >> 24) & 0xFF),
+        static_cast<uint8_t>((addr >> 16) & 0xFF),
+        static_cast<uint8_t>((addr >> 8) & 0xFF),
+        static_cast<uint8_t>(addr & 0xFF),
+        mask_bits));
+  }
+  return ret;
+}
+
+bool IPV4RangeSet::check(uint32_t addr) const {
+  auto it = this->ranges.upper_bound(addr);
+  if (it == this->ranges.begin()) {
+    return false; // addr is before any range
+  }
+  const auto& range = *(--it);
+  return (((range.first ^ addr) & (0xFFFFFFFF << (32 - range.second))) == 0);
+}
+
+bool IPV4RangeSet::check(const struct sockaddr_storage& ss) const {
+  if (ss.ss_family != AF_INET) {
+    return false;
+  }
+  const sockaddr_in* sin = reinterpret_cast<const sockaddr_in*>(&ss);
+  return this->check(ntohl(sin->sin_addr.s_addr));
+}

src/IPV4RangeSet.hh

@@ -0,0 +1,18 @@
+#pragma once
+
+#include <phosg/JSON.hh>
+#include <set>
+
+class IPV4RangeSet {
+public:
+  IPV4RangeSet() = default;
+  explicit IPV4RangeSet(const JSON& json);
+
+  JSON json() const;
+
+  bool check(uint32_t addr) const;
+  bool check(const struct sockaddr_storage& ss) const;
+
+protected:
+  std::map<uint32_t, uint8_t> ranges; // {addr: mask_bits}
+};

src/IntegralExpression.cc

@@ -1,4 +1,4 @@
-#include "QuestAvailabilityExpression.hh"
+#include "IntegralExpression.hh"
 
 #include <algorithm>
 #include <mutex>
@@ -21,16 +21,16 @@
 
 using namespace std;
 
-QuestAvailabilityExpression::QuestAvailabilityExpression(const string& text)
+IntegralExpression::IntegralExpression(const string& text)
     : root(this->parse_expr(text)) {}
 
-QuestAvailabilityExpression::BinaryOperatorNode::BinaryOperatorNode(
+IntegralExpression::BinaryOperatorNode::BinaryOperatorNode(
     Type type, unique_ptr<const Node>&& left, unique_ptr<const Node>&& right)
     : type(type),
       left(std::move(left)),
       right(std::move(right)) {}
 
-bool QuestAvailabilityExpression::BinaryOperatorNode::operator==(const Node& other) const {
+bool IntegralExpression::BinaryOperatorNode::operator==(const Node& other) const {
   try {
     const BinaryOperatorNode& other_bin = dynamic_cast<const BinaryOperatorNode&>(other);
     return other_bin.type == this->type && *other_bin.left == *this->left && *other_bin.right == *this->right;
@@ -39,7 +39,7 @@ bool QuestAvailabilityExpression::BinaryOperatorNode::operator==(const Node& oth
   }
 }
 
-int64_t QuestAvailabilityExpression::BinaryOperatorNode::evaluate(const Env& env) const {
+int64_t IntegralExpression::BinaryOperatorNode::evaluate(const Env& env) const {
   switch (this->type) {
     case Type::LOGICAL_OR:
       return this->left->evaluate(env) || this->right->evaluate(env);
@@ -82,7 +82,7 @@ int64_t QuestAvailabilityExpression::BinaryOperatorNode::evaluate(const Env& env
   }
 }
 
-string QuestAvailabilityExpression::BinaryOperatorNode::str() const {
+string IntegralExpression::BinaryOperatorNode::str() const {
   switch (this->type) {
     case Type::LOGICAL_OR:
       return "(" + this->left->str() + ") || (" + this->right->str() + ")";
@@ -125,11 +125,11 @@ string QuestAvailabilityExpression::BinaryOperatorNode::str() const {
   }
 }
 
-QuestAvailabilityExpression::UnaryOperatorNode::UnaryOperatorNode(Type type, unique_ptr<const Node>&& sub)
+IntegralExpression::UnaryOperatorNode::UnaryOperatorNode(Type type, unique_ptr<const Node>&& sub)
     : type(type),
       sub(std::move(sub)) {}
 
-bool QuestAvailabilityExpression::UnaryOperatorNode::operator==(const Node& other) const {
+bool IntegralExpression::UnaryOperatorNode::operator==(const Node& other) const {
   try {
     const UnaryOperatorNode& other_un = dynamic_cast<const UnaryOperatorNode&>(other);
     return other_un.type == this->type && *other_un.sub == *this->sub;
@@ -138,7 +138,7 @@ bool QuestAvailabilityExpression::UnaryOperatorNode::operator==(const Node& othe
   }
 }
 
-int64_t QuestAvailabilityExpression::UnaryOperatorNode::evaluate(const Env& env) const {
+int64_t IntegralExpression::UnaryOperatorNode::evaluate(const Env& env) const {
   switch (this->type) {
     case Type::LOGICAL_NOT:
       return !this->sub->evaluate(env);
@@ -151,7 +151,7 @@ int64_t QuestAvailabilityExpression::UnaryOperatorNode::evaluate(const Env& env)
   }
 }
 
-string QuestAvailabilityExpression::UnaryOperatorNode::str() const {
+string IntegralExpression::UnaryOperatorNode::str() const {
   switch (this->type) {
     case Type::LOGICAL_NOT:
       return "!(" + this->sub->str() + ")";
@@ -164,10 +164,10 @@ string QuestAvailabilityExpression::UnaryOperatorNode::str() const {
   }
 }
 
-QuestAvailabilityExpression::FlagLookupNode::FlagLookupNode(uint16_t flag_index)
+IntegralExpression::FlagLookupNode::FlagLookupNode(uint16_t flag_index)
     : flag_index(flag_index) {}
 
-bool QuestAvailabilityExpression::FlagLookupNode::operator==(const Node& other) const {
+bool IntegralExpression::FlagLookupNode::operator==(const Node& other) const {
   try {
     const FlagLookupNode& other_flag = dynamic_cast<const FlagLookupNode&>(other);
     return other_flag.flag_index == this->flag_index;
@@ -176,20 +176,23 @@ bool QuestAvailabilityExpression::FlagLookupNode::operator==(const Node& other)
   }
 }
 
-int64_t QuestAvailabilityExpression::FlagLookupNode::evaluate(const Env& env) const {
+int64_t IntegralExpression::FlagLookupNode::evaluate(const Env& env) const {
+  if (!env.flags) {
+    throw runtime_error("quest flags not available");
+  }
   return env.flags->get(this->flag_index) ? 1 : 0;
 }
 
-string QuestAvailabilityExpression::FlagLookupNode::str() const {
+string IntegralExpression::FlagLookupNode::str() const {
   return string_printf("F_%04hX", this->flag_index);
 }
 
-QuestAvailabilityExpression::ChallengeCompletionLookupNode::ChallengeCompletionLookupNode(
+IntegralExpression::ChallengeCompletionLookupNode::ChallengeCompletionLookupNode(
     Episode episode, uint8_t stage_index)
     : episode(episode),
       stage_index(stage_index) {}
 
-bool QuestAvailabilityExpression::ChallengeCompletionLookupNode::operator==(const Node& other) const {
+bool IntegralExpression::ChallengeCompletionLookupNode::operator==(const Node& other) const {
   try {
     const ChallengeCompletionLookupNode& other_cc = dynamic_cast<const ChallengeCompletionLookupNode&>(other);
     return other_cc.episode == this->episode && other_cc.stage_index == this->stage_index;
@@ -198,7 +201,10 @@ bool QuestAvailabilityExpression::ChallengeCompletionLookupNode::operator==(cons
   }
 }
 
-int64_t QuestAvailabilityExpression::ChallengeCompletionLookupNode::evaluate(const Env& env) const {
+int64_t IntegralExpression::ChallengeCompletionLookupNode::evaluate(const Env& env) const {
+  if (!env.challenge_records) {
+    throw runtime_error("challenge records not available");
+  }
   if (this->episode == Episode::EP1) {
     return env.challenge_records->times_ep1_online.at(this->stage_index).has_value();
   } else if (this->episode == Episode::EP2) {
@@ -207,14 +213,14 @@ int64_t QuestAvailabilityExpression::ChallengeCompletionLookupNode::evaluate(con
   return false;
 }
 
-string QuestAvailabilityExpression::ChallengeCompletionLookupNode::str() const {
+string IntegralExpression::ChallengeCompletionLookupNode::str() const {
   return string_printf("CC_%s_%hhu", abbreviation_for_episode(this->episode), static_cast<uint8_t>(this->stage_index + 1));
 }
 
-QuestAvailabilityExpression::TeamRewardLookupNode::TeamRewardLookupNode(const string& reward_name)
+IntegralExpression::TeamRewardLookupNode::TeamRewardLookupNode(const string& reward_name)
     : reward_name(reward_name) {}
 
-bool QuestAvailabilityExpression::TeamRewardLookupNode::operator==(const Node& other) const {
+bool IntegralExpression::TeamRewardLookupNode::operator==(const Node& other) const {
   try {
     const TeamRewardLookupNode& other_team_reward = dynamic_cast<const TeamRewardLookupNode&>(other);
     return other_team_reward.reward_name == this->reward_name;
@@ -223,46 +229,60 @@ bool QuestAvailabilityExpression::TeamRewardLookupNode::operator==(const Node& o
   }
 }
 
-int64_t QuestAvailabilityExpression::TeamRewardLookupNode::evaluate(const Env& env) const {
+int64_t IntegralExpression::TeamRewardLookupNode::evaluate(const Env& env) const {
   return (env.team && env.team->has_reward(this->reward_name)) ? 1 : 0;
 }
 
-string QuestAvailabilityExpression::TeamRewardLookupNode::str() const {
+string IntegralExpression::TeamRewardLookupNode::str() const {
   return "T_" + this->reward_name;
 }
 
-QuestAvailabilityExpression::NumPlayersLookupNode::NumPlayersLookupNode() {}
+IntegralExpression::NumPlayersLookupNode::NumPlayersLookupNode() {}
 
-bool QuestAvailabilityExpression::NumPlayersLookupNode::operator==(const Node& other) const {
+bool IntegralExpression::NumPlayersLookupNode::operator==(const Node& other) const {
   return dynamic_cast<const NumPlayersLookupNode*>(&other) != nullptr;
 }
 
-int64_t QuestAvailabilityExpression::NumPlayersLookupNode::evaluate(const Env& env) const {
+int64_t IntegralExpression::NumPlayersLookupNode::evaluate(const Env& env) const {
   return env.num_players;
 }
 
-string QuestAvailabilityExpression::NumPlayersLookupNode::str() const {
+string IntegralExpression::NumPlayersLookupNode::str() const {
   return "V_NumPlayers";
 }
 
-QuestAvailabilityExpression::EventLookupNode::EventLookupNode() {}
+IntegralExpression::EventLookupNode::EventLookupNode() {}
 
-bool QuestAvailabilityExpression::EventLookupNode::operator==(const Node& other) const {
+bool IntegralExpression::EventLookupNode::operator==(const Node& other) const {
   return dynamic_cast<const EventLookupNode*>(&other) != nullptr;
 }
 
-int64_t QuestAvailabilityExpression::EventLookupNode::evaluate(const Env& env) const {
-  return env.num_players;
+int64_t IntegralExpression::EventLookupNode::evaluate(const Env& env) const {
+  return env.event;
 }
 
-string QuestAvailabilityExpression::EventLookupNode::str() const {
+string IntegralExpression::EventLookupNode::str() const {
   return "V_Event";
 }
 
-QuestAvailabilityExpression::ConstantNode::ConstantNode(bool value)
+IntegralExpression::V1PresenceLookupNode::V1PresenceLookupNode() {}
+
+bool IntegralExpression::V1PresenceLookupNode::operator==(const Node& other) const {
+  return dynamic_cast<const V1PresenceLookupNode*>(&other) != nullptr;
+}
+
+int64_t IntegralExpression::V1PresenceLookupNode::evaluate(const Env& env) const {
+  return env.v1_present ? 1 : 0;
+}
+
+string IntegralExpression::V1PresenceLookupNode::str() const {
+  return "V_V1Present";
+}
+
+IntegralExpression::ConstantNode::ConstantNode(int64_t value)
     : value(value) {}
 
-bool QuestAvailabilityExpression::ConstantNode::operator==(const Node& other) const {
+bool IntegralExpression::ConstantNode::operator==(const Node& other) const {
   try {
     const ConstantNode& other_const = dynamic_cast<const ConstantNode&>(other);
     return other_const.value == this->value;
@@ -271,15 +291,15 @@ bool QuestAvailabilityExpression::ConstantNode::operator==(const Node& other) co
   }
 }
 
-int64_t QuestAvailabilityExpression::ConstantNode::evaluate(const Env&) const {
+int64_t IntegralExpression::ConstantNode::evaluate(const Env&) const {
   return this->value;
 }
 
-string QuestAvailabilityExpression::ConstantNode::str() const {
+string IntegralExpression::ConstantNode::str() const {
   return string_printf("%" PRId64, this->value);
 }
 
-unique_ptr<const QuestAvailabilityExpression::Node> QuestAvailabilityExpression::parse_expr(string_view text) {
+unique_ptr<const IntegralExpression::Node> IntegralExpression::parse_expr(string_view text) {
   // Strip off spaces and fully-enclosing parentheses
   for (;;) {
     size_t starting_size = text.size();
@@ -320,13 +340,13 @@ unique_ptr<const QuestAvailabilityExpression::Node> QuestAvailabilityExpression:
   // Check for unary operators
   if (text[0] == '!') {
     return make_unique<UnaryOperatorNode>(UnaryOperatorNode::Type::LOGICAL_NOT,
-        QuestAvailabilityExpression::parse_expr(text.substr(1)));
+        IntegralExpression::parse_expr(text.substr(1)));
   } else if (text[0] == '~') {
     return make_unique<UnaryOperatorNode>(UnaryOperatorNode::Type::BITWISE_NOT,
-        QuestAvailabilityExpression::parse_expr(text.substr(1)));
+        IntegralExpression::parse_expr(text.substr(1)));
   } else if (text[0] == '-') {
     return make_unique<UnaryOperatorNode>(UnaryOperatorNode::Type::NEGATIVE,
-        QuestAvailabilityExpression::parse_expr(text.substr(1)));
+        IntegralExpression::parse_expr(text.substr(1)));
   }
 
   // Check for binary operators at the root level
@@ -363,8 +383,8 @@ unique_ptr<const QuestAvailabilityExpression::Node> QuestAvailabilityExpression:
               ((z < oper.first.size()) || (text.compare(z - oper.first.size(), oper.first.size(), oper.first) != 0)) &&
               (text.compare(z, oper.first.size(), oper.first) == 0) &&
               (text.compare(z + oper.first.size(), oper.first.size(), oper.first) != 0)) {
-            auto left = QuestAvailabilityExpression::parse_expr(text.substr(0, z));
-            auto right = QuestAvailabilityExpression::parse_expr(text.substr(z + oper.first.size()));
+            auto left = IntegralExpression::parse_expr(text.substr(0, z));
+            auto right = IntegralExpression::parse_expr(text.substr(z + oper.first.size()));
             return make_unique<BinaryOperatorNode>(oper.second, std::move(left), std::move(right));
           }
         }
@@ -412,6 +432,9 @@ unique_ptr<const QuestAvailabilityExpression::Node> QuestAvailabilityExpression:
   if (text == "V_Event") {
     return make_unique<EventLookupNode>();
   }
+  if (text == "V_V1Present") {
+    return make_unique<V1PresenceLookupNode>();
+  }
 
   // Check for constants
   if (text == "true") {

src/IntegralExpression.hh

@@ -13,22 +13,23 @@
 #include "StaticGameData.hh"
 #include "TeamIndex.hh"
 
-class QuestAvailabilityExpression {
+class IntegralExpression {
 public:
   struct Env {
     const QuestFlagsForDifficulty* flags;
-    const PlayerRecordsBB_Challenge* challenge_records;
+    const PlayerRecordsChallengeBB* challenge_records;
     std::shared_ptr<const TeamIndex::Team> team;
     size_t num_players;
     uint8_t event;
+    bool v1_present;
   };
 
-  QuestAvailabilityExpression(const std::string& text);
-  ~QuestAvailabilityExpression() = default;
-  inline bool operator==(const QuestAvailabilityExpression& other) const {
+  IntegralExpression(const std::string& text);
+  ~IntegralExpression() = default;
+  inline bool operator==(const IntegralExpression& other) const {
     return this->root->operator==(*other.root);
   }
-  inline bool operator!=(const QuestAvailabilityExpression& other) const {
+  inline bool operator!=(const IntegralExpression& other) const {
     return !this->operator==(other);
   }
   inline int64_t evaluate(const Env& env) const {
@@ -160,9 +161,18 @@ protected:
     virtual std::string str() const;
   };
 
+  class V1PresenceLookupNode : public Node {
+  public:
+    V1PresenceLookupNode();
+    virtual ~V1PresenceLookupNode() = default;
+    virtual bool operator==(const Node& other) const;
+    virtual int64_t evaluate(const Env& env) const;
+    virtual std::string str() const;
+  };
+
   class ConstantNode : public Node {
   public:
-    ConstantNode(bool value);
+    ConstantNode(int64_t value);
     virtual ~ConstantNode() = default;
     virtual bool operator==(const Node& other) const;
     virtual int64_t evaluate(const Env& env) const;

src/ItemCreator.cc

@@ -18,15 +18,16 @@ ItemCreator::ItemCreator(
     shared_ptr<const WeaponRandomSet> weapon_random_set,
     shared_ptr<const TekkerAdjustmentSet> tekker_adjustment_set,
     shared_ptr<const ItemParameterTable> item_parameter_table,
-    Version version,
+    std::shared_ptr<const ItemData::StackLimits> stack_limits,
     Episode episode,
     GameMode mode,
     uint8_t difficulty,
     uint8_t section_id,
-    uint32_t random_seed,
+    std::shared_ptr<PSOLFGEncryption> opt_rand_crypt,
     shared_ptr<const BattleRules> restrictions)
-    : log(string_printf("[ItemCreator:%s/%s/%s/%c/%hhu] ", name_for_enum(version), abbreviation_for_episode(episode), abbreviation_for_mode(mode), abbreviation_for_difficulty(difficulty), section_id), lobby_log.min_level),
-      version(version),
+    : log(string_printf("[ItemCreator:%s/%s/%s/%c/%hhu] ", name_for_enum(stack_limits->version), abbreviation_for_episode(episode), abbreviation_for_mode(mode), abbreviation_for_difficulty(difficulty), section_id), lobby_log.min_level),
+      logic_version(stack_limits->version),
+      stack_limits(stack_limits),
       episode(episode),
       mode(mode),
       difficulty(difficulty),
@@ -37,18 +38,27 @@ ItemCreator::ItemCreator(
       weapon_random_set(weapon_random_set),
       tekker_adjustment_set(tekker_adjustment_set),
       item_parameter_table(item_parameter_table),
+      common_item_set(common_item_set),
       pt(common_item_set->get_table(this->episode, this->mode, this->difficulty, this->section_id)),
       restrictions(restrictions),
-      random_crypt(random_seed) {
+      opt_rand_crypt(opt_rand_crypt ? make_shared<PSOV2Encryption>(opt_rand_crypt->seed()) : nullptr) {
   this->generate_unit_stars_tables();
 }
 
-void ItemCreator::set_random_state(uint32_t seed, uint32_t absolute_offset) {
-  if ((this->random_crypt.seed() != seed) || (this->random_crypt.absolute_offset() > absolute_offset)) {
-    this->random_crypt = PSOV2Encryption(seed);
-  }
-  while (this->random_crypt.absolute_offset() < absolute_offset) {
-    this->random_crypt.next();
+void ItemCreator::set_random_crypt(shared_ptr<PSOLFGEncryption> new_random_crypt) {
+  this->opt_rand_crypt = new_random_crypt;
+}
+
+void ItemCreator::set_section_id(uint8_t new_section_id) {
+  if (this->section_id != new_section_id) {
+    this->section_id = new_section_id;
+    this->log.prefix = string_printf("[ItemCreator:%s/%s/%s/%c/%hhu] ",
+        name_for_enum(stack_limits->version),
+        abbreviation_for_episode(episode),
+        abbreviation_for_mode(mode),
+        abbreviation_for_difficulty(difficulty),
+        this->section_id);
+    this->pt = common_item_set->get_table(this->episode, this->mode, this->difficulty, this->section_id);
   }
 }
 
@@ -56,8 +66,9 @@ bool ItemCreator::are_rare_drops_allowed() const {
   // Note: The client has an additional check here, which appears to be a subtle
   // anti-cheating measure. There is a flag on the client, initially zero, which
   // is set to 1 when certain unexpected item-related things happen (for
-  // example, a player possessing a mag with a level above 200). When the flag
-  // is set, this function returns false, which prevents all rare item drops.
+  // example, a player possessing a mag with a level above 200, or a stack of
+  // consumables with an amount above the stack size limit). When the flag is
+  // set, this function returns false, which prevents all rare item drops.
   // newserv intentionally does not implement this flag.
   return (this->mode != GameMode::CHALLENGE);
 }
@@ -122,16 +133,29 @@ uint8_t ItemCreator::normalize_area_number(uint8_t area) const {
 }
 
 ItemCreator::DropResult ItemCreator::on_box_item_drop(uint8_t area) {
-  return this->on_box_item_drop_with_area_norm(this->normalize_area_number(area));
+  try {
+    return this->on_box_item_drop_with_area_norm(this->normalize_area_number(area));
+  } catch (const exception& e) {
+    this->log.error("Exception in item creation: %s", e.what());
+    return DropResult();
+  }
 }
 
 ItemCreator::DropResult ItemCreator::on_monster_item_drop(uint32_t enemy_type, uint8_t area) {
-  return this->on_monster_item_drop_with_area_norm(enemy_type, this->normalize_area_number(area));
+  try {
+    return this->on_monster_item_drop_with_area_norm(enemy_type, this->normalize_area_number(area));
+  } catch (const exception& e) {
+    this->log.error("Exception in item creation: %s", e.what());
+    return DropResult();
+  }
 }
 
 ItemCreator::DropResult ItemCreator::on_box_item_drop_with_area_norm(uint8_t area_norm) {
-  this->log.info("Box drop checks for area_norm %02hhX; random state: %08" PRIX32 " %08" PRIX32,
-      area_norm, this->random_crypt.seed(), this->random_crypt.absolute_offset());
+  this->log.info("Box drop checks for area_norm %02hhX", area_norm);
+  if (this->opt_rand_crypt) {
+    this->log.info("Random state: %08" PRIX32 " %08" PRIX32,
+        this->opt_rand_crypt->seed(), this->opt_rand_crypt->absolute_offset());
+  }
   DropResult res;
   res.item = this->check_rare_specs_and_create_rare_box_item(area_norm);
   if (!res.item.empty()) {
@@ -178,7 +202,11 @@ ItemCreator::DropResult ItemCreator::on_monster_item_drop_with_area_norm(uint32_
     this->log.warning("Invalid enemy type: %" PRIX32, enemy_type);
     return DropResult();
   }
-  this->log.info("Enemy type: %" PRIX32 "; random state: %08" PRIX32 " %08" PRIX32, enemy_type, this->random_crypt.seed(), this->random_crypt.absolute_offset());
+  this->log.info("Enemy type: %" PRIX32 "", enemy_type);
+  if (this->opt_rand_crypt) {
+    this->log.info("Random state: %08" PRIX32 " %08" PRIX32,
+        this->opt_rand_crypt->seed(), this->opt_rand_crypt->absolute_offset());
+  }
 
   uint8_t type_drop_prob = this->pt->enemy_type_drop_probs.at(enemy_type);
   uint8_t drop_sample = this->rand_int(100);
@@ -259,23 +287,27 @@ ItemData ItemCreator::check_rare_specs_and_create_rare_box_item(uint8_t area_nor
   for (const auto& spec : rare_specs) {
     item = this->check_rate_and_create_rare_item(spec, area_norm);
     if (!item.empty()) {
-      this->log.info("Box spec %08" PRIX32 " produced item %02hhX%02hhX%02hhX",
-          spec.probability, spec.item_code[0], spec.item_code[1], spec.item_code[2]);
+      if (this->log.should_log(LogLevel::INFO)) {
+        auto hex = spec.data.hex();
+        this->log.info("Box spec %08" PRIX32 " produced item %s", spec.probability, hex.c_str());
+      }
       break;
     }
-    this->log.info("Box spec %08" PRIX32 " did not produce item %02hhX%02hhX%02hhX",
-        spec.probability, spec.item_code[0], spec.item_code[1], spec.item_code[2]);
+    if (this->log.should_log(LogLevel::INFO)) {
+      auto hex = spec.data.hex();
+      this->log.info("Box spec %08" PRIX32 " did not produce item %s", spec.probability, hex.c_str());
+    }
   }
   return item;
 }
 
 uint32_t ItemCreator::rand_int(uint64_t max) {
-  return this->random_crypt.next() % max;
+  return random_from_optional_crypt(this->opt_rand_crypt) % max;
 }
 
 float ItemCreator::rand_float_0_1_from_crypt() {
   // This lacks some precision, but matches the original implementation.
-  return (static_cast<double>(this->random_crypt.next() >> 16) / 65536.0);
+  return (static_cast<double>(random_from_optional_crypt(this->opt_rand_crypt) >> 16) / 65536.0);
 }
 
 template <size_t NumRanges>
@@ -316,12 +348,16 @@ ItemData ItemCreator::check_rare_spec_and_create_rare_enemy_item(uint32_t enemy_
     for (const auto& spec : rare_specs) {
       item = this->check_rate_and_create_rare_item(spec, area_norm);
       if (!item.empty()) {
-        this->log.info("Enemy spec %08" PRIX32 " produced item %02hhX%02hhX%02hhX",
-            spec.probability, spec.item_code[0], spec.item_code[1], spec.item_code[2]);
+        if (this->log.should_log(LogLevel::INFO)) {
+          auto hex = spec.data.hex();
+          this->log.info("Enemy spec %08" PRIX32 " produced item %s", spec.probability, hex.c_str());
+        }
         break;
       }
-      this->log.info("Enemy spec %08" PRIX32 " did not produce item %02hhX%02hhX%02hhX",
-          spec.probability, spec.item_code[0], spec.item_code[1], spec.item_code[2]);
+      if (this->log.should_log(LogLevel::INFO)) {
+        auto hex = spec.data.hex();
+        this->log.info("Enemy spec %08" PRIX32 " did not produce item %s", spec.probability, hex.c_str());
+      }
     }
   }
   return item;
@@ -338,37 +374,36 @@ ItemData ItemCreator::check_rate_and_create_rare_item(const RareItemSet::Expande
     return ItemData();
   }
 
-  ItemData item;
-  item.data1[0] = drop.item_code[0];
-  item.data1[1] = drop.item_code[1];
-  item.data1[2] = drop.item_code[2];
-  switch (item.data1[0]) {
-    case 0:
-      if (this->pt->has_rare_bonus_value_prob_table) {
-        this->generate_rare_weapon_bonuses(item, this->rand_int(10));
-      } else {
-        this->generate_common_weapon_bonuses(item, area_norm);
-      }
-      this->set_item_unidentified_flag_if_not_challenge(item);
-      break;
-    case 1:
-      this->generate_common_armor_slots_and_bonuses(item);
-      break;
-    case 2:
-      this->generate_common_mag_variances(item);
-      break;
-    case 3:
-      this->clear_tool_item_if_invalid(item);
-      this->set_tool_item_amount_to_1(item);
-      break;
-    case 4:
-      break;
-    default:
-      throw logic_error("invalid item class");
+  ItemData item = drop.data;
+  if (item.can_be_encoded_in_rel_rare_table()) {
+    switch (item.data1[0]) {
+      case 0:
+        if (this->pt->has_rare_bonus_value_prob_table) {
+          this->generate_rare_weapon_bonuses(item, this->rand_int(10));
+        } else {
+          this->generate_common_weapon_bonuses(item, area_norm);
+        }
+        this->set_item_unidentified_flag_if_not_challenge(item);
+        break;
+      case 1:
+        this->generate_common_armor_slots_and_bonuses(item);
+        break;
+      case 2:
+        this->generate_common_mag_variances(item);
+        break;
+      case 3:
+        this->clear_tool_item_if_invalid(item);
+        this->set_tool_item_amount_to_1(item);
+        break;
+      case 4:
+        break;
+      default:
+        throw logic_error("invalid item class");
+    }
+    this->set_item_kill_count_if_unsealable(item);
   }
 
   this->clear_item_if_restricted(item);
-  this->set_item_kill_count_if_unsealable(item);
   return item;
 }
 
@@ -448,23 +483,21 @@ void ItemCreator::set_item_unidentified_flag_if_not_challenge(ItemData& item) co
   if (item.data1[0] != 0x00) {
     return;
   }
-  // On V3, all rare weapons and weapons with specials are untekked when
-  // created; on V2, only rares that are not in the standard item classes are
-  // untekked when created.
-  if (this->is_v3()) {
-    if (this->item_parameter_table->is_item_rare(item) || (item.data1[4] != 0)) {
-      item.data1[4] |= 0x80;
-    }
-  } else {
-    if (this->item_parameter_table->is_item_rare(item) ? (item.data1[1] > 0x0C) : (item.data1[4] != 0)) {
-      item.data1[4] |= 0x80;
-    }
+  // On V1, V3, and V4, all rare weapons and weapons with specials are untekked
+  // when created; on V2, only rares that are not in the standard item classes
+  // are untekked when created.
+  bool is_rare = this->item_parameter_table->is_item_rare(item);
+  bool use_v2_logic = is_v2(this->logic_version) && (this->logic_version != Version::GC_NTE);
+  if (use_v2_logic
+          ? (is_rare ? (item.data1[1] > 0x0C) : (item.data1[4] != 0))
+          : (is_rare || (item.data1[4] != 0))) {
+    item.data1[4] |= 0x80;
   }
 }
 
 void ItemCreator::set_tool_item_amount_to_1(ItemData& item) const {
   if (item.data1[0] == 0x03) {
-    item.set_tool_item_amount(this->version, 1);
+    item.set_tool_item_amount(*this->stack_limits, 1);
   }
 }
 
@@ -644,7 +677,7 @@ void ItemCreator::generate_common_tool_variances(uint32_t area_norm, ItemData& i
   item.clear();
 
   uint8_t tool_class = this->get_rand_from_weighted_tables_2d_vertical(this->pt->tool_class_prob_table, area_norm);
-  if (this->is_v3() && (tool_class == 0x1A)) {
+  if ((!is_v1_or_v2(this->logic_version) || (this->logic_version == Version::GC_NTE)) && (tool_class == 0x1A)) {
     tool_class = 0x73;
   }
   this->log.info("Generating tool with class %02hhX", tool_class);
@@ -692,12 +725,12 @@ void ItemCreator::generate_common_mag_variances(ItemData& item) {
 
     // The original code (on PSO GC) assigns the mag color as 0x0E. We assign
     // a random color instead.
-    if (is_pre_v1(this->version)) {
+    if (is_pre_v1(this->logic_version)) {
       item.data2[3] = 0x00;
-    } else if (is_v1_or_v2(this->version)) {
-      item.data2[3] = this->random_crypt.next() % 0x0E;
+    } else if (is_v1_or_v2(this->logic_version)) {
+      item.data2[3] = random_from_optional_crypt(this->opt_rand_crypt) % 0x0E;
     } else {
-      item.data2[3] = this->random_crypt.next() % 0x12;
+      item.data2[3] = random_from_optional_crypt(this->opt_rand_crypt) % 0x12;
     }
   }
 }
@@ -810,7 +843,7 @@ void ItemCreator::generate_unit_stars_tables() {
 
   size_t star_base_index;
   uint8_t num_units;
-  switch (this->version) {
+  switch (this->logic_version) {
     case Version::PC_PATCH:
     case Version::BB_PATCH:
     case Version::GC_EP3_NTE:
@@ -1007,8 +1040,7 @@ bool ItemCreator::shop_does_not_contain_duplicate_item_by_data1_0_1_2(
   return true;
 }
 
-void ItemCreator::generate_armor_shop_armors(
-    vector<ItemData>& shop, size_t player_level) {
+void ItemCreator::generate_armor_shop_armors(vector<ItemData>& shop, size_t player_level) {
   size_t num_items;
   if (player_level < 11) {
     num_items = 4;
@@ -1028,7 +1060,7 @@ void ItemCreator::generate_armor_shop_armors(
       pt.push(src_table.first[z].value);
     }
   }
-  pt.shuffle(this->random_crypt);
+  pt.shuffle(this->opt_rand_crypt);
 
   for (size_t items_generated = 0; items_generated < num_items;) {
     ItemData item;
@@ -1072,7 +1104,7 @@ void ItemCreator::generate_armor_shop_shields(vector<ItemData>& shop, size_t pla
       pt.push(src_table.first[z].value);
     }
   }
-  pt.shuffle(this->random_crypt);
+  pt.shuffle(this->opt_rand_crypt);
 
   for (size_t items_generated = 0; items_generated < num_items;) {
     ItemData item;
@@ -1115,7 +1147,7 @@ void ItemCreator::generate_armor_shop_units(vector<ItemData>& shop, size_t playe
       pt.push(src_table.first[z].value);
     }
   }
-  pt.shuffle(this->random_crypt);
+  pt.shuffle(this->opt_rand_crypt);
 
   for (size_t items_generated = 0; items_generated < num_items;) {
     ItemData item;
@@ -1218,7 +1250,7 @@ void ItemCreator::generate_rare_tool_shop_recovery_items(
       pt.push(e.value);
     }
   }
-  pt.shuffle(this->random_crypt);
+  pt.shuffle(this->opt_rand_crypt);
 
   size_t effective_num_items = num_items;
   size_t items_generated = 0;
@@ -1261,7 +1293,7 @@ void ItemCreator::generate_tool_shop_tech_disks(vector<ItemData>& shop, size_t p
       pt.push(e.value);
     }
   }
-  pt.shuffle(this->random_crypt);
+  pt.shuffle(this->opt_rand_crypt);
 
   static const array<uint8_t, 0x13> tech_num_map = {
       0x00, 0x03, 0x06, 0x0F, 0x10, 0x0D, 0x0A, 0x0B, 0x0C, 0x01, 0x04, 0x07,
@@ -1362,7 +1394,7 @@ vector<ItemData> ItemCreator::generate_weapon_shop_contents(size_t player_level)
       pt.push(e.value);
     }
   }
-  pt.shuffle(this->random_crypt);
+  pt.shuffle(this->opt_rand_crypt);
 
   vector<ItemData> shop;
   while (shop.size() < num_items) {
@@ -1562,7 +1594,7 @@ void ItemCreator::generate_weapon_shop_item_special(ItemData& item, size_t playe
 
   // Note: The original code shuffles pt and then pops a single value from it.
   // For simplicity, we just sample a single value (and don't pop it) instead.
-  switch (pt.sample(this->random_crypt)) {
+  switch (pt.sample(this->opt_rand_crypt)) {
     case 0:
       item.data1[4] = 0;
       break;
@@ -1614,7 +1646,7 @@ void ItemCreator::generate_weapon_shop_item_bonus1(
 
   // Note: The original code shuffles pt and then pops a single value from it.
   // For simplicity, we just sample a single value (and don't pop it) instead.
-  item.data1[6] = pt.sample(this->random_crypt);
+  item.data1[6] = pt.sample(this->opt_rand_crypt);
   if (item.data1[6] == 0) {
     item.data1[7] = 0;
 
@@ -1655,7 +1687,7 @@ void ItemCreator::generate_weapon_shop_item_bonus2(ItemData& item, size_t player
       pt.push(e.value);
     }
   }
-  pt.shuffle(this->random_crypt);
+  pt.shuffle(this->opt_rand_crypt);
 
   do {
     item.data1[8] = pt.pop();
@@ -1713,7 +1745,7 @@ ItemData ItemCreator::base_item_for_specialized_box(uint32_t def0, uint32_t def1
       if (item.data1[1] == 0x02) {
         item.data1[4] = def0 & 0xFF;
       }
-      item.set_tool_item_amount(this->version, 1);
+      item.set_tool_item_amount(*this->stack_limits, 1);
       break;
     case 0x04:
       item.data2d = ((def1 >> 0x10) & 0xFFFF) * 10;
@@ -1741,7 +1773,7 @@ ssize_t ItemCreator::apply_tekker_deltas(ItemData& item, uint8_t section_id) {
   // Adjust the weapon's special
   {
     const auto& prob_table = this->tekker_adjustment_set->get_special_upgrade_prob_table(section_id, favored);
-    uint8_t delta_index = prob_table.sample(this->random_crypt);
+    uint8_t delta_index = prob_table.sample(this->opt_rand_crypt);
     int8_t delta = delta_table.at(delta_index);
     this->log.info("(Special) Delta index %hhu, delta %hhd", delta_index, delta);
     // Note: The original code checks specifically for -1 and +1 here, but the
@@ -1777,7 +1809,7 @@ ssize_t ItemCreator::apply_tekker_deltas(ItemData& item, uint8_t section_id) {
   if (!this->item_parameter_table->is_item_rare(item)) {
     const auto& weapon_def = this->item_parameter_table->get_weapon(item.data1[1], item.data1[2]);
     const auto& prob_table = this->tekker_adjustment_set->get_grind_delta_prob_table(section_id, favored);
-    uint8_t delta_index = prob_table.sample(this->random_crypt);
+    uint8_t delta_index = prob_table.sample(this->opt_rand_crypt);
     int8_t delta = delta_table.at(delta_index);
     this->log.info("(Grind) Delta index %hhu, delta %hhd", delta_index, delta);
     int16_t new_grind = static_cast<int16_t>(item.data1[3]) + static_cast<int16_t>(delta);
@@ -1793,7 +1825,7 @@ ssize_t ItemCreator::apply_tekker_deltas(ItemData& item, uint8_t section_id) {
     const auto& prob_table = this->tekker_adjustment_set->get_bonus_delta_prob_table(section_id, favored);
     // Note: The original code really does use the same delta for all three
     // bonuses.
-    uint8_t delta_index = prob_table.sample(this->random_crypt);
+    uint8_t delta_index = prob_table.sample(this->opt_rand_crypt);
     int8_t delta = delta_table.at(delta_index);
     this->log.info("(Bonuses) Delta index %hhu, delta %hhd", delta_index, delta);
     // Note: The original code doesn't check if there's actually a bonus in each

src/ItemCreator.hh

@@ -19,16 +19,16 @@ public:
       std::shared_ptr<const WeaponRandomSet> weapon_random_set,
       std::shared_ptr<const TekkerAdjustmentSet> tekker_adjustment_set,
       std::shared_ptr<const ItemParameterTable> item_parameter_table,
-      Version version,
+      std::shared_ptr<const ItemData::StackLimits> stack_limits,
       Episode episode,
       GameMode mode,
       uint8_t difficulty,
       uint8_t section_id,
-      uint32_t random_seed,
+      std::shared_ptr<PSOLFGEncryption> opt_rand_crypt,
       std::shared_ptr<const BattleRules> restrictions = nullptr);
   ~ItemCreator() = default;
 
-  void set_random_state(uint32_t seed, uint32_t absolute_offset);
+  void set_random_crypt(std::shared_ptr<PSOLFGEncryption> new_random_crypt);
 
   struct DropResult {
     ItemData item;
@@ -52,10 +52,15 @@ public:
   inline void set_restrictions(std::shared_ptr<const BattleRules> restrictions) {
     this->restrictions = restrictions;
   }
+  inline uint8_t get_section_id() const {
+    return this->section_id;
+  }
+  void set_section_id(uint8_t new_section_id);
 
 private:
   PrefixedLogger log;
-  Version version;
+  Version logic_version;
+  std::shared_ptr<const ItemData::StackLimits> stack_limits;
   Episode episode;
   GameMode mode;
   uint8_t difficulty;
@@ -66,22 +71,19 @@ private:
   std::shared_ptr<const WeaponRandomSet> weapon_random_set;
   std::shared_ptr<const TekkerAdjustmentSet> tekker_adjustment_set;
   std::shared_ptr<const ItemParameterTable> item_parameter_table;
+  std::shared_ptr<const CommonItemSet> common_item_set;
   std::shared_ptr<const CommonItemSet::Table> pt;
   std::shared_ptr<const BattleRules> restrictions;
 
   struct UnitResult {
     uint8_t unit;
     int8_t modifier;
-  } __attribute__((packed));
+  } __packed_ws__(UnitResult, 2);
   std::array<std::vector<UnitResult>, 13> unit_results_by_star_count;
 
   // Note: The original implementation uses 17 different random states for some
   // reason. We forego that and use only one for simplicity.
-  PSOV2Encryption random_crypt;
-
-  inline bool is_v3() const {
-    return !is_v1_or_v2(this->version);
-  }
+  std::shared_ptr<PSOLFGEncryption> opt_rand_crypt;
 
   bool are_rare_drops_allowed() const;
   uint8_t normalize_area_number(uint8_t area) const;

src/ItemData.cc

@@ -8,6 +8,39 @@
 
 using namespace std;
 
+const vector<uint8_t> ItemData::StackLimits::DEFAULT_TOOL_LIMITS_DC_11_2000(
+    {10});
+const vector<uint8_t> ItemData::StackLimits::DEFAULT_TOOL_LIMITS_V1_V2(
+    {10, 10, 1, 10, 10, 10, 10, 10, 10, 1});
+const vector<uint8_t> ItemData::StackLimits::DEFAULT_TOOL_LIMITS_V3_V4(
+    {10, 10, 1, 10, 10, 10, 10, 10, 10, 1, 1, 1, 1, 1, 1, 1, 99, 1});
+
+ItemData::StackLimits::StackLimits(
+    Version version, const vector<uint8_t>& max_tool_stack_sizes_by_data1_1, uint32_t max_meseta_stack_size)
+    : version(version),
+      max_tool_stack_sizes_by_data1_1(max_tool_stack_sizes_by_data1_1),
+      max_meseta_stack_size(max_meseta_stack_size) {}
+
+ItemData::StackLimits::StackLimits(Version version, const JSON& json)
+    : version(version) {
+  this->max_tool_stack_sizes_by_data1_1.clear();
+  for (const auto& limit_json : json.at("ToolLimits").as_list()) {
+    this->max_tool_stack_sizes_by_data1_1.emplace_back(limit_json->as_int());
+  }
+  this->max_meseta_stack_size = json.at("MesetaLimit").as_int();
+}
+
+uint8_t ItemData::StackLimits::get(uint8_t data1_0, uint8_t data1_1) const {
+  if (data1_0 == 4) {
+    return this->max_meseta_stack_size;
+  }
+  if (data1_0 == 3) {
+    const auto& vec = this->max_tool_stack_sizes_by_data1_1;
+    return vec.at(min<size_t>(data1_1, vec.size() - 1));
+  }
+  return 1;
+}
+
 ItemData::ItemData() {
   this->clear();
 }
@@ -68,6 +101,14 @@ bool ItemData::empty() const {
 }
 
 uint32_t ItemData::primary_identifier() const {
+  // Primary identifiers are like:
+  // - 00TTSS00 = weapon (T = type, S = subtype; subtype is 0 for ES weapons)
+  // - 01TTSS00 = armor/shield/unit
+  // - 02TT0000 = mag
+  // - 0302ZZLL = tech disk (Z = tech number, L = level)
+  // - 03TTSS00 = tool
+  // - 04000000 = meseta
+
   // The game treats any item starting with 04 as Meseta, and ignores the rest
   // of data1 (the value is in data2)
   if (this->data1[0] == 0x04) {
@@ -85,7 +126,7 @@ uint32_t ItemData::primary_identifier() const {
   }
 }
 
-bool ItemData::is_wrapped(Version version) const {
+bool ItemData::is_wrapped(const StackLimits& limits) const {
   switch (this->data1[0]) {
     case 0:
     case 1:
@@ -93,7 +134,7 @@ bool ItemData::is_wrapped(Version version) const {
     case 2:
       return this->data2[2] & 0x40;
     case 3:
-      return !this->is_stackable(version) && (this->data1[3] & 0x40);
+      return !this->is_stackable(limits) && (this->data1[3] & 0x40);
     case 4:
       return false;
     default:
@@ -101,7 +142,7 @@ bool ItemData::is_wrapped(Version version) const {
   }
 }
 
-void ItemData::wrap(Version version) {
+void ItemData::wrap(const StackLimits& limits) {
   switch (this->data1[0]) {
     case 0:
     case 1:
@@ -111,7 +152,7 @@ void ItemData::wrap(Version version) {
       this->data2[2] |= 0x40;
       break;
     case 3:
-      if (!this->is_stackable(version)) {
+      if (!this->is_stackable(limits)) {
         this->data1[3] |= 0x40;
       }
       break;
@@ -122,7 +163,7 @@ void ItemData::wrap(Version version) {
   }
 }
 
-void ItemData::unwrap(Version version) {
+void ItemData::unwrap(const StackLimits& limits) {
   switch (this->data1[0]) {
     case 0:
     case 1:
@@ -132,7 +173,7 @@ void ItemData::unwrap(Version version) {
       this->data2[2] &= 0xBF;
       break;
     case 3:
-      if (!this->is_stackable(version)) {
+      if (!this->is_stackable(limits)) {
         this->data1[3] &= 0xBF;
       }
       break;
@@ -143,23 +184,23 @@ void ItemData::unwrap(Version version) {
   }
 }
 
-bool ItemData::is_stackable(Version version) const {
-  return this->max_stack_size(version) > 1;
+bool ItemData::is_stackable(const StackLimits& limits) const {
+  return this->max_stack_size(limits) > 1;
 }
 
-size_t ItemData::stack_size(Version version) const {
-  if (max_stack_size_for_item(version, this->data1[0], this->data1[1]) > 1) {
+size_t ItemData::stack_size(const StackLimits& limits) const {
+  if (this->max_stack_size(limits) > 1) {
     return this->data1[5];
   }
   return 1;
 }
 
-size_t ItemData::max_stack_size(Version version) const {
-  return max_stack_size_for_item(version, this->data1[0], this->data1[1]);
+size_t ItemData::max_stack_size(const StackLimits& limits) const {
+  return limits.get(this->data1[0], this->data1[1]);
 }
 
-void ItemData::enforce_min_stack_size(Version version) {
-  if (this->stack_size(version) == 0) {
+void ItemData::enforce_min_stack_size(const StackLimits& limits) {
+  if (this->stack_size(limits) == 0) {
     this->data1[5] = 1;
   }
 }
@@ -306,7 +347,7 @@ void ItemData::add_mag_photon_blast(uint8_t pb_num) {
 
 void ItemData::decode_for_version(Version from_version) {
   uint8_t encoded_v2_data = this->get_encoded_v2_data();
-  bool should_decode_v2_data = (is_v1(from_version) || is_v2(from_version)) &&
+  bool should_decode_v2_data = (is_v1(from_version) || is_v2(from_version)) && (from_version != Version::GC_NTE) &&
       (encoded_v2_data != 0x00) && this->has_encoded_v2_data();
 
   switch (this->data1[0]) {
@@ -330,13 +371,9 @@ void ItemData::decode_for_version(Version from_version) {
         this->data1[1] = encoded_v2_data + 0x2B;
       }
 
-      if (is_big_endian(from_version)) {
-        // PSO GC erroneously byteswaps the data2d field, even though it's actually
-        // just four individual bytes, so we correct for that here.
-        this->data2d = bswap32(this->data2d);
-
-      } else if (is_v1(from_version) || is_v2(from_version)) {
-        // PSO PC encodes mags in a tediously annoying manner. The first four bytes are the same, but then...
+      if (is_v1(from_version) || is_v2(from_version)) {
+        // PSO PC and GC NTE encode mags in a tediously annoying manner. The
+        // first four bytes are the same, but then...
         // V2: pHHHHHHHHHHHHHHc pIIIIIIIIIIIIIIc JJJJJJJJJJJJJJJc KKKKKKKKKKKKKKKc QQQQQQQQ QQQQQQQQ YYYYYYYY pYYYYYYY
         // V3: HHHHHHHHHHHHHHHH IIIIIIIIIIIIIIII JJJJJJJJJJJJJJJJ KKKKKKKKKKKKKKKK YYYYYYYY QQQQQQQQ PPPPPPPP CCCCCCCC
         // c = color in V2 (4 bits; low bit first)
@@ -352,10 +389,18 @@ void ItemData::decode_for_version(Version from_version) {
         this->data2[0] = this->data2w[1] & 0x7FFF; // Synchro
         this->data2[2] = ((this->data2[3] >> 7) & 1) | ((this->data1w[2] >> 14) & 2) | ((this->data1w[3] >> 13) & 4); // PB flags
         this->data2[3] = (this->data1w[2] & 1) | ((this->data1w[3] & 1) << 1) | ((this->data1w[4] & 1) << 2) | ((this->data1w[5] & 1) << 3); // Color
+        // 01000080
         this->data1w[2] &= 0x7FFE;
         this->data1w[3] &= 0x7FFE;
         this->data1w[4] &= 0xFFFE;
         this->data1w[5] &= 0xFFFE;
+
+      } else if (is_big_endian(from_version)) {
+        // PSO GC (but not GC NTE, which uses the above logic) byteswaps the
+        // data2d field, since internally it's actually a uint32_t. We treat it
+        // as individual bytes instead, so we correct for the client's
+        // byteswapping here.
+        this->data2d = bswap32(this->data2d);
       }
       break;
 
@@ -386,7 +431,7 @@ void ItemData::decode_for_version(Version from_version) {
 }
 
 void ItemData::encode_for_version(Version to_version, shared_ptr<const ItemParameterTable> item_parameter_table) {
-  bool should_encode_v2_data = (is_v1(to_version) || is_v2(to_version)) && !this->has_encoded_v2_data();
+  bool should_encode_v2_data = (is_v1(to_version) || is_v2(to_version)) && (to_version != Version::GC_NTE) && !this->has_encoded_v2_data();
 
   switch (this->data1[0]) {
     case 0x00:
@@ -425,9 +470,7 @@ void ItemData::encode_for_version(Version to_version, shared_ptr<const ItemParam
       // This logic is the inverse of the corresponding logic in
       // decode_for_version; see that function for a description of what's
       // going on here.
-      if (is_big_endian(to_version)) {
-        this->data2d = bswap32(this->data2d);
-      } else if (is_v1(to_version) || is_v2(to_version)) {
+      if (is_v1(to_version) || is_v2(to_version)) {
         this->data1w[2] = (this->data1w[2] & 0x7FFE) | ((this->data2[2] << 14) & 0x8000) | (this->data2[3] & 1);
         this->data1w[3] = (this->data1w[3] & 0x7FFE) | ((this->data2[2] << 13) & 0x8000) | ((this->data2[3] >> 1) & 1);
         this->data1w[4] = (this->data1w[4] & 0xFFFE) | ((this->data2[3] >> 2) & 1);
@@ -435,6 +478,8 @@ void ItemData::encode_for_version(Version to_version, shared_ptr<const ItemParam
         // Order is important; data2w[0] must not be written before data2[0] is read
         this->data2w[1] = this->data2[0] | ((this->data2[2] << 15) & 0x8000);
         this->data2w[0] = this->data2[1];
+      } else if (is_big_endian(to_version)) {
+        this->data2d = bswap32(this->data2d);
       }
       break;
 
@@ -502,12 +547,12 @@ void ItemData::set_sealed_item_kill_count(uint16_t v) {
   }
 }
 
-uint8_t ItemData::get_tool_item_amount(Version version) const {
-  return this->is_stackable(version) ? this->data1[5] : 1;
+uint8_t ItemData::get_tool_item_amount(const StackLimits& limits) const {
+  return this->is_stackable(limits) ? this->data1[5] : 1;
 }
 
-void ItemData::set_tool_item_amount(Version version, uint8_t amount) {
-  if (this->is_stackable(version)) {
+void ItemData::set_tool_item_amount(const StackLimits& limits, uint8_t amount) {
+  if (this->is_stackable(limits)) {
     this->data1[5] = amount;
   } else if (this->data1[0] == 0x03) {
     this->data1[5] = 0x00;
@@ -628,6 +673,10 @@ bool ItemData::can_be_equipped_in_slot(EquipSlot slot) const {
   }
 }
 
+bool ItemData::can_be_encoded_in_rel_rare_table() const {
+  return !(this->data1[3] || this->data1d[1] || this->data1d[2] || this->data2d);
+}
+
 bool ItemData::compare_for_sort(const ItemData& a, const ItemData& b) {
   for (size_t z = 0; z < 12; z++) {
     if (a.data1[z] < b.data1[z]) {
@@ -667,7 +716,7 @@ ItemData ItemData::from_data(const string& data) {
   return ret;
 }
 
-ItemData ItemData::from_primary_identifier(Version version, uint32_t primary_identifier) {
+ItemData ItemData::from_primary_identifier(const StackLimits& limits, uint32_t primary_identifier) {
   ItemData ret;
   if (primary_identifier > 0x04000000) {
     throw runtime_error("invalid item class");
@@ -680,15 +729,24 @@ ItemData ItemData::from_primary_identifier(Version version, uint32_t primary_ide
   } else {
     ret.data1[2] = (primary_identifier >> 8) & 0xFF;
   }
-  ret.set_tool_item_amount(version, 1);
+  ret.set_tool_item_amount(limits, 1);
   return ret;
 }
 
 string ItemData::hex() const {
-  return string_printf("%02hhX%02hhX%02hhX%02hhX %02hhX%02hhX%02hhX%02hhX %02hhX%02hhX%02hhX%02hhX (%08" PRIX32 ") %02hhX%02hhX%02hhX%02hhX",
-      this->data1[0], this->data1[1], this->data1[2], this->data1[3],
-      this->data1[4], this->data1[5], this->data1[6], this->data1[7],
-      this->data1[8], this->data1[9], this->data1[10], this->data1[11],
-      this->id.load(),
-      this->data2[0], this->data2[1], this->data2[2], this->data2[3]);
+  return string_printf("%08" PRIX32 " %08" PRIX32 " %08" PRIX32 " (%08" PRIX32 ") %08" PRIX32,
+      this->data1db[0].load(), this->data1db[1].load(), this->data1db[2].load(), this->id.load(), this->data2db.load());
+}
+
+string ItemData::short_hex() const {
+  auto ret = string_printf("%08" PRIX32 "%08" PRIX32 "%08" PRIX32 "%08" PRIX32,
+      this->data1db[0].load(), this->data1db[1].load(), this->data1db[2].load(), this->data2db.load());
+  size_t offset = ret.find_last_not_of('0');
+  if (offset != string::npos) {
+    offset += (offset & 1) ? 1 : 2;
+    if (offset < ret.size()) {
+      ret.resize(offset);
+    }
+  }
+  return ret;
 }

src/ItemData.hh

@@ -1,6 +1,7 @@
 #pragma once
 
 #include <phosg/Encoding.hh>
+#include <phosg/JSON.hh>
 #include <string>
 
 #include "Text.hh"
@@ -54,7 +55,26 @@ struct ItemMagStats {
   }
 };
 
-struct ItemData { // 0x14 bytes
+struct ItemData {
+  struct StackLimits {
+    Version version;
+    std::vector<uint8_t> max_tool_stack_sizes_by_data1_1;
+    uint32_t max_meseta_stack_size;
+
+    StackLimits(Version version, const std::vector<uint8_t>& max_tool_stack_sizes_by_data1_1, uint32_t max_meseta_stack_size);
+    StackLimits(Version version, const JSON& json);
+    StackLimits(const StackLimits& other) = default;
+    StackLimits(StackLimits&& other) = default;
+    StackLimits& operator=(const StackLimits& other) = default;
+    StackLimits& operator=(StackLimits&& other) = default;
+
+    uint8_t get(uint8_t data1_0, uint8_t data1_1) const;
+
+    static const std::vector<uint8_t> DEFAULT_TOOL_LIMITS_DC_11_2000;
+    static const std::vector<uint8_t> DEFAULT_TOOL_LIMITS_V1_V2;
+    static const std::vector<uint8_t> DEFAULT_TOOL_LIMITS_V3_V4;
+  };
+
   // QUICK ITEM FORMAT REFERENCE
   //           data1/0  data1/4  data1/8  data2
   //   Weapon: 00ZZZZGG SS00AABB AABBAABB 00000000
@@ -101,7 +121,7 @@ struct ItemData { // 0x14 bytes
     parray<be_uint16_t, 6> data1wb;
     parray<le_uint32_t, 3> data1d;
     parray<be_uint32_t, 3> data1db;
-  } __attribute__((packed));
+  } __packed__;
   le_uint32_t id;
   union {
     parray<uint8_t, 4> data2;
@@ -109,7 +129,7 @@ struct ItemData { // 0x14 bytes
     parray<be_uint16_t, 2> data2wb;
     le_uint32_t data2d;
     be_uint32_t data2db;
-  } __attribute__((packed));
+  } __packed__;
 
   ItemData();
   ItemData(const ItemData& other);
@@ -124,18 +144,19 @@ struct ItemData { // 0x14 bytes
   void clear();
 
   static ItemData from_data(const std::string& data);
-  static ItemData from_primary_identifier(Version version, uint32_t primary_identifier);
+  static ItemData from_primary_identifier(const StackLimits& limits, uint32_t primary_identifier);
   std::string hex() const;
+  std::string short_hex() const;
   uint32_t primary_identifier() const;
 
-  bool is_wrapped(Version version) const;
-  void wrap(Version version);
-  void unwrap(Version version);
+  bool is_wrapped(const StackLimits& limits) const;
+  void wrap(const StackLimits& limits);
+  void unwrap(const StackLimits& limits);
 
-  bool is_stackable(Version version) const;
-  size_t stack_size(Version version) const;
-  size_t max_stack_size(Version version) const;
-  void enforce_min_stack_size(Version version);
+  bool is_stackable(const StackLimits& limits) const;
+  size_t stack_size(const StackLimits& limits) const;
+  size_t max_stack_size(const StackLimits& limits) const;
+  void enforce_min_stack_size(const StackLimits& limits);
 
   static bool is_common_consumable(uint32_t primary_identifier);
   bool is_common_consumable() const;
@@ -154,8 +175,8 @@ struct ItemData { // 0x14 bytes
 
   uint16_t get_sealed_item_kill_count() const;
   void set_sealed_item_kill_count(uint16_t v);
-  uint8_t get_tool_item_amount(Version version) const;
-  void set_tool_item_amount(Version version, uint8_t amount);
+  uint8_t get_tool_item_amount(const StackLimits& limits) const;
+  void set_tool_item_amount(const StackLimits& limits, uint8_t amount);
   int16_t get_armor_or_shield_defense_bonus() const;
   void set_armor_or_shield_defense_bonus(int16_t bonus);
   int16_t get_common_armor_evasion_bonus() const;
@@ -169,7 +190,9 @@ struct ItemData { // 0x14 bytes
   EquipSlot default_equip_slot() const;
   bool can_be_equipped_in_slot(EquipSlot slot) const;
 
+  bool can_be_encoded_in_rel_rare_table() const;
+
   bool empty() const;
 
   static bool compare_for_sort(const ItemData& a, const ItemData& b);
-} __attribute__((packed));
+} __packed_ws__(ItemData, 0x14);

src/ItemNameIndex.cc

@@ -5,16 +5,16 @@
 using namespace std;
 
 ItemNameIndex::ItemNameIndex(
-    Version version,
     std::shared_ptr<const ItemParameterTable> item_parameter_table,
+    std::shared_ptr<const ItemData::StackLimits> limits,
     const std::vector<std::string>& name_coll)
-    : version(version),
-      item_parameter_table(item_parameter_table) {
+    : item_parameter_table(item_parameter_table),
+      limits(limits) {
 
   for (uint32_t primary_identifier : item_parameter_table->compute_all_valid_primary_identifiers()) {
     const string* name = nullptr;
     try {
-      ItemData item = ItemData::from_primary_identifier(this->version, primary_identifier);
+      ItemData item = ItemData::from_primary_identifier(*this->limits, primary_identifier);
       name = &name_coll.at(item_parameter_table->get_item_id(item));
     } catch (const out_of_range&) {
     }
@@ -97,9 +97,7 @@ const array<const char*, 0x11> name_for_s_rank_special = {
     "King\'s",
 };
 
-std::string ItemNameIndex::describe_item(
-    const ItemData& item,
-    bool include_color_escapes) const {
+std::string ItemNameIndex::describe_item(const ItemData& item, bool include_color_escapes) const {
   if (item.data1[0] == 0x04) {
     return string_printf("%s%" PRIu32 " Meseta", include_color_escapes ? "$C7" : "", item.data2d.load());
   }
@@ -107,8 +105,9 @@ std::string ItemNameIndex::describe_item(
   vector<string> ret_tokens;
 
   // For weapons, specials appear before the weapon name
+  bool is_unidentified = false;
   if ((item.data1[0] == 0x00) && (item.data1[4] != 0x00) && !item.is_s_rank_weapon()) {
-    bool is_unidentified = item.data1[4] & 0x80;
+    is_unidentified = item.data1[4] & 0x80;
     bool is_present = item.data1[4] & 0x40;
     uint8_t special_id = item.data1[4] & 0x3F;
     if (is_present) {
@@ -138,7 +137,7 @@ std::string ItemNameIndex::describe_item(
   // flags in a different location.
   if (((item.data1[1] == 0x01) && (item.data1[4] & 0x40)) ||
       ((item.data1[0] == 0x02) && (item.data2[2] & 0x40)) ||
-      ((item.data1[0] == 0x03) && !item.is_stackable(this->version) && (item.data1[3] & 0x40))) {
+      ((item.data1[0] == 0x03) && !item.is_stackable(*this->limits) && (item.data1[3] & 0x40))) {
     ret_tokens.emplace_back("Wrapped");
   }
 
@@ -167,7 +166,7 @@ std::string ItemNameIndex::describe_item(
     }
   }
 
-  // For weapons, add the grind and percentages, or S-rank name if applicable
+  // For weapons, add the grind and bonuses, or S-rank name if applicable
   if (item.data1[0] == 0x00) {
     if (item.data1[3] > 0) {
       ret_tokens.emplace_back(string_printf("+%hhu", item.data1[3]));
@@ -202,7 +201,7 @@ std::string ItemNameIndex::describe_item(
       }
 
     } else { // Not S-rank (extended name bits not set)
-      parray<int8_t, 5> percentages(0);
+      parray<int8_t, 5> bonuses(0);
       for (size_t x = 0; x < 3; x++) {
         uint8_t which = item.data1[6 + 2 * x];
         uint8_t value = item.data1[7 + 2 * x];
@@ -212,12 +211,20 @@ std::string ItemNameIndex::describe_item(
         if (which > 5) {
           ret_tokens.emplace_back(string_printf("!PC:%02hhX%02hhX", which, value));
         } else {
-          percentages[which - 1] = value;
+          bonuses[which - 1] = value;
         }
       }
-      if (!percentages.is_filled_with(0)) {
-        ret_tokens.emplace_back(string_printf("%hhd/%hhd/%hhd/%hhd/%hhd",
-            percentages[0], percentages[1], percentages[2], percentages[3], percentages[4]));
+      if (!bonuses.is_filled_with(0)) {
+        bool should_include_hit = (bonuses[4] != 0);
+        bool should_highlight_hit = include_color_escapes && (bonuses[4] > 0);
+        if (should_include_hit) {
+          ret_tokens.emplace_back(string_printf("%hhd/%hhd/%hhd/%hhd/%s%hhd",
+              bonuses[0], bonuses[1], bonuses[2], bonuses[3],
+              (should_highlight_hit ? "$CG" : ""), bonuses[4]));
+        } else {
+          ret_tokens.emplace_back(string_printf("%hhd/%hhd/%hhd/%hhd",
+              bonuses[0], bonuses[1], bonuses[2], bonuses[3]));
+        }
       }
     }
 
@@ -318,14 +325,16 @@ std::string ItemNameIndex::describe_item(
 
     // For tools, add the amount (if applicable)
   } else if (item.data1[0] == 0x03) {
-    if (item.max_stack_size(this->version) > 1) {
+    if (item.max_stack_size(*this->limits) > 1) {
       ret_tokens.emplace_back(string_printf("x%hhu", item.data1[5]));
     }
   }
 
   string ret = join(ret_tokens, " ");
   if (include_color_escapes) {
-    if (item.is_s_rank_weapon()) {
+    if (is_unidentified) {
+      return "$C3" + ret;
+    } else if (item.is_s_rank_weapon()) {
       return "$C4" + ret;
     } else if (this->item_parameter_table->is_item_rare(item)) {
       return "$C6" + ret;
@@ -360,7 +369,7 @@ ItemData ItemNameIndex::parse_item_description(const std::string& desc) const {
       }
     }
   }
-  ret.enforce_min_stack_size(this->version);
+  ret.enforce_min_stack_size(*this->limits);
   return ret;
 }
 
@@ -407,6 +416,16 @@ ItemData ItemNameIndex::parse_item_description_phase(const std::string& descript
   if (is_wrapped) {
     desc = desc.substr(8);
   }
+  bool is_unidentified = starts_with(desc, "?");
+  if (is_unidentified) {
+    size_t z;
+    for (z = 1; z < desc.size(); z++) {
+      if (desc[z] != ' ' && desc[z] != '?') {
+        break;
+      }
+    }
+    desc = desc.substr(z);
+  }
 
   // TODO: It'd be nice to be able to parse S-rank weapon specials here too.
   uint8_t weapon_special = 0;
@@ -458,7 +477,7 @@ ItemData ItemNameIndex::parse_item_description_phase(const std::string& descript
 
   if (ret.data1[0] == 0x00) {
     // Weapons: add special, grind and percentages (or name, if S-rank)
-    ret.data1[4] = weapon_special | (is_wrapped ? 0x40 : 0x00);
+    ret.data1[4] = weapon_special | (is_wrapped ? 0x40 : 0x00) | (is_unidentified ? 0x80 : 0x00);
 
     auto tokens = split(desc, ' ');
     for (auto& token : tokens) {
@@ -596,7 +615,7 @@ ItemData ItemNameIndex::parse_item_description_phase(const std::string& descript
       ret.data2[2] |= 0x40;
     }
   } else if (ret.data1[0] == 0x03) {
-    if (ret.max_stack_size(this->version) > 1) {
+    if (ret.max_stack_size(*this->limits) > 1) {
       if (starts_with(desc, "x")) {
         ret.data1[5] = stoul(desc.substr(1), nullptr, 10);
       } else {
@@ -607,7 +626,7 @@ ItemData ItemNameIndex::parse_item_description_phase(const std::string& descript
     }
 
     if (is_wrapped) {
-      if (ret.is_stackable(this->version)) {
+      if (ret.is_stackable(*this->limits)) {
         throw runtime_error("stackable items cannot be wrapped");
       } else {
         ret.data1[3] |= 0x40;
@@ -816,7 +835,7 @@ void ItemNameIndex::print_table(FILE* stream) const {
       item.data1[0] = 0x03;
       item.data1[1] = data1_1;
       item.data1[(data1_1 == 0x02) ? 4 : 2] = data1_2;
-      item.set_tool_item_amount(this->version, 1);
+      item.set_tool_item_amount(*this->limits, 1);
       string name = this->describe_item(item);
 
       fprintf(stream, "03%02zX%02zX => %08" PRIX32 " %04hX %04hX %6" PRIu32 " %5hu %04hX %6" PRId32 " %08" PRIX32 " %2hhu* %s %s\n",

src/ItemNameIndex.hh

@@ -20,8 +20,8 @@ public:
   };
 
   ItemNameIndex(
-      Version version,
       std::shared_ptr<const ItemParameterTable> pmt,
+      std::shared_ptr<const ItemData::StackLimits> limits,
       const std::vector<std::string>& name_coll);
 
   inline size_t entry_count() const {
@@ -43,8 +43,8 @@ public:
 private:
   ItemData parse_item_description_phase(const std::string& description, bool skip_special) const;
 
-  Version version;
   std::shared_ptr<const ItemParameterTable> item_parameter_table;
+  std::shared_ptr<const ItemData::StackLimits> limits;
 
   std::unordered_map<uint32_t, std::shared_ptr<const ItemMetadata>> primary_identifier_index;
   std::map<std::string, std::shared_ptr<const ItemMetadata>> name_index;

src/ItemParameterTable.cc

@@ -79,9 +79,9 @@ ItemParameterTable::ItemParameterTable(shared_ptr<const string> data, Version ve
     case Version::GC_V3:
     case Version::XB_V3: {
       if (is_big_endian(this->version)) {
-        this->offsets_v3_be = &this->r.pget<TableOffsetsV3V4<true>>(offset_table_offset);
+        this->offsets_v3_be = &this->r.pget<TableOffsetsV3V4BE>(offset_table_offset);
       } else {
-        this->offsets_v3_le = &this->r.pget<TableOffsetsV3V4<false>>(offset_table_offset);
+        this->offsets_v3_le = &this->r.pget<TableOffsetsV3V4>(offset_table_offset);
       }
       this->num_weapon_classes = 0xAA;
       this->num_tool_classes = 0x18;
@@ -93,7 +93,7 @@ ItemParameterTable::ItemParameterTable(shared_ptr<const string> data, Version ve
     }
 
     case Version::BB_V4: {
-      this->offsets_v4 = &this->r.pget<TableOffsetsV3V4<false>>(offset_table_offset);
+      this->offsets_v4 = &this->r.pget<TableOffsetsV3V4>(offset_table_offset);
       this->num_weapon_classes = 0xED;
       this->num_tool_classes = 0x1B;
       this->item_stars_first_id = 0xB1;
@@ -207,7 +207,7 @@ ItemParameterTable::WeaponV4 ItemParameterTable::WeaponGCNTE::to_v4() const {
 }
 
 template <bool IsBigEndian>
-ItemParameterTable::WeaponV4 ItemParameterTable::WeaponV3<IsBigEndian>::to_v4() const {
+ItemParameterTable::WeaponV4 ItemParameterTable::WeaponV3T<IsBigEndian>::to_v4() const {
   WeaponV4 ret;
   ret.base.id = this->base.id.load();
   ret.base.type = this->base.type.load();
@@ -282,7 +282,7 @@ ItemParameterTable::ArmorOrShieldV4 ItemParameterTable::ArmorOrShieldV1V2::to_v4
 }
 
 template <bool IsBigEndian>
-ItemParameterTable::ArmorOrShieldV4 ItemParameterTable::ArmorOrShieldV3<IsBigEndian>::to_v4() const {
+ItemParameterTable::ArmorOrShieldV4 ItemParameterTable::ArmorOrShieldV3T<IsBigEndian>::to_v4() const {
   ArmorOrShieldV4 ret;
   ret.base.id = this->base.id.load();
   ret.base.type = this->base.type.load();
@@ -324,7 +324,7 @@ ItemParameterTable::UnitV4 ItemParameterTable::UnitV1V2::to_v4() const {
 }
 
 template <bool IsBigEndian>
-ItemParameterTable::UnitV4 ItemParameterTable::UnitV3<IsBigEndian>::to_v4() const {
+ItemParameterTable::UnitV4 ItemParameterTable::UnitV3T<IsBigEndian>::to_v4() const {
   UnitV4 ret;
   ret.base.id = this->base.id.load();
   ret.base.type = this->base.type.load();
@@ -371,7 +371,7 @@ ItemParameterTable::MagV4 ItemParameterTable::MagV2::to_v4() const {
 }
 
 template <bool IsBigEndian>
-ItemParameterTable::MagV4 ItemParameterTable::MagV3<IsBigEndian>::to_v4() const {
+ItemParameterTable::MagV4 ItemParameterTable::MagV3T<IsBigEndian>::to_v4() const {
   MagV4 ret;
   ret.base.id = this->base.id.load();
   ret.base.type = this->base.type.load();
@@ -402,7 +402,7 @@ ItemParameterTable::ToolV4 ItemParameterTable::ToolV1V2::to_v4() const {
 }
 
 template <bool IsBigEndian>
-ItemParameterTable::ToolV4 ItemParameterTable::ToolV3<IsBigEndian>::to_v4() const {
+ItemParameterTable::ToolV4 ItemParameterTable::ToolV3T<IsBigEndian>::to_v4() const {
   ToolV4 ret;
   ret.base.id = this->base.id.load();
   ret.base.type = this->base.type.load();
@@ -416,13 +416,13 @@ ItemParameterTable::ToolV4 ItemParameterTable::ToolV3<IsBigEndian>::to_v4() cons
 
 template <bool IsBigEndian>
 size_t indirect_lookup_2d_count(const StringReader& r, size_t root_offset, size_t co_index) {
-  using ArrayRefT = typename std::conditional_t<IsBigEndian, ItemParameterTable::ArrayRefBE, ItemParameterTable::ArrayRefLE>;
+  using ArrayRefT = typename std::conditional_t<IsBigEndian, ItemParameterTable::ArrayRefBE, ItemParameterTable::ArrayRef>;
   return r.pget<ArrayRefT>(root_offset + sizeof(ArrayRefT) * co_index).count;
 }
 
 template <typename T, bool IsBigEndian>
 const T& indirect_lookup_2d(const StringReader& r, size_t root_offset, size_t co_index, size_t item_index) {
-  using ArrayRefT = typename std::conditional_t<IsBigEndian, ItemParameterTable::ArrayRefBE, ItemParameterTable::ArrayRefLE>;
+  using ArrayRefT = typename std::conditional_t<IsBigEndian, ItemParameterTable::ArrayRefBE, ItemParameterTable::ArrayRef>;
 
   const auto& co = r.pget<ArrayRefT>(root_offset + sizeof(ArrayRefT) * co_index);
   if (item_index >= co.count) {
@@ -473,9 +473,9 @@ const ItemParameterTable::WeaponV4& ItemParameterTable::get_weapon(uint8_t data1
     } else if (this->offsets_gc_nte) {
       def_v4 = indirect_lookup_2d<WeaponGCNTE, true>(this->r, this->offsets_gc_nte->weapon_table, data1_1, data1_2).to_v4();
     } else if (this->offsets_v3_le) {
-      def_v4 = indirect_lookup_2d<WeaponV3<false>, false>(this->r, this->offsets_v3_le->weapon_table, data1_1, data1_2).to_v4();
+      def_v4 = indirect_lookup_2d<WeaponV3, false>(this->r, this->offsets_v3_le->weapon_table, data1_1, data1_2).to_v4();
     } else if (this->offsets_v3_be) {
-      def_v4 = indirect_lookup_2d<WeaponV3<true>, true>(this->r, this->offsets_v3_be->weapon_table, data1_1, data1_2).to_v4();
+      def_v4 = indirect_lookup_2d<WeaponV3BE, true>(this->r, this->offsets_v3_be->weapon_table, data1_1, data1_2).to_v4();
     } else {
       throw logic_error("table is not v2, v3, or v4");
     }
@@ -528,11 +528,11 @@ const ItemParameterTable::ArmorOrShieldV4& ItemParameterTable::get_armor_or_shie
     } else if (this->offsets_v1_v2) {
       def_v4 = indirect_lookup_2d<ArmorOrShieldV1V2, false>(this->r, this->offsets_v1_v2->armor_table, data1_1 - 1, data1_2).to_v4();
     } else if (this->offsets_gc_nte) {
-      def_v4 = indirect_lookup_2d<ArmorOrShieldV3<true>, true>(this->r, this->offsets_gc_nte->armor_table, data1_1 - 1, data1_2).to_v4();
+      def_v4 = indirect_lookup_2d<ArmorOrShieldV3BE, true>(this->r, this->offsets_gc_nte->armor_table, data1_1 - 1, data1_2).to_v4();
     } else if (this->offsets_v3_le) {
-      def_v4 = indirect_lookup_2d<ArmorOrShieldV3<false>, false>(this->r, this->offsets_v3_le->armor_table, data1_1 - 1, data1_2).to_v4();
+      def_v4 = indirect_lookup_2d<ArmorOrShieldV3, false>(this->r, this->offsets_v3_le->armor_table, data1_1 - 1, data1_2).to_v4();
     } else if (this->offsets_v3_be) {
-      def_v4 = indirect_lookup_2d<ArmorOrShieldV3<true>, true>(this->r, this->offsets_v3_be->armor_table, data1_1 - 1, data1_2).to_v4();
+      def_v4 = indirect_lookup_2d<ArmorOrShieldV3BE, true>(this->r, this->offsets_v3_be->armor_table, data1_1 - 1, data1_2).to_v4();
     } else {
       throw logic_error("table is not v2, v3, or v4");
     }
@@ -580,11 +580,11 @@ const ItemParameterTable::UnitV4& ItemParameterTable::get_unit(uint8_t data1_2)
     } else if (this->offsets_v1_v2) {
       def_v4 = indirect_lookup_2d<UnitV1V2, false>(this->r, this->offsets_v1_v2->unit_table, 0, data1_2).to_v4();
     } else if (this->offsets_gc_nte) {
-      def_v4 = indirect_lookup_2d<UnitV3<true>, true>(this->r, this->offsets_gc_nte->unit_table, 0, data1_2).to_v4();
+      def_v4 = indirect_lookup_2d<UnitV3BE, true>(this->r, this->offsets_gc_nte->unit_table, 0, data1_2).to_v4();
     } else if (this->offsets_v3_le) {
-      def_v4 = indirect_lookup_2d<UnitV3<false>, false>(this->r, this->offsets_v3_le->unit_table, 0, data1_2).to_v4();
+      def_v4 = indirect_lookup_2d<UnitV3, false>(this->r, this->offsets_v3_le->unit_table, 0, data1_2).to_v4();
     } else if (this->offsets_v3_be) {
-      def_v4 = indirect_lookup_2d<UnitV3<true>, true>(this->r, this->offsets_v3_be->unit_table, 0, data1_2).to_v4();
+      def_v4 = indirect_lookup_2d<UnitV3BE, true>(this->r, this->offsets_v3_be->unit_table, 0, data1_2).to_v4();
     } else {
       throw logic_error("table is not v2, v3, or v4");
     }
@@ -636,11 +636,11 @@ const ItemParameterTable::MagV4& ItemParameterTable::get_mag(uint8_t data1_1) co
         def_v4 = indirect_lookup_2d<MagV2, false>(this->r, this->offsets_v1_v2->mag_table, 0, data1_1).to_v4();
       }
     } else if (this->offsets_gc_nte) {
-      def_v4 = indirect_lookup_2d<MagV3<true>, true>(this->r, this->offsets_gc_nte->mag_table, 0, data1_1).to_v4();
+      def_v4 = indirect_lookup_2d<MagV3BE, true>(this->r, this->offsets_gc_nte->mag_table, 0, data1_1).to_v4();
     } else if (this->offsets_v3_le) {
-      def_v4 = indirect_lookup_2d<MagV3<false>, false>(this->r, this->offsets_v3_le->mag_table, 0, data1_1).to_v4();
+      def_v4 = indirect_lookup_2d<MagV3, false>(this->r, this->offsets_v3_le->mag_table, 0, data1_1).to_v4();
     } else if (this->offsets_v3_be) {
-      def_v4 = indirect_lookup_2d<MagV3<true>, true>(this->r, this->offsets_v3_be->mag_table, 0, data1_1).to_v4();
+      def_v4 = indirect_lookup_2d<MagV3BE, true>(this->r, this->offsets_v3_be->mag_table, 0, data1_1).to_v4();
     } else {
       throw logic_error("table is not v2, v3, or v4");
     }
@@ -693,11 +693,11 @@ const ItemParameterTable::ToolV4& ItemParameterTable::get_tool(uint8_t data1_1,
     } else if (this->offsets_v1_v2) {
       def_v4 = indirect_lookup_2d<ToolV1V2, false>(this->r, this->offsets_v1_v2->tool_table, data1_1, data1_2).to_v4();
     } else if (this->offsets_gc_nte) {
-      def_v4 = indirect_lookup_2d<ToolV3<true>, true>(this->r, this->offsets_gc_nte->tool_table, data1_1, data1_2).to_v4();
+      def_v4 = indirect_lookup_2d<ToolV3BE, true>(this->r, this->offsets_gc_nte->tool_table, data1_1, data1_2).to_v4();
     } else if (this->offsets_v3_le) {
-      def_v4 = indirect_lookup_2d<ToolV3<false>, false>(this->r, this->offsets_v3_le->tool_table, data1_1, data1_2).to_v4();
+      def_v4 = indirect_lookup_2d<ToolV3, false>(this->r, this->offsets_v3_le->tool_table, data1_1, data1_2).to_v4();
     } else if (this->offsets_v3_be) {
-      def_v4 = indirect_lookup_2d<ToolV3<true>, true>(this->r, this->offsets_v3_be->tool_table, data1_1, data1_2).to_v4();
+      def_v4 = indirect_lookup_2d<ToolV3BE, true>(this->r, this->offsets_v3_be->tool_table, data1_1, data1_2).to_v4();
     } else {
       throw logic_error("table is not v2, v3, or v4");
     }
@@ -706,13 +706,13 @@ const ItemParameterTable::ToolV4& ItemParameterTable::get_tool(uint8_t data1_1,
   }
 }
 
-template <typename ToolT, bool IsBigEndian>
+template <typename ToolDefT, bool IsBigEndian>
 pair<uint8_t, uint8_t> ItemParameterTable::find_tool_by_id_t(uint32_t tool_table_offset, uint32_t item_id) const {
-  const auto* cos = &this->r.pget<ArrayRef<IsBigEndian>>(
-      tool_table_offset, this->num_tool_classes * sizeof(ArrayRef<IsBigEndian>));
+  const auto* cos = &this->r.pget<ArrayRefT<IsBigEndian>>(
+      tool_table_offset, this->num_tool_classes * sizeof(ArrayRefT<IsBigEndian>));
   for (size_t z = 0; z < this->num_tool_classes; z++) {
     const auto& co = cos[z];
-    const auto* defs = &this->r.pget<ToolT>(co.offset, sizeof(ToolT) * co.count);
+    const auto* defs = &this->r.pget<ToolDefT>(co.offset, sizeof(ToolDefT) * co.count);
     for (size_t y = 0; y < co.count; y++) {
       if (defs[y].base.id == item_id) {
         return make_pair(z, y);
@@ -728,11 +728,11 @@ pair<uint8_t, uint8_t> ItemParameterTable::find_tool_by_id(uint32_t item_id) con
   } else if (this->offsets_v1_v2) {
     return this->find_tool_by_id_t<ToolV1V2, false>(this->offsets_v1_v2->tool_table, item_id);
   } else if (this->offsets_gc_nte) {
-    return this->find_tool_by_id_t<ToolV3<true>, true>(this->offsets_gc_nte->tool_table, item_id);
+    return this->find_tool_by_id_t<ToolV3BE, true>(this->offsets_gc_nte->tool_table, item_id);
   } else if (this->offsets_v3_le) {
-    return this->find_tool_by_id_t<ToolV3<false>, false>(this->offsets_v3_le->tool_table, item_id);
+    return this->find_tool_by_id_t<ToolV3, false>(this->offsets_v3_le->tool_table, item_id);
   } else if (this->offsets_v3_be) {
-    return this->find_tool_by_id_t<ToolV3<true>, true>(this->offsets_v3_be->tool_table, item_id);
+    return this->find_tool_by_id_t<ToolV3BE, true>(this->offsets_v3_be->tool_table, item_id);
   } else if (this->offsets_v4) {
     return this->find_tool_by_id_t<ToolV4, false>(this->offsets_v4->tool_table, item_id);
   } else {
@@ -752,7 +752,7 @@ float ItemParameterTable::get_sale_divisor_t(const OffsetsT* offsets, uint8_t da
       return this->r.pget<FloatT>(offsets->weapon_sale_divisor_table + data1_1 * sizeof(FloatT));
 
     case 1: {
-      const auto& divisors = this->r.pget<NonWeaponSaleDivisors<IsBigEndian>>(offsets->sale_divisor_table);
+      const auto& divisors = this->r.pget<NonWeaponSaleDivisorsT<IsBigEndian>>(offsets->sale_divisor_table);
       switch (data1_1) {
         case 1:
           return divisors.armor_divisor;
@@ -765,7 +765,7 @@ float ItemParameterTable::get_sale_divisor_t(const OffsetsT* offsets, uint8_t da
     }
 
     case 2: {
-      const auto& divisors = this->r.pget<NonWeaponSaleDivisors<IsBigEndian>>(offsets->sale_divisor_table);
+      const auto& divisors = this->r.pget<NonWeaponSaleDivisorsT<IsBigEndian>>(offsets->sale_divisor_table);
       return divisors.mag_divisor;
     }
 
@@ -803,22 +803,22 @@ const ItemParameterTable::MagFeedResult& ItemParameterTable::get_mag_feed_result
 
   uint32_t offset;
   if (this->offsets_dc_protos) {
-    const auto& table_offsets = this->r.pget<MagFeedResultsListOffsets<false>>(this->offsets_dc_protos->mag_feed_table);
+    const auto& table_offsets = this->r.pget<MagFeedResultsListOffsets>(this->offsets_dc_protos->mag_feed_table);
     offset = table_offsets.offsets[table_index];
   } else if (this->offsets_v1_v2) {
-    const auto& table_offsets = this->r.pget<MagFeedResultsListOffsets<false>>(this->offsets_v1_v2->mag_feed_table);
+    const auto& table_offsets = this->r.pget<MagFeedResultsListOffsets>(this->offsets_v1_v2->mag_feed_table);
     offset = table_offsets.offsets[table_index];
   } else if (this->offsets_gc_nte) {
-    const auto& table_offsets = this->r.pget<MagFeedResultsListOffsets<true>>(this->offsets_gc_nte->mag_feed_table);
+    const auto& table_offsets = this->r.pget<MagFeedResultsListOffsetsBE>(this->offsets_gc_nte->mag_feed_table);
     offset = table_offsets.offsets[table_index];
   } else if (this->offsets_v3_le) {
-    const auto& table_offsets = this->r.pget<MagFeedResultsListOffsets<false>>(this->offsets_v3_le->mag_feed_table);
+    const auto& table_offsets = this->r.pget<MagFeedResultsListOffsets>(this->offsets_v3_le->mag_feed_table);
     offset = table_offsets.offsets[table_index];
   } else if (this->offsets_v3_be) {
-    const auto& table_offsets = this->r.pget<MagFeedResultsListOffsets<true>>(this->offsets_v3_be->mag_feed_table);
+    const auto& table_offsets = this->r.pget<MagFeedResultsListOffsetsBE>(this->offsets_v3_be->mag_feed_table);
     offset = table_offsets.offsets[table_index];
   } else if (this->offsets_v4) {
-    const auto& table_offsets = this->r.pget<MagFeedResultsListOffsets<false>>(this->offsets_v4->mag_feed_table);
+    const auto& table_offsets = this->r.pget<MagFeedResultsListOffsets>(this->offsets_v4->mag_feed_table);
     offset = table_offsets.offsets[table_index];
   } else {
     throw logic_error("table is not v2, v3, or v4");
@@ -856,24 +856,24 @@ uint8_t ItemParameterTable::get_special_stars(uint8_t special) const {
       : 0;
 }
 
-const ItemParameterTable::Special<false>& ItemParameterTable::get_special(uint8_t special) const {
+const ItemParameterTable::Special& ItemParameterTable::get_special(uint8_t special) const {
   special &= 0x3F;
   if (special >= this->num_specials) {
     throw out_of_range("invalid special index");
   }
 
   if (this->offsets_dc_protos) {
-    return this->r.pget<Special<false>>(this->offsets_dc_protos->special_data_table + sizeof(Special<false>) * special);
+    return this->r.pget<Special>(this->offsets_dc_protos->special_data_table + sizeof(Special) * special);
   } else if (this->offsets_v1_v2) {
-    return this->r.pget<Special<false>>(this->offsets_v1_v2->special_data_table + sizeof(Special<false>) * special);
+    return this->r.pget<Special>(this->offsets_v1_v2->special_data_table + sizeof(Special) * special);
   } else if (this->offsets_v3_le) {
-    return this->r.pget<Special<false>>(this->offsets_v3_le->special_data_table + sizeof(Special<false>) * special);
+    return this->r.pget<Special>(this->offsets_v3_le->special_data_table + sizeof(Special) * special);
   } else if (this->offsets_gc_nte) {
     if ((special >= this->parsed_specials.size()) || (this->parsed_specials[special].type != 0xFFFF)) {
       if (special >= this->parsed_specials.size()) {
         this->parsed_specials.resize(special + 1);
       }
-      const auto& sp_be = this->r.pget<Special<true>>(this->offsets_gc_nte->special_data_table + sizeof(Special<true>) * special);
+      const auto& sp_be = this->r.pget<SpecialBE>(this->offsets_gc_nte->special_data_table + sizeof(SpecialBE) * special);
       this->parsed_specials[special].type = sp_be.type.load();
       this->parsed_specials[special].amount = sp_be.amount.load();
     }
@@ -883,13 +883,13 @@ const ItemParameterTable::Special<false>& ItemParameterTable::get_special(uint8_
       if (special >= this->parsed_specials.size()) {
         this->parsed_specials.resize(special + 1);
       }
-      const auto& sp_be = this->r.pget<Special<true>>(this->offsets_v3_be->special_data_table + sizeof(Special<true>) * special);
+      const auto& sp_be = this->r.pget<SpecialBE>(this->offsets_v3_be->special_data_table + sizeof(SpecialBE) * special);
       this->parsed_specials[special].type = sp_be.type.load();
       this->parsed_specials[special].amount = sp_be.amount.load();
     }
     return this->parsed_specials[special];
   } else if (this->offsets_v4) {
-    return this->r.pget<Special<false>>(this->offsets_v4->special_data_table + sizeof(Special<false>) * special);
+    return this->r.pget<Special>(this->offsets_v4->special_data_table + sizeof(Special) * special);
   } else {
     throw logic_error("table is not v2, v3, or v4");
   }
@@ -1014,14 +1014,15 @@ uint8_t ItemParameterTable::get_item_base_stars(const ItemData& item) const {
   }
 }
 
-uint8_t ItemParameterTable::get_item_adjusted_stars(const ItemData& item) const {
+uint8_t ItemParameterTable::get_item_adjusted_stars(const ItemData& item, bool ignore_unidentified) const {
   uint8_t ret = this->get_item_base_stars(item);
   if (item.data1[0] == 0) {
+    bool is_unidentified = (!ignore_unidentified) && (item.data1[4] & 0x80);
     if (ret < 9) {
-      if (!(item.data1[4] & 0x80)) {
+      if (!is_unidentified) {
         ret += this->get_special_stars(item.data1[4]);
       }
-    } else if (item.data1[4] & 0x80) {
+    } else if (is_unidentified) {
       ret = 0;
     }
   } else if (item.data1[0] == 1) {
@@ -1038,7 +1039,11 @@ uint8_t ItemParameterTable::get_item_adjusted_stars(const ItemData& item) const
 }
 
 bool ItemParameterTable::is_item_rare(const ItemData& item) const {
-  return (this->get_item_base_stars(item) >= 9);
+  try {
+    return (this->get_item_base_stars(item) >= 9);
+  } catch (const out_of_range&) {
+    return false;
+  }
 }
 
 bool ItemParameterTable::is_unsealable_item(uint8_t data1_0, uint8_t data1_1, uint8_t data1_2) const {
@@ -1046,7 +1051,7 @@ bool ItemParameterTable::is_unsealable_item(uint8_t data1_0, uint8_t data1_1, ui
   if (this->offsets_dc_protos || this->offsets_v1_v2 || this->offsets_gc_nte) {
     return false;
   } else if (this->offsets_v3_le) {
-    const auto& co = this->r.pget<ArrayRefLE>(this->offsets_v3_le->unsealable_table);
+    const auto& co = this->r.pget<ArrayRef>(this->offsets_v3_le->unsealable_table);
     offset = co.offset;
     count = co.count;
   } else if (this->offsets_v3_be) {
@@ -1054,7 +1059,7 @@ bool ItemParameterTable::is_unsealable_item(uint8_t data1_0, uint8_t data1_1, ui
     offset = co.offset;
     count = co.count;
   } else if (this->offsets_v4) {
-    const auto& co = this->r.pget<ArrayRefLE>(this->offsets_v4->unsealable_table);
+    const auto& co = this->r.pget<ArrayRef>(this->offsets_v4->unsealable_table);
     offset = co.offset;
     count = co.count;
   } else {
@@ -1106,7 +1111,7 @@ const std::map<uint32_t, std::vector<ItemParameterTable::ItemCombination>>& Item
       static const std::map<uint32_t, std::vector<ItemParameterTable::ItemCombination>> empty_map;
       return empty_map;
     } else if (this->offsets_v3_le) {
-      const auto& co = this->r.pget<ArrayRefLE>(this->offsets_v3_le->combination_table);
+      const auto& co = this->r.pget<ArrayRef>(this->offsets_v3_le->combination_table);
       offset = co.offset;
       count = co.count;
     } else if (this->offsets_v3_be) {
@@ -1114,7 +1119,7 @@ const std::map<uint32_t, std::vector<ItemParameterTable::ItemCombination>>& Item
       offset = co.offset;
       count = co.count;
     } else if (this->offsets_v4) {
-      const auto& co = this->r.pget<ArrayRefLE>(this->offsets_v4->combination_table);
+      const auto& co = this->r.pget<ArrayRef>(this->offsets_v4->combination_table);
       offset = co.offset;
       count = co.count;
     } else {
@@ -1133,7 +1138,7 @@ const std::map<uint32_t, std::vector<ItemParameterTable::ItemCombination>>& Item
 
 template <bool IsBigEndian>
 size_t ItemParameterTable::num_events_t(uint32_t base_offset) const {
-  return this->r.pget<ArrayRef<IsBigEndian>>(base_offset).count;
+  return this->r.pget<ArrayRefT<IsBigEndian>>(base_offset).count;
 }
 
 size_t ItemParameterTable::num_events() const {
@@ -1153,11 +1158,11 @@ size_t ItemParameterTable::num_events() const {
 template <bool IsBigEndian>
 std::pair<const ItemParameterTable::EventItem*, size_t> ItemParameterTable::get_event_items_t(
     uint32_t base_offset, uint8_t event_number) const {
-  const auto& co = this->r.pget<ArrayRef<IsBigEndian>>(base_offset);
+  const auto& co = this->r.pget<ArrayRefT<IsBigEndian>>(base_offset);
   if (event_number >= co.count) {
     throw out_of_range("invalid event number");
   }
-  const auto& event_co = this->r.pget<ArrayRef<IsBigEndian>>(co.offset + sizeof(ArrayRef<IsBigEndian>) * event_number);
+  const auto& event_co = this->r.pget<ArrayRefT<IsBigEndian>>(co.offset + sizeof(ArrayRefT<IsBigEndian>) * event_number);
   const auto* defs = &this->r.pget<EventItem>(event_co.offset, event_co.count * sizeof(EventItem));
   return make_pair(defs, event_co.count);
 }

src/ItemParameterTable.hh

@@ -19,42 +19,43 @@ public:
   // being null or not in each public function. Rewrite this and make it better.
 
   template <bool IsBigEndian>
-  struct ArrayRef {
+  struct ArrayRefT {
     using U32T = typename std::conditional<IsBigEndian, be_uint32_t, le_uint32_t>::type;
     /* 00 */ U32T count;
     /* 04 */ U32T offset;
     /* 08 */
-  } __attribute__((packed));
-  struct ArrayRefLE : ArrayRef<false> {
-  } __attribute__((packed));
-  struct ArrayRefBE : ArrayRef<true> {
-  } __attribute__((packed));
+  } __packed__;
+
+  using ArrayRef = ArrayRefT<false>;
+  using ArrayRefBE = ArrayRefT<true>;
+  check_struct_size(ArrayRef, 8);
+  check_struct_size(ArrayRefBE, 8);
 
   template <bool IsBigEndian>
-  struct ItemBaseV2 {
+  struct ItemBaseV2T {
     using U32T = typename std::conditional<IsBigEndian, be_uint32_t, le_uint32_t>::type;
     // id specifies several things; notably, it doubles as the index of the
     // item's name in the text archive (e.g. TextEnglish) collection 0.
     /* 00 */ U32T id = 0xFFFFFFFF;
     /* 04 */
-  } __attribute__((packed));
+  } __packed__;
   template <bool IsBigEndian>
-  struct ItemBaseV3 : ItemBaseV2<IsBigEndian> {
+  struct ItemBaseV3T : ItemBaseV2T<IsBigEndian> {
     using U16T = typename std::conditional<IsBigEndian, be_uint16_t, le_uint16_t>::type;
     /* 04 */ U16T type = 0;
     /* 06 */ U16T skin = 0;
     /* 08 */
-  } __attribute__((packed));
+  } __packed__;
   template <bool IsBigEndian>
-  struct ItemBaseV4 : ItemBaseV3<IsBigEndian> {
+  struct ItemBaseV4T : ItemBaseV3T<IsBigEndian> {
     using U32T = typename std::conditional<IsBigEndian, be_uint32_t, le_uint32_t>::type;
     /* 08 */ U32T team_points = 0;
     /* 0C */
-  } __attribute__((packed));
+  } __packed__;
 
   struct WeaponV4;
   struct WeaponDCProtos {
-    /* 00 */ ItemBaseV2<false> base;
+    /* 00 */ ItemBaseV2T<false> base;
     /* 04 */ le_uint16_t class_flags = 0;
     /* 06 */ le_uint16_t atp_min = 0;
     /* 08 */ le_uint16_t atp_max = 0;
@@ -68,10 +69,10 @@ public:
     /* 14 */
 
     WeaponV4 to_v4() const;
-  } __attribute__((packed));
+  } __packed_ws__(WeaponDCProtos, 0x14);
 
   struct WeaponV1V2 {
-    /* 00 */ ItemBaseV2<false> base;
+    /* 00 */ ItemBaseV2T<false> base;
     /* 04 */ le_uint16_t class_flags = 0;
     /* 06 */ le_uint16_t atp_min = 0;
     /* 08 */ le_uint16_t atp_max = 0;
@@ -87,10 +88,10 @@ public:
     /* 18 */
 
     WeaponV4 to_v4() const;
-  } __attribute__((packed));
+  } __packed_ws__(WeaponV1V2, 0x18);
 
   struct WeaponGCNTE {
-    /* 00 */ ItemBaseV3<true> base;
+    /* 00 */ ItemBaseV3T<true> base;
     /* 08 */ be_uint16_t class_flags = 0;
     /* 0A */ be_uint16_t atp_min = 0;
     /* 0C */ be_uint16_t atp_max = 0;
@@ -115,12 +116,12 @@ public:
     /* 24 */
 
     WeaponV4 to_v4() const;
-  } __attribute__((packed));
+  } __packed_ws__(WeaponGCNTE, 0x24);
 
   template <bool IsBigEndian>
-  struct WeaponV3 {
+  struct WeaponV3T {
     using U16T = typename std::conditional<IsBigEndian, be_uint16_t, le_uint16_t>::type;
-    /* 00 */ ItemBaseV3<IsBigEndian> base;
+    /* 00 */ ItemBaseV3T<IsBigEndian> base;
     /* 08 */ U16T class_flags = 0;
     /* 0A */ U16T atp_min = 0;
     /* 0C */ U16T atp_max = 0;
@@ -149,10 +150,15 @@ public:
     /* 28 */
 
     WeaponV4 to_v4() const;
-  } __attribute__((packed));
+  } __packed__;
+
+  using WeaponV3 = WeaponV3T<false>;
+  using WeaponV3BE = WeaponV3T<true>;
+  check_struct_size(WeaponV3, 0x28);
+  check_struct_size(WeaponV3BE, 0x28);
 
   struct WeaponV4 {
-    /* 00 */ ItemBaseV4<false> base;
+    /* 00 */ ItemBaseV4T<false> base;
     /* 0C */ le_uint16_t class_flags = 0x00FF;
     /* 0E */ le_uint16_t atp_min = 0;
     /* 10 */ le_uint16_t atp_max = 0;
@@ -179,10 +185,10 @@ public:
     /* 2A */ uint8_t tech_boost = 0;
     /* 2B */ uint8_t combo_type = 0;
     /* 2C */
-  } __attribute__((packed));
+  } __packed_ws__(WeaponV4, 0x2C);
 
   template <typename BaseT, bool IsBigEndian>
-  struct ArmorOrShield {
+  struct ArmorOrShieldT {
     using U16T = typename std::conditional<IsBigEndian, be_uint16_t, le_uint16_t>::type;
     /* V1/V2 offsets */
     /* 00 */ BaseT base;
@@ -200,33 +206,36 @@ public:
     /* 12 */ uint8_t dfp_range = 0;
     /* 13 */ uint8_t evp_range = 0;
     /* 14 */
-  } __attribute__((packed));
+  } __packed__;
 
-  struct ArmorOrShieldV4;
-  struct ArmorOrShieldDCProtos : ArmorOrShield<ItemBaseV2<false>, false> {
-    ArmorOrShieldV4 to_v4() const;
-  } __attribute__((packed));
   template <typename BaseT, bool IsBigEndian>
-  struct ArmorOrShieldFinal : ArmorOrShield<BaseT, IsBigEndian> {
+  struct ArmorOrShieldFinalT : ArmorOrShieldT<BaseT, IsBigEndian> {
     using U16T = typename std::conditional<IsBigEndian, be_uint16_t, le_uint16_t>::type;
     /* 14 */ uint8_t stat_boost = 0;
     /* 15 */ uint8_t tech_boost = 0;
     /* 16 */ U16T unknown_a2 = 0;
     /* 18 */
-  } __attribute__((packed));
+  } __packed__;
+  using ArmorOrShieldV4 = ArmorOrShieldFinalT<ItemBaseV4T<false>, false>;
+  check_struct_size(ArmorOrShieldV4, 0x20);
+  struct ArmorOrShieldDCProtos : ArmorOrShieldT<ItemBaseV2T<false>, false> {
+    ArmorOrShieldV4 to_v4() const;
+  } __packed_ws__(ArmorOrShieldDCProtos, 0x14);
 
-  struct ArmorOrShieldV1V2 : ArmorOrShieldFinal<ItemBaseV2<false>, false> {
+  struct ArmorOrShieldV1V2 : ArmorOrShieldFinalT<ItemBaseV2T<false>, false> {
     ArmorOrShieldV4 to_v4() const;
-  } __attribute__((packed));
+  } __packed_ws__(ArmorOrShieldV1V2, 0x18);
   template <bool IsBigEndian>
-  struct ArmorOrShieldV3 : ArmorOrShieldFinal<ItemBaseV3<IsBigEndian>, IsBigEndian> {
+  struct ArmorOrShieldV3T : ArmorOrShieldFinalT<ItemBaseV3T<IsBigEndian>, IsBigEndian> {
     ArmorOrShieldV4 to_v4() const;
-  } __attribute__((packed));
-  struct ArmorOrShieldV4 : ArmorOrShieldFinal<ItemBaseV4<false>, false> {
-  } __attribute__((packed));
+  } __packed__;
+  using ArmorOrShieldV3 = ArmorOrShieldV3T<false>;
+  using ArmorOrShieldV3BE = ArmorOrShieldV3T<true>;
+  check_struct_size(ArmorOrShieldV3, 0x1C);
+  check_struct_size(ArmorOrShieldV3BE, 0x1C);
 
   template <typename BaseT, bool IsBigEndian>
-  struct Unit {
+  struct UnitT {
     using U16T = typename std::conditional<IsBigEndian, be_uint16_t, le_uint16_t>::type;
     using S16T = typename std::conditional<IsBigEndian, be_int16_t, le_int16_t>::type;
     /* V1/V2 offsets */
@@ -234,31 +243,34 @@ public:
     /* 04 */ U16T stat = 0;
     /* 06 */ U16T stat_amount = 0;
     /* 08 */
-  } __attribute__((packed));
+  } __packed__;
 
-  struct UnitV4;
-  struct UnitDCProtos : Unit<ItemBaseV2<false>, false> {
-    UnitV4 to_v4() const;
-  } __attribute__((packed));
   template <typename BaseT, bool IsBigEndian>
-  struct UnitFinal : Unit<BaseT, IsBigEndian> {
+  struct UnitFinalT : UnitT<BaseT, IsBigEndian> {
     using S16T = typename std::conditional<IsBigEndian, be_int16_t, le_int16_t>::type;
     /* 08 */ S16T modifier_amount = 0;
     /* 0A */ parray<uint8_t, 2> unused;
     /* 0C */
-  } __attribute__((packed));
-  struct UnitV1V2 : UnitFinal<ItemBaseV2<false>, false> {
+  } __packed__;
+  using UnitV4 = UnitFinalT<ItemBaseV4T<false>, false>;
+  check_struct_size(UnitV4, 0x14);
+  struct UnitDCProtos : UnitT<ItemBaseV2T<false>, false> {
     UnitV4 to_v4() const;
-  } __attribute__((packed));
+  } __packed_ws__(UnitDCProtos, 0x08);
+  struct UnitV1V2 : UnitFinalT<ItemBaseV2T<false>, false> {
+    UnitV4 to_v4() const;
+  } __packed_ws__(UnitV1V2, 0x0C);
   template <bool IsBigEndian>
-  struct UnitV3 : UnitFinal<ItemBaseV3<IsBigEndian>, IsBigEndian> {
+  struct UnitV3T : UnitFinalT<ItemBaseV3T<IsBigEndian>, IsBigEndian> {
     UnitV4 to_v4() const;
-  } __attribute__((packed));
-  struct UnitV4 : UnitFinal<ItemBaseV4<false>, false> {
-  } __attribute__((packed));
+  } __packed__;
+  using UnitV3 = UnitV3T<false>;
+  using UnitV3BE = UnitV3T<true>;
+  check_struct_size(UnitV3, 0x10);
+  check_struct_size(UnitV3BE, 0x10);
 
   template <typename BaseT, bool IsBigEndian>
-  struct Mag {
+  struct MagT {
     using U16T = typename std::conditional<IsBigEndian, be_uint16_t, le_uint16_t>::type;
     /* V1/V2 offsets */
     /* 00 */ BaseT base;
@@ -289,36 +301,38 @@ public:
     /* 0E */ uint8_t on_death_flag = 0;
     /* 0F */ uint8_t on_boss_flag = 0;
     /* 10 */
-  } __attribute__((packed));
+  } __packed__;
 
-  struct MagV4;
-  struct MagV1 : Mag<ItemBaseV2<false>, false> {
+  struct MagV4 : MagT<ItemBaseV4T<false>, false> {
+    le_uint16_t class_flags = 0x00FF;
+    parray<uint8_t, 2> unused;
+  } __packed_ws__(MagV4, 0x1C);
+  struct MagV1 : MagT<ItemBaseV2T<false>, false> {
     MagV4 to_v4() const;
-  } __attribute__((packed));
-  struct MagV2 : Mag<ItemBaseV2<false>, false> {
+  } __packed_ws__(MagV1, 0x10);
+  struct MagV2 : MagT<ItemBaseV2T<false>, false> {
     /* 10 */ le_uint16_t class_flags = 0x00FF;
     /* 12 */ parray<uint8_t, 2> unused;
     /* 14 */
 
     MagV4 to_v4() const;
-  } __attribute__((packed));
+  } __packed_ws__(MagV2, 0x14);
   template <bool IsBigEndian>
-  struct MagV3 : Mag<ItemBaseV3<IsBigEndian>, IsBigEndian> {
+  struct MagV3T : MagT<ItemBaseV3T<IsBigEndian>, IsBigEndian> {
     using U16T = typename std::conditional<IsBigEndian, be_uint16_t, le_uint16_t>::type;
     /* 10 */ U16T class_flags = 0x00FF;
     /* 12 */ parray<uint8_t, 2> unused;
     /* 14 */
 
     MagV4 to_v4() const;
-  } __attribute__((packed));
-  struct MagV4 : Mag<ItemBaseV4<false>, false> {
-    /* 10 */ le_uint16_t class_flags = 0x00FF;
-    /* 12 */ parray<uint8_t, 2> unused;
-    /* 14 */
-  } __attribute__((packed));
+  } __packed__;
+  using MagV3 = MagV3T<false>;
+  using MagV3BE = MagV3T<true>;
+  check_struct_size(MagV3, 0x18);
+  check_struct_size(MagV3BE, 0x18);
 
   template <typename BaseT, bool IsBigEndian>
-  struct Tool {
+  struct ToolT {
     using U16T = typename std::conditional<IsBigEndian, be_uint16_t, le_uint16_t>::type;
     using S32T = typename std::conditional<IsBigEndian, be_int32_t, le_int32_t>::type;
     using U32T = typename std::conditional<IsBigEndian, be_uint32_t, le_uint32_t>::type;
@@ -329,18 +343,21 @@ public:
     /* 08 */ S32T cost = 0;
     /* 0C */ U32T item_flag = 0;
     /* 10 */
-  } __attribute__((packed));
+  } __packed__;
 
-  struct ToolV4;
-  struct ToolV1V2 : Tool<ItemBaseV2<false>, false> {
+  using ToolV4 = ToolT<ItemBaseV4T<false>, false>;
+  check_struct_size(ToolV4, 0x18);
+  struct ToolV1V2 : ToolT<ItemBaseV2T<false>, false> {
     ToolV4 to_v4() const;
-  } __attribute__((packed));
+  } __packed_ws__(ToolV1V2, 0x10);
   template <bool IsBigEndian>
-  struct ToolV3 : Tool<ItemBaseV3<IsBigEndian>, IsBigEndian> {
+  struct ToolV3T : ToolT<ItemBaseV3T<IsBigEndian>, IsBigEndian> {
     ToolV4 to_v4() const;
-  } __attribute__((packed));
-  struct ToolV4 : Tool<ItemBaseV4<false>, false> {
-  } __attribute__((packed));
+  } __packed__;
+  using ToolV3 = ToolV3T<false>;
+  using ToolV3BE = ToolV3T<true>;
+  check_struct_size(ToolV3, 0x14);
+  check_struct_size(ToolV3BE, 0x14);
 
   struct MagFeedResult {
     int8_t def = 0;
@@ -350,31 +367,43 @@ public:
     int8_t iq = 0;
     int8_t synchro = 0;
     parray<uint8_t, 2> unused;
-  } __attribute__((packed));
+  } __packed_ws__(MagFeedResult, 8);
 
   using MagFeedResultsList = parray<MagFeedResult, 11>;
 
   template <bool IsBigEndian>
-  struct MagFeedResultsListOffsets {
+  struct MagFeedResultsListOffsetsT {
     using U32T = typename std::conditional<IsBigEndian, be_uint32_t, le_uint32_t>::type;
     parray<U32T, 8> offsets; // Offsets of MagFeedResultsList objects
-  } __attribute__((packed));
+  } __packed__;
+  using MagFeedResultsListOffsets = MagFeedResultsListOffsetsT<false>;
+  using MagFeedResultsListOffsetsBE = MagFeedResultsListOffsetsT<true>;
+  check_struct_size(MagFeedResultsListOffsets, 0x20);
+  check_struct_size(MagFeedResultsListOffsetsBE, 0x20);
 
   template <bool IsBigEndian>
-  struct Special {
+  struct SpecialT {
     using U16T = typename std::conditional<IsBigEndian, be_uint16_t, le_uint16_t>::type;
     U16T type = 0xFFFF;
     U16T amount = 0;
-  } __attribute__((packed));
+  } __packed__;
+  using Special = SpecialT<false>;
+  using SpecialBE = SpecialT<true>;
+  check_struct_size(Special, 4);
+  check_struct_size(SpecialBE, 4);
 
   template <bool IsBigEndian>
-  struct StatBoost {
+  struct StatBoostT {
     using U16T = typename std::conditional<IsBigEndian, be_uint16_t, le_uint16_t>::type;
     uint8_t stat1 = 0;
     uint8_t stat2 = 0;
     U16T amount1 = 0;
     U16T amount2 = 0;
-  } __attribute__((packed));
+  } __packed__;
+  using StatBoost = StatBoostT<false>;
+  using StatBoostBE = StatBoostT<true>;
+  check_struct_size(StatBoost, 6);
+  check_struct_size(StatBoostBE, 6);
 
   // Indexed as [tech_num][char_class]
   using MaxTechniqueLevels = parray<parray<uint8_t, 12>, 19>;
@@ -388,10 +417,10 @@ public:
     uint8_t level = 0;
     uint8_t char_class = 0;
     parray<uint8_t, 3> unused;
-  } __attribute__((packed));
+  } __packed_ws__(ItemCombination, 0x10);
 
   template <bool IsBigEndian>
-  struct TechniqueBoost {
+  struct TechniqueBoostT {
     using U32T = typename std::conditional<IsBigEndian, be_uint32_t, le_uint32_t>::type;
     using FloatT = typename std::conditional<IsBigEndian, be_float, le_float>::type;
     U32T tech1 = 0;
@@ -400,26 +429,34 @@ public:
     FloatT boost2 = 0.0f;
     U32T tech3 = 0;
     FloatT boost3 = 0.0f;
-  } __attribute__((packed));
+  } __packed__;
+  using TechniqueBoost = TechniqueBoostT<false>;
+  using TechniqueBoostBE = TechniqueBoostT<true>;
+  check_struct_size(TechniqueBoost, 0x18);
+  check_struct_size(TechniqueBoostBE, 0x18);
 
   struct EventItem {
     parray<uint8_t, 3> item;
     uint8_t probability = 0;
-  } __attribute__((packed));
+  } __packed_ws__(EventItem, 4);
 
   struct UnsealableItem {
     parray<uint8_t, 3> item;
     uint8_t unused = 0;
-  } __attribute__((packed));
+  } __packed_ws__(UnsealableItem, 4);
 
   template <bool IsBigEndian>
-  struct NonWeaponSaleDivisors {
+  struct NonWeaponSaleDivisorsT {
     using FloatT = typename std::conditional<IsBigEndian, be_float, le_float>::type;
     FloatT armor_divisor = 0.0f;
     FloatT shield_divisor = 0.0f;
     FloatT unit_divisor = 0.0f;
     FloatT mag_divisor = 0.0f;
-  } __attribute__((packed));
+  } __packed__;
+  using NonWeaponSaleDivisors = NonWeaponSaleDivisorsT<false>;
+  using NonWeaponSaleDivisorsBE = NonWeaponSaleDivisorsT<true>;
+  check_struct_size(NonWeaponSaleDivisors, 0x10);
+  check_struct_size(NonWeaponSaleDivisorsBE, 0x10);
 
   ItemParameterTable(std::shared_ptr<const std::string> data, Version version);
   ~ItemParameterTable() = default;
@@ -443,14 +480,14 @@ public:
   const MagFeedResult& get_mag_feed_result(uint8_t table_index, uint8_t which) const;
   uint8_t get_item_stars(uint32_t id) const;
   uint8_t get_special_stars(uint8_t special) const;
-  const Special<false>& get_special(uint8_t special) const;
+  const Special& get_special(uint8_t special) const;
   uint8_t get_max_tech_level(uint8_t char_class, uint8_t tech_num) const;
   uint8_t get_weapon_v1_replacement(uint8_t data1_1) const;
 
   uint32_t get_item_id(const ItemData& item) const;
   uint32_t get_item_team_points(const ItemData& item) const;
   uint8_t get_item_base_stars(const ItemData& item) const;
-  uint8_t get_item_adjusted_stars(const ItemData& item) const;
+  uint8_t get_item_adjusted_stars(const ItemData& item, bool ignore_unidentified = false) const;
   bool is_item_rare(const ItemData& item) const;
   bool is_unsealable_item(uint8_t data1_0, uint8_t data1_1, uint8_t data1_2) const;
   bool is_unsealable_item(const ItemData& param_1) const;
@@ -493,7 +530,7 @@ private:
     /* 44 / 0668 / 0668 */ le_uint32_t unknown_a2;
     /* 48 / 030C / 030C */ le_uint32_t unknown_a3;
     /* 4C / 2CE4 / 2D78 */ le_uint32_t unknown_a4;
-  } __attribute__((packed));
+  } __packed_ws__(TableOffsetsDCProtos, 0x50);
 
   struct TableOffsetsV1V2 {
     // TODO: Is weapon count 0x89 or 0x8A? It could be that the last entry in
@@ -516,7 +553,7 @@ private:
     /* 38 / 2C12 / 4540 */ le_uint32_t special_data_table; // -> [Special](0x29)
     /* 3C / 2CB8 / 58DC */ le_uint32_t stat_boost_table; // -> [StatBoost]
     /* 40 / 3198 / 5704 */ le_uint32_t shield_effect_table; // -> [8-byte structs]
-  } __attribute__((packed));
+  } __packed_ws__(TableOffsetsV1V2, 0x44);
 
   struct TableOffsetsGCNTE {
     /* 00 / 6F0C */ be_uint32_t weapon_table; // -> [{count, offset -> [WeaponV3/WeaponV4]}](0xED)
@@ -539,10 +576,10 @@ private:
     /* 44 / 737C */ be_uint32_t combination_table; // -> {count, offset -> [ItemCombination]}
     /* 48 / 68B0 */ be_uint32_t unknown_a1;
     /* 4C / 6B1C */ be_uint32_t tech_boost_table; // -> [TechniqueBoost] (always 0x2C of them? from counts struct?)
-  } __attribute__((packed));
+  } __packed_ws__(TableOffsetsGCNTE, 0x50);
 
   template <bool IsBigEndian>
-  struct TableOffsetsV3V4 {
+  struct TableOffsetsV3V4T {
     using U32T = typename std::conditional<IsBigEndian, be_uint32_t, le_uint32_t>::type;
     /* ## / GC   / BB */
     /* 00 / F078 / 14884 */ U32T weapon_table; // -> [{count, offset -> [WeaponV3/WeaponV4]}](0xED)
@@ -568,7 +605,11 @@ private:
     /* 50 / F5F0 / 15014 */ U32T unwrap_table; // -> {count, offset -> [{count, offset -> [EventItem]}]}
     /* 54 / F5F8 / 1501C */ U32T unsealable_table; // -> {count, offset -> [UnsealableItem]}
     /* 58 / F600 / 15024 */ U32T ranged_special_table; // -> {count, offset -> [4-byte structs]}
-  } __attribute__((packed));
+  } __packed__;
+  using TableOffsetsV3V4 = TableOffsetsV3V4T<false>;
+  using TableOffsetsV3V4BE = TableOffsetsV3V4T<true>;
+  check_struct_size(TableOffsetsV3V4, 0x5C);
+  check_struct_size(TableOffsetsV3V4BE, 0x5C);
 
   Version version;
   std::shared_ptr<const std::string> data;
@@ -576,9 +617,9 @@ private:
   const TableOffsetsDCProtos* offsets_dc_protos;
   const TableOffsetsV1V2* offsets_v1_v2;
   const TableOffsetsGCNTE* offsets_gc_nte;
-  const TableOffsetsV3V4<false>* offsets_v3_le;
-  const TableOffsetsV3V4<true>* offsets_v3_be;
-  const TableOffsetsV3V4<false>* offsets_v4;
+  const TableOffsetsV3V4* offsets_v3_le;
+  const TableOffsetsV3V4BE* offsets_v3_be;
+  const TableOffsetsV3V4* offsets_v4;
 
   // These are unused if offsets_v4 is not null (in that case, we just return
   // references pointing inside the data string)
@@ -588,13 +629,13 @@ private:
   mutable std::vector<UnitV4> parsed_units;
   mutable std::vector<MagV4> parsed_mags;
   mutable std::unordered_map<uint16_t, ToolV4> parsed_tools;
-  mutable std::vector<Special<false>> parsed_specials;
+  mutable std::vector<Special> parsed_specials;
 
   // Key is used_item. We can't index on (used_item, equipped_item) because
   // equipped_item may contain wildcards, and the matching order matters.
   mutable std::map<uint32_t, std::vector<ItemCombination>> item_combination_index;
 
-  template <typename ToolT, bool IsBigEndian>
+  template <typename ToolDefT, bool IsBigEndian>
   std::pair<uint8_t, uint8_t> find_tool_by_id_t(uint32_t tool_table_offset, uint32_t id) const;
   template <bool IsBigEndian, typename OffsetsT>
   float get_sale_divisor_t(const OffsetsT* offsets, uint8_t data1_0, uint8_t data1_1) const;
@@ -607,17 +648,18 @@ private:
 class MagEvolutionTable {
 public:
   struct TableOffsets {
-    /* 00 / 0400 */ le_uint32_t unknown_a1; // -> [offset -> (0xC-byte struct)[0x53], offset -> (same as first offset)]
-    /* 04 / 0408 */ le_uint32_t unknown_a2; // -> (2-byte struct, or single word)[0x53]
+    // num_mags = 0x53 in BB, 0x43 in V3
+    /* 00 / 0400 */ le_uint32_t unknown_a1; // -> [offset -> (0xC-byte struct)[num_mags], offset -> (same as first offset)]
+    /* 04 / 0408 */ le_uint32_t unknown_a2; // -> (2-byte struct, or single word)[num_mags]
     /* 08 / 04AE */ le_uint32_t unknown_a3; // -> (0xA8 bytes; possibly (8-byte struct)[0x15])
-    /* 0C / 0556 */ le_uint32_t unknown_a4; // -> (uint8_t)[0x53]
+    /* 0C / 0556 */ le_uint32_t unknown_a4; // -> (uint8_t)[num_mags]
     /* 10 / 05AC */ le_uint32_t unknown_a5; // -> (float)[0x48]
-    /* 14 / 06CC */ le_uint32_t evolution_number; // -> (uint8_t)[0x53]
-  } __attribute__((packed));
+    /* 14 / 06CC */ le_uint32_t evolution_number; // -> (uint8_t)[num_mags]
+  } __packed_ws__(TableOffsets, 0x18);
 
   struct EvolutionNumberTable {
     parray<uint8_t, 0x53> values;
-  } __attribute__((packed));
+  } __packed_ws__(EvolutionNumberTable, 0x53);
 
   MagEvolutionTable(std::shared_ptr<const std::string> data);
   ~MagEvolutionTable() = default;

src/Items.cc

@@ -6,7 +6,7 @@
 
 using namespace std;
 
-void player_use_item(shared_ptr<Client> c, size_t item_index, shared_ptr<PSOLFGEncryption> random_crypt) {
+void player_use_item(shared_ptr<Client> c, size_t item_index, shared_ptr<PSOLFGEncryption> opt_rand_crypt) {
   auto s = c->require_server_state();
 
   // On PC (and presumably DC), the client sends a 6x29 after this to delete the
@@ -81,7 +81,7 @@ void player_use_item(shared_ptr<Client> c, size_t item_index, shared_ptr<PSOLFGE
         break;
       case 6: // Hit Material (v1/v2) or Luck Material (v3/v4)
         type = Type::LUCK;
-        if (!is_v3_or_later) {
+        if (!is_v3_or_later && (c->version() != Version::GC_NTE)) {
           // Hit material doesn't exist on v3/v4, but we'll ignore type anyway
           // in this case because track_non_hp_tp_materials is false
           p->disp.stats.char_stats.ata += 2;
@@ -91,7 +91,7 @@ void player_use_item(shared_ptr<Client> c, size_t item_index, shared_ptr<PSOLFGE
         break;
       case 7: // Luck Material (v1/v2)
         type = Type::LUCK;
-        if (!is_v3_or_later) {
+        if (!is_v3_or_later && (c->version() != Version::GC_NTE)) {
           p->disp.stats.char_stats.lck += 2;
         } else {
           throw runtime_error("unknown material used");
@@ -111,9 +111,9 @@ void player_use_item(shared_ptr<Client> c, size_t item_index, shared_ptr<PSOLFGE
     }
     armor.data.data1[5]++;
 
-  } else if (item.data.is_wrapped(c->version())) {
+  } else if (item.data.is_wrapped(*s->item_stack_limits(c->version()))) {
     // Unwrap present
-    item.data.unwrap(c->version());
+    item.data.unwrap(*s->item_stack_limits(c->version()));
     should_delete_item = false;
 
   } else if (primary_identifier == 0x00330000) {
@@ -177,7 +177,7 @@ void player_use_item(shared_ptr<Client> c, size_t item_index, shared_ptr<PSOLFGE
     if (sum == 0) {
       throw runtime_error("no unwrap results available for event");
     }
-    size_t det = random_crypt->next() % sum;
+    size_t det = random_from_optional_crypt(opt_rand_crypt) % sum;
     for (size_t z = 0; z < table.second; z++) {
       const auto& entry = table.first[z];
       if (det > entry.probability) {
@@ -250,7 +250,7 @@ void player_use_item(shared_ptr<Client> c, size_t item_index, shared_ptr<PSOLFGE
   if (should_delete_item) {
     // Allow overdrafting meseta if the client is not BB, since the server isn't
     // informed when meseta is added or removed from the bank.
-    player->remove_item(item.data.id, 1, c->version());
+    player->remove_item(item.data.id, 1, *s->item_stack_limits(c->version()));
   }
 }
 

src/Items.hh

@@ -12,7 +12,7 @@
 #include "ServerState.hh"
 #include "StaticGameData.hh"
 
-void player_use_item(std::shared_ptr<Client> c, size_t item_index, std::shared_ptr<PSOLFGEncryption> random_crypt);
+void player_use_item(std::shared_ptr<Client> c, size_t item_index, std::shared_ptr<PSOLFGEncryption> opt_rand_crypt);
 void player_feed_mag(std::shared_ptr<Client> c, size_t mag_item_index, size_t fed_item_index);
 
 void apply_mag_feed_result(

src/LevelTable.cc

@@ -9,26 +9,26 @@
 
 using namespace std;
 
-void PlayerStats::reset_to_base(uint8_t char_class, shared_ptr<const LevelTable> level_table) {
-  this->level = 0;
-  this->experience = 0;
-  this->char_stats = level_table->base_stats_for_class(char_class);
+void LevelTable::reset_to_base(PlayerStats& stats, uint8_t char_class) const {
+  stats.level = 0;
+  stats.experience = 0;
+  stats.char_stats = this->base_stats_for_class(char_class);
 }
 
-void PlayerStats::advance_to_level(uint8_t char_class, uint32_t level, shared_ptr<const LevelTable> level_table) {
-  for (; this->level < level; this->level++) {
-    const auto& level_stats = level_table->stats_delta_for_level(char_class, this->level + 1);
+void LevelTable::advance_to_level(PlayerStats& stats, uint32_t level, uint8_t char_class) const {
+  for (; stats.level < level; stats.level++) {
+    const auto& level_stats = this->stats_delta_for_level(char_class, stats.level + 1);
     // The original code clamps the resulting stat values to [0, max_stat]; we
     // don't have max_stat handy so we just allow them to be unbounded
-    this->char_stats.atp += level_stats.atp;
-    this->char_stats.mst += level_stats.mst;
-    this->char_stats.evp += level_stats.evp;
-    this->char_stats.hp += level_stats.hp;
-    this->char_stats.dfp += level_stats.dfp;
-    this->char_stats.ata += level_stats.ata;
+    stats.char_stats.atp += level_stats.atp;
+    stats.char_stats.mst += level_stats.mst;
+    stats.char_stats.evp += level_stats.evp;
+    stats.char_stats.hp += level_stats.hp;
+    stats.char_stats.dfp += level_stats.dfp;
+    stats.char_stats.ata += level_stats.ata;
     // Note: It is not a bug that lck is ignored here; the original code
     // ignores it too.
-    this->experience = level_stats.experience;
+    stats.experience = level_stats.experience;
   }
 }
 
@@ -52,7 +52,7 @@ LevelTableV2::LevelTableV2(const string& data, bool compressed) {
     le_uint32_t unknown_a10; // -> u32[3] -> (0x10-byte struct)[0x0C]
     le_uint32_t unknown_a11; // -> u32[3] -> (0x30-bytes)
     le_uint32_t unknown_a12; // -> u32[3] -> (0x14-byte struct)[0x0F]
-  } __attribute__((packed));
+  } __packed_ws__(Offsets, 0x40);
 
   StringReader r;
   string decompressed_data;
@@ -158,7 +158,7 @@ LevelTableV4::LevelTableV4(const string& data, bool compressed) {
   struct Offsets {
     le_uint32_t base_stats; // -> u32[12] -> CharacterStats
     le_uint32_t level_deltas; // -> u32[12] -> LevelStatsDelta[200]
-  } __attribute__((packed));
+  } __packed_ws__(Offsets, 8);
 
   StringReader r;
   string decompressed_data;

src/LevelTable.hh

@@ -7,35 +7,74 @@
 #include <phosg/Encoding.hh>
 #include <string>
 
+#include "Text.hh"
+
 class LevelTable;
 
-struct CharacterStats {
-  /* 00 */ le_uint16_t atp = 0;
-  /* 02 */ le_uint16_t mst = 0;
-  /* 04 */ le_uint16_t evp = 0;
-  /* 06 */ le_uint16_t hp = 0;
-  /* 08 */ le_uint16_t dfp = 0;
-  /* 0A */ le_uint16_t ata = 0;
-  /* 0C */ le_uint16_t lck = 0;
+template <bool IsBigEndian>
+struct CharacterStatsT {
+  using U16T = typename std::conditional<IsBigEndian, be_uint16_t, le_uint16_t>::type;
+
+  /* 00 */ U16T atp = 0;
+  /* 02 */ U16T mst = 0;
+  /* 04 */ U16T evp = 0;
+  /* 06 */ U16T hp = 0;
+  /* 08 */ U16T dfp = 0;
+  /* 0A */ U16T ata = 0;
+  /* 0C */ U16T lck = 0;
   /* 0E */
-} __attribute__((packed));
 
-struct PlayerStats {
-  /* 00 */ CharacterStats char_stats;
-  /* 0E */ le_uint16_t unknown_a1 = 0;
-  /* 10 */ le_float height = 0.0;
-  /* 14 */ le_float unknown_a3 = 0.0;
-  /* 18 */ le_uint32_t level = 0;
-  /* 1C */ le_uint32_t experience = 0;
-  /* 20 */ le_uint32_t meseta = 0;
-  /* 24 */
-
-  void reset_to_base(uint8_t char_class, std::shared_ptr<const LevelTable> level_table);
-  void advance_to_level(uint8_t char_class, uint32_t level, std::shared_ptr<const LevelTable> level_table);
-} __attribute__((packed));
+  operator CharacterStatsT<!IsBigEndian>() const {
+    CharacterStatsT<!IsBigEndian> ret;
+    ret.atp = this->atp.load();
+    ret.mst = this->mst.load();
+    ret.evp = this->evp.load();
+    ret.hp = this->hp.load();
+    ret.dfp = this->dfp.load();
+    ret.ata = this->ata.load();
+    ret.lck = this->lck.load();
+    return ret;
+  }
+} __packed__;
+using CharacterStats = CharacterStatsT<false>;
+using CharacterStatsBE = CharacterStatsT<true>;
+check_struct_size(CharacterStats, 0x0E);
+check_struct_size(CharacterStatsBE, 0x0E);
 
 template <bool IsBigEndian>
-struct LevelStatsDeltaBase {
+struct PlayerStatsT {
+  using U16T = typename std::conditional<IsBigEndian, be_uint16_t, le_uint16_t>::type;
+  using U32T = typename std::conditional<IsBigEndian, be_uint32_t, le_uint32_t>::type;
+  using F32T = typename std::conditional<IsBigEndian, be_float, le_float>::type;
+
+  /* 00 */ CharacterStatsT<IsBigEndian> char_stats;
+  /* 0E */ U16T esp = 0;
+  /* 10 */ F32T height = 0.0;
+  /* 14 */ F32T unknown_a3 = 0.0;
+  /* 18 */ U32T level = 0;
+  /* 1C */ U32T experience = 0;
+  /* 20 */ U32T meseta = 0;
+  /* 24 */
+
+  operator PlayerStatsT<!IsBigEndian>() const {
+    PlayerStatsT<!IsBigEndian> ret;
+    ret.char_stats = this->char_stats;
+    ret.esp = this->esp.load();
+    ret.height = this->height.load();
+    ret.unknown_a3 = this->unknown_a3.load();
+    ret.level = this->level.load();
+    ret.experience = this->experience.load();
+    ret.meseta = this->meseta.load();
+    return ret;
+  }
+} __packed__;
+using PlayerStats = PlayerStatsT<false>;
+using PlayerStatsBE = PlayerStatsT<true>;
+check_struct_size(PlayerStats, 0x24);
+check_struct_size(PlayerStatsBE, 0x24);
+
+template <bool IsBigEndian>
+struct LevelStatsDeltaT {
   using U32T = typename std::conditional<IsBigEndian, be_uint32_t, le_uint32_t>::type;
 
   /* 00 */ uint8_t atp;
@@ -58,12 +97,11 @@ struct LevelStatsDeltaBase {
     ps.mst += this->mst;
     ps.lck += this->lck;
   }
-} __attribute__((packed));
-
-struct LevelStatsDelta : LevelStatsDeltaBase<false> {
-} __attribute__((packed));
-struct LevelStatsDeltaBE : LevelStatsDeltaBase<true> {
-} __attribute__((packed));
+} __packed__;
+using LevelStatsDelta = LevelStatsDeltaT<false>;
+using LevelStatsDeltaBE = LevelStatsDeltaT<true>;
+check_struct_size(LevelStatsDelta, 0x0C);
+check_struct_size(LevelStatsDeltaBE, 0x0C);
 
 class LevelTable {
   // This is the base class for all the LevelTable implementations. The public
@@ -76,6 +114,9 @@ public:
   virtual const CharacterStats& base_stats_for_class(uint8_t char_class) const = 0;
   virtual const LevelStatsDelta& stats_delta_for_level(uint8_t char_class, uint8_t level) const = 0;
 
+  void reset_to_base(PlayerStats& stats, uint8_t char_class) const;
+  void advance_to_level(PlayerStats& stats, uint32_t level, uint8_t char_class) const;
+
 protected:
   LevelTable() = default;
 };
@@ -89,7 +130,7 @@ public:
     /* 14 */ le_float unknown_a3 = 0.0;
     /* 18 */ le_uint32_t level = 0;
     /* 1C */
-  } __attribute__((packed));
+  } __packed_ws__(Level100Entry, 0x1C);
 
   LevelTableV2(const std::string& data, bool compressed);
   virtual ~LevelTableV2() = default;

src/License.cc

@@ -1,358 +0,0 @@
-#include <inttypes.h>
-#include <stdio.h>
-#include <string.h>
-
-#include <phosg/Filesystem.hh>
-#include <phosg/Hash.hh>
-#include <phosg/Time.hh>
-
-#include "License.hh"
-
-using namespace std;
-
-License::License(const JSON& json)
-    : serial_number(0),
-      flags(0),
-      ban_end_time(0),
-      ep3_current_meseta(0),
-      ep3_total_meseta_earned(0),
-      bb_team_id(0) {
-  this->serial_number = json.get_int("SerialNumber");
-  this->access_key = json.get_string("AccessKey", "");
-  this->gc_password = json.get_string("GCPassword", "");
-  this->xb_gamertag = json.get_string("XBGamerTag", "");
-  this->xb_user_id = json.get_int("XBUserID", 0);
-  this->xb_account_id = json.get_int("XBAccountID", 0);
-  this->bb_username = json.get_string("BBUsername", "");
-  this->bb_password = json.get_string("BBPassword", "");
-  this->flags = json.get_int("Flags", 0);
-  this->ban_end_time = json.get_int("BanEndTime", 0);
-  this->last_player_name = json.get_string("LastPlayerName", "");
-  this->auto_reply_message = json.get_string("AutoReplyMessage", "");
-  this->ep3_current_meseta = json.get_int("Ep3CurrentMeseta", 0);
-  this->ep3_total_meseta_earned = json.get_int("Ep3TotalMesetaEarned", 0);
-  this->bb_team_id = json.get_int("BBTeamID", 0);
-}
-
-JSON License::json() const {
-  return JSON::dict({
-      {"SerialNumber", this->serial_number},
-      {"AccessKey", this->access_key},
-      {"GCPassword", this->gc_password},
-      {"XBGamerTag", this->xb_gamertag},
-      {"XBUserID", this->xb_user_id},
-      {"XBAccountID", this->xb_account_id},
-      {"BBUsername", this->bb_username},
-      {"BBPassword", this->bb_password},
-      {"Flags", this->flags},
-      {"BanEndTime", this->ban_end_time},
-      {"LastPlayerName", this->last_player_name},
-      {"AutoReplyMessage", this->auto_reply_message},
-      {"Ep3CurrentMeseta", this->ep3_current_meseta},
-      {"Ep3TotalMesetaEarned", this->ep3_total_meseta_earned},
-      {"BBTeamID", this->bb_team_id},
-  });
-}
-
-void License::save() const {}
-void License::delete_file() const {}
-
-string License::str() const {
-  vector<string> tokens;
-  tokens.emplace_back(string_printf("serial_number=%010" PRIu32 "/%08" PRIX32, this->serial_number, this->serial_number));
-  if (!this->access_key.empty()) {
-    tokens.emplace_back("access_key=" + this->access_key);
-  }
-  if (!this->gc_password.empty()) {
-    tokens.emplace_back("gc_password=" + this->gc_password);
-  }
-  if (!this->xb_gamertag.empty()) {
-    tokens.emplace_back("xb_gamertag=" + this->xb_gamertag);
-  }
-  if (this->xb_user_id != 0) {
-    tokens.emplace_back(string_printf("xb_user_id=%016" PRIX64, this->xb_user_id));
-  }
-  if (this->xb_account_id != 0) {
-    tokens.emplace_back(string_printf("xb_account_id=%016" PRIX64, this->xb_account_id));
-  }
-  if (!this->bb_username.empty()) {
-    tokens.emplace_back("bb_username=" + this->bb_username);
-  }
-  if (!this->bb_password.empty()) {
-    tokens.emplace_back("bb_password=" + this->bb_password);
-  }
-  tokens.emplace_back(string_printf("flags=%08" PRIX32, this->flags));
-  if (this->ban_end_time) {
-    tokens.emplace_back(string_printf("ban_end_time=%016" PRIX64, this->ban_end_time));
-  }
-  if (this->ep3_current_meseta) {
-    tokens.emplace_back(string_printf("ep3_current_meseta=%" PRIu32, this->ep3_current_meseta));
-  }
-  if (this->ep3_total_meseta_earned) {
-    tokens.emplace_back(string_printf("ep3_total_meseta_earned=%" PRIu32, this->ep3_total_meseta_earned));
-  }
-  return "[License: " + join(tokens, ", ") + "]";
-}
-
-DiskLicense::DiskLicense(const JSON& json) : License(json) {}
-
-void DiskLicense::save() const {
-  auto json = this->json();
-  string json_data = json.serialize(JSON::SerializeOption::FORMAT | JSON::SerializeOption::HEX_INTEGERS);
-  string filename = string_printf("system/licenses/%010" PRIu32 ".json", this->serial_number);
-  save_file(filename, json_data);
-}
-
-void DiskLicense::delete_file() const {
-  string filename = string_printf("system/licenses/%010" PRIu32 ".json", this->serial_number);
-  remove(filename.c_str());
-}
-
-shared_ptr<License> LicenseIndex::create_license() const {
-  return make_shared<License>();
-}
-
-shared_ptr<License> LicenseIndex::create_temporary_license() const {
-  return make_shared<License>();
-}
-
-size_t LicenseIndex::count() const {
-  return this->serial_number_to_license.size();
-}
-
-shared_ptr<License> LicenseIndex::get(uint32_t serial_number) const {
-  try {
-    return this->serial_number_to_license.at(serial_number);
-  } catch (const out_of_range&) {
-    throw missing_license();
-  }
-}
-
-shared_ptr<License> LicenseIndex::get_by_bb_username(const string& bb_username) const {
-  try {
-    return this->bb_username_to_license.at(bb_username);
-  } catch (const out_of_range&) {
-    throw missing_license();
-  }
-}
-
-vector<shared_ptr<License>> LicenseIndex::all() const {
-  vector<shared_ptr<License>> ret;
-  ret.reserve(this->serial_number_to_license.size());
-  for (const auto& it : this->serial_number_to_license) {
-    ret.emplace_back(it.second);
-  }
-  return ret;
-}
-
-void LicenseIndex::add(shared_ptr<License> l) {
-  this->serial_number_to_license[l->serial_number] = l;
-  if (!l->bb_username.empty()) {
-    this->bb_username_to_license[l->bb_username] = l;
-  }
-  if (!l->xb_gamertag.empty()) {
-    this->xb_gamertag_to_license[l->xb_gamertag] = l;
-  }
-}
-
-void LicenseIndex::remove(uint32_t serial_number) {
-  auto l = this->serial_number_to_license.at(serial_number);
-  this->serial_number_to_license.erase(l->serial_number);
-  if (!l->bb_username.empty()) {
-    this->bb_username_to_license.erase(l->bb_username);
-  }
-  if (!l->xb_gamertag.empty()) {
-    this->xb_gamertag_to_license.erase(l->xb_gamertag);
-  }
-}
-
-shared_ptr<License> LicenseIndex::verify_v1_v2(
-    uint32_t serial_number,
-    const string& access_key,
-    const string& character_name) const {
-  if (serial_number == 0) {
-    throw no_username();
-  }
-  try {
-    auto& license = this->serial_number_to_license.at(serial_number);
-    if (license->ban_end_time && (license->ban_end_time >= now())) {
-      throw invalid_argument("user is banned");
-    }
-    if (license->check_flag(License::Flag::IS_SHARED_SERIAL)) {
-      return this->create_temporary_license_for_shared_license(license->flags, serial_number, access_key, "", character_name);
-    }
-    if (license->access_key.compare(0, 8, access_key) != 0) {
-      throw incorrect_access_key();
-    }
-    return license;
-  } catch (const out_of_range&) {
-    throw missing_license();
-  }
-}
-
-shared_ptr<License> LicenseIndex::verify_gc_no_password(
-    uint32_t serial_number,
-    const string& access_key,
-    const string& character_name) const {
-  if (serial_number == 0) {
-    throw no_username();
-  }
-  try {
-    auto& license = this->serial_number_to_license.at(serial_number);
-    if (license->ban_end_time && (license->ban_end_time >= now())) {
-      throw invalid_argument("user is banned");
-    }
-    if (license->check_flag(License::Flag::IS_SHARED_SERIAL)) {
-      return this->create_temporary_license_for_shared_license(license->flags, serial_number, access_key, "", character_name);
-    }
-    if (license->access_key != access_key) {
-      throw incorrect_access_key();
-    }
-    return license;
-  } catch (const out_of_range&) {
-    throw missing_license();
-  }
-}
-
-shared_ptr<License> LicenseIndex::verify_gc_with_password(
-    uint32_t serial_number,
-    const string& access_key,
-    const string& password,
-    const string& character_name) const {
-  if (serial_number == 0) {
-    throw no_username();
-  }
-  try {
-    auto& license = this->serial_number_to_license.at(serial_number);
-    if (license->ban_end_time && (license->ban_end_time >= now())) {
-      throw invalid_argument("user is banned");
-    }
-    if (license->check_flag(License::Flag::IS_SHARED_SERIAL)) {
-      return this->create_temporary_license_for_shared_license(license->flags, serial_number, access_key, password, character_name);
-    }
-    if (license->access_key != access_key) {
-      throw incorrect_access_key();
-    }
-    if (license->gc_password != password) {
-      throw incorrect_password();
-    }
-    return license;
-  } catch (const out_of_range&) {
-    throw missing_license();
-  }
-}
-
-shared_ptr<License> LicenseIndex::verify_xb(const string& gamertag, uint64_t user_id, uint64_t account_id) const {
-  if (user_id == 0 || account_id == 0) {
-    throw incorrect_access_key();
-  }
-  try {
-    auto& license = this->xb_gamertag_to_license.at(gamertag);
-    if (license->check_flag(License::Flag::IS_SHARED_SERIAL)) {
-      throw missing_license(); // XB users cannot use shared serials
-    }
-    if (license->ban_end_time && (license->ban_end_time >= now())) {
-      throw invalid_argument("user is banned");
-    }
-    if (license->xb_user_id && (license->xb_user_id != user_id)) {
-      throw incorrect_access_key();
-    }
-    if (license->xb_account_id && (license->xb_account_id != account_id)) {
-      throw incorrect_access_key();
-    }
-    return license;
-  } catch (const out_of_range&) {
-    throw missing_license();
-  }
-}
-
-shared_ptr<License> LicenseIndex::verify_bb(const string& username, const string& password) const {
-  if (username.empty() || password.empty()) {
-    throw no_username();
-  }
-  try {
-    auto& license = this->bb_username_to_license.at(username);
-    if (license->check_flag(License::Flag::IS_SHARED_SERIAL)) {
-      throw missing_license(); // BB users cannot use shared serials
-    }
-    if (license->ban_end_time && (license->ban_end_time >= now())) {
-      throw invalid_argument("user is banned");
-    }
-    if (license->bb_password != password) {
-      throw incorrect_password();
-    }
-    return license;
-  } catch (const out_of_range&) {
-    throw missing_license();
-  }
-}
-
-shared_ptr<License> LicenseIndex::create_temporary_license_for_shared_license(
-    uint32_t base_flags,
-    uint32_t serial_number,
-    const string& access_key,
-    const string& password,
-    const string& character_name) const {
-  uint32_t temp_serial_number = fnv1a32(&serial_number, sizeof(serial_number));
-  temp_serial_number = fnv1a32(access_key, temp_serial_number);
-  temp_serial_number = fnv1a32(password, temp_serial_number);
-  temp_serial_number = fnv1a32(character_name, temp_serial_number);
-  auto ret = this->create_temporary_license();
-  ret->serial_number = temp_serial_number & 0x7FFFFFFF;
-  ret->flags = base_flags;
-  ret->set_flag(License::Flag::IS_SHARED_SERIAL);
-  return ret;
-}
-
-DiskLicenseIndex::DiskLicenseIndex() {
-  struct BinaryLicense {
-    pstring<TextEncoding::ASCII, 0x14> username; // BB username (max. 16 chars; should technically be Unicode)
-    pstring<TextEncoding::ASCII, 0x14> bb_password; // BB password (max. 16 chars)
-    uint32_t serial_number; // PC/GC serial number. MUST BE PRESENT FOR BB LICENSES TOO; this is also the player's guild card number.
-    pstring<TextEncoding::ASCII, 0x10> access_key; // PC/GC access key. (to log in using PC on a GC license, just enter the first 8 characters of the GC access key)
-    pstring<TextEncoding::ASCII, 0x0C> gc_password; // GC password
-    uint32_t privileges; // privilege level
-    uint64_t ban_end_time; // end time of ban (zero = not banned)
-  } __attribute__((packed));
-
-  if (!isdir("system/licenses")) {
-    mkdir("system/licenses", 0755);
-  }
-
-  // Convert binary licenses to JSON licenses and save them
-  if (isfile("system/licenses.nsi")) {
-    auto bin_licenses = load_vector_file<BinaryLicense>("system/licenses.nsi");
-    for (const auto& bin_license : bin_licenses) {
-      // Only add licenses from the binary file if there isn't a JSON version of
-      // the same license
-      try {
-        this->get(bin_license.serial_number);
-      } catch (const missing_license&) {
-        License license;
-        license.serial_number = bin_license.serial_number;
-        license.access_key = bin_license.access_key.decode();
-        license.gc_password = bin_license.gc_password.decode();
-        license.bb_username = bin_license.username.decode();
-        license.bb_password = bin_license.bb_password.decode();
-        license.flags = bin_license.privileges;
-        license.ban_end_time = bin_license.ban_end_time;
-        license.ep3_current_meseta = 0;
-        license.ep3_total_meseta_earned = 0;
-        license.save();
-      }
-    }
-    ::remove("system/licenses.nsi");
-  }
-
-  for (const auto& item : list_directory("system/licenses")) {
-    if (ends_with(item, ".json")) {
-      JSON json = JSON::parse(load_file("system/licenses/" + item));
-      auto license = make_shared<DiskLicense>(json);
-      this->add(license);
-    }
-  }
-}
-
-shared_ptr<License> DiskLicenseIndex::create_license() const {
-  return make_shared<DiskLicense>();
-}

src/License.hh

@@ -1,161 +0,0 @@
-#pragma once
-
-#include <memory>
-#include <phosg/JSON.hh>
-#include <string>
-#include <unordered_map>
-#include <vector>
-
-#include "Text.hh"
-
-class LicenseIndex;
-
-class License {
-public:
-  enum class Flag : uint32_t {
-    // clang-format off
-    KICK_USER                  = 0x00000001,
-    BAN_USER                   = 0x00000002,
-    SILENCE_USER               = 0x00000004,
-    CHANGE_EVENT               = 0x00000010,
-    ANNOUNCE                   = 0x00000020,
-    FREE_JOIN_GAMES            = 0x00000040,
-    DEBUG                      = 0x01000000,
-    CHEAT_ANYWHERE             = 0x02000000,
-    DISABLE_QUEST_REQUIREMENTS = 0x04000000,
-    MODERATOR                  = 0x00000007,
-    ADMINISTRATOR              = 0x000000FF,
-    ROOT                       = 0x7FFFFFFF,
-    IS_SHARED_SERIAL           = 0x80000000,
-    // NOTE: When adding or changing license flags, don't forget to change the
-    // documentation in the shell's help text.
-
-    UNUSED_BITS                = 0x78FFFF00,
-    // clang-format on
-  };
-
-  uint32_t serial_number = 0;
-  std::string access_key;
-  std::string gc_password;
-  std::string xb_gamertag;
-  uint64_t xb_user_id = 0;
-  uint64_t xb_account_id = 0;
-  std::string bb_username;
-  std::string bb_password;
-
-  uint32_t flags = 0;
-  uint64_t ban_end_time = 0; // 0 = not banned
-  std::string last_player_name;
-  std::string auto_reply_message;
-
-  uint32_t ep3_current_meseta = 0;
-  uint32_t ep3_total_meseta_earned = 0;
-
-  uint32_t bb_team_id = 0;
-
-  License() = default;
-  explicit License(const JSON& json);
-  virtual ~License() = default;
-
-  JSON json() const;
-  virtual void save() const;
-  virtual void delete_file() const;
-
-  [[nodiscard]] inline bool check_flag(Flag flag) const {
-    return !!(this->flags & static_cast<uint32_t>(flag));
-  }
-  inline void set_flag(Flag flag) {
-    this->flags |= static_cast<uint32_t>(flag);
-  }
-  inline void clear_flag(Flag flag) {
-    this->flags &= (~static_cast<uint32_t>(flag));
-  }
-  inline void toggle_flag(Flag flag) {
-    this->flags ^= static_cast<uint32_t>(flag);
-  }
-  inline void replace_all_flags(Flag mask) {
-    this->flags = static_cast<uint32_t>(mask);
-  }
-
-  std::string str() const;
-};
-
-class DiskLicense : public License {
-public:
-  DiskLicense() = default;
-  explicit DiskLicense(const JSON& json);
-  virtual ~DiskLicense() = default;
-
-  virtual void save() const;
-  virtual void delete_file() const;
-};
-
-class LicenseIndex {
-public:
-  class no_username : public std::invalid_argument {
-  public:
-    no_username() : invalid_argument("serial number is zero or username is missing") {}
-  };
-  class incorrect_password : public std::invalid_argument {
-  public:
-    incorrect_password() : invalid_argument("incorrect password") {}
-  };
-  class incorrect_access_key : public std::invalid_argument {
-  public:
-    incorrect_access_key() : invalid_argument("incorrect access key") {}
-  };
-  class missing_license : public std::invalid_argument {
-  public:
-    missing_license() : invalid_argument("missing license") {}
-  };
-
-  LicenseIndex() = default;
-  virtual ~LicenseIndex() = default;
-
-  virtual std::shared_ptr<License> create_license() const;
-  virtual std::shared_ptr<License> create_temporary_license() const;
-
-  size_t count() const;
-  std::shared_ptr<License> get(uint32_t serial_number) const;
-  std::shared_ptr<License> get_by_bb_username(const std::string& bb_username) const;
-  std::vector<std::shared_ptr<License>> all() const;
-
-  void add(std::shared_ptr<License> l);
-  void remove(uint32_t serial_number);
-
-  std::shared_ptr<License> verify_v1_v2(
-      uint32_t serial_number,
-      const std::string& access_key,
-      const std::string& character_name) const;
-  std::shared_ptr<License> verify_gc_no_password(
-      uint32_t serial_number,
-      const std::string& access_key,
-      const std::string& character_name) const;
-  std::shared_ptr<License> verify_gc_with_password(
-      uint32_t serial_number,
-      const std::string& access_key,
-      const std::string& password,
-      const std::string& character_name) const;
-  std::shared_ptr<License> verify_xb(const std::string& gamertag, uint64_t user_id, uint64_t account_id) const;
-  std::shared_ptr<License> verify_bb(const std::string& username, const std::string& password) const;
-
-protected:
-  std::unordered_map<std::string, std::shared_ptr<License>> bb_username_to_license;
-  std::unordered_map<std::string, std::shared_ptr<License>> xb_gamertag_to_license;
-  std::unordered_map<uint32_t, std::shared_ptr<License>> serial_number_to_license;
-
-  std::shared_ptr<License> create_temporary_license_for_shared_license(
-      uint32_t base_flags,
-      uint32_t serial_number,
-      const std::string& access_key,
-      const std::string& password,
-      const std::string& character_name) const;
-};
-
-class DiskLicenseIndex : public LicenseIndex {
-public:
-  DiskLicenseIndex();
-  virtual ~DiskLicenseIndex() = default;
-
-  virtual std::shared_ptr<License> create_license() const;
-};

src/Lobby.cc

@@ -12,7 +12,7 @@
 using namespace std;
 
 bool Lobby::FloorItem::visible_to_client(uint8_t client_id) const {
-  return this->visibility_flags & (1 << client_id);
+  return this->flags & (1 << client_id);
 }
 
 Lobby::FloorItemManager::FloorItemManager(uint32_t lobby_id, uint8_t floor)
@@ -27,18 +27,18 @@ shared_ptr<Lobby::FloorItem> Lobby::FloorItemManager::find(uint32_t item_id) con
   return this->items.at(item_id);
 }
 
-void Lobby::FloorItemManager::add(const ItemData& item, float x, float z, uint16_t visibility_flags) {
+void Lobby::FloorItemManager::add(const ItemData& item, float x, float z, uint16_t flags) {
   auto fi = make_shared<FloorItem>();
   fi->data = item;
   fi->x = x;
   fi->z = z;
   fi->drop_number = this->next_drop_number++;
-  fi->visibility_flags = visibility_flags & 0x0FFF;
+  fi->flags = flags;
   this->add(fi);
 }
 
 void Lobby::FloorItemManager::add(shared_ptr<Lobby::FloorItem> fi) {
-  if (fi->visibility_flags == 0) {
+  if (fi->flags == 0) {
     throw logic_error("floor item is not visible to any player");
   }
 
@@ -51,8 +51,8 @@ void Lobby::FloorItemManager::add(shared_ptr<Lobby::FloorItem> fi) {
       this->queue_for_client[z].emplace(fi->drop_number, fi);
     }
   }
-  this->log.info("Added floor item %08" PRIX32 " at %g, %g with drop number %" PRIu64 " visible to %03hX",
-      fi->data.id.load(), fi->x, fi->z, fi->drop_number, fi->visibility_flags);
+  this->log.info("Added floor item %08" PRIX32 " at %g, %g with drop number %" PRIu64 " with flags %03hX",
+      fi->data.id.load(), fi->x, fi->z, fi->drop_number, fi->flags);
 }
 
 std::shared_ptr<Lobby::FloorItem> Lobby::FloorItemManager::remove(uint32_t item_id, uint8_t client_id) {
@@ -70,8 +70,8 @@ std::shared_ptr<Lobby::FloorItem> Lobby::FloorItemManager::remove(uint32_t item_
     }
   }
   this->items.erase(item_it);
-  this->log.info("Removed floor item %08" PRIX32 " at %g, %g with drop number %" PRIu64 " visible to %03hX",
-      fi->data.id.load(), fi->x, fi->z, fi->drop_number, fi->visibility_flags);
+  this->log.info("Removed floor item %08" PRIX32 " at %g, %g with drop number %" PRIu64 " with flags %03hX",
+      fi->data.id.load(), fi->x, fi->z, fi->drop_number, fi->flags);
   return fi;
 }
 
@@ -89,7 +89,7 @@ std::unordered_set<std::shared_ptr<Lobby::FloorItem>> Lobby::FloorItemManager::e
 void Lobby::FloorItemManager::clear_inaccessible(uint16_t remaining_clients_mask) {
   unordered_set<uint32_t> item_ids_to_delete;
   for (const auto& it : this->items) {
-    if ((it.second->visibility_flags & remaining_clients_mask) == 0) {
+    if ((it.second->flags & remaining_clients_mask) == 0) {
       item_ids_to_delete.emplace(it.first);
     }
   }
@@ -102,7 +102,7 @@ void Lobby::FloorItemManager::clear_inaccessible(uint16_t remaining_clients_mask
 void Lobby::FloorItemManager::clear_private() {
   unordered_set<uint32_t> item_ids_to_delete;
   for (const auto& it : this->items) {
-    if ((it.second->visibility_flags & 0x00F) != 0x00F) {
+    if ((it.second->flags & 0x00F) != 0x00F) {
       item_ids_to_delete.emplace(it.first);
     }
   }
@@ -144,7 +144,7 @@ Lobby::Lobby(shared_ptr<ServerState> s, uint32_t id, bool is_game)
       next_game_item_id(0xCC000000),
       base_version(Version::GC_V3),
       allowed_versions(0x0000),
-      section_id(0),
+      override_section_id(0xFF),
       episode(Episode::NONE),
       mode(GameMode::NORMAL),
       difficulty(0),
@@ -252,15 +252,44 @@ void Lobby::create_item_creator() {
       s->weapon_random_sets.at(this->difficulty),
       s->tekker_adjustment_set,
       s->item_parameter_table(this->base_version),
-      this->base_version,
+      s->item_stack_limits(this->base_version),
       this->episode,
       (this->mode == GameMode::SOLO) ? GameMode::NORMAL : this->mode,
       this->difficulty,
-      this->section_id,
-      this->random_seed,
+      this->effective_section_id(),
+      this->opt_rand_crypt,
       this->quest ? this->quest->battle_rules : nullptr);
 }
 
+void Lobby::change_section_id() {
+  if (this->item_creator) {
+    uint8_t new_section_id = this->effective_section_id();
+    if (this->item_creator->get_section_id() != new_section_id) {
+      this->log.info("Changing section ID to %s", name_for_section_id(new_section_id));
+      this->item_creator->set_section_id(new_section_id);
+      for (const auto& c : this->clients) {
+        if (c && c->config.check_flag(Client::Flag::DEBUG_ENABLED)) {
+          send_text_message_printf(c, "$C5Section ID changed\nto %s (%hhu)", name_for_section_id(new_section_id), new_section_id);
+        }
+      }
+    }
+  }
+}
+
+uint8_t Lobby::effective_section_id() const {
+  if (this->override_section_id != 0xFF) {
+    return this->override_section_id;
+  }
+  if (this->check_flag(Lobby::Flag::USE_CREATOR_SECTION_ID)) {
+    return this->creator_section_id;
+  }
+  auto leader = this->clients.at(this->leader_id);
+  if (leader) {
+    return leader->character()->disp.visual.section_id;
+  }
+  return 0;
+}
+
 shared_ptr<Map> Lobby::load_maps(
     Version version,
     Episode episode,
@@ -268,18 +297,16 @@ shared_ptr<Map> Lobby::load_maps(
     uint8_t event,
     uint32_t lobby_id,
     shared_ptr<const Map::RareEnemyRates> rare_rates,
-    shared_ptr<PSOLFGEncryption> random_crypt,
-    shared_ptr<const VersionedQuest> vq) {
-  if (!vq->dat_contents_decompressed) {
-    throw runtime_error("quest does not have DAT data");
-  }
-  auto map = make_shared<Map>(version, lobby_id, random_crypt);
-  map->add_enemies_and_objects_from_quest_data(
+    uint32_t random_seed,
+    shared_ptr<PSOLFGEncryption> opt_rand_crypt,
+    shared_ptr<const string> quest_dat_contents_decompressed) {
+  auto map = make_shared<Map>(version, lobby_id, random_seed, opt_rand_crypt);
+  map->add_entities_from_quest_data(
       episode,
       difficulty,
       event,
-      vq->dat_contents_decompressed->data(),
-      vq->dat_contents_decompressed->size(),
+      quest_dat_contents_decompressed->data(),
+      quest_dat_contents_decompressed->size(),
       rare_rates);
   return map;
 }
@@ -294,17 +321,34 @@ shared_ptr<Map> Lobby::load_maps(
     shared_ptr<const SetDataTableBase> sdt,
     function<shared_ptr<const string>(Version, const string&)> get_file_data,
     shared_ptr<const Map::RareEnemyRates> rare_rates,
-    shared_ptr<PSOLFGEncryption> random_crypt,
+    uint32_t random_seed,
+    shared_ptr<PSOLFGEncryption> opt_rand_crypt,
     const parray<le_uint32_t, 0x20>& variations,
     const PrefixedLogger* log) {
-  auto enemy_filenames = sdt->map_filenames_for_variations(variations, episode, mode, true);
-  auto object_filenames = sdt->map_filenames_for_variations(variations, episode, mode, false);
-  return Lobby::load_maps(enemy_filenames, object_filenames, version, episode, mode, difficulty, event, lobby_id, get_file_data, rare_rates, random_crypt, log);
+  auto enemy_filenames = sdt->map_filenames_for_variations(variations, episode, mode, SetDataTable::FilenameType::ENEMIES);
+  auto object_filenames = sdt->map_filenames_for_variations(variations, episode, mode, SetDataTable::FilenameType::OBJECTS);
+  auto event_filenames = sdt->map_filenames_for_variations(variations, episode, mode, SetDataTable::FilenameType::EVENTS);
+  return Lobby::load_maps(
+      enemy_filenames,
+      object_filenames,
+      event_filenames,
+      version,
+      episode,
+      mode,
+      difficulty,
+      event,
+      lobby_id,
+      get_file_data,
+      rare_rates,
+      random_seed,
+      opt_rand_crypt,
+      log);
 }
 
 shared_ptr<Map> Lobby::load_maps(
     const vector<string>& enemy_filenames,
     const vector<string>& object_filenames,
+    const vector<string>& event_filenames,
     Version version,
     Episode episode,
     GameMode mode,
@@ -313,9 +357,10 @@ shared_ptr<Map> Lobby::load_maps(
     uint32_t lobby_id,
     function<shared_ptr<const string>(Version, const string&)> get_file_data,
     shared_ptr<const Map::RareEnemyRates> rare_rates,
-    shared_ptr<PSOLFGEncryption> random_crypt,
+    uint32_t rare_seed,
+    shared_ptr<PSOLFGEncryption> opt_rand_crypt,
     const PrefixedLogger* log) {
-  auto map = make_shared<Map>(version, lobby_id, random_crypt);
+  auto map = make_shared<Map>(version, lobby_id, rare_seed, opt_rand_crypt);
 
   // Don't load free-roam maps in Challenge mode, since players can't go to
   // Ragol without a quest loaded
@@ -360,6 +405,21 @@ shared_ptr<Map> Lobby::load_maps(
     } else if (log) {
       log->info("No objects to load for floor %02zX", floor);
     }
+
+    const auto& floor_event_filename = event_filenames.at(floor);
+    if (!floor_event_filename.empty()) {
+      auto map_data = get_file_data(version, floor_event_filename);
+      if (map_data) {
+        map->add_events_from_map_data(floor, map_data->data(), map_data->size());
+        if (log) {
+          log->info("Loaded events map %s for floor %02zX", floor_event_filename.c_str(), floor);
+        }
+      } else if (log) {
+        log->info("Events map %s for floor %02zX cannot be used; skipping", floor_event_filename.c_str(), floor);
+      }
+    } else if (log) {
+      log->info("No events to load for floor %02zX", floor);
+    }
   }
 
   return map;
@@ -377,6 +437,9 @@ void Lobby::load_maps() {
     }
 
     auto vq = this->quest->version(this->base_version, leader_c->language());
+    if (!vq->dat_contents_decompressed) {
+      throw runtime_error("quest does not have DAT data");
+    }
     this->map = this->load_maps(
         this->base_version,
         this->episode,
@@ -384,8 +447,9 @@ void Lobby::load_maps() {
         this->event,
         this->lobby_id,
         rare_rates,
-        this->random_crypt,
-        vq);
+        this->random_seed,
+        this->opt_rand_crypt,
+        vq->dat_contents_decompressed);
 
   } else if (this->mode != GameMode::CHALLENGE) {
     auto s = this->require_server_state();
@@ -399,12 +463,13 @@ void Lobby::load_maps() {
         s->set_data_table(this->base_version, this->episode, this->mode, this->difficulty),
         bind(&ServerState::load_map_file, s.get(), placeholders::_1, placeholders::_2),
         rare_rates,
-        this->random_crypt,
+        this->random_seed,
+        this->opt_rand_crypt,
         this->variations,
         &this->log);
 
   } else {
-    this->map = make_shared<Map>(this->base_version, this->lobby_id, this->random_crypt);
+    this->map = make_shared<Map>(this->base_version, this->lobby_id, this->random_seed, this->opt_rand_crypt);
   }
 
   this->log.info("Generated objects list (%zu entries):", this->map->objects.size());
@@ -417,6 +482,11 @@ void Lobby::load_maps() {
     string e_str = this->map->enemies[z].str();
     this->log.info("(E-%zX) %s", z, e_str.c_str());
   }
+  this->log.info("Generated events list (%zu entries):", this->map->events.size());
+  for (size_t z = 0; z < this->map->events.size(); z++) {
+    string e_str = this->map->events[z].str();
+    this->log.info("%s", e_str.c_str());
+  }
   this->log.info("Loaded maps contain %zu object entries and %zu enemy entries overall (%zu as rares)",
       this->map->objects.size(), this->map->enemies.size(), this->map->rare_enemy_indexes.size());
 }
@@ -434,7 +504,7 @@ void Lobby::create_ep3_server() {
       .card_index = is_nte ? s->ep3_card_index_trial : s->ep3_card_index,
       .map_index = s->ep3_map_index,
       .behavior_flags = s->ep3_behavior_flags,
-      .random_crypt = this->random_crypt,
+      .opt_rand_crypt = this->opt_rand_crypt,
       .tournament = tourn,
       .trap_card_ids = s->ep3_trap_card_ids,
   };
@@ -452,8 +522,9 @@ void Lobby::reassign_leader_on_client_departure(size_t leaving_client_index) {
     if (x == leaving_client_index) {
       continue;
     }
-    if (this->clients[x].get()) {
+    if (this->clients[x]) {
       this->leader_id = x;
+      this->change_section_id();
       return;
     }
   }
@@ -478,14 +549,27 @@ bool Lobby::any_client_loading() const {
 size_t Lobby::count_clients() const {
   size_t ret = 0;
   for (size_t x = 0; x < this->max_clients; x++) {
-    if (this->clients[x].get()) {
+    if (this->clients[x]) {
       ret++;
     }
   }
   return ret;
 }
 
+bool Lobby::any_v1_clients_present() const {
+  for (size_t x = 0; x < this->max_clients; x++) {
+    if (this->clients[x] && is_v1(this->clients[x]->version())) {
+      return true;
+    }
+  }
+  return false;
+}
+
 void Lobby::add_client(shared_ptr<Client> c, ssize_t required_client_id) {
+  if (!c->login) {
+    throw runtime_error("client is not logged in");
+  }
+
   ssize_t index;
   ssize_t min_client_id = this->check_flag(Lobby::Flag::IS_SPECTATOR_TEAM) ? 4 : 0;
 
@@ -531,6 +615,7 @@ void Lobby::add_client(shared_ptr<Client> c, ssize_t required_client_id) {
   }
   if (leader_index >= this->max_clients) {
     this->leader_id = c->lobby_client_id;
+    this->change_section_id();
   }
 
   // If this is a lobby or no one was here before this, reassign all the floor
@@ -564,12 +649,12 @@ void Lobby::add_client(shared_ptr<Client> c, ssize_t required_client_id) {
     auto p = c->character();
     PlayerLobbyDataDCGC lobby_data;
     lobby_data.player_tag = 0x00010000;
-    lobby_data.guild_card_number = c->license->serial_number;
+    lobby_data.guild_card_number = c->login->account->account_id;
     lobby_data.name.encode(p->disp.name.decode(c->language()), c->language());
     this->battle_record->add_player(
         lobby_data,
         p->inventory,
-        p->disp.to_dcpcv3(c->language(), c->language()),
+        p->disp.to_dcpcv3<false>(c->language(), c->language()),
         c->ep3_config ? (c->ep3_config->online_clv_exp / 100) : 0);
   }
 
@@ -686,14 +771,13 @@ void Lobby::move_client_to_lobby(
   dest_lobby->add_client(c, required_client_id);
 }
 
-shared_ptr<Client> Lobby::find_client(const string* identifier, uint64_t serial_number) {
+shared_ptr<Client> Lobby::find_client(const string* identifier, uint64_t account_id) {
   for (size_t x = 0; x < this->max_clients; x++) {
     auto lc = this->clients[x];
     if (!lc) {
       continue;
     }
-    if (serial_number && lc->license &&
-        (lc->license->serial_number == serial_number)) {
+    if (account_id && lc->login && (lc->login->account->account_id == account_id)) {
       return lc;
     }
     if (identifier && (lc->character()->disp.name.eq(*identifier, lc->language()))) {
@@ -721,7 +805,7 @@ Lobby::JoinError Lobby::join_error_for_client(std::shared_ptr<Client> c, const s
     if (this->mode == GameMode::SOLO) {
       return JoinError::SOLO;
     }
-    if (!c->license->check_flag(License::Flag::FREE_JOIN_GAMES)) {
+    if (!c->login->account->check_flag(Account::Flag::FREE_JOIN_GAMES)) {
       if (password && !this->password.empty() && (*password != this->password)) {
         return JoinError::INCORRECT_PASSWORD;
       }
@@ -734,8 +818,10 @@ Lobby::JoinError Lobby::join_error_for_client(std::shared_ptr<Client> c, const s
       }
       if (this->quest) {
         size_t num_clients = this->count_clients() + 1;
-        if (!c->can_see_quest(this->quest, this->event, this->difficulty, num_clients) ||
-            !c->can_play_quest(this->quest, this->event, this->difficulty, num_clients)) {
+        bool v1_present = is_v1(c->version()) || this->any_v1_clients_present();
+        auto this_sh = this->shared_from_this();
+        if (!c->can_see_quest(this->quest, this_sh, this->event, this->difficulty, num_clients, v1_present) ||
+            !c->can_play_quest(this->quest, this_sh, this->event, this->difficulty, num_clients, v1_present)) {
           return JoinError::NO_ACCESS_TO_QUEST;
         }
       }
@@ -749,19 +835,6 @@ Lobby::JoinError Lobby::join_error_for_client(std::shared_ptr<Client> c, const s
   return JoinError::ALLOWED;
 }
 
-uint8_t Lobby::game_event_for_lobby_event(uint8_t lobby_event) {
-  if (lobby_event > 7) {
-    return 0;
-  }
-  if (lobby_event == 7) {
-    return 2;
-  }
-  if (lobby_event == 2) {
-    return 0;
-  }
-  return lobby_event;
-}
-
 bool Lobby::item_exists(uint8_t floor, uint32_t item_id) const {
   if (floor >= this->floor_item_managers.size()) {
     return false;
@@ -773,9 +846,9 @@ shared_ptr<Lobby::FloorItem> Lobby::find_item(uint8_t floor, uint32_t item_id) c
   return this->floor_item_managers.at(floor).find(item_id);
 }
 
-void Lobby::add_item(uint8_t floor, const ItemData& data, float x, float z, uint16_t visibility_flags) {
+void Lobby::add_item(uint8_t floor, const ItemData& data, float x, float z, uint16_t flags) {
   auto& m = this->floor_item_managers.at(floor);
-  m.add(data, x, z, visibility_flags);
+  m.add(data, x, z, flags);
   this->evict_items_from_floor(floor);
 }
 
@@ -845,11 +918,11 @@ void Lobby::assign_inventory_and_bank_item_ids(shared_ptr<Client> c, bool consum
   }
 }
 
-unordered_map<uint32_t, shared_ptr<Client>> Lobby::clients_by_serial_number() const {
+unordered_map<uint32_t, shared_ptr<Client>> Lobby::clients_by_account_id() const {
   unordered_map<uint32_t, shared_ptr<Client>> ret;
   for (auto c : this->clients) {
-    if (c) {
-      ret.emplace(c->license->serial_number, c);
+    if (c && c->login) {
+      ret.emplace(c->login->account->account_id, c);
     }
   }
   return ret;
@@ -857,13 +930,15 @@ unordered_map<uint32_t, shared_ptr<Client>> Lobby::clients_by_serial_number() co
 
 QuestIndex::IncludeCondition Lobby::quest_include_condition() const {
   size_t num_players = this->count_clients();
-  return [this, num_players](shared_ptr<const Quest> q) -> QuestIndex::IncludeState {
+  bool v1_present = this->any_v1_clients_present();
+  return [this, num_players, v1_present](shared_ptr<const Quest> q) -> QuestIndex::IncludeState {
     bool is_enabled = true;
     for (const auto& lc : this->clients) {
-      if (lc && !lc->can_see_quest(q, this->event, this->difficulty, num_players)) {
+      auto this_sh = this->shared_from_this();
+      if (lc && !lc->can_see_quest(q, this_sh, this->event, this->difficulty, num_players, v1_present)) {
         return QuestIndex::IncludeState::HIDDEN;
       }
-      if (lc && !lc->can_play_quest(q, this->event, this->difficulty, num_players)) {
+      if (lc && !lc->can_play_quest(q, this_sh, this->event, this->difficulty, num_players, v1_present)) {
         is_enabled = false;
       }
     }

src/Lobby.hh

@@ -29,7 +29,13 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
     float x;
     float z;
     uint64_t drop_number;
-    uint16_t visibility_flags;
+    // The low 12 bits of flags are visibility flags, specifying which clients
+    // can see the item. (In practice, only the lowest 4 of these bits are used,
+    // but the game has fields for 12 players so we do too.)
+    // The 13th bit (0x1000) specifies whether a rare item notification should
+    // be sent to all players when the item is picked up. This has no effect for
+    // non-rare items.
+    uint16_t flags;
 
     bool visible_to_client(uint8_t client_id) const;
   };
@@ -46,7 +52,7 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
 
     bool exists(uint32_t item_id) const;
     std::shared_ptr<FloorItem> find(uint32_t item_id) const;
-    void add(const ItemData& item, float x, float z, uint16_t visibility_flags);
+    void add(const ItemData& item, float x, float z, uint16_t flags);
     void add(std::shared_ptr<FloorItem> fi);
     std::shared_ptr<FloorItem> remove(uint32_t item_id, uint8_t client_id);
     std::unordered_set<std::shared_ptr<FloorItem>> evict();
@@ -56,23 +62,24 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
     uint32_t reassign_all_item_ids(uint32_t next_item_id);
   };
   enum class Flag {
-    GAME = 0x00000001,
-    PERSISTENT = 0x00000002,
-
+    // clang-format off
+    GAME                            = 0x00000001,
+    PERSISTENT                      = 0x00000002,
     // Flags used only for games
-    CHEATS_ENABLED = 0x00000100,
-    QUEST_IN_PROGRESS = 0x00000200,
-    BATTLE_IN_PROGRESS = 0x00000400,
-    JOINABLE_QUEST_IN_PROGRESS = 0x00000800,
-    IS_SPECTATOR_TEAM = 0x00002000, // episode must be EP3 also
-    SPECTATORS_FORBIDDEN = 0x00004000,
+    CHEATS_ENABLED                  = 0x00000100,
+    QUEST_IN_PROGRESS               = 0x00000200,
+    BATTLE_IN_PROGRESS              = 0x00000400,
+    JOINABLE_QUEST_IN_PROGRESS      = 0x00000800,
+    IS_SPECTATOR_TEAM               = 0x00002000, // episode must be EP3 also
+    SPECTATORS_FORBIDDEN            = 0x00004000,
     START_BATTLE_PLAYER_IMMEDIATELY = 0x00008000,
-    CANNOT_CHANGE_CHEAT_MODE = 0x00010000,
-
+    CANNOT_CHANGE_CHEAT_MODE        = 0x00010000,
+    USE_CREATOR_SECTION_ID          = 0x00020000,
     // Flags used only for lobbies
-    PUBLIC = 0x01000000,
-    DEFAULT = 0x02000000,
-    IS_OVERFLOW = 0x08000000,
+    PUBLIC                          = 0x01000000,
+    DEFAULT                         = 0x02000000,
+    IS_OVERFLOW                     = 0x08000000,
+    // clang-format on
   };
   enum class DropMode {
     DISABLED = 0,
@@ -91,15 +98,16 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
   uint32_t min_level;
   uint32_t max_level;
 
-  // Item state
+  // Game state
   std::array<uint32_t, 12> next_item_id_for_client;
   uint32_t next_game_item_id;
   std::vector<FloorItemManager> floor_item_managers;
-
-  // Map state
   std::shared_ptr<const Map::RareEnemyRates> rare_enemy_rates;
   std::shared_ptr<Map> map;
   parray<le_uint32_t, 0x20> variations;
+  std::unique_ptr<QuestFlags> quest_flags_known; // If null, ALL quest flags are known
+  std::unique_ptr<QuestFlags> quest_flag_values;
+  std::unique_ptr<SwitchFlags> switch_flags;
 
   // Game config
   Version base_version;
@@ -107,7 +115,8 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
   // bits are indexed as (1 << version), where version is a value from the
   // Version enum.
   uint16_t allowed_versions;
-  uint8_t section_id;
+  uint8_t creator_section_id;
+  uint8_t override_section_id;
   Episode episode;
   GameMode mode;
   uint8_t difficulty; // 0-3
@@ -117,7 +126,7 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
   std::string name;
   // This seed is also sent to the client for rare enemy generation
   uint32_t random_seed;
-  std::shared_ptr<PSOLFGEncryption> random_crypt;
+  std::shared_ptr<PSOLFGEncryption> opt_rand_crypt;
   uint8_t allowed_drop_modes;
   DropMode drop_mode;
   std::shared_ptr<ItemCreator> item_creator;
@@ -195,6 +204,8 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
   std::shared_ptr<ChallengeParameters> require_challenge_params() const;
   void set_drop_mode(DropMode new_mode);
   void create_item_creator();
+  void change_section_id();
+  uint8_t effective_section_id() const;
   static std::shared_ptr<Map> load_maps(
       Version version,
       Episode episode,
@@ -202,8 +213,9 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
       uint8_t event,
       uint32_t lobby_id,
       std::shared_ptr<const Map::RareEnemyRates> rare_rates,
-      std::shared_ptr<PSOLFGEncryption> random_crypt,
-      std::shared_ptr<const VersionedQuest> vq);
+      uint32_t random_seed,
+      std::shared_ptr<PSOLFGEncryption> opt_rand_crypt,
+      std::shared_ptr<const std::string> quest_dat_contents_decompressed);
   static std::shared_ptr<Map> load_maps(
       Version version,
       Episode episode,
@@ -214,12 +226,14 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
       std::shared_ptr<const SetDataTableBase> sdt,
       std::function<std::shared_ptr<const std::string>(Version, const std::string&)> get_file_data,
       std::shared_ptr<const Map::RareEnemyRates> rare_rates,
-      std::shared_ptr<PSOLFGEncryption> random_crypt,
+      uint32_t random_seed,
+      std::shared_ptr<PSOLFGEncryption> opt_rand_crypt,
       const parray<le_uint32_t, 0x20>& variations,
       const PrefixedLogger* log = nullptr);
   static std::shared_ptr<Map> load_maps(
       const std::vector<std::string>& enemy_filenames,
       const std::vector<std::string>& object_filenames,
+      const std::vector<std::string>& event_filenames,
       Version version,
       Episode episode,
       GameMode mode,
@@ -228,7 +242,8 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
       uint32_t lobby_id,
       std::function<std::shared_ptr<const std::string>(Version, const std::string&)> get_file_data,
       std::shared_ptr<const Map::RareEnemyRates> rare_rates,
-      std::shared_ptr<PSOLFGEncryption> random_crypt,
+      uint32_t random_seed,
+      std::shared_ptr<PSOLFGEncryption> opt_rand_crypt,
       const PrefixedLogger* log = nullptr);
   void load_maps();
   void create_ep3_server();
@@ -249,6 +264,7 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
 
   void reassign_leader_on_client_departure(size_t leaving_client_id);
   size_t count_clients() const;
+  bool any_v1_clients_present() const;
   bool any_client_loading() const;
 
   void add_client(std::shared_ptr<Client> c, ssize_t required_client_id = -1);
@@ -259,9 +275,7 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
       std::shared_ptr<Client> c,
       ssize_t required_client_id = -1);
 
-  std::shared_ptr<Client> find_client(
-      const std::string* identifier = nullptr,
-      uint64_t serial_number = 0);
+  std::shared_ptr<Client> find_client(const std::string* identifier = nullptr, uint64_t account_id = 0);
 
   enum class JoinError {
     ALLOWED = 0,
@@ -280,7 +294,7 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
 
   bool item_exists(uint8_t floor, uint32_t item_id) const;
   std::shared_ptr<FloorItem> find_item(uint8_t floor, uint32_t item_id) const;
-  void add_item(uint8_t floor, const ItemData& item, float x, float z, uint16_t visibility_flags);
+  void add_item(uint8_t floor, const ItemData& item, float x, float z, uint16_t flags);
   void add_item(uint8_t floor, std::shared_ptr<FloorItem>);
   void evict_items_from_floor(uint8_t floor);
   std::shared_ptr<FloorItem> remove_item(uint8_t floor, uint32_t item_id, uint8_t requesting_client_id);
@@ -291,9 +305,7 @@ struct Lobby : public std::enable_shared_from_this<Lobby> {
 
   QuestIndex::IncludeCondition quest_include_condition() const;
 
-  static uint8_t game_event_for_lobby_event(uint8_t lobby_event);
-
-  std::unordered_map<uint32_t, std::shared_ptr<Client>> clients_by_serial_number() const;
+  std::unordered_map<uint32_t, std::shared_ptr<Client>> clients_by_account_id() const;
 
   static void dispatch_on_idle_timeout(evutil_socket_t, short, void* ctx);
 

src/Map.cc

@@ -14,6 +14,10 @@ using namespace std;
 
 static constexpr float UINT32_MAX_AS_FLOAT = 4294967296.0f;
 
+static uint64_t section_index_key(uint8_t floor, uint16_t section, uint16_t wave_number) {
+  return (static_cast<uint64_t>(floor) << 32) | (static_cast<uint64_t>(section) << 16) | static_cast<uint64_t>(wave_number);
+}
+
 const char* Map::name_for_object_type(uint16_t type) {
   switch (type) {
     case 0x0000:
@@ -603,8 +607,10 @@ JSON Map::RareEnemyRates::json() const {
 }
 
 string Map::ObjectEntry::str() const {
-  return string_printf("[ObjectEntry type=%04hX flags=%04hX index=%04hX a2=%04hX entity_id=%04hX group=%04hX section=%04hX a3=%04hX x=%g y=%g z=%g x_angle=%08" PRIX32 " y_angle=%08" PRIX32 " z_angle=%08" PRIX32 " params=[%g %g %g %08" PRIX32 " %08" PRIX32 " %08" PRIX32 "] unused=%08" PRIX32 "]",
+  string name_str = Map::name_for_object_type(this->base_type);
+  return string_printf("[ObjectEntry type=%04hX \"%s\" flags=%04hX index=%04hX a2=%04hX entity_id=%04hX group=%04hX section=%04hX a3=%04hX x=%g y=%g z=%g x_angle=%08" PRIX32 " y_angle=%08" PRIX32 " z_angle=%08" PRIX32 " params=[%g %g %g %08" PRIX32 " %08" PRIX32 " %08" PRIX32 "] unused=%08" PRIX32 "]",
       this->base_type.load(),
+      name_str.c_str(),
       this->flags.load(),
       this->index.load(),
       this->unknown_a2.load(),
@@ -655,24 +661,44 @@ string Map::EnemyEntry::str() const {
       this->unused.load());
 }
 
-Map::Enemy::Enemy(uint16_t enemy_id, size_t source_index, uint8_t floor, EnemyType type)
+Map::Enemy::Enemy(
+    uint16_t enemy_id,
+    size_t source_index,
+    size_t set_index,
+    uint8_t floor,
+    uint16_t section,
+    uint16_t wave_number,
+    EnemyType type)
     : source_index(source_index),
+      set_index(set_index),
       enemy_id(enemy_id),
+      total_damage(0),
+      game_flags(0),
+      section(section),
+      wave_number(wave_number),
       type(type),
       floor(floor),
-      state_flags(0),
-      last_hit_by_client_id(0) {
-}
+      state_flags(0) {}
 
 string Map::Enemy::str() const {
-  return string_printf("[Map::Enemy E-%hX source %zX %s%s floor=%02hhX flags=%02hhX last_hit_by_client_id=%hu]",
+  return string_printf("[Map::Enemy E-%hX source %zX %s%s floor=%02hhX section=%04hX wave_number=%04hX flags=%02hhX]",
       this->enemy_id,
       this->source_index,
       name_for_enum(this->type),
       enemy_type_is_rare(this->type) ? " RARE" : "",
       this->floor,
-      this->state_flags,
-      this->last_hit_by_client_id);
+      this->section,
+      this->wave_number,
+      this->state_flags);
+}
+
+string Map::Event::str() const {
+  return string_printf("[Map::Event W-%02hhX-%" PRIX32 " flags=%04hX floor=%02hhX action_stream_offset=%" PRIX32 "]",
+      this->floor,
+      this->event_id,
+      this->flags,
+      this->floor,
+      this->action_stream_offset);
 }
 
 string Map::Object::str() const {
@@ -689,10 +715,11 @@ string Map::Object::str() const {
       this->item_drop_checked ? "true" : "false");
 }
 
-Map::Map(Version version, uint32_t lobby_id, std::shared_ptr<PSOLFGEncryption> random_crypt)
+Map::Map(Version version, uint32_t lobby_id, uint32_t rare_seed, std::shared_ptr<PSOLFGEncryption> opt_rand_crypt)
     : log(string_printf("[Lobby:%08" PRIX32 ":map] ", lobby_id), lobby_log.min_level),
       version(version),
-      random_crypt(random_crypt) {}
+      rare_seed(rare_seed),
+      opt_rand_crypt(opt_rand_crypt) {}
 
 void Map::clear() {
   this->objects.clear();
@@ -711,17 +738,22 @@ void Map::add_objects_from_map_data(uint8_t floor, const void* data, size_t size
     uint16_t object_id = this->objects.size();
     this->objects.emplace_back(Object{
         .source_index = z,
-        .object_id = object_id,
         .floor = floor,
+        .object_id = object_id,
         .base_type = objects[z].base_type,
         .section = objects[z].section,
+        .group = objects[z].group,
         .param1 = objects[z].param1,
         .param3 = objects[z].param3,
         .param4 = objects[z].param4,
         .param5 = objects[z].param5,
         .param6 = objects[z].param6,
+        .game_flags = 0,
+        .set_flags = 0,
         .item_drop_checked = false,
     });
+    uint64_t k = section_index_key(floor, objects[z].section, objects[z].group);
+    this->floor_section_and_group_to_object_index.emplace(k, object_id);
   }
 }
 
@@ -735,7 +767,7 @@ bool Map::check_and_log_rare_enemy(bool default_is_rare, uint32_t rare_rate) {
   // versions, we must match the client's logic, even though it's more
   // computationally expensive.
   if (this->version == Version::BB_V4) {
-    if ((this->rare_enemy_indexes.size() < 0x10) && (this->random_crypt->next() < rare_rate)) {
+    if ((this->rare_enemy_indexes.size() < 0x10) && (random_from_optional_crypt(this->opt_rand_crypt) < rare_rate)) {
       this->rare_enemy_indexes.emplace_back(this->enemies.size());
       return true;
     }
@@ -744,7 +776,7 @@ bool Map::check_and_log_rare_enemy(bool default_is_rare, uint32_t rare_rate) {
     // TODO: We only need the first value from this crypt, so it's unfortunate
     // that we have to initialize the entire thing. Find a way to make this
     // faster.
-    PSOV2Encryption crypt(this->random_crypt->seed() + 0x1000 + this->enemies.size());
+    PSOV2Encryption crypt(this->rare_seed + 0x1000 + this->enemies.size());
     float det = (static_cast<float>((crypt.next() >> 16) & 0xFFFF) / 65536.0f);
     // On v1 and v2 (and GC NTE), the rare rate is 0.1% instead of 0.2%.
     float threshold = is_v1_or_v2(this->version) ? 0.001f : 0.002f;
@@ -762,12 +794,17 @@ void Map::add_enemy(
     uint8_t difficulty,
     uint8_t event,
     uint8_t floor,
-    size_t index,
+    size_t source_index,
     const EnemyEntry& e,
     std::shared_ptr<const RareEnemyRates> rare_rates) {
+  size_t set_index = this->enemy_set_flags.size();
+  this->enemy_set_flags.emplace_back(0);
+
   auto add = [&](EnemyType type) -> void {
     uint16_t enemy_id = this->enemies.size();
-    this->enemies.emplace_back(enemy_id, index, floor, type);
+    this->enemies.emplace_back(enemy_id, source_index, set_index, floor, e.section, e.wave_number, type);
+    uint64_t k = section_index_key(floor, e.section, e.wave_number);
+    this->floor_section_and_wave_number_to_enemy_index.emplace(k, enemy_id);
   };
 
   EnemyType child_type = EnemyType::UNKNOWN;
@@ -903,7 +940,7 @@ void Map::add_enemy(
       if ((episode == Episode::EP2) && (e.floor == 0x11)) {
         add(EnemyType::DEL_LILY);
       } else {
-        add(this->check_and_log_rare_enemy((this->version == Version::BB_V4) && (e.uparam1 & 1), rare_rates->nar_lily)
+        add(this->check_and_log_rare_enemy(false, rare_rates->nar_lily)
                 ? EnemyType::NAR_LILY
                 : EnemyType::POISON_LILY);
       }
@@ -1183,16 +1220,13 @@ void Map::add_enemy(
     case 0x00C7: // TBoss3VoloptHiraisin
     case 0x0118:
       add(EnemyType::UNKNOWN);
-      this->log.warning(
-          "(Entry %zu, offset %zX in file) Unknown enemy type %04hX",
-          index, index * sizeof(EnemyEntry), e.base_type.load());
       break;
 
     default:
       add(EnemyType::UNKNOWN);
       this->log.warning(
           "(Entry %zu, offset %zX in file) Invalid enemy type %04hX",
-          index, index * sizeof(EnemyEntry), e.base_type.load());
+          source_index, source_index * sizeof(EnemyEntry), e.base_type.load());
       break;
   }
 
@@ -1314,6 +1348,10 @@ void Map::add_random_enemies_from_map_data(
   }
   wave_events_segment_r.go(wave_events_header.entries_offset);
 
+  size_t action_stream_base_offset = this->event_action_stream.size();
+  this->event_action_stream += wave_events_segment_r.pread(
+      wave_events_header.action_stream_offset, wave_events_segment_r.size() - wave_events_header.action_stream_offset);
+
   const auto& locations_header = locations_segment_r.get<RandomEnemyLocationsHeader>();
   const auto& definitions_header = definitions_segment_r.get<RandomEnemyDefinitionsHeader>();
   auto definitions_r = definitions_segment_r.sub(
@@ -1330,6 +1368,7 @@ void Map::add_random_enemies_from_map_data(
     size_t remaining_waves = random_state->rand_int_biased(1, entry.max_waves);
     // Trace: at 0080E125 EAX is wave count
 
+    le_uint32_t wave_next_event_id = entry.event_id;
     uint32_t wave_number = entry.wave_number;
     while (remaining_waves) {
       remaining_waves--;
@@ -1416,17 +1455,71 @@ void Map::add_random_enemies_from_map_data(
         }
       }
       if (remaining_waves) {
-        // We don't generate the event stream here, but the client does, and in
-        // doing so, it uses one value from random to determine the delay
-        // parameter of the event. To keep our state in sync with what the
-        // client would do, we skip a random value here.
-        random_state->random.next();
+        /* ev.delay = */ random_state->rand_int_biased(entry.min_delay, entry.max_delay);
+        this->add_event(wave_next_event_id, entry.flags, floor, entry.section, wave_number, this->event_action_stream.size());
+        this->event_action_stream.push_back(0x0C);
+        wave_next_event_id = entry.event_id + wave_number + 10000;
+        this->event_action_stream.append(reinterpret_cast<const char*>(&wave_next_event_id), sizeof(wave_next_event_id));
+        this->event_action_stream.push_back(0x01);
         wave_number++;
       }
     }
 
-    // For the same reason as above, we need to skip another random value here.
-    random_state->random.next();
+    /* ev.delay = */ random_state->rand_int_biased(entry.min_delay, entry.max_delay);
+    this->add_event(wave_next_event_id, entry.flags, floor, entry.section, wave_number, action_stream_base_offset + entry.action_stream_offset);
+    wave_number++;
+  }
+}
+
+void Map::add_event(uint32_t event_id, uint16_t flags, uint8_t floor, uint16_t section, uint16_t wave_number, uint32_t action_stream_offset) {
+  size_t index = this->events.size();
+  auto& ev = this->events.emplace_back();
+  ev.event_id = event_id;
+  ev.section = section;
+  ev.wave_number = wave_number;
+  ev.flags = flags;
+  ev.floor = floor;
+  ev.action_stream_offset = action_stream_offset;
+
+  uint64_t k = (static_cast<uint64_t>(floor) << 32) | event_id;
+  this->floor_and_event_id_to_index.emplace(k, index);
+  k = section_index_key(floor, section, wave_number);
+  this->floor_section_and_wave_number_to_event_index.emplace(k, index);
+}
+
+vector<Map::Event*> Map::get_events(uint8_t floor, uint32_t event_id) {
+  uint64_t k = (static_cast<uint64_t>(floor) << 32) | event_id;
+  vector<Event*> ret;
+  for (auto its = this->floor_and_event_id_to_index.equal_range(k); its.first != its.second; its.first++) {
+    ret.emplace_back(&this->events.at(its.first->second));
+  }
+  return ret;
+}
+
+vector<const Map::Event*> Map::get_events(uint8_t floor, uint32_t event_id) const {
+  uint64_t k = (static_cast<uint64_t>(floor) << 32) | event_id;
+  vector<const Event*> ret;
+  for (auto its = this->floor_and_event_id_to_index.equal_range(k); its.first != its.second; its.first++) {
+    ret.emplace_back(&this->events.at(its.first->second));
+  }
+  return ret;
+}
+
+void Map::add_events_from_map_data(uint8_t floor, const void* data, size_t size) {
+  StringReader r(data, size);
+  const auto& header = r.get<EventsSectionHeader>();
+  if (header.format != 0) {
+    throw runtime_error("events section format is not zero");
+  }
+
+  size_t action_stream_base_offset = this->event_action_stream.size();
+  this->event_action_stream += r.pread(header.action_stream_offset, r.size() - header.action_stream_offset);
+
+  this->events.reserve(this->events.size() + header.entry_count);
+  auto events_r = r.sub(header.entries_offset, sizeof(Event1Entry) * header.entry_count);
+  while (!events_r.eof()) {
+    const auto& entry = events_r.get<Event1Entry>();
+    this->add_event(entry.event_id, entry.flags, floor, entry.section, entry.wave_number, entry.action_stream_offset + action_stream_base_offset);
   }
 }
 
@@ -1491,7 +1584,7 @@ vector<Map::DATSectionsForFloor> Map::collect_quest_map_data_sections(const void
   return ret;
 }
 
-void Map::add_enemies_and_objects_from_quest_data(
+void Map::add_entities_from_quest_data(
     Episode episode,
     uint8_t difficulty,
     uint8_t event,
@@ -1513,28 +1606,15 @@ void Map::add_enemies_and_objects_from_quest_data(
       this->add_objects_from_map_data(floor, r.pgetv(floor_sections.objects + sizeof(header), header.data_size), header.data_size);
     }
 
-    if (floor_sections.enemies != 0xFFFFFFFF) {
-      const auto& header = r.pget<SectionHeader>(floor_sections.enemies);
-      if (header.data_size % sizeof(EnemyEntry)) {
-        throw runtime_error("quest layout enemy section size is not a multiple of enemy entry size");
-      }
-      this->add_enemies_from_map_data(
-          episode,
-          difficulty,
-          event,
-          floor,
-          r.pgetv(floor_sections.enemies + sizeof(header), header.data_size),
-          header.data_size,
-          rare_rates);
-
-    } else if ((floor_sections.wave_events != 0xFFFFFFFF) &&
+    if ((floor_sections.wave_events != 0xFFFFFFFF) &&
         (floor_sections.random_enemy_locations != 0xFFFFFFFF) &&
         (floor_sections.random_enemy_definitions != 0xFFFFFFFF)) {
+      // Challenge Mode random enemy waves
       const auto& wave_events_header = r.pget<SectionHeader>(floor_sections.wave_events);
       const auto& random_enemy_locations_header = r.pget<SectionHeader>(floor_sections.random_enemy_locations);
       const auto& random_enemy_definitions_header = r.pget<SectionHeader>(floor_sections.random_enemy_definitions);
       if (!random_state) {
-        random_state = make_shared<DATParserRandomState>(this->random_crypt->seed());
+        random_state = make_shared<DATParserRandomState>(this->rare_seed);
       }
       this->add_random_enemies_from_map_data(
           episode,
@@ -1546,6 +1626,29 @@ void Map::add_enemies_and_objects_from_quest_data(
           r.sub(floor_sections.random_enemy_definitions + sizeof(SectionHeader), random_enemy_definitions_header.data_size),
           random_state,
           rare_rates);
+
+    } else {
+      // Non-Challenge (standard) enemies
+      if (floor_sections.enemies != 0xFFFFFFFF) {
+        const auto& header = r.pget<SectionHeader>(floor_sections.enemies);
+        if (header.data_size % sizeof(EnemyEntry)) {
+          throw runtime_error("quest layout enemy section size is not a multiple of enemy entry size");
+        }
+        this->add_enemies_from_map_data(
+            episode,
+            difficulty,
+            event,
+            floor,
+            r.pgetv(floor_sections.enemies + sizeof(header), header.data_size),
+            header.data_size,
+            rare_rates);
+      }
+
+      if (floor_sections.wave_events != 0xFFFFFFFF) {
+        const auto& wave_events_header = r.pget<SectionHeader>(floor_sections.wave_events);
+        const void* data = r.pgetv(floor_sections.wave_events + sizeof(SectionHeader), wave_events_header.data_size);
+        this->add_events_from_map_data(floor, data, wave_events_header.data_size);
+      }
     }
   }
 }
@@ -1568,6 +1671,154 @@ Map::Enemy& Map::find_enemy(uint8_t floor, EnemyType type) {
   throw out_of_range("enemy not found");
 }
 
+std::vector<Map::Object*> Map::get_objects(uint8_t floor, uint16_t section, uint16_t group) {
+  uint64_t k = section_index_key(floor, section, group);
+  vector<Object*> ret;
+  for (auto its = this->floor_section_and_group_to_object_index.equal_range(k); its.first != its.second; its.first++) {
+    ret.emplace_back(&this->objects.at(its.first->second));
+  }
+  return ret;
+}
+
+std::vector<Map::Enemy*> Map::get_enemies(uint8_t floor, uint16_t section, uint16_t wave_number) {
+  uint64_t k = section_index_key(floor, section, wave_number);
+  vector<Enemy*> ret;
+  for (auto its = this->floor_section_and_wave_number_to_enemy_index.equal_range(k); its.first != its.second; its.first++) {
+    ret.emplace_back(&this->enemies.at(its.first->second));
+  }
+  return ret;
+}
+
+std::vector<Map::Event*> Map::get_events(uint8_t floor, uint16_t section, uint16_t wave_number) {
+  uint64_t k = section_index_key(floor, section, wave_number);
+  vector<Event*> ret;
+  for (auto its = this->floor_section_and_wave_number_to_event_index.equal_range(k); its.first != its.second; its.first++) {
+    ret.emplace_back(&this->events.at(its.first->second));
+  }
+  return ret;
+}
+
+std::vector<Map::Event*> Map::get_events(uint8_t floor) {
+  uint64_t k_start = (static_cast<uint64_t>(floor) << 32);
+  uint64_t k_end = (static_cast<uint64_t>(floor + 1) << 32);
+  vector<Event*> ret;
+  for (auto it = this->floor_and_event_id_to_index.lower_bound(k_start);
+       (it != this->floor_and_event_id_to_index.end()) && (it->first < k_end);
+       it++) {
+    ret.emplace_back(&this->events.at(it->second));
+  }
+  return ret;
+}
+
+template <typename EntryT>
+static string disassemble_vector_file_t(const void* data, size_t size, size_t* entry_number, char type_ch) {
+  deque<string> ret;
+  StringReader r(data, size);
+
+  size_t local_entry_number = 0;
+  if (!entry_number) {
+    entry_number = &local_entry_number;
+  }
+
+  while (r.remaining() >= sizeof(EntryT)) {
+    string o_str = r.get<EntryT>().str();
+    ret.emplace_back(string_printf("/* %c-%zX */ %s", type_ch, (*entry_number)++, o_str.c_str()));
+  }
+  if (r.remaining()) {
+    ret.emplace_back("// Warning: section size is not a multiple of entry size");
+    size_t size = r.remaining();
+    ret.emplace_back(format_data(r.getv(size), size));
+  }
+  return join(ret, "\n");
+}
+
+string Map::disassemble_objects_data(const void* data, size_t size, size_t* object_number) {
+  return disassemble_vector_file_t<ObjectEntry>(data, size, object_number, 'K');
+}
+
+string Map::disassemble_enemies_data(const void* data, size_t size, size_t* enemy_number) {
+  return disassemble_vector_file_t<EnemyEntry>(data, size, enemy_number, 'S');
+}
+
+string Map::disassemble_wave_events_data(const void* data, size_t size, uint8_t floor) {
+  deque<string> ret;
+  StringReader r(data, size);
+
+  const auto& evt_header = r.get<EventsSectionHeader>();
+  if (evt_header.format == 0x65767432) { // 'evt2'
+    ret.emplace_back(".evt2_format"); // TODO
+    size_t size = r.remaining();
+    ret.emplace_back(format_data(r.getv(size), size));
+  } else {
+    auto action_stream_r = r.sub(evt_header.action_stream_offset);
+    for (size_t z = 0; z < evt_header.entry_count; z++) {
+      const auto& entry = r.get<Event1Entry>();
+      ret.emplace_back(string_printf("/* W-%02hhX-%" PRIX32 " */ [Event1Entry flags=%04hX type=%04hX section=%04hX wave_number=%04hX delay=%" PRIu32 "]",
+          floor,
+          entry.event_id.load(),
+          entry.flags.load(),
+          entry.event_type.load(),
+          entry.section.load(),
+          entry.wave_number.load(),
+          entry.delay.load()));
+      auto ev_actions_r = action_stream_r.sub(entry.action_stream_offset);
+      bool should_continue = true;
+      while (!ev_actions_r.eof() && should_continue) {
+        uint8_t opcode = ev_actions_r.get_u8();
+        switch (opcode) {
+          case 0x00:
+            ret.emplace_back(string_printf("  00            nop"));
+            break;
+          case 0x01:
+            ret.emplace_back(string_printf("  01            stop"));
+            should_continue = false;
+            break;
+          case 0x08: {
+            uint16_t section = ev_actions_r.get_u16l();
+            uint16_t group = ev_actions_r.get_u16l();
+            ret.emplace_back(string_printf("  08 %04hX %04hX  construct_objects       section=%04hX group=%04hX",
+                section, group, section, group));
+            break;
+          }
+          case 0x09: {
+            uint16_t section = ev_actions_r.get_u16l();
+            uint16_t wave_number = ev_actions_r.get_u16l();
+            ret.emplace_back(string_printf("  09 %04hX %04hX  construct_enemies       section=%04hX wave_number=%04hX",
+                section, wave_number, section, wave_number));
+            break;
+          }
+          case 0x0A: {
+            uint16_t id = ev_actions_r.get_u16l();
+            ret.emplace_back(string_printf("  0A %04hX       enable_switch_flag      id=%04hX", id, id));
+            break;
+          }
+          case 0x0B: {
+            uint16_t id = ev_actions_r.get_u16l();
+            ret.emplace_back(string_printf("  0B %04hX       disable_switch_flag     id=%04hX", id, id));
+            break;
+          }
+          case 0x0C: {
+            uint32_t event_id = ev_actions_r.get_u32l();
+            ret.emplace_back(string_printf("  0C %08" PRIX32 "   trigger_event           event_id=%08" PRIX32, event_id, event_id));
+            break;
+          }
+          case 0x0D: {
+            uint16_t section = ev_actions_r.get_u16l();
+            uint16_t wave_number = ev_actions_r.get_u16l();
+            ret.emplace_back(string_printf("  0D %04hX %04hX  construct_enemies_stop  section=%04hX wave_number=%04hX",
+                section, wave_number, section, wave_number));
+            break;
+          }
+          default:
+            ret.emplace_back(string_printf("  %02hhX            .invalid", opcode));
+        }
+      }
+    }
+  }
+
+  return join(ret, "\n");
+}
+
 string Map::disassemble_quest_data(const void* data, size_t size) {
   auto all_floor_sections = Map::collect_quest_map_data_sections(data, size);
 
@@ -1581,56 +1832,34 @@ string Map::disassemble_quest_data(const void* data, size_t size) {
     if (floor_sections.objects != 0xFFFFFFFF) {
       ret.emplace_back(string_printf(".objects %zu", floor));
       const auto& header = r.pget<SectionHeader>(floor_sections.objects);
-      auto sub_r = r.sub(floor_sections.objects + sizeof(SectionHeader), header.data_size);
-      while (sub_r.remaining() >= sizeof(ObjectEntry)) {
-        string o_str = sub_r.get<ObjectEntry>().str();
-        ret.emplace_back(string_printf("/* K-%zX */ %s", object_number++, o_str.c_str()));
-      }
-      if (sub_r.remaining()) {
-        ret.emplace_back("// Warning: object section size is not a multiple of object entry size");
-        size_t offset = floor_sections.objects + sizeof(SectionHeader) + r.where();
-        size_t bytes = r.remaining();
-        ret.emplace_back(format_data(r.getv(r.remaining()), bytes, offset));
-      }
+      size_t offset = floor_sections.objects + sizeof(SectionHeader);
+      ret.emplace_back(Map::disassemble_objects_data(r.pgetv(offset, header.data_size), header.data_size, &object_number));
     }
-
     if (floor_sections.enemies != 0xFFFFFFFF) {
       ret.emplace_back(string_printf(".enemies %zu", floor));
       const auto& header = r.pget<SectionHeader>(floor_sections.enemies);
-      auto sub_r = r.sub(floor_sections.enemies + sizeof(SectionHeader), header.data_size);
-      while (sub_r.remaining() >= sizeof(EnemyEntry)) {
-        string e_str = sub_r.get<EnemyEntry>().str();
-        ret.emplace_back(string_printf("/* entry %zX */ %s", enemy_number++, e_str.c_str()));
-      }
-      if (sub_r.remaining()) {
-        ret.emplace_back("// Warning: enemy section size is not a multiple of enemy entry size");
-        size_t offset = floor_sections.objects + sizeof(SectionHeader) + r.where();
-        size_t bytes = r.remaining();
-        ret.emplace_back(format_data(r.getv(r.remaining()), bytes, offset));
-      }
+      size_t offset = floor_sections.enemies + sizeof(SectionHeader);
+      ret.emplace_back(Map::disassemble_enemies_data(r.pgetv(offset, header.data_size), header.data_size, &enemy_number));
     }
-
-    // TODO: Add disassembly for these section types
     if (floor_sections.wave_events != 0xFFFFFFFF) {
       ret.emplace_back(string_printf(".wave_events %zu", floor));
       const auto& header = r.pget<SectionHeader>(floor_sections.wave_events);
       size_t offset = floor_sections.wave_events + sizeof(SectionHeader);
-      auto sub_r = r.sub(offset, header.data_size);
-      ret.emplace_back(format_data(r.getv(r.remaining()), header.data_size, offset));
+      ret.emplace_back(Map::disassemble_wave_events_data(r.pgetv(offset, header.data_size), header.data_size, floor));
     }
     if (floor_sections.random_enemy_locations != 0xFFFFFFFF) {
       ret.emplace_back(string_printf(".random_enemy_locations %zu", floor));
       const auto& header = r.pget<SectionHeader>(floor_sections.random_enemy_locations);
       size_t offset = floor_sections.random_enemy_locations + sizeof(SectionHeader);
       auto sub_r = r.sub(offset, header.data_size);
-      ret.emplace_back(format_data(r.getv(r.remaining()), header.data_size, offset));
+      ret.emplace_back(format_data(sub_r.getv(sub_r.remaining()), header.data_size, offset));
     }
     if (floor_sections.random_enemy_definitions != 0xFFFFFFFF) {
       ret.emplace_back(string_printf(".random_enemy_definitions %zu", floor));
       const auto& header = r.pget<SectionHeader>(floor_sections.random_enemy_definitions);
       size_t offset = floor_sections.random_enemy_definitions + sizeof(SectionHeader);
       auto sub_r = r.sub(offset, header.data_size);
-      ret.emplace_back(format_data(r.getv(r.remaining()), header.data_size, offset));
+      ret.emplace_back(format_data(sub_r.getv(sub_r.remaining()), header.data_size, offset));
     }
   }
 
@@ -1640,25 +1869,25 @@ string Map::disassemble_quest_data(const void* data, size_t size) {
 SetDataTableBase::SetDataTableBase(Version version) : version(version) {}
 
 parray<le_uint32_t, 0x20> SetDataTableBase::generate_variations(
-    Episode episode, bool is_solo, std::shared_ptr<PSOLFGEncryption> random_crypt) const {
+    Episode episode, bool is_solo, std::shared_ptr<PSOLFGEncryption> opt_rand_crypt) const {
   parray<le_uint32_t, 0x20> ret;
   for (size_t floor = 0; floor < 0x10; floor++) {
     auto num_vars = this->num_free_roam_variations_for_floor(episode, is_solo, floor);
-    ret[floor * 2] = (num_vars.first > 1) ? (random_crypt->next() % num_vars.first) : 0;
-    ret[floor * 2 + 1] = (num_vars.second > 1) ? (random_crypt->next() % num_vars.second) : 0;
+    ret[floor * 2] = (num_vars.first > 1) ? (random_from_optional_crypt(opt_rand_crypt) % num_vars.first) : 0;
+    ret[floor * 2 + 1] = (num_vars.second > 1) ? (random_from_optional_crypt(opt_rand_crypt) % num_vars.second) : 0;
   }
   return ret;
 }
 
 vector<string> SetDataTableBase::map_filenames_for_variations(
-    const parray<le_uint32_t, 0x20>& variations, Episode episode, GameMode mode, bool is_enemies) const {
+    const parray<le_uint32_t, 0x20>& variations, Episode episode, GameMode mode, FilenameType type) const {
   vector<string> ret;
   for (uint8_t floor = 0; floor < 0x10; floor++) {
     ret.emplace_back(this->map_filename_for_variation(
-        floor, variations[floor * 2], variations[floor * 2 + 1], episode, mode, is_enemies));
+        floor, variations[floor * 2], variations[floor * 2 + 1], episode, mode, type));
   }
   for (uint8_t floor = 0x10; floor < 0x12; floor++) {
-    ret.emplace_back(this->map_filename_for_variation(floor, 0, 0, episode, mode, is_enemies));
+    ret.emplace_back(this->map_filename_for_variation(floor, 0, 0, episode, mode, type));
   }
   return ret;
 }
@@ -1711,7 +1940,8 @@ void SetDataTable::load_table_t(const string& data) {
     U32T unknown_a6; // == 0
     U32T unknown_a7; // == 0
     U32T unknown_a8; // == 0
-  } __attribute__((packed));
+  } __packed_ws__(Footer, 0x20);
+
   if (r.size() < sizeof(Footer)) {
     throw runtime_error("set data table is too small");
   }
@@ -1732,8 +1962,8 @@ void SetDataTable::load_table_t(const string& data) {
       while (!var2_r.eof()) {
         auto& entry = var2_v.emplace_back();
         entry.object_list_basename = r.pget_cstr(var2_r.get<U32T>());
-        entry.enemy_list_basename = r.pget_cstr(var2_r.get<U32T>());
-        entry.event_list_basename = r.pget_cstr(var2_r.get<U32T>());
+        entry.enemy_and_event_list_basename = r.pget_cstr(var2_r.get<U32T>());
+        entry.area_setup_filename = r.pget_cstr(var2_r.get<U32T>());
       }
     }
   }
@@ -1744,7 +1974,10 @@ pair<uint32_t, uint32_t> SetDataTable::num_available_variations_for_floor(Episod
   if (area == 0xFF) {
     return make_pair(1, 1);
   } else {
-    const auto& e = this->entries.at(area);
+    if (area >= this->entries.size()) {
+      return make_pair(1, 1);
+    }
+    const auto& e = this->entries[area];
     return make_pair(e.size(), e.at(0).size());
   }
 }
@@ -1783,7 +2016,7 @@ pair<uint32_t, uint32_t> SetDataTable::num_free_roam_variations_for_floor(Episod
 }
 
 string SetDataTable::map_filename_for_variation(
-    uint8_t floor, uint32_t var1, uint32_t var2, Episode episode, GameMode mode, bool is_enemies) const {
+    uint8_t floor, uint32_t var1, uint32_t var2, Episode episode, GameMode mode, FilenameType type) const {
   uint8_t area = this->default_area_for_floor(episode, floor);
   if (area == 0xFF) {
     return "";
@@ -1794,22 +2027,35 @@ string SetDataTable::map_filename_for_variation(
   }
 
   const auto& entry = this->entries.at(area).at(var1).at(var2);
-  string filename = is_enemies ? entry.enemy_list_basename : entry.object_list_basename;
 
-  filename += (is_enemies ? "e" : "o");
+  string filename;
+  switch (type) {
+    case FilenameType::OBJECTS:
+      filename = entry.object_list_basename + "o";
+      break;
+    case FilenameType::ENEMIES:
+      filename = entry.enemy_and_event_list_basename + "e";
+      break;
+    case FilenameType::EVENTS:
+      filename = entry.enemy_and_event_list_basename;
+      break;
+    default:
+      throw logic_error("invalid map filename type");
+  }
 
+  bool is_events = (type == FilenameType::EVENTS);
   switch ((floor != 0) ? GameMode::NORMAL : mode) {
     case GameMode::NORMAL:
-      filename += ".dat";
+      filename += is_events ? ".evt" : ".dat";
       break;
     case GameMode::SOLO:
-      filename += "_s.dat";
+      filename += is_events ? "_s.evt" : "_s.dat";
       break;
     case GameMode::CHALLENGE:
-      filename += "_c1.dat";
+      filename += is_events ? "_c1.evt" : "_c1.dat";
       break;
     case GameMode::BATTLE:
-      filename += "_d.dat";
+      filename += is_events ? "_d.evt" : "_d.dat";
       break;
     default:
       throw logic_error("invalid game mode");
@@ -1820,14 +2066,15 @@ string SetDataTable::map_filename_for_variation(
 
 string SetDataTable::str() const {
   vector<string> lines;
-  lines.emplace_back(string_printf("FL/V1/V2 => ----------------------OBJECT -----------------------ENEMY -----------------------EVENT\n"));
+  lines.emplace_back(string_printf("FL/V1/V2 => ----------------------OBJECT -----------------ENEMY+EVENT -----------------------SETUP\n"));
   for (size_t a = 0; a < this->entries.size(); a++) {
     const auto& v1_v = this->entries[a];
     for (size_t v1 = 0; v1 < v1_v.size(); v1++) {
       const auto& v2_v = v1_v[v1];
       for (size_t v2 = 0; v2 < v2_v.size(); v2++) {
         const auto& e = v2_v[v2];
-        lines.emplace_back(string_printf("%02zX/%02zX/%02zX => %28s %28s %28s\n", a, v1, v2, e.object_list_basename.c_str(), e.enemy_list_basename.c_str(), e.event_list_basename.c_str()));
+        lines.emplace_back(string_printf("%02zX/%02zX/%02zX => %28s %28s %28s\n",
+            a, v1, v2, e.object_list_basename.c_str(), e.enemy_and_event_list_basename.c_str(), e.area_setup_filename.c_str()));
       }
     }
   }
@@ -1848,7 +2095,7 @@ struct AreaMapFileInfo {
         variation2_values(variation2_values) {}
 };
 
-const array<vector<vector<string>>, 0x10> SetDataTableDCNTE::NAMES = {{
+const array<vector<vector<string>>, 0x12> SetDataTableDCNTE::NAMES = {{
     /* 00 */ {{"map_city00_00"}},
     /* 01 */ {{"map_forest01_00", "map_forest01_01"}},
     /* 02 */ {{"map_forest02_00", "map_forest02_03"}},
@@ -1865,12 +2112,15 @@ const array<vector<vector<string>>, 0x10> SetDataTableDCNTE::NAMES = {{
     /* 0D */ {{"map_boss03"}},
     /* 0E */ {{"map_boss04"}},
     /* 0F */ {{"map_visuallobby"}, {"map_visuallobby"}, {"map_visuallobby"}, {"map_visuallobby"}, {"map_visuallobby"}, {"map_visuallobby"}, {"map_visuallobby"}, {"map_visuallobby"}, {"map_visuallobby"}, {"map_visuallobby"}},
+    /* 10 */ {},
+    /* 11 */ {},
 }};
 
 SetDataTableDCNTE::SetDataTableDCNTE() : SetDataTableBase(Version::DC_NTE) {}
 
 pair<uint32_t, uint32_t> SetDataTableDCNTE::num_available_variations_for_floor(Episode, uint8_t floor) const {
-  return make_pair(this->NAMES[floor].size(), this->NAMES[floor][0].size());
+  const auto& floor_names = this->NAMES.at(floor);
+  return make_pair(floor_names.size(), floor_names.empty() ? 0 : this->NAMES.at(floor)[0].size());
 }
 
 pair<uint32_t, uint32_t> SetDataTableDCNTE::num_free_roam_variations_for_floor(Episode episode, bool, uint8_t floor) const {
@@ -1878,14 +2128,29 @@ pair<uint32_t, uint32_t> SetDataTableDCNTE::num_free_roam_variations_for_floor(E
 }
 
 string SetDataTableDCNTE::map_filename_for_variation(
-    uint8_t floor, uint32_t var1, uint32_t var2, Episode, GameMode, bool is_enemies) const {
-  if (floor >= this->NAMES.size()) {
+    uint8_t floor, uint32_t var1, uint32_t var2, Episode, GameMode, FilenameType type) const {
+  try {
+    string basename = this->NAMES.at(floor).at(var1).at(var2);
+    switch (type) {
+      case FilenameType::ENEMIES:
+        basename += "e.dat";
+        break;
+      case FilenameType::OBJECTS:
+        basename += "o.dat";
+        break;
+      case FilenameType::EVENTS:
+        basename += ".evt";
+        break;
+      default:
+        throw logic_error("invalid map filename type");
+    }
+    return basename;
+  } catch (const out_of_range&) {
     return "";
   }
-  return this->NAMES.at(floor).at(var1).at(var2) + (is_enemies ? "e.dat" : "o.dat");
 }
 
-const array<vector<vector<string>>, 0x10> SetDataTableDC112000::NAMES = {{
+const array<vector<vector<string>>, 0x12> SetDataTableDC112000::NAMES = {{
     /* 00 */ {{"map_city00_00"}},
     /* 01 */ {{"map_forest01_00", "map_forest01_01", "map_forest01_02", "map_forest01_03", "map_forest01_04"}},
     /* 02 */ {{"map_forest02_00", "map_forest02_01", "map_forest02_02", "map_forest02_03", "map_forest02_04"}},
@@ -1902,12 +2167,15 @@ const array<vector<vector<string>>, 0x10> SetDataTableDC112000::NAMES = {{
     /* 0D */ {{"map_boss03"}},
     /* 0E */ {{"map_boss04"}},
     /* 0F */ {{"map_visuallobby"}, {"map_visuallobby"}, {"map_visuallobby"}, {"map_visuallobby"}, {"map_visuallobby"}, {"map_visuallobby"}, {"map_visuallobby"}, {"map_visuallobby"}, {"map_visuallobby"}, {"map_visuallobby"}},
+    /* 10 */ {},
+    /* 11 */ {},
 }};
 
 SetDataTableDC112000::SetDataTableDC112000() : SetDataTableBase(Version::DC_V1_11_2000_PROTOTYPE) {}
 
 pair<uint32_t, uint32_t> SetDataTableDC112000::num_available_variations_for_floor(Episode, uint8_t floor) const {
-  return make_pair(this->NAMES[floor].size(), this->NAMES[floor][0].size());
+  const auto& floor_names = this->NAMES.at(floor);
+  return make_pair(floor_names.size(), floor_names.empty() ? 0 : this->NAMES.at(floor)[0].size());
 }
 
 pair<uint32_t, uint32_t> SetDataTableDC112000::num_free_roam_variations_for_floor(Episode episode, bool, uint8_t floor) const {
@@ -1915,11 +2183,26 @@ pair<uint32_t, uint32_t> SetDataTableDC112000::num_free_roam_variations_for_floo
 }
 
 string SetDataTableDC112000::map_filename_for_variation(
-    uint8_t floor, uint32_t var1, uint32_t var2, Episode, GameMode, bool is_enemies) const {
-  if (floor >= this->NAMES.size()) {
+    uint8_t floor, uint32_t var1, uint32_t var2, Episode, GameMode, FilenameType type) const {
+  try {
+    string basename = this->NAMES.at(floor).at(var1).at(var2);
+    switch (type) {
+      case FilenameType::ENEMIES:
+        basename += "e.dat";
+        break;
+      case FilenameType::OBJECTS:
+        basename += "o.dat";
+        break;
+      case FilenameType::EVENTS:
+        basename += ".evt";
+        break;
+      default:
+        throw logic_error("invalid map filename type");
+    }
+    return basename;
+  } catch (const out_of_range&) {
     return "";
   }
-  return this->NAMES.at(floor).at(var1).at(var2) + (is_enemies ? "e.dat" : "o.dat");
 }
 
 static const vector<AreaMapFileInfo> map_file_info_dc_nte = {

src/Map.hh

@@ -34,7 +34,7 @@ struct Map {
     inline Type type() const {
       return static_cast<Type>(this->le_type.load());
     }
-  } __attribute__((packed));
+  } __packed_ws__(SectionHeader, 0x10);
 
   struct ObjectEntry { // Section type 1 (OBJECTS)
     /* 00 */ le_uint16_t base_type;
@@ -61,7 +61,7 @@ struct Map {
     /* 44 */
 
     std::string str() const;
-  } __attribute__((packed));
+  } __packed_ws__(ObjectEntry, 0x44);
 
   struct EnemyEntry { // Section type 2 (ENEMIES)
     /* 00 */ le_uint16_t base_type;
@@ -91,31 +91,35 @@ struct Map {
     /* 48 */
 
     std::string str() const;
-  } __attribute__((packed));
+  } __packed_ws__(EnemyEntry, 0x48);
 
   struct EventsSectionHeader { // Section type 3 (WAVE_EVENTS)
-    /* 00 */ le_uint32_t footer_offset;
+    /* 00 */ le_uint32_t action_stream_offset;
     /* 04 */ le_uint32_t entries_offset;
     /* 08 */ le_uint32_t entry_count;
     /* 0C */ be_uint32_t format; // 0 or 'evt2'
     /* 10 */
-  } __attribute__((packed));
+  } __packed_ws__(EventsSectionHeader, 0x10);
 
   struct Event1Entry { // Section type 3 (WAVE_EVENTS) if format == 0
     /* 00 */ le_uint32_t event_id;
+    // Bits in flags:
+    //   0004 = is active
+    //   0008 = post-wave actions have been run
+    //   0010 = all enemies killed
     /* 04 */ le_uint16_t flags;
-    /* 06 */ le_uint16_t unknown_a2;
+    /* 06 */ le_uint16_t event_type;
     /* 08 */ le_uint16_t section;
     /* 0A */ le_uint16_t wave_number;
     /* 0C */ le_uint32_t delay;
-    /* 10 */ le_uint32_t clear_events_index;
+    /* 10 */ le_uint32_t action_stream_offset;
     /* 14 */
-  } __attribute__((packed));
+  } __packed_ws__(Event1Entry, 0x14);
 
   struct Event2Entry { // Section type 3 (WAVE_EVENTS) if format == 'evt2'
     /* 00 */ le_uint32_t event_id;
     /* 04 */ le_uint16_t flags;
-    /* 06 */ le_uint16_t unknown_a2;
+    /* 06 */ le_uint16_t event_type;
     /* 08 */ le_uint16_t section;
     /* 0A */ le_uint16_t wave_number;
     /* 0C */ le_uint16_t min_delay;
@@ -123,23 +127,23 @@ struct Map {
     /* 10 */ uint8_t min_enemies;
     /* 11 */ uint8_t max_enemies;
     /* 12 */ le_uint16_t max_waves;
-    /* 14 */ le_uint32_t clear_events_index;
+    /* 14 */ le_uint32_t action_stream_offset;
     /* 18 */
-  } __attribute__((packed));
+  } __packed_ws__(Event2Entry, 0x18);
 
   struct RandomEnemyLocationsHeader { // Section type 4 (RANDOM_ENEMY_LOCATIONS)
     /* 00 */ le_uint32_t section_table_offset; // Offset to RandomEnemyLocationSegment structs, from start of this struct
     /* 04 */ le_uint32_t entries_offset; // Offset to RandomEnemyLocationEntry structs, from start of this struct
     /* 08 */ le_uint32_t num_sections;
     /* 0C */
-  } __attribute__((packed));
+  } __packed_ws__(RandomEnemyLocationsHeader, 0x0C);
 
   struct RandomEnemyLocationSection { // Section type 4 (RANDOM_ENEMY_LOCATIONS)
     /* 00 */ le_uint16_t section;
     /* 02 */ le_uint16_t count;
     /* 04 */ le_uint32_t offset;
     /* 08 */
-  } __attribute__((packed));
+  } __packed_ws__(RandomEnemyLocationSection, 8);
 
   struct RandomEnemyLocationEntry { // Section type 4 (RANDOM_ENEMY_LOCATIONS)
     /* 00 */ le_float x;
@@ -151,7 +155,7 @@ struct Map {
     /* 18 */ uint16_t unknown_a9;
     /* 1A */ uint16_t unknown_a10;
     /* 1C */
-  } __attribute__((packed));
+  } __packed_ws__(RandomEnemyLocationEntry, 0x1C);
 
   struct RandomEnemyDefinitionsHeader { // Section type 5 (RANDOM_ENEMY_DEFINITIONS)
     /* 00 */ le_uint32_t entries_offset; // Offset to RandomEnemyDefinition structs, from start of this struct
@@ -159,7 +163,7 @@ struct Map {
     /* 08 */ le_uint32_t entry_count;
     /* 0C */ le_uint32_t weight_entry_count;
     /* 10 */
-  } __attribute__((packed));
+  } __packed_ws__(RandomEnemyDefinitionsHeader, 0x10);
 
   struct RandomEnemyDefinition { // Section type 5 (RANDOM_ENEMY_DEFINITIONS)
     // All fields through entry_num map to the corresponding fields in
@@ -175,7 +179,7 @@ struct Map {
     /* 1C */ le_uint16_t min_children;
     /* 1E */ le_uint16_t max_children;
     /* 20 */
-  } __attribute__((packed));
+  } __packed_ws__(RandomEnemyDefinition, 0x20);
 
   struct RandomEnemyWeight { // Section type 5 (RANDOM_ENEMY_DEFINITIONS)
     /* 00 */ uint8_t base_type_index;
@@ -183,7 +187,7 @@ struct Map {
     /* 02 */ uint8_t weight;
     /* 03 */ uint8_t unknown_a4;
     /* 04 */
-  } __attribute__((packed));
+  } __packed_ws__(RandomEnemyWeight, 4);
 
   struct RareEnemyRates {
     uint32_t hildeblue; // HILDEBEAR -> HILDEBLUE
@@ -208,15 +212,20 @@ struct Map {
     // TODO: Add more fields in here if we ever care about them. Currently we
     // only care about boxes with fixed item drops.
     size_t source_index;
-    uint16_t object_id;
     uint8_t floor;
+    uint16_t object_id;
     uint16_t base_type;
     uint16_t section;
+    uint16_t group;
     float param1; // If <= 0, this is a specialized box, and the specialization is in param4/5/6
     float param3; // If == 0, the item should be varied by difficulty and area
     uint32_t param4;
     uint32_t param5;
     uint32_t param6;
+    uint16_t game_flags;
+    // Technically set_flags shouldn't be part of the Object struct, but all
+    // object entries always generate exactly one object, so we store it here.
+    uint16_t set_flags;
     bool item_drop_checked;
 
     std::string str() const;
@@ -231,16 +240,39 @@ struct Map {
       ITEM_DROPPED = 0x10,
     };
     size_t source_index;
+    size_t set_index;
     uint16_t enemy_id;
+    uint16_t total_damage;
+    uint32_t game_flags; // From 6x0A
+    uint16_t section;
+    uint16_t wave_number;
     EnemyType type;
     uint8_t floor;
     uint8_t state_flags;
-    uint8_t last_hit_by_client_id;
 
-    Enemy(uint16_t enemy_id, size_t source_index, uint8_t floor, EnemyType type);
+    Enemy(
+        uint16_t enemy_id,
+        size_t source_index,
+        size_t set_index,
+        uint8_t floor,
+        uint16_t section,
+        uint16_t wave_number,
+        EnemyType type);
 
     std::string str() const;
-  } __attribute__((packed));
+  };
+
+  struct Event {
+    uint32_t event_id;
+    uint16_t flags;
+    uint16_t section;
+    uint16_t wave_number;
+    uint8_t floor;
+    uint32_t action_stream_offset;
+    std::vector<size_t> enemy_indexes;
+
+    std::string str() const;
+  };
 
   struct DATParserRandomState {
     PSOV2Encryption random;
@@ -256,7 +288,7 @@ struct Map {
     void generate_shuffled_location_table(const Map::RandomEnemyLocationsHeader& header, StringReader r, uint16_t section);
   };
 
-  Map(Version version, uint32_t lobby_id, std::shared_ptr<PSOLFGEncryption> random_crypt);
+  Map(Version version, uint32_t lobby_id, uint32_t rare_seed, std::shared_ptr<PSOLFGEncryption> opt_rand_crypt);
   ~Map() = default;
 
   void clear();
@@ -291,6 +323,17 @@ struct Map {
       std::shared_ptr<DATParserRandomState> random_state,
       std::shared_ptr<const RareEnemyRates> rare_rates = DEFAULT_RARE_ENEMIES);
 
+  void add_event(
+      uint32_t event_id,
+      uint16_t flags,
+      uint8_t floor,
+      uint16_t section,
+      uint16_t wave_number,
+      uint32_t action_stream_offset);
+  std::vector<Event*> get_events(uint8_t floor, uint32_t event_id);
+  std::vector<const Event*> get_events(uint8_t floor, uint32_t event_id) const;
+  void add_events_from_map_data(uint8_t floor, const void* data, size_t size);
+
   struct DATSectionsForFloor {
     uint32_t objects = 0xFFFFFFFF;
     uint32_t enemies = 0xFFFFFFFF;
@@ -300,7 +343,7 @@ struct Map {
   };
   static std::vector<DATSectionsForFloor> collect_quest_map_data_sections(const void* data, size_t size);
 
-  void add_enemies_and_objects_from_quest_data(
+  void add_entities_from_quest_data(
       Episode episode,
       uint8_t difficulty,
       uint8_t event,
@@ -310,29 +353,53 @@ struct Map {
 
   const Enemy& find_enemy(uint8_t floor, EnemyType type) const;
   Enemy& find_enemy(uint8_t floor, EnemyType type);
+  std::vector<Object*> get_objects(uint8_t floor, uint16_t section, uint16_t wave_number);
+  std::vector<Enemy*> get_enemies(uint8_t floor, uint16_t section, uint16_t wave_number);
+  std::vector<Event*> get_events(uint8_t floor, uint16_t section, uint16_t wave_number);
+  std::vector<Event*> get_events(uint8_t floor);
 
+  static std::string disassemble_objects_data(const void* data, size_t size, size_t* object_number = nullptr);
+  static std::string disassemble_enemies_data(const void* data, size_t size, size_t* enemy_number = nullptr);
+  static std::string disassemble_wave_events_data(const void* data, size_t size, uint8_t floor = 0xFF);
   static std::string disassemble_quest_data(const void* data, size_t size);
 
   PrefixedLogger log;
   Version version;
-  std::shared_ptr<PSOLFGEncryption> random_crypt;
+  uint32_t rare_seed;
+  std::shared_ptr<PSOLFGEncryption> opt_rand_crypt;
   std::vector<Object> objects;
   std::vector<Enemy> enemies;
+  std::vector<uint16_t> enemy_set_flags;
   std::vector<size_t> rare_enemy_indexes;
+  std::vector<Event> events;
+  std::string event_action_stream;
+  std::multimap<uint64_t, size_t> floor_and_event_id_to_index;
+  std::unordered_multimap<uint64_t, size_t> floor_section_and_group_to_object_index;
+  std::unordered_multimap<uint64_t, size_t> floor_section_and_wave_number_to_enemy_index;
+  std::unordered_multimap<uint64_t, size_t> floor_section_and_wave_number_to_event_index;
 };
 
 class SetDataTableBase {
 public:
   virtual ~SetDataTableBase() = default;
 
-  parray<le_uint32_t, 0x20> generate_variations(Episode episode, bool is_solo, std::shared_ptr<PSOLFGEncryption> random_crypt) const;
+  parray<le_uint32_t, 0x20> generate_variations(
+      Episode episode,
+      bool is_solo,
+      std::shared_ptr<PSOLFGEncryption> opt_rand_crypt = nullptr) const;
   virtual std::pair<uint32_t, uint32_t> num_available_variations_for_floor(Episode episode, uint8_t floor) const = 0;
   virtual std::pair<uint32_t, uint32_t> num_free_roam_variations_for_floor(Episode episode, bool is_solo, uint8_t floor) const = 0;
 
+  enum class FilenameType {
+    OBJECTS = 0,
+    ENEMIES,
+    EVENTS,
+  };
+
   virtual std::string map_filename_for_variation(
-      uint8_t floor, uint32_t var1, uint32_t var2, Episode episode, GameMode mode, bool is_enemies) const = 0;
+      uint8_t floor, uint32_t var1, uint32_t var2, Episode episode, GameMode mode, FilenameType type) const = 0;
   std::vector<std::string> map_filenames_for_variations(
-      const parray<le_uint32_t, 0x20>& variations, Episode episode, GameMode mode, bool is_enemies) const;
+      const parray<le_uint32_t, 0x20>& variations, Episode episode, GameMode mode, FilenameType type) const;
 
   uint8_t default_area_for_floor(Episode episode, uint8_t floor) const;
 
@@ -346,8 +413,8 @@ class SetDataTable : public SetDataTableBase {
 public:
   struct SetEntry {
     std::string object_list_basename;
-    std::string enemy_list_basename;
-    std::string event_list_basename;
+    std::string enemy_and_event_list_basename;
+    std::string area_setup_filename;
   };
 
   SetDataTable(Version version, const std::string& data);
@@ -356,7 +423,7 @@ public:
   virtual std::pair<uint32_t, uint32_t> num_available_variations_for_floor(Episode episode, uint8_t floor) const;
   virtual std::pair<uint32_t, uint32_t> num_free_roam_variations_for_floor(Episode episode, bool is_solo, uint8_t floor) const;
   virtual std::string map_filename_for_variation(
-      uint8_t floor, uint32_t var1, uint32_t var2, Episode episode, GameMode mode, bool is_enemies) const;
+      uint8_t floor, uint32_t var1, uint32_t var2, Episode episode, GameMode mode, FilenameType type) const;
 
   std::string str() const;
 
@@ -377,10 +444,10 @@ public:
   virtual std::pair<uint32_t, uint32_t> num_available_variations_for_floor(Episode episode, uint8_t floor) const;
   virtual std::pair<uint32_t, uint32_t> num_free_roam_variations_for_floor(Episode episode, bool is_solo, uint8_t floor) const;
   virtual std::string map_filename_for_variation(
-      uint8_t floor, uint32_t var1, uint32_t var2, Episode episode, GameMode mode, bool is_enemies) const;
+      uint8_t floor, uint32_t var1, uint32_t var2, Episode episode, GameMode mode, FilenameType type) const;
 
 private:
-  static const std::array<std::vector<std::vector<std::string>>, 0x10> NAMES;
+  static const std::array<std::vector<std::vector<std::string>>, 0x12> NAMES;
 };
 
 class SetDataTableDC112000 : public SetDataTableBase {
@@ -391,10 +458,10 @@ public:
   virtual std::pair<uint32_t, uint32_t> num_available_variations_for_floor(Episode episode, uint8_t floor) const;
   virtual std::pair<uint32_t, uint32_t> num_free_roam_variations_for_floor(Episode episode, bool is_solo, uint8_t floor) const;
   virtual std::string map_filename_for_variation(
-      uint8_t floor, uint32_t var1, uint32_t var2, Episode episode, GameMode mode, bool is_enemies) const;
+      uint8_t floor, uint32_t var1, uint32_t var2, Episode episode, GameMode mode, FilenameType type) const;
 
 private:
-  static const std::array<std::vector<std::vector<std::string>>, 0x10> NAMES;
+  static const std::array<std::vector<std::vector<std::string>>, 0x12> NAMES;
 };
 
 void generate_variations_deprecated(

src/Menu.hh

@@ -14,6 +14,7 @@
 
 namespace MenuID {
 constexpr uint32_t MAIN = 0x11000011;
+constexpr uint32_t CLEAR_LICENSE_CONFIRMATION = 0x11111111;
 constexpr uint32_t INFORMATION = 0x22000022;
 constexpr uint32_t LOBBY = 0x33000033;
 constexpr uint32_t GAME = 0x44000044;
@@ -23,6 +24,7 @@ constexpr uint32_t QUEST_CATEGORIES = 0x66010166;
 constexpr uint32_t PROXY_DESTINATIONS = 0x77000077;
 constexpr uint32_t PROGRAMS = 0x88000088;
 constexpr uint32_t PATCHES = 0x99000099;
+constexpr uint32_t PATCH_SWITCHES = 0x99010199;
 constexpr uint32_t PROXY_OPTIONS = 0xAA0000AA;
 constexpr uint32_t TOURNAMENTS = 0xBB0000BB;
 constexpr uint32_t TOURNAMENTS_FOR_SPEC = 0xBB1111BB;
@@ -35,11 +37,17 @@ constexpr uint32_t INFORMATION = 0x11333311;
 constexpr uint32_t DOWNLOAD_QUESTS = 0x11444411;
 constexpr uint32_t PROXY_DESTINATIONS = 0x11555511;
 constexpr uint32_t PATCHES = 0x11666611;
+constexpr uint32_t PATCH_SWITCHES = 0x11676711;
 constexpr uint32_t PROGRAMS = 0x11777711;
 constexpr uint32_t DISCONNECT = 0x11888811;
 constexpr uint32_t CLEAR_LICENSE = 0x11999911;
 } // namespace MainMenuItemID
 
+namespace ClearLicenseConfirmationMenuItemID {
+constexpr uint32_t CANCEL = 0x01010101;
+constexpr uint32_t CLEAR_LICENSE = 0x02020202;
+} // namespace ClearLicenseConfirmationMenuItemID
+
 namespace InformationMenuItemID {
 constexpr uint32_t GO_BACK = 0x22FFFF22;
 }

src/NetworkAddresses.cc

@@ -10,6 +10,7 @@
 
 #include <memory>
 #include <phosg/Encoding.hh>
+#include <phosg/Network.hh>
 #include <phosg/Strings.hh>
 #include <stdexcept>
 
@@ -23,8 +24,7 @@ uint32_t resolve_address(const char* address) {
         "can\'t resolve hostname %s: %s", address, e.c_str()));
   }
 
-  std::unique_ptr<struct addrinfo, void (*)(struct addrinfo*)> res0_unique(
-      res0, freeaddrinfo);
+  std::unique_ptr<struct addrinfo, void (*)(struct addrinfo*)> res0_unique(res0, freeaddrinfo);
   struct addrinfo* res4 = nullptr;
   for (struct addrinfo* res = res0; res; res = res->ai_next) {
     if (res->ai_family == AF_INET) {
@@ -88,3 +88,17 @@ string string_for_address(uint32_t address) {
 uint32_t address_for_string(const char* address) {
   return ntohl(inet_addr(address));
 }
+
+uint64_t devolution_phone_number_for_netloc(uint32_t addr, uint16_t port) {
+  // It seems the address part of the number is fixed-width, but the port is
+  // not. Why did they do it this way?
+  if (port & 0xF000) {
+    return (static_cast<uint64_t>(addr) << 16) | port;
+  } else if (port & 0x0F00) {
+    return (static_cast<uint64_t>(addr) << 12) | port;
+  } else if (port & 0x00F0) {
+    return (static_cast<uint64_t>(addr) << 8) | port;
+  } else {
+    return (static_cast<uint64_t>(addr) << 4) | port;
+  }
+}

src/NetworkAddresses.hh

@@ -17,3 +17,5 @@ bool is_local_address(const sockaddr_storage& daddr);
 
 std::string string_for_address(uint32_t address);
 uint32_t address_for_string(const char* address);
+
+uint64_t devolution_phone_number_for_netloc(uint32_t addr, uint16_t port);

src/PSOEncryption.hh

@@ -5,6 +5,7 @@
 
 #include <memory>
 #include <phosg/Encoding.hh>
+#include <phosg/Random.hh>
 #include <stdexcept>
 #include <string>
 #include <vector>
@@ -113,13 +114,13 @@ public:
       parray<le_uint32_t, 0x12> as32;
       InitialKeys() : as32() {}
       InitialKeys(const InitialKeys& other) : as32(other.as32) {}
-    } __attribute__((packed));
+    } __packed_ws__(InitialKeys, 0x48);
     union PrivateKeys {
       parray<uint8_t, 0x1000> as8;
       parray<le_uint32_t, 0x400> as32;
       PrivateKeys() : as32() {}
       PrivateKeys(const PrivateKeys& other) : as32(other.as32) {}
-    } __attribute__((packed));
+    } __packed_ws__(PrivateKeys, 0x1000);
     InitialKeys initial_keys;
     PrivateKeys private_keys;
     // This field only really needs to be one byte, but annoyingly, some
@@ -128,7 +129,7 @@ public:
     // this structure's size from not matching the .nsk files' sizes, we use
     // an unnecessarily large size for this field.
     le_uint64_t subtype;
-  } __attribute__((packed));
+  } __packed_ws__(KeyFile, 0x1050);
 
   PSOBBEncryption(const KeyFile& key, const void* seed, size_t seed_size);
 
@@ -253,39 +254,49 @@ uint32_t encrypt_challenge_time(uint16_t value);
 uint16_t decrypt_challenge_time(uint32_t value);
 
 template <bool IsBigEndian>
-class ChallengeTime {
+class ChallengeTimeT {
 private:
   using U32T = typename std::conditional<IsBigEndian, be_uint32_t, le_uint32_t>::type;
   U32T value;
 
 public:
-  ChallengeTime() = default;
-  ChallengeTime(uint16_t v) {
-    this->store(v);
+  ChallengeTimeT() = default;
+  ChallengeTimeT(uint16_t v) {
+    this->encode(v);
   }
-  ChallengeTime(const ChallengeTime& other) = default;
-  ChallengeTime(ChallengeTime&& other) = default;
-  ChallengeTime& operator=(const ChallengeTime& other) = default;
-  ChallengeTime& operator=(ChallengeTime&& other) = default;
+  ChallengeTimeT(const ChallengeTimeT& other) = default;
+  ChallengeTimeT(ChallengeTimeT&& other) = default;
+  ChallengeTimeT& operator=(const ChallengeTimeT& other) = default;
+  ChallengeTimeT& operator=(ChallengeTimeT&& other) = default;
 
   bool has_value() const {
     return this->value != 0;
   }
 
-  uint16_t load() const {
+  uint32_t load_raw() const {
+    return this->value;
+  }
+  void store_raw(uint32_t value) {
+    this->value = value;
+  }
+
+  uint16_t decode() const {
     return decrypt_challenge_time(this->value);
   }
-  operator uint16_t() const {
-    return this->load();
-  }
-  void store(uint16_t v) {
+  void encode(uint16_t v) {
     this->value = ((v == 0) || (v == 0xFFFF)) ? 0 : encrypt_challenge_time(v);
   }
-  ChallengeTime& operator=(uint16_t v) {
-    this->store(v);
-    return *this;
+
+  operator ChallengeTimeT<!IsBigEndian>() const {
+    ChallengeTimeT<!IsBigEndian> ret;
+    ret.store_raw(this->value);
+    return ret;
   }
-} __attribute__((packed));
+} __packed__;
+using ChallengeTime = ChallengeTimeT<false>;
+using ChallengeTimeBE = ChallengeTimeT<true>;
+check_struct_size(ChallengeTime, 4);
+check_struct_size(ChallengeTimeBE, 4);
 
 std::string decrypt_v2_registry_value(const void* data, size_t size);
 
@@ -342,3 +353,7 @@ std::string encrypt_pr2_data(const std::string& data, size_t decompressed_size,
   }
   return ret;
 }
+
+inline uint32_t random_from_optional_crypt(std::shared_ptr<PSOLFGEncryption> random_crypt) {
+  return random_crypt ? random_crypt->next() : random_object<uint32_t>();
+}

src/PSOGCObjectGraph.cc

@@ -11,7 +11,7 @@ struct TObjectVTable {
   be_uint32_t update;
   be_uint32_t render;
   be_uint32_t render_shadow;
-} __attribute__((packed));
+} __packed_ws__(TObjectVTable, 0x18);
 
 struct TObject {
   be_uint32_t type_name_addr;
@@ -22,7 +22,7 @@ struct TObject {
   be_uint32_t parent_addr;
   be_uint32_t children_head_addr;
   be_uint32_t vtable_addr;
-} __attribute__((packed));
+} __packed_ws__(TObject, 0x1C);
 
 PSOGCObjectGraph::PSOGCObjectGraph(
     const string& memory_data, uint32_t root_address) {

src/PSOProtocol.hh

@@ -13,19 +13,19 @@ struct PSOCommandHeaderPC {
   le_uint16_t size;
   uint8_t command;
   uint8_t flag;
-} __attribute__((packed));
+} __packed_ws__(PSOCommandHeaderPC, 4);
 
 struct PSOCommandHeaderDCV3 {
   uint8_t command;
   uint8_t flag;
   le_uint16_t size;
-} __attribute__((packed));
+} __packed_ws__(PSOCommandHeaderDCV3, 4);
 
 struct PSOCommandHeaderBB {
   le_uint16_t size;
   le_uint16_t command;
   le_uint32_t flag;
-} __attribute__((packed));
+} __packed_ws__(PSOCommandHeaderBB, 8);
 
 union PSOCommandHeader {
   PSOCommandHeaderDCV3 dc;
@@ -45,7 +45,7 @@ union PSOCommandHeader {
   }
 
   PSOCommandHeader();
-} __attribute__((packed));
+} __packed_ws__(PSOCommandHeader, 8);
 
 // This function is used in a lot of places to check received command sizes and
 // cast them to the appropriate type

src/PatchServer.cc

@@ -0,0 +1,478 @@
+#include "PatchServer.hh"
+
+#include <arpa/inet.h>
+#include <ctype.h>
+#include <errno.h>
+#include <event2/buffer.h>
+#include <event2/bufferevent.h>
+#include <event2/event.h>
+#include <event2/listener.h>
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+
+#include <algorithm>
+#include <iostream>
+#include <phosg/Encoding.hh>
+#include <phosg/Network.hh>
+#include <phosg/Strings.hh>
+#include <phosg/Time.hh>
+
+#include "EventUtils.hh"
+#include "Loggers.hh"
+#include "PSOProtocol.hh"
+#include "ReceiveCommands.hh"
+
+using namespace std;
+
+static atomic<uint64_t> next_id(1);
+
+PatchServer::Client::Client(
+    shared_ptr<PatchServer> server,
+    struct bufferevent* bev,
+    Version version,
+    uint64_t idle_timeout_usecs,
+    bool hide_data_from_logs)
+    : server(server),
+      id(next_id++),
+      log(string_printf("[C-%" PRIX64 "] ", this->id), client_log.min_level),
+      channel(bev, 0, version, 1, nullptr, nullptr, this, string_printf("C-%" PRIX64, this->id), TerminalFormat::FG_YELLOW, TerminalFormat::FG_GREEN),
+      idle_timeout_usecs(idle_timeout_usecs),
+      idle_timeout_event(
+          event_new(bufferevent_get_base(bev), -1, EV_TIMEOUT, &PatchServer::Client::dispatch_idle_timeout, this),
+          event_free) {
+  this->reschedule_timeout_event();
+
+  // Don't print data sent to patch clients to the logs. The patch server
+  // protocol is fully understood and data logs for patch clients are generally
+  // more annoying than helpful at this point.
+  if (hide_data_from_logs) {
+    this->channel.terminal_recv_color = TerminalFormat::END;
+    this->channel.terminal_send_color = TerminalFormat::END;
+  }
+
+  this->log.info("Created");
+}
+
+void PatchServer::Client::reschedule_timeout_event() {
+  struct timeval idle_tv = usecs_to_timeval(this->idle_timeout_usecs);
+  event_add(this->idle_timeout_event.get(), &idle_tv);
+}
+
+void PatchServer::Client::dispatch_idle_timeout(evutil_socket_t, short, void* ctx) {
+  reinterpret_cast<Client*>(ctx)->idle_timeout();
+}
+
+void PatchServer::Client::idle_timeout() {
+  this->log.info("Idle timeout expired");
+  auto s = this->server.lock();
+  if (s) {
+    auto c = this->shared_from_this();
+    s->disconnect_client(c);
+  } else {
+    this->channel.disconnect();
+    this->log.info("Server is deleted; cannot disconnect client");
+  }
+}
+
+void PatchServer::send_server_init(shared_ptr<Client> c) const {
+  uint32_t server_key = random_object<uint32_t>();
+  uint32_t client_key = random_object<uint32_t>();
+
+  S_ServerInit_Patch_02 cmd;
+  cmd.copyright.encode("Patch Server. Copyright SonicTeam, LTD. 2001");
+  cmd.server_key = server_key;
+  cmd.client_key = client_key;
+  c->channel.send(0x02, 0x00, cmd);
+
+  c->channel.crypt_out = make_shared<PSOV2Encryption>(server_key);
+  c->channel.crypt_in = make_shared<PSOV2Encryption>(client_key);
+}
+
+void PatchServer::send_message_box(shared_ptr<Client> c, const string& text) const {
+  StringWriter w;
+  try {
+    if (c->version() == Version::PC_PATCH) {
+      w.write(tt_encode_marked_optional(text, c->channel.language, true));
+    } else if (c->version() == Version::BB_PATCH) {
+      w.write(tt_encode_marked_optional(add_color(text), c->channel.language, true));
+    } else {
+      throw logic_error("non-patch client on patch server");
+    }
+  } catch (const runtime_error& e) {
+    log_warning("Failed to encode message for patch message box command: %s", e.what());
+    return;
+  }
+  w.put_u16(0);
+  while (w.str().size() & 3) {
+    w.put_u8(0);
+  }
+  c->channel.send(0x13, 0x00, w.str());
+}
+
+void PatchServer::send_enter_directory(shared_ptr<Client> c, const string& dir) const {
+  S_EnterDirectory_Patch_09 cmd = {{dir, 1}};
+  c->channel.send(0x09, 0x00, cmd);
+}
+
+void PatchServer::on_02(shared_ptr<Client> c, string& data) {
+  check_size_v(data.size(), 0);
+  c->channel.send(0x04, 0x00); // This requests the user's login information
+}
+
+void PatchServer::change_to_directory(
+    shared_ptr<Client> c,
+    vector<string>& client_path_directories,
+    const vector<string>& file_path_directories) const {
+  // First, exit all leaf directories that don't match the desired path
+  while (!client_path_directories.empty() &&
+      ((client_path_directories.size() > file_path_directories.size()) ||
+          (client_path_directories.back() != file_path_directories[client_path_directories.size() - 1]))) {
+    c->channel.send(0x0A, 0x00);
+    client_path_directories.pop_back();
+  }
+
+  // At this point, client_path_directories should be a prefix of
+  // file_path_directories (or should match exactly)
+  if (client_path_directories.size() > file_path_directories.size()) {
+    throw logic_error("did not exit all necessary directories");
+  }
+  for (size_t x = 0; x < client_path_directories.size(); x++) {
+    if (client_path_directories[x] != file_path_directories[x]) {
+      throw logic_error("intermediate path is not a prefix of final path");
+    }
+  }
+
+  // Second, enter all necessary leaf directories
+  while (client_path_directories.size() < file_path_directories.size()) {
+    const string& dir = file_path_directories[client_path_directories.size()];
+    this->send_enter_directory(c, dir);
+    client_path_directories.emplace_back(dir);
+  }
+}
+
+void PatchServer::on_04(shared_ptr<Client> c, string& data) {
+  const auto& cmd = check_size_t<C_Login_Patch_04>(data);
+
+  string username = cmd.username.decode();
+  string password = cmd.password.decode();
+
+  // There are 3 cases here:
+  // - No login information at all: just proceed without checking credentials
+  // - Username: check that account exists if allow_unregistered_users is off
+  // - Username and password: call verify_bb
+  if (!username.empty() && !password.empty()) {
+    try {
+      this->config->account_index->from_bb_credentials(username, &password, false);
+
+    } catch (const AccountIndex::incorrect_password& e) {
+      this->send_message_box(c, string_printf("Login failed: %s", e.what()));
+      this->disconnect_client(c);
+      return;
+
+    } catch (const AccountIndex::missing_account& e) {
+      if (!this->config->allow_unregistered_users) {
+        this->send_message_box(c, string_printf("Login failed: %s", e.what()));
+        this->disconnect_client(c);
+        return;
+      }
+    }
+
+  } else if (!username.empty() && !this->config->allow_unregistered_users) {
+    try {
+      this->config->account_index->from_bb_credentials(username, nullptr, false);
+    } catch (const AccountIndex::missing_account& e) {
+      this->send_message_box(c, string_printf("Login failed: %s", e.what()));
+      this->disconnect_client(c);
+      return;
+    }
+  }
+
+  if (!this->config->message.empty()) {
+    this->send_message_box(c, this->config->message.c_str());
+  }
+
+  const auto& index = this->config->patch_file_index;
+  if (index.get()) {
+    c->channel.send(0x0B, 0x00); // Start patch session; go to root directory
+
+    vector<string> path_directories;
+    for (const auto& file : index->all_files()) {
+      this->change_to_directory(c, path_directories, file->path_directories);
+
+      S_FileChecksumRequest_Patch_0C req = {c->patch_file_checksum_requests.size(), {file->name, 1}};
+      c->channel.send(0x0C, 0x00, req);
+      c->patch_file_checksum_requests.emplace_back(file);
+    }
+    this->change_to_directory(c, path_directories, {});
+
+    c->channel.send(0x0D, 0x00); // End of checksum requests
+
+  } else {
+    // No patch index present: just do something that will satisfy the client
+    // without actually checking or downloading any files
+    this->send_enter_directory(c, ".");
+    this->send_enter_directory(c, "data");
+    this->send_enter_directory(c, "scene");
+    c->channel.send(0x0A, 0x00);
+    c->channel.send(0x0A, 0x00);
+    c->channel.send(0x0A, 0x00);
+    c->channel.send(0x12, 0x00);
+  }
+}
+
+void PatchServer::on_0F(shared_ptr<Client> c, string& data) {
+  auto& cmd = check_size_t<C_FileInformation_Patch_0F>(data);
+  auto& req = c->patch_file_checksum_requests.at(cmd.request_id);
+  req.crc32 = cmd.checksum;
+  req.size = cmd.size;
+  req.response_received = true;
+}
+
+void PatchServer::on_10(shared_ptr<Client> c, string&) {
+  S_StartFileDownloads_Patch_11 start_cmd = {0, 0};
+  for (const auto& req : c->patch_file_checksum_requests) {
+    if (!req.response_received) {
+      throw runtime_error("client did not respond to checksum request");
+    }
+    if (req.needs_update()) {
+      c->log.info("File %s needs update (CRC: %08" PRIX32 "/%08" PRIX32 ", size: %" PRIu32 "/%" PRIu32 ")",
+          req.file->name.c_str(), req.file->crc32, req.crc32, req.file->size, req.size);
+      start_cmd.total_bytes += req.file->size;
+      start_cmd.num_files++;
+    } else {
+      c->log.info("File %s is up to date", req.file->name.c_str());
+    }
+  }
+
+  if (start_cmd.num_files) {
+    c->channel.send(0x11, 0x00, start_cmd);
+    vector<string> path_directories;
+    for (const auto& req : c->patch_file_checksum_requests) {
+      if (req.needs_update()) {
+        this->change_to_directory(c, path_directories, req.file->path_directories);
+
+        S_OpenFile_Patch_06 open_cmd = {0, req.file->size, {req.file->name, 1}};
+        c->channel.send(0x06, 0x00, open_cmd);
+
+        for (size_t x = 0; x < req.file->chunk_crcs.size(); x++) {
+          auto data = req.file->load_data();
+          size_t chunk_size = min<uint32_t>(req.file->size - (x * 0x4000), 0x4000);
+
+          vector<pair<const void*, size_t>> blocks;
+          S_WriteFileHeader_Patch_07 cmd_header = {x, req.file->chunk_crcs[x], chunk_size};
+          blocks.emplace_back(&cmd_header, sizeof(cmd_header));
+          blocks.emplace_back(data->data() + (x * 0x4000), chunk_size);
+          c->channel.send(0x07, 0x00, blocks);
+        }
+
+        S_CloseCurrentFile_Patch_08 close_cmd = {0};
+        c->channel.send(0x08, 0x00, close_cmd);
+      }
+    }
+    this->change_to_directory(c, path_directories, {});
+  }
+
+  c->channel.send(0x12, 0x00);
+}
+
+void PatchServer::disconnect_client(shared_ptr<Client> c) {
+  if (c->channel.virtual_network_id) {
+    server_log.info("Client disconnected: C-%" PRIX64 " on N-%" PRIu64, c->id, c->channel.virtual_network_id);
+  } else if (c->channel.bev) {
+    server_log.info("Client disconnected: C-%" PRIX64, c->id);
+  } else {
+    server_log.info("Client C-%" PRIX64 " removed from patch server", c->id);
+  }
+
+  this->channel_to_client.erase(&c->channel);
+  c->channel.disconnect();
+
+  // We can't just let c be destroyed here, since disconnect_client can be
+  // called from within the client's channel's receive handler. So, we instead
+  // move it to another set, which we'll clear in an immediately-enqueued
+  // callback after the current event. This will also call the client's
+  // disconnect hooks (if any).
+  this->clients_to_destroy.insert(std::move(c));
+  this->enqueue_destroy_clients();
+}
+
+void PatchServer::enqueue_destroy_clients() {
+  auto tv = usecs_to_timeval(0);
+  event_add(this->destroy_clients_ev.get(), &tv);
+}
+
+void PatchServer::dispatch_destroy_clients(evutil_socket_t, short, void* ctx) {
+  reinterpret_cast<PatchServer*>(ctx)->clients_to_destroy.clear();
+}
+
+void PatchServer::dispatch_on_listen_accept(
+    struct evconnlistener* listener, evutil_socket_t fd,
+    struct sockaddr* address, int socklen, void* ctx) {
+  reinterpret_cast<PatchServer*>(ctx)->on_listen_accept(listener, fd, address, socklen);
+}
+
+void PatchServer::dispatch_on_listen_error(
+    struct evconnlistener* listener, void* ctx) {
+  reinterpret_cast<PatchServer*>(ctx)->on_listen_error(listener);
+}
+
+void PatchServer::on_listen_accept(struct evconnlistener* listener, evutil_socket_t fd, struct sockaddr*, int) {
+  struct sockaddr_storage remote_addr;
+  get_socket_addresses(fd, nullptr, &remote_addr);
+  if (this->config->banned_ipv4_ranges->check(remote_addr)) {
+    close(fd);
+    return;
+  }
+
+  int listen_fd = evconnlistener_get_fd(listener);
+  ListeningSocket* listening_socket;
+  try {
+    listening_socket = &this->listening_sockets.at(listen_fd);
+  } catch (const out_of_range& e) {
+    server_log.warning("Can\'t determine version for socket %d; disconnecting client", listen_fd);
+    close(fd);
+    return;
+  }
+
+  struct bufferevent* bev = bufferevent_socket_new(this->base.get(), fd, BEV_OPT_CLOSE_ON_FREE | BEV_OPT_DEFER_CALLBACKS);
+  auto c = make_shared<Client>(
+      this->shared_from_this(),
+      bev,
+      listening_socket->version,
+      this->config->idle_timeout_usecs,
+      this->config->hide_data_from_logs);
+  c->channel.on_command_received = PatchServer::on_client_input;
+  c->channel.on_error = PatchServer::on_client_error;
+  c->channel.context_obj = this;
+  this->channel_to_client.emplace(&c->channel, c);
+
+  server_log.info("Patch client connected: U-%" PRIX64 " on fd %d via %d (%s)",
+      c->id, fd, listen_fd, listening_socket->addr_str.c_str());
+
+  this->send_server_init(c);
+}
+
+void PatchServer::on_listen_error(struct evconnlistener* listener) {
+  int err = EVUTIL_SOCKET_ERROR();
+  server_log.error("Failure on listening socket %d: %d (%s)",
+      evconnlistener_get_fd(listener), err, evutil_socket_error_to_string(err));
+  event_base_loopexit(this->base.get(), nullptr);
+}
+
+void PatchServer::on_client_input(Channel& ch, uint16_t command, uint32_t, std::string& data) {
+  PatchServer* server = reinterpret_cast<PatchServer*>(ch.context_obj);
+  shared_ptr<Client> c = server->channel_to_client.at(&ch);
+
+  try {
+    switch (command) {
+      case 0x02:
+        server->on_02(c, data);
+        break;
+      case 0x04:
+        server->on_04(c, data);
+        break;
+      case 0x0F:
+        server->on_0F(c, data);
+        break;
+      case 0x10:
+        server->on_10(c, data);
+        break;
+      default:
+        throw runtime_error("invalid command");
+    }
+  } catch (const exception& e) {
+    server_log.warning("Error processing client command: %s", e.what());
+  }
+}
+
+void PatchServer::on_client_error(Channel& ch, short events) {
+  PatchServer* server = reinterpret_cast<PatchServer*>(ch.context_obj);
+  shared_ptr<Client> c = server->channel_to_client.at(&ch);
+
+  if (events & BEV_EVENT_ERROR) {
+    int err = EVUTIL_SOCKET_ERROR();
+    server_log.warning("Client caused error %d (%s)", err, evutil_socket_error_to_string(err));
+  }
+  if (events & (BEV_EVENT_EOF | BEV_EVENT_ERROR)) {
+    server->disconnect_client(c);
+  }
+}
+
+PatchServer::PatchServer(shared_ptr<const Config> config)
+    : config(config) {
+  if (config->shared_base) {
+    this->base = config->shared_base;
+    this->base_is_shared = true;
+  } else {
+    this->base.reset(event_base_new(), event_base_free);
+    this->base_is_shared = false;
+  }
+  this->destroy_clients_ev.reset(
+      event_new(this->base.get(), -1, EV_TIMEOUT, &PatchServer::dispatch_destroy_clients, this), event_free);
+  if (!this->base_is_shared) {
+    this->th = thread(&PatchServer::thread_fn, this);
+  }
+}
+
+void PatchServer::schedule_stop() {
+  if (!this->base_is_shared) {
+    event_base_loopexit(this->base.get(), nullptr);
+  }
+}
+
+void PatchServer::wait_for_stop() {
+  if (!this->base_is_shared) {
+    this->th.join();
+  }
+}
+
+void PatchServer::listen(const std::string& addr_str, const string& socket_path, Version version) {
+  int fd = ::listen(socket_path, 0, SOMAXCONN);
+  server_log.info("Listening on Unix socket %s on fd %d as %s", socket_path.c_str(), fd, addr_str.c_str());
+  this->add_socket(addr_str, fd, version);
+}
+
+void PatchServer::listen(const std::string& addr_str, const string& addr, int port, Version version) {
+  if (port == 0) {
+    this->listen(addr_str, addr, version);
+  } else {
+    int fd = ::listen(addr, port, SOMAXCONN);
+    string netloc_str = render_netloc(addr, port);
+    server_log.info("Listening on TCP interface %s on fd %d as %s", netloc_str.c_str(), fd, addr_str.c_str());
+    this->add_socket(addr_str, fd, version);
+  }
+}
+
+void PatchServer::listen(const std::string& addr_str, int port, Version version) {
+  this->listen(addr_str, "", port, version);
+}
+
+PatchServer::ListeningSocket::ListeningSocket(PatchServer* s, const std::string& addr_str, int fd, Version version)
+    : addr_str(addr_str),
+      fd(fd),
+      version(version),
+      listener(evconnlistener_new(s->base.get(), PatchServer::dispatch_on_listen_accept, s, LEV_OPT_REUSEABLE, 0, this->fd),
+          evconnlistener_free) {
+  evconnlistener_set_error_cb(this->listener.get(), PatchServer::dispatch_on_listen_error);
+}
+
+void PatchServer::add_socket(const std::string& addr_str, int fd, Version version) {
+  this->listening_sockets.emplace(piecewise_construct, forward_as_tuple(fd), forward_as_tuple(this, addr_str, fd, version));
+}
+
+void PatchServer::thread_fn() {
+  event_base_loop(this->base.get(), EVLOOP_NO_EXIT_ON_EMPTY);
+}
+
+void PatchServer::set_config(std::shared_ptr<const Config> config) {
+  if (this->base_is_shared) {
+    this->config = config;
+  } else {
+    forward_to_event_thread(this->base, [s = this->shared_from_this(), config = std::move(config)]() {
+      s->config = config;
+    });
+  }
+}