add HTML rare table generator

README.md

@@ -54,11 +54,11 @@ At the time of its inception, Aeon was also called newserv, and you may find som
 
 Independently of this project, there are many other PSO servers out there. Those that I know of that are (or were) public are listed here in approximate chronological order:
 
-* (Early 2000s) **[Schtserv](https://schtserv.com/)**: The first public-access PSO server; written in Delphi by Schthack. Still active and popular as of this writing (early 2024). Schtserv is also the only other unofficial server to support all versions of PSO, including Episode 3.
+* (Early 2000s) **[Schtserv](https://schtserv.com/)**: The first public-access PSO server; written in Delphi by Schthack. Still active and popular as of early 2025. Schtserv is also the only other unofficial server to support all versions of PSO, including Episode 3. (Their implementation of Episode 3 is based on newserv's, which is itself based on Sega's.)
 * (2005) **Khyller**: An early attempt of mine to support PSO PC, GC, and BB. See above for more details.
 * (2006) **Aeon**: My second attempt. Better than Khyller, but still unreliable.
 * (2008) **Tethealla**: A fairly extensive implementation of PSOBB, written in C by Sodaboy. The public version of Tethealla has been [officially disowned](https://www.pioneer2.net/community/threads/tethealla-server-forums-removal.26365/) (as it is now more than 15 years old), but closed-source development continues. [Ephinea](https://ephinea.pioneer2.net/), currently the most popular PSOBB server, is the continuation of this project. Several other modern PSOBB servers are forks of the initial public version of Tethealla as well.
-* (2008) **[Sylverant](https://sylverant.net/)** [(source)](https://sourceforge.net/projects/sylverant/): The second public-access PSO server; written in C by BlueCrab. Still active and popular as of this writing (early 2024).
+* (2008) **[Sylverant](https://sylverant.net/)** [(source)](https://sourceforge.net/projects/sylverant/): The second public-access PSO server; written in C by BlueCrab. Still active and popular as of early 2025.
 * (2015) **[Archon](https://github.com/dcrodman/archon)**: A PSOBB server written in Go by Drew Rodman.
 * (2015) **[Idola](https://github.com/HybridEidolon/idolapsoserv)**: A PSOBB server written in Rust by HybridEidolon. Functionality status unknown; the project has been archived.
 * (2017) **[Aselia](https://github.com/Solybum/Aselia)**: A PSOBB server written written in C# by Soly. It seems this was planned to be open-source at some point, but that has not (yet) happened.
@@ -71,12 +71,14 @@ Independently of this project, there are many other PSO servers out there. Those
 
 There is a lot of code in this project that could be useful as a reference. Some of the more notable files are:
 * **src/CommandFormats.hh**: Complete listing of all network commands used in all known versions of the game, and their formats
+* **src/CommonItemSet.hh/cc**: Format of ItemPT files, shop definition files, and tekker adjustment tables
 * **src/DCSerialNumbers.hh/cc**: PSO DC serial number validation algorithm and serial number generator
 * **src/ItemData.hh**: Item format reference
 * **src/ItemCreator.hh/cc**: Reverse-engineered item generator from Episodes 1&2 (used for all versions)
 * **src/ItemParameterTable.hh**: Format of many structures in ItemPMT.prs
 * **src/Map.hh/cc**: Map file (.dat) structure and reverse-engineered Challenge Mode random enemy generation algorithm
 * **src/QuestScript.cc**: Complete listing of all quest opcodes on all versions, along with their arguments and behavior
+* **src/RareItemSet.hh/cc**: Format of ItemRT files (rare item drop tables)
 * **src/SaveFileFormats.hh**: Definitions of save file structures for all versions
 * **src/Episode3/DataIndexes.hh**: Episode 3 file structures, including card definition format and map/quest format
 * **system/item-tables/names-v4.json**: Names of all items, indexed by the first 3 bytes of data1
@@ -291,12 +293,12 @@ Within the category directories, quest files should be named like `q###-VERSION-
 
 For .dat files, the `LANGUAGE` token may be omitted. If it's present, then that .dat file will only be used for that language of the quest; if omitted, then that .dat file will be used for all languages of the quest.
 
-Some quests (mostly battle and challenge mode quests) have additional JSON metadata files that describe how the server should handle them. This includes flags that can be used to hide the quest unless a preceding quest has been cleared, or to hide the quest unless purchased as a team reward. These metadata files are generally named similarly to their .bin and .dat counterparts, except the `VERSION` token may also be omitted if the metadata applies to all languages of the quest on all PSO versions. See system/quests/battle/b88001.json for documentation on the exact format of the JSON file.
+For example, the GameCube version of Lost HEAT SWORD is in two files named `q058-gc-e.bin` and `q058-gc.dat`. newserv knows these files are quests because they're in the system/quests/ directory, it knows they're for PSO GC because the filenames contain `-gc`, it knows this is the English version of the quest because the .bin filename ends with `-e` (even though the .dat filename does not), and it puts them in the Retrieval category because the files are within the retrieval/ directory within system/quests/.
+
+Some quests (mostly battle and challenge mode quests) have additional JSON metadata files that describe how the server should handle them. These files include flags that can be used to hide the quest unless a preceding quest has been cleared, or to hide the quest unless purchased as a BB team reward. These metadata files are generally named similarly to their .bin and .dat counterparts, except the `VERSION` token may also be omitted if the metadata applies to all languages of the quest on all PSO versions. See system/quests/battle/b88001.json for documentation on the exact format of the JSON file.
 
 Some quests may also include a .pvr file, which contains an image used in the quest. These files are named similarly to their .bin and .dat counterparts.
 
-For example, the GameCube version of Lost HEAT SWORD is in two files named `q058-gc-e.bin` and `q058-gc.dat`. newserv knows these files are quests because they're in the system/quests/ directory, it knows they're for PSO GC because the filenames contain `-gc`, it knows this is the English version of the quest because the .bin filename ends with `-e` (even though the .dat filename does not), and it puts them in the Retrieval category because the files are within the retrieval/ directory within system/quests/.
-
 The GameCube and Xbox quest formats are very similar, but newserv treats them as different. If you want to use the same quest file for GameCube and Xbox clients, you can make one a symbolic link to the other.
 
 There are multiple PSO quest formats out there; newserv supports all of them. It can also decode any known format to standard .bin/.dat format. Specifically:
@@ -574,7 +576,7 @@ Some commands only work on the game server and not on the proxy server. The chat
 
 * Personal state commands
     * `$arrow <color-id>`: Change your lobby arrow color.
-    * `$secid <section-id>`: Set your override section ID. After running this command, any games you create will use your override section ID for rare drops instead of your character's actual section ID. If you're in a game and you are the leader of the game, this also immediately changes the item tables used by the server when creating items. To revert to your actual section id, run `$secid` with no name after it. On the proxy server, this will not work if the remote server controls item drops (e.g. on BB, or on Schtserv with server drops enabled). If the server does not allow cheat mode anywhere (that is, "CheatModeBehavior" is "Off" in config.json), this command does nothing.
+    * `$secid <section-id>`: Set your override section ID. After running this command, any games you create will use your override section ID for rare drops instead of your character's actual section ID. If you're in a game and you are the leader of the game, this also immediately changes the item tables used by the server when creating items. To revert to your actual section id, run `$secid` with no name after it. On the proxy server, this will not work if the remote server controls item drops. If the server does not allow cheat mode anywhere (that is, "CheatModeBehavior" is "Off" in config.json), this command does nothing.
     * `$rand <seed>`: Set your override random seed (specified as a 32-bit hex value). This will make any games you create use the given seed for rare enemies. This also makes item drops deterministic in Blue Burst games hosted by newserv. On the proxy server, this command can cause desyncs with other players in the same game, since they will not see the overridden random seed. To remove the override, run `$rand` with no arguments. If the server does not allow cheat mode anywhere (that is, "CheatModeBehavior" is "Off" in config.json), this command does nothing.
     * `$ln [name-or-type]`: Set the lobby number. Visible only to you. This command exists because some non-lobby maps can be loaded as lobbies with invalid lobby numbers. See the "GC lobby types" and "Ep3 lobby types" entries in the information menu for acceptable values here. Note that non-lobby maps do not have a lobby counter, so there's no way to exit the lobby without using either `$ln` again or `$exit`. On the game server, `$ln` reloads the lobby immediately; on the proxy server, it doesn't take effect until you load another lobby yourself (which means you'll like have to use `$exit` to escape). Run this command with no argument to return to the default lobby.
     * `$swa`: Enable or disable switch assist. When enabled, the server will unlock two-player and four-player doors in non-quest games when you step on any of the required switches.
@@ -719,33 +721,33 @@ newserv has many CLI options, which can be used to access functionality other th
 
 The data formats that newserv can convert to/from are:
 
-| Format                         | Encode/compress action    | Decode/extract action        |
+| Format                              | Encode/compress action    | Decode/extract action        |
-|--------------------------------|---------------------------|------------------------------|
+|-------------------------------------|---------------------------|------------------------------|
-| PRS compression                | `compress-prs`            | `decompress-prs`             |
+| PRS compression                     | `compress-prs`            | `decompress-prs`             |
-| PR2/PRC compression            | `compress-pr2`            | `decompress-pr2`             |
+| PR2/PRC compression                 | `compress-pr2`            | `decompress-pr2`             |
-| BC0 compression                | `compress-bc0`            | `decompress-bc0`             |
+| BC0 compression                     | `compress-bc0`            | `decompress-bc0`             |
-| Raw encrypted data             | `encrypt-data`            | `decrypt-data`               |
+| Raw encrypted data                  | `encrypt-data`            | `decrypt-data`               |
-| Episode 3 command mask         | `encrypt-trivial-data`    | `decrypt-trivial-data`       |
+| Episode 3 command mask              | `encrypt-trivial-data`    | `decrypt-trivial-data`       |
-| Challenge Mode rank text       | `encrypt-challenge-data`  | `decrypt-challenge-data`     |
+| Challenge Mode rank text            | `encrypt-challenge-data`  | `decrypt-challenge-data`     |
-| PSO DC quest file (.vms)       | None                      | `decode-vms`                 |
+| PSO DC quest file (.vms)            | None                      | `decode-vms`                 |
-| PSO GC quest file (.gci)       | None                      | `decode-gci`                 |
+| PSO GC quest file (.gci)            | None                      | `decode-gci`                 |
-| Download quest file (.dlq)     | None                      | `decode-dlq`                 |
+| Download quest file (.dlq)          | None                      | `decode-dlq`                 |
-| Server quest file (.qst)       | `encode-qst`              | `decode-qst`                 |
+| Server quest file (.qst)            | `encode-qst`              | `decode-qst`                 |
-| PSO DC save file (.vms)        | `encrypt-vms-save`        | `decrypt-vms-save`           |
+| PSO DC save file (.vms)             | `encrypt-vms-save`        | `decrypt-vms-save`           |
-| PSO PC save file               | `encrypt-pc-save`         | `decrypt-pc-save`            |
+| PSO PC save file                    | `encrypt-pc-save`         | `decrypt-pc-save`            |
-| PSO GC save file (.gci)        | `encrypt-gci-save`        | `decrypt-gci-save`           |
+| PSO GC save file (.gci)             | `encrypt-gci-save`        | `decrypt-gci-save`           |
-| PSO GC snapshot file           | None                      | `decode-gci-snapshot`        |
+| PSO GC snapshot file                | None                      | `decode-gci-snapshot`        |
-| Quest script (.bin)            | `assemble-quest-script`   | `disassemble-quest-script`   |
+| Quest script (.bin)                 | `assemble-quest-script`   | `disassemble-quest-script`   |
-| Quest map (.dat)               | None                      | `disassemble-quest-map`      |
+| Quest map (.dat)                    | None                      | `disassemble-quest-map`      |
-| AFS archive                    | None                      | `extract-afs`                |
+| AFS archive                         | None                      | `extract-afs`                |
-| BML archive                    | None                      | `extract-bml`                |
+| BML archive                         | None                      | `extract-bml`                |
-| GSL archive                    | `generate-gsl`            | `extract-gsl`                |
+| GSL archive                         | `generate-gsl`            | `extract-gsl`                |
-| GVM texture                    | `encode-gvm`              | None                         |
+| GVM texture                         | `encode-gvm`              | None                         |
-| Text archive                   | `encode-text-archive`     | `decode-text-archive`        |
+| Text archive                        | `encode-text-archive`     | `decode-text-archive`        |
-| Unicode text set               | `encode-unicode-text-set` | `decode-unicode-text-set`    |
+| Unicode text set                    | `encode-unicode-text-set` | `decode-unicode-text-set`    |
-| Word Select data set           | None                      | `decode-word-select-set`     |
+| Word Select data set                | None                      | `decode-word-select-set`     |
-| Set data table                 | None                      | `disassemble-set-data-table` |
+| Set data table                      | None                      | `disassemble-set-data-table` |
-| Rare item table (AFS/GSL/JSON) | `convert-rare-item-set`   | `convert-rare-item-set`      |
+| Rare item table (AFS/GSL/JSON/HTML) | `convert-rare-item-set`   | `convert-rare-item-set`      |
 
 There are several actions that don't fit well into the table above, which let you do other things:
 

src/Client.cc

@@ -784,7 +784,7 @@ void Client::load_all_files() {
     if (this->character_data) {
       player_data_log.info("Using loaded character file %s", char_filename.c_str());
     } else if (phosg::isfile(char_filename)) {
-      auto psochar = load_psochar(char_filename, !this->system_data);
+      auto psochar = PSOCHARFile::load_shared(char_filename, !this->system_data);
       this->character_data = psochar.character_file;
       files_manager->set_character(char_filename, this->character_data);
       player_data_log.info("Loaded character data from %s", char_filename.c_str());
@@ -966,7 +966,7 @@ void Client::save_character_file(
     const string& filename,
     shared_ptr<const PSOBBBaseSystemFile> system,
     shared_ptr<const PSOBBCharacterFile> character) {
-  save_psochar(filename, system, character);
+  PSOCHARFile::save(filename, system, character);
   player_data_log.info("Saved character file %s", filename.c_str());
 }
 
@@ -1008,7 +1008,7 @@ void Client::save_guild_card_file() const {
 
 void Client::load_backup_character(uint32_t account_id, size_t index) {
   string filename = this->backup_character_filename(account_id, index, false);
-  this->character_data = load_psochar(filename, false).character_file;
+  this->character_data = PSOCHARFile::load_shared(filename, false).character_file;
   this->update_character_data_after_load(this->character_data);
   this->v1_v2_last_reported_disp.reset();
 }
@@ -1096,7 +1096,7 @@ void Client::use_character_bank(int8_t index) {
       this->external_bank_character_index = index;
       player_data_log.info("Using loaded character file %s for external bank", filename.c_str());
     } else if (phosg::isfile(filename)) {
-      this->external_bank_character = load_psochar(filename, false).character_file;
+      this->external_bank_character = PSOCHARFile::load_shared(filename, false).character_file;
       this->update_character_data_after_load(this->external_bank_character);
       this->external_bank_character_index = index;
       files_manager->set_character(filename, this->external_bank_character);

src/EnemyType.hh

@@ -7,9 +7,9 @@
 #include "StaticGameData.hh"
 #include "Types.hh"
 
-enum class EnemyType {
+enum class EnemyType : uint8_t {
-  UNKNOWN = -1,
+  UNKNOWN = 0,
-  NONE = 0,
+  NONE,
   NON_ENEMY_NPC,
   AL_RAPPY,
   ASTARK,
@@ -40,8 +40,8 @@ enum class EnemyType {
   DE_ROL_LE_MINE,
   DEATH_GUNNER,
   DEL_LILY,
-  DEL_RAPPY,
+  DEL_RAPPY_CRATER,
-  DEL_RAPPY_ALT,
+  DEL_RAPPY_DESERT,
   DELBITER,
   DELDEPTH,
   DELSABER,
@@ -54,10 +54,10 @@ enum class EnemyType {
   DUBCHIC,
   DUBWITCH, // Has no entry in battle params
   EGG_RAPPY,
-  EPSIGUARD,
+  EPSIGARD,
   EPSILON,
   EVIL_SHARK,
-  GAEL,
+  GAEL_OR_GIEL,
   GAL_GRYPHON,
   GARANZ,
   GEE,
@@ -98,8 +98,8 @@ enum class EnemyType {
   OLGA_FLOW_2,
   PAL_SHARK,
   PAN_ARMS,
-  PAZUZU,
+  PAZUZU_CRATER,
-  PAZUZU_ALT,
+  PAZUZU_DESERT,
   PIG_RAY,
   POFUILLY_SLIME,
   POUILLY_SLIME,
@@ -108,12 +108,12 @@ enum class EnemyType {
   RAG_RAPPY,
   RECOBOX,
   RECON,
-  SAINT_MILLION,
+  SAINT_MILION,
   SAINT_RAPPY,
-  SAND_RAPPY,
+  SAND_RAPPY_CRATER,
-  SAND_RAPPY_ALT,
+  SAND_RAPPY_DESERT,
-  SATELLITE_LIZARD,
+  SATELLITE_LIZARD_CRATER,
-  SATELLITE_LIZARD_ALT,
+  SATELLITE_LIZARD_DESERT,
   SAVAGE_WOLF,
   SHAMBERTIN,
   SINOW_BEAT,
@@ -130,23 +130,53 @@ enum class EnemyType {
   VOL_OPT_CORE,
   VOL_OPT_MONITOR,
   VOL_OPT_PILLAR,
-  YOWIE,
+  YOWIE_CRATER,
-  YOWIE_ALT,
+  YOWIE_DESERT,
   ZE_BOOTA,
   ZOL_GIBBON,
-  ZU,
+  ZU_CRATER,
-  ZU_ALT,
+  ZU_DESERT,
   MAX_ENEMY_TYPE,
 };
 
+struct EnemyTypeDefinition {
+  enum Flag : uint8_t {
+    VALID_EP1 = 0x01,
+    VALID_EP2 = 0x02,
+    VALID_EP4 = 0x04,
+    IS_RARE = 0x08,
+  };
+  EnemyType type;
+  uint8_t flags;
+  uint8_t rt_index; // 0xFF if not valid (e.g. not an enemy)
+  uint8_t bp_index; // 0xFF if not valid (e.g. not an enemy)
+  const char* enum_name;
+  const char* in_game_name;
+  const char* ultimate_name; // May be null if same as in_game_name
+
+  inline bool valid_in_episode(Episode ep) const {
+    switch (ep) {
+      case Episode::EP1:
+        return (this->flags & Flag::VALID_EP1);
+      case Episode::EP2:
+        return (this->flags & Flag::VALID_EP2);
+      case Episode::EP4:
+        return (this->flags & Flag::VALID_EP4);
+      default:
+        throw std::logic_error("invalid episode number");
+    }
+  }
+  inline bool is_rare() const {
+    return (this->flags & Flag::IS_RARE);
+  }
+  EnemyType rare_type(Episode episode, uint8_t event, uint8_t floor) const;
+};
+
+const EnemyTypeDefinition& type_definition_for_enemy(EnemyType type);
+
 template <>
 const char* phosg::name_for_enum<EnemyType>(EnemyType type);
 template <>
 EnemyType phosg::enum_for_name<EnemyType>(const char* name);
 
-bool enemy_type_valid_for_episode(Episode episode, EnemyType enemy_type);
-uint8_t battle_param_index_for_enemy_type(Episode episode, EnemyType enemy_type);
-uint8_t rare_table_index_for_enemy_type(EnemyType enemy_type);
 const std::vector<EnemyType>& enemy_types_for_rare_table_index(Episode episode, uint8_t rt_index);
-bool enemy_type_is_rare(EnemyType type);
-EnemyType rare_type_for_enemy_type(EnemyType base_type, Episode episode, uint8_t event, uint8_t floor);

src/ItemNameIndex.cc

@@ -97,7 +97,7 @@ const array<const char*, 0x11> name_for_s_rank_special = {
     "King\'s",
 };
 
-std::string ItemNameIndex::describe_item(const ItemData& item, bool include_color_escapes) const {
+std::string ItemNameIndex::describe_item(const ItemData& item, bool include_color_escapes, bool hide_mag_stats) const {
   if (item.data1[0] == 0x04) {
     return phosg::string_printf("%s%" PRIu32 " Meseta", include_color_escapes ? "$C7" : "", item.data2d.load());
   }
@@ -166,8 +166,8 @@ std::string ItemNameIndex::describe_item(const ItemData& item, bool include_colo
     }
   }
 
-  // For weapons, add the grind and bonuses, or S-rank name if applicable
   if (item.data1[0] == 0x00) {
+    // For weapons, add the grind and bonuses, or S-rank name if applicable
     if (item.data1[3] > 0) {
       ret_tokens.emplace_back(phosg::string_printf("+%hhu", item.data1[3]));
     }
@@ -232,8 +232,8 @@ std::string ItemNameIndex::describe_item(const ItemData& item, bool include_colo
       }
     }
 
-    // For armors, add the slots, unit modifiers, and/or DEF/EVP bonuses
   } else if (item.data1[0] == 0x01) {
+    // For armors, add the slots, unit modifiers, and/or DEF/EVP bonuses
     if (item.data1[1] == 0x03) { // Units
       int16_t modifier = item.data1w[3];
       if (modifier == 1 || modifier == 2) {
@@ -266,8 +266,8 @@ std::string ItemNameIndex::describe_item(const ItemData& item, bool include_colo
       }
     }
 
+  } else if (!hide_mag_stats && (item.data1[0] == 0x02)) {
     // For mags, add tons of info
-  } else if (item.data1[0] == 0x02) {
     ret_tokens.emplace_back(phosg::string_printf("LV%hhu", item.data1[2]));
 
     uint16_t def = item.data1w[2];
@@ -327,8 +327,8 @@ std::string ItemNameIndex::describe_item(const ItemData& item, bool include_colo
       ret_tokens.emplace_back(phosg::string_printf("(!CL:%02hhX)", item.data2[3]));
     }
 
-    // For tools, add the amount (if applicable)
   } else if (item.data1[0] == 0x03) {
+    // For tools, add the amount (if applicable)
     if (item.max_stack_size(*this->limits) > 1) {
       ret_tokens.emplace_back(phosg::string_printf("x%hhu", item.data1[5]));
     }

src/ItemNameIndex.hh

@@ -38,7 +38,7 @@ public:
   inline bool exists(const ItemData& item) const {
     return this->primary_identifier_index.count(item.primary_identifier());
   }
-  std::string describe_item(const ItemData& item, bool include_color_escapes = false) const;
+  std::string describe_item(const ItemData& item, bool include_color_escapes = false, bool hide_mag_stats = false) const;
   ItemData parse_item_description(const std::string& description) const;
 
   void print_table(FILE* stream) const;

src/Main.cc

@@ -1456,23 +1456,25 @@ Action a_disassemble_quest_map(
       if (!args.get<bool>("decompressed")) {
         *data = prs_decompress(*data);
       }
-      string result = MapFile(data).disassemble();
+      bool reassembly = args.get<bool>("reassembly");
+      string result = MapFile(data).disassemble(reassembly);
       write_output_data(args, result.data(), result.size(), "txt");
     });
 Action a_disassemble_free_map(
     "disassemble-free-map", "\
   disassemble-free-map INPUT-FILENAME [OUTPUT-FILENAME]\n\
     Disassemble the input free-play map (.dat or .evt file) into a text\n\
-    representation of the data it contains. Unlike othe disassembly actions,\n\
+    representation of the data it contains. Unlike other disassembly actions,\n\
     this action expects its input to be already decompressed. If the input is\n\
     compressed, use the --compressed option. Also unlike other options, the\n\
     input must be from a file (that is, INPUT-FILENAME is required and cannot\n\
     be \"-\").\n",
     +[](phosg::Arguments& args) {
       const string& input_filename = args.get<string>(1, true);
-      bool is_events = phosg::ends_with(input_filename, ".evt");
+      string input_filename_lower = phosg::tolower(input_filename);
-      bool is_enemies = phosg::ends_with(input_filename, "e.dat") || phosg::ends_with(input_filename, "e_s.dat") || phosg::ends_with(input_filename, "e_c1.dat") || phosg::ends_with(input_filename, "e_d.dat");
+      bool is_events = phosg::ends_with(input_filename_lower, ".evt");
-      bool is_objects = phosg::ends_with(input_filename, "o.dat") || phosg::ends_with(input_filename, "o_s.dat") || phosg::ends_with(input_filename, "o_c1.dat") || phosg::ends_with(input_filename, "o_d.dat");
+      bool is_enemies = phosg::ends_with(input_filename_lower, "e.dat") || phosg::ends_with(input_filename_lower, "e_s.dat") || phosg::ends_with(input_filename_lower, "e_c1.dat") || phosg::ends_with(input_filename_lower, "e_d.dat");
+      bool is_objects = phosg::ends_with(input_filename_lower, "o.dat") || phosg::ends_with(input_filename_lower, "o_s.dat") || phosg::ends_with(input_filename_lower, "o_c1.dat") || phosg::ends_with(input_filename_lower, "o_d.dat");
       if (!is_objects && !is_enemies && !is_events) {
         throw runtime_error("cannot determine input file type");
       }
@@ -1483,13 +1485,14 @@ Action a_disassemble_free_map(
       }
 
       uint8_t floor = args.get<uint8_t>("floor", 0);
+      bool reassembly = args.get<bool>("reassembly");
       string result;
       if (is_objects) {
-        result = MapFile(floor, data, nullptr, nullptr).disassemble();
+        result = MapFile(floor, data, nullptr, nullptr).disassemble(reassembly);
       } else if (is_enemies) {
-        result = MapFile(floor, nullptr, data, nullptr).disassemble();
+        result = MapFile(floor, nullptr, data, nullptr).disassemble(reassembly);
       } else if (is_events) {
-        result = MapFile(floor, nullptr, nullptr, data).disassemble();
+        result = MapFile(floor, nullptr, nullptr, data).disassemble(reassembly);
       } else {
         throw logic_error("unhandled input type");
       }
@@ -1867,7 +1870,7 @@ Action a_download_files(
       }
       shared_ptr<struct event_base> base(event_base_new(), event_base_free);
       auto remote = phosg::make_sockaddr_storage(phosg::parse_netloc(args.get<string>(1))).first;
-      auto character = load_psochar(args.get<string>("character", true), false).character_file;
+      auto character = PSOCHARFile::load_shared(args.get<string>("character", true), false).character_file;
       auto ship_menu_selections_str = args.get<string>("ship-menu-selections", false);
 
       unordered_set<string> ship_menu_selections;
@@ -1924,11 +1927,10 @@ Action a_convert_rare_item_set(
       .gslb (PSO GC big-endian GSL archive)\n\
       .afs (PSO V2 little-endian AFS archive)\n\
       .rel (Schtserv rare table; cannot be used in output filename)\n\
+      .html (HTML rare table; cannot be used in input filename)\n\
     If the --multiply=X option is given, multiplies all drop rates by X (given\n\
     as a decimal value).\n",
     +[](phosg::Arguments& args) {
-      auto version = get_cli_version(args);
-
       double rate_factor = args.get<double>("multiply", 1.0);
       auto s = make_shared<ServerState>(get_config_filename(args));
       s->load_config_early();
@@ -1942,17 +1944,18 @@ Action a_convert_rare_item_set(
         throw runtime_error("input filename must be given");
       }
 
+      string input_filename_lower = phosg::tolower(input_filename);
       auto data = make_shared<string>(read_input_data(args));
       shared_ptr<RareItemSet> rs;
-      if (phosg::ends_with(input_filename, ".json")) {
+      if (phosg::ends_with(input_filename_lower, ".json")) {
-        rs = make_shared<RareItemSet>(phosg::JSON::parse(*data), s->item_name_index_opt(version));
+        rs = make_shared<RareItemSet>(phosg::JSON::parse(*data), s->item_name_index_opt(get_cli_version(args, Version::BB_V4)));
-      } else if (phosg::ends_with(input_filename, ".gsl")) {
+      } else if (phosg::ends_with(input_filename_lower, ".gsl")) {
         rs = make_shared<RareItemSet>(GSLArchive(data, false), false);
-      } else if (phosg::ends_with(input_filename, ".gslb")) {
+      } else if (phosg::ends_with(input_filename_lower, ".gslb")) {
         rs = make_shared<RareItemSet>(GSLArchive(data, true), true);
-      } else if (phosg::ends_with(input_filename, ".afs")) {
+      } else if (phosg::ends_with(input_filename_lower, ".afs")) {
-        rs = make_shared<RareItemSet>(AFSArchive(data), is_v1(version));
+        rs = make_shared<RareItemSet>(AFSArchive(data), is_v1(get_cli_version(args, Version::DC_V2)));
-      } else if (phosg::ends_with(input_filename, ".rel")) {
+      } else if (phosg::ends_with(input_filename_lower, ".rel")) {
         rs = make_shared<RareItemSet>(*data, true);
       } else {
         throw runtime_error("cannot determine input format; use a filename ending with .json, .gsl, .gslb, .afs, or .rel");
@@ -1963,22 +1966,38 @@ Action a_convert_rare_item_set(
       }
 
       string output_filename = args.get<string>(2, false);
+      string output_filename_lower = phosg::tolower(output_filename);
       if (output_filename.empty() || (output_filename == "-")) {
-        rs->print_all_collections(stdout, s->item_name_index_opt(version));
+        rs->print_all_collections(stdout, s->item_name_index_opt(get_cli_version(args, Version::BB_V4)));
-      } else if (phosg::ends_with(output_filename, ".json")) {
+      } else if (phosg::ends_with(output_filename_lower, ".json")) {
-        auto json = rs->json(s->item_name_index_opt(version));
+        auto json = rs->json(s->item_name_index_opt(get_cli_version(args, Version::BB_V4)));
         string data = json.serialize(phosg::JSON::SerializeOption::FORMAT | phosg::JSON::SerializeOption::HEX_INTEGERS | phosg::JSON::SerializeOption::SORT_DICT_KEYS);
         write_output_data(args, data.data(), data.size(), nullptr);
-      } else if (phosg::ends_with(output_filename, ".gsl")) {
+      } else if (phosg::ends_with(output_filename_lower, ".gsl")) {
         string data = rs->serialize_gsl(args.get<bool>("big-endian"));
         write_output_data(args, data.data(), data.size(), nullptr);
-      } else if (phosg::ends_with(output_filename, ".gslb")) {
+      } else if (phosg::ends_with(output_filename_lower, ".gslb")) {
         string data = rs->serialize_gsl(true);
         write_output_data(args, data.data(), data.size(), nullptr);
-      } else if (phosg::ends_with(output_filename, ".afs")) {
+      } else if (phosg::ends_with(output_filename_lower, ".afs")) {
-        bool is_v1 = ::is_v1(get_cli_version(args, Version::GC_V3));
+        bool is_v1 = ::is_v1(get_cli_version(args, Version::DC_V2));
         string data = rs->serialize_afs(is_v1);
         write_output_data(args, data.data(), data.size(), nullptr);
+      } else if (phosg::ends_with(output_filename_lower, ".html")) {
+        bool is_v1 = ::is_v1(get_cli_version(args, Version::BB_V4));
+        static const array<GameMode, 4> modes = {GameMode::NORMAL, GameMode::BATTLE, GameMode::CHALLENGE, GameMode::SOLO};
+        for (GameMode mode : modes) {
+          static const array<Episode, 3> episodes = {Episode::EP1, Episode::EP2, Episode::EP4};
+          for (Episode episode : episodes) {
+            for (size_t difficulty = 0; difficulty < (is_v1 ? 3 : 4); difficulty++) {
+              auto item_name_index = s->item_name_index(get_cli_version(args, Version::BB_V4));
+              string data = rs->serialize_html(mode, episode, difficulty, item_name_index);
+              string out_filename = output_filename.substr(0, output_filename.size() - 5) + "." + name_for_mode(mode) + "." + name_for_episode(episode) + "." + name_for_difficulty(difficulty) + output_filename.substr(output_filename.size() - 5);
+              phosg::save_file(out_filename, data);
+              phosg::log_info("... %s", out_filename.c_str());
+            }
+          }
+        }
       } else {
         throw runtime_error("cannot determine output format; use a filename ending with .json, .gsl, .gslb, or .afs");
       }

src/Map.cc

@@ -1678,35 +1678,59 @@ string MapFile::disassemble_action_stream(const void* data, size_t size) {
   return phosg::join(ret, "\n");
 }
 
-string MapFile::disassemble() const {
+string MapFile::disassemble(bool reassembly) const {
   deque<string> ret;
   for (uint8_t floor = 0; floor < this->sections_for_floor.size(); floor++) {
     const auto& sf = this->sections_for_floor[floor];
     phosg::StringReader as_r(sf.event_action_stream, sf.event_action_stream_bytes);
 
     if (sf.object_sets) {
-      ret.emplace_back(phosg::string_printf(".object_sets %hhu /* 0x%zX in file; 0x%zX bytes */",
+      if (reassembly) {
-          floor, sf.object_sets_file_offset, sf.object_sets_file_size));
+        ret.emplace_back(phosg::string_printf(".object_sets %hhu", floor));
+      } else {
+        ret.emplace_back(phosg::string_printf(".object_sets %hhu /* 0x%zX in file; 0x%zX bytes */",
+            floor, sf.object_sets_file_offset, sf.object_sets_file_size));
+      }
       for (size_t z = 0; z < sf.object_set_count; z++) {
-        size_t k_id = z + sf.first_object_set_index;
+        if (reassembly) {
-        ret.emplace_back(phosg::string_printf("/* K-%03zX */ ", k_id) + sf.object_sets[z].str());
+          ret.emplace_back(sf.object_sets[z].str());
+        } else {
+          size_t k_id = z + sf.first_object_set_index;
+          ret.emplace_back(phosg::string_printf("/* K-%03zX */ ", k_id) + sf.object_sets[z].str());
+        }
       }
     }
     if (sf.enemy_sets) {
-      ret.emplace_back(phosg::string_printf(".enemy_sets %hhu /* 0x%zX in file; 0x%zX bytes */",
+      if (reassembly) {
-          floor, sf.enemy_sets_file_offset, sf.enemy_sets_file_size));
+        ret.emplace_back(phosg::string_printf(".enemy_sets %hhu", floor));
+      } else {
+        ret.emplace_back(phosg::string_printf(".enemy_sets %hhu /* 0x%zX in file; 0x%zX bytes */",
+            floor, sf.enemy_sets_file_offset, sf.enemy_sets_file_size));
+      }
       for (size_t z = 0; z < sf.enemy_set_count; z++) {
-        size_t s_id = z + sf.first_enemy_set_index;
+        if (reassembly) {
-        ret.emplace_back(phosg::string_printf("/* S-%03zX */ ", s_id) + sf.enemy_sets[z].str());
+          ret.emplace_back(sf.enemy_sets[z].str());
+        } else {
+          size_t s_id = z + sf.first_enemy_set_index;
+          ret.emplace_back(phosg::string_printf("/* S-%03zX */ ", s_id) + sf.enemy_sets[z].str());
+        }
       }
     }
     if (sf.events1) {
-      ret.emplace_back(phosg::string_printf(".events %hhu /* 0x%zX in file; 0x%zX bytes; 0x%zX bytes in action stream */",
+      if (reassembly) {
-          floor, sf.events_file_offset, sf.events_file_size, sf.event_action_stream_bytes));
+        ret.emplace_back(phosg::string_printf(".events %hhu", floor));
+      } else {
+        ret.emplace_back(phosg::string_printf(".events %hhu /* 0x%zX in file; 0x%zX bytes; 0x%zX bytes in action stream */",
+            floor, sf.events_file_offset, sf.events_file_size, sf.event_action_stream_bytes));
+      }
       for (size_t z = 0; z < sf.event_count; z++) {
         const auto& ev = sf.events1[z];
-        size_t w_id = z + sf.first_event_set_index;
+        if (reassembly) {
-        ret.emplace_back(phosg::string_printf("/* W-%03zX */ ", w_id) + ev.str());
+          ret.emplace_back(ev.str());
+        } else {
+          size_t w_id = z + sf.first_event_set_index;
+          ret.emplace_back(phosg::string_printf("/* W-%03zX */ ", w_id) + ev.str());
+        }
         if (ev.action_stream_offset >= sf.event_action_stream_bytes) {
           ret.emplace_back(phosg::string_printf(
               "  // WARNING: Event action stream offset (0x%" PRIX32 ") is outside of this section",
@@ -1717,11 +1741,20 @@ string MapFile::disassemble() const {
       }
     }
     if (sf.events2) {
-      ret.emplace_back(phosg::string_printf(".random_events %hhu /* 0x%zX in file; 0x%zX bytes; 0x%zX bytes in action stream */",
+      if (reassembly) {
-          floor, sf.events_file_offset, sf.events_file_size, sf.event_action_stream_bytes));
+        ret.emplace_back(phosg::string_printf(".random_events %hhu", floor));
+      } else {
+        ret.emplace_back(phosg::string_printf(
+            ".random_events %hhu /* 0x%zX in file; 0x%zX bytes; 0x%zX bytes in action stream */",
+            floor, sf.events_file_offset, sf.events_file_size, sf.event_action_stream_bytes));
+      }
       for (size_t z = 0; z < sf.event_count; z++) {
         const auto& ev = sf.events2[z];
-        ret.emplace_back(phosg::string_printf("/* index %zu */", z) + ev.str());
+        if (reassembly) {
+          ret.emplace_back(ev.str());
+        } else {
+          ret.emplace_back(phosg::string_printf("/* index %zu */", z) + ev.str());
+        }
         if (ev.action_stream_offset >= sf.event_action_stream_bytes) {
           ret.emplace_back(phosg::string_printf(
               "  // WARNING: Event action stream offset (0x%" PRIX32 ") is outside of this section",
@@ -1732,13 +1765,21 @@ string MapFile::disassemble() const {
       }
     }
     if (sf.random_enemy_locations_data) {
-      ret.emplace_back(phosg::string_printf(".random_enemy_locations %hhu /* 0x%zX in file; 0x%zX bytes */",
+      if (reassembly) {
-          floor, sf.random_enemy_locations_file_offset, sf.random_enemy_locations_file_size));
+        ret.emplace_back(phosg::string_printf(".random_enemy_locations %hhu", floor));
+      } else {
+        ret.emplace_back(phosg::string_printf(".random_enemy_locations %hhu /* 0x%zX in file; 0x%zX bytes */",
+            floor, sf.random_enemy_locations_file_offset, sf.random_enemy_locations_file_size));
+      }
       ret.emplace_back(phosg::format_data(sf.random_enemy_locations_data, sf.random_enemy_locations_data_size));
     }
     if (sf.random_enemy_definitions_data) {
-      ret.emplace_back(phosg::string_printf(".random_enemy_definitions %hhu /* 0x%zX in file; 0x%zX bytes */",
+      if (reassembly) {
-          floor, sf.random_enemy_definitions_file_offset, sf.random_enemy_definitions_file_size));
+        ret.emplace_back(phosg::string_printf(".random_enemy_definitions %hhu", floor));
+      } else {
+        ret.emplace_back(phosg::string_printf(".random_enemy_definitions %hhu /* 0x%zX in file; 0x%zX bytes */",
+            floor, sf.random_enemy_definitions_file_offset, sf.random_enemy_definitions_file_size));
+      }
       ret.emplace_back(phosg::format_data(sf.random_enemy_definitions_data, sf.random_enemy_definitions_data_size));
     }
   }
@@ -2050,7 +2091,7 @@ shared_ptr<SuperMap::Enemy> SuperMap::add_enemy_and_children(
           add(EnemyType::RAG_RAPPY, is_rare_v123, is_rare_bb);
           break;
         case Episode::EP4:
-          add((floor > 0x05) ? EnemyType::SAND_RAPPY_ALT : EnemyType::SAND_RAPPY, is_rare_v123, is_rare_bb);
+          add((floor > 0x05) ? EnemyType::SAND_RAPPY_DESERT : EnemyType::SAND_RAPPY_CRATER, is_rare_v123, is_rare_bb);
           break;
         default:
           throw logic_error("invalid episode");
@@ -2290,7 +2331,7 @@ shared_ptr<SuperMap::Enemy> SuperMap::add_enemy_and_children(
       if ((episode == Episode::EP2) && (floor > 0x0F)) {
         add(EnemyType::EPSILON);
         default_num_children = 4;
-        child_type = EnemyType::EPSIGUARD;
+        child_type = EnemyType::EPSIGARD;
       } else {
         add((set_entry->uparam1 & 0x01) ? EnemyType::SINOW_ZELE : EnemyType::SINOW_ZOA);
       }
@@ -2303,9 +2344,9 @@ shared_ptr<SuperMap::Enemy> SuperMap::add_enemy_and_children(
       break;
     case 0x0111:
       if (floor > 0x05) {
-        add(set_entry->fparam2 ? EnemyType::YOWIE_ALT : EnemyType::SATELLITE_LIZARD_ALT);
+        add(set_entry->fparam2 ? EnemyType::YOWIE_DESERT : EnemyType::SATELLITE_LIZARD_DESERT);
       } else {
-        add(set_entry->fparam2 ? EnemyType::YOWIE : EnemyType::SATELLITE_LIZARD);
+        add(set_entry->fparam2 ? EnemyType::YOWIE_CRATER : EnemyType::SATELLITE_LIZARD_CRATER);
       }
       break;
     case 0x0112: {
@@ -2318,7 +2359,7 @@ shared_ptr<SuperMap::Enemy> SuperMap::add_enemy_and_children(
       break;
     case 0x0114: {
       bool is_rare = (set_entry->uparam1 & 0x01);
-      add((floor > 0x05) ? EnemyType::ZU_ALT : EnemyType::ZU, is_rare, is_rare);
+      add((floor > 0x05) ? EnemyType::ZU_DESERT : EnemyType::ZU_CRATER, is_rare, is_rare);
       break;
     }
     case 0x0115:
@@ -2341,7 +2382,7 @@ shared_ptr<SuperMap::Enemy> SuperMap::add_enemy_and_children(
     case 0x0119: {
       // TODO: It appears BB doesn't have a way to force Kondrieu to appear via
       // constructor args. Is this true?
-      add((set_entry->uparam1 & 1) ? EnemyType::SHAMBERTIN : EnemyType::SAINT_MILLION);
+      add((set_entry->uparam1 & 1) ? EnemyType::SHAMBERTIN : EnemyType::SAINT_MILION);
       default_num_children = 0x18;
       break;
     }
@@ -3378,8 +3419,8 @@ uint32_t MapState::RareEnemyRates::for_enemy_type(EnemyType type) const {
     case EnemyType::HILDEBEAR:
       return this->hildeblue;
     case EnemyType::RAG_RAPPY:
-    case EnemyType::SAND_RAPPY:
+    case EnemyType::SAND_RAPPY_CRATER:
-    case EnemyType::SAND_RAPPY_ALT:
+    case EnemyType::SAND_RAPPY_DESERT:
       return this->rappy;
     case EnemyType::POISON_LILY:
       return this->nar_lily;
@@ -3389,12 +3430,12 @@ uint32_t MapState::RareEnemyRates::for_enemy_type(EnemyType type) const {
       return this->mericarand;
     case EnemyType::MERISSA_A:
       return this->merissa_aa;
-    case EnemyType::ZU:
+    case EnemyType::ZU_CRATER:
-    case EnemyType::ZU_ALT:
+    case EnemyType::ZU_DESERT:
       return this->pazuzu;
     case EnemyType::DORPHON:
       return this->dorphon_eclair;
-    case EnemyType::SAINT_MILLION:
+    case EnemyType::SAINT_MILION:
     case EnemyType::SHAMBERTIN:
       return this->kondrieu;
     default:
@@ -3643,7 +3684,7 @@ void MapState::index_super_map(const FloorConfig& fc, shared_ptr<PSOLFGEncryptio
         type = ene->type;
     }
 
-    auto rare_type = rare_type_for_enemy_type(type, fc.super_map->get_episode(), this->event, ene->floor);
+    auto rare_type = type_definition_for_enemy(type).rare_type(fc.super_map->get_episode(), this->event, ene->floor);
     if ((type == EnemyType::MERICARAND) || (rare_type != type)) {
       unordered_map<uint32_t, float> det_cache;
       uint32_t bb_rare_rate = this->bb_rare_rates->for_enemy_type(type);

src/Map.hh

@@ -417,7 +417,7 @@ public:
   size_t count_events() const;
 
   static std::string disassemble_action_stream(const void* data, size_t size);
-  std::string disassemble() const;
+  std::string disassemble(bool reassembly = false) const;
 
 protected:
   void link_data(std::shared_ptr<const std::string> data);
@@ -661,7 +661,7 @@ public:
     uint32_t merissa_aa; // MERISSA_A -> MERISSA_AA
     uint32_t pazuzu; // ZU -> PAZUZU (and _ALT variants)
     uint32_t dorphon_eclair; // DORPHON -> DORPHON_ECLAIR
-    uint32_t kondrieu; // {SAINT_MILLION, SHAMBERTIN} -> KONDRIEU
+    uint32_t kondrieu; // {SAINT_MILION, SHAMBERTIN} -> KONDRIEU
 
     RareEnemyRates(uint32_t enemy_rate, uint32_t mericarand_rate, uint32_t boss_rate);
     explicit RareEnemyRates(const phosg::JSON& json);
@@ -768,7 +768,7 @@ public:
         }
       } else {
         return this->is_rare(version)
-            ? rare_type_for_enemy_type(this->super_ene->type, episode, event, this->super_ene->floor)
+            ? type_definition_for_enemy(this->super_ene->type).rare_type(episode, event, this->super_ene->floor)
             : this->super_ene->type;
       }
     }

src/RareItemSet.cc

@@ -330,11 +330,14 @@ RareItemSet::RareItemSet(const phosg::JSON& json, shared_ptr<const ItemNameIndex
               }
               target = &collection.box_area_to_specs[area];
             } else {
-              size_t index = rare_table_index_for_enemy_type(phosg::enum_for_name<EnemyType>(item_it.first.c_str()));
+              size_t rt_index = type_definition_for_enemy(phosg::enum_for_name<EnemyType>(item_it.first.c_str())).rt_index;
-              if (collection.rt_index_to_specs.size() <= index) {
+              if (rt_index == 0xFF) {
-                collection.rt_index_to_specs.resize(index + 1);
+                throw runtime_error("enemy type " + item_it.first + " does not have an rt_index");
               }
-              target = &collection.rt_index_to_specs[index];
+              if (collection.rt_index_to_specs.size() <= rt_index) {
+                collection.rt_index_to_specs.resize(rt_index + 1);
+              }
+              target = &collection.rt_index_to_specs[rt_index];
             }
 
             for (const auto& spec_json : item_it.second->as_list()) {
@@ -422,6 +425,260 @@ std::string RareItemSet::serialize_gsl(bool big_endian) const {
   return GSLArchive::generate(files, big_endian);
 }
 
+string RareItemSet::serialize_html(
+    GameMode mode,
+    Episode episode,
+    uint8_t difficulty,
+    shared_ptr<const ItemNameIndex> name_index) const {
+
+  struct ZoneTypes {
+    const char* name;
+    vector<uint8_t> floors;
+    vector<EnemyType> types;
+  };
+
+  // clang-format off
+  static const std::map<Episode, std::vector<ZoneTypes>> zone_types_for_episode{
+    {Episode::EP1, {
+      {"Forest", {0x01, 0x02, 0x0B}, {
+        EnemyType::BOOMA, EnemyType::GOBOOMA, EnemyType::GIGOBOOMA,
+        EnemyType::SAVAGE_WOLF, EnemyType::BARBAROUS_WOLF,
+        EnemyType::RAG_RAPPY, EnemyType::AL_RAPPY,
+        EnemyType::MONEST, EnemyType::MOTHMANT,
+        EnemyType::HILDEBEAR, EnemyType::HILDEBLUE,
+        EnemyType::DRAGON,
+      }},
+      {"Caves", {0x03, 0x04, 0x05, 0x0C}, {
+        EnemyType::EVIL_SHARK, EnemyType::PAL_SHARK, EnemyType::GUIL_SHARK,
+        EnemyType::POISON_LILY, EnemyType::NAR_LILY,
+        EnemyType::POFUILLY_SLIME, EnemyType::POUILLY_SLIME,
+        EnemyType::NANO_DRAGON,
+        EnemyType::GRASS_ASSASSIN,
+        EnemyType::PAN_ARMS, EnemyType::HIDOOM, EnemyType::MIGIUM,
+        EnemyType::DE_ROL_LE_BODY, EnemyType::DE_ROL_LE_MINE, EnemyType::DE_ROL_LE,
+      }},
+      {"Mines", {0x06, 0x07, 0x0D}, {
+        EnemyType::GILLCHIC, EnemyType::DUBCHIC, EnemyType::DUBWITCH,
+        EnemyType::CANADINE, EnemyType::CANADINE_GROUP, EnemyType::CANANE,
+        EnemyType::SINOW_BEAT, EnemyType::SINOW_GOLD,
+        EnemyType::GARANZ,
+        EnemyType::VOL_OPT_AMP, EnemyType::VOL_OPT_CORE, EnemyType::VOL_OPT_MONITOR, EnemyType::VOL_OPT_PILLAR,   EnemyType::VOL_OPT_1, EnemyType::VOL_OPT_2,
+      }},
+      {"Ruins", {0x08, 0x09, 0x0A, 0x0E}, {
+        EnemyType::DIMENIAN, EnemyType::LA_DIMENIAN, EnemyType::SO_DIMENIAN,
+        EnemyType::CLAW, EnemyType::BULK, EnemyType::BULCLAW,
+        EnemyType::DELSABER,
+        EnemyType::CHAOS_SORCERER, EnemyType::BEE_L, EnemyType::BEE_R,
+        EnemyType::DARK_BELRA,
+        EnemyType::DARK_GUNNER, EnemyType::DEATH_GUNNER,
+        EnemyType::CHAOS_BRINGER,
+        EnemyType::DARVANT, EnemyType::DARVANT_ULTIMATE, EnemyType::DARK_FALZ_1, EnemyType::DARK_FALZ_2, EnemyType::DARK_FALZ_3,
+      }},
+  }},
+  {Episode::EP2, {
+      {"VR Temple", {0x01, 0x02, 0x0E}, {
+        EnemyType::RAG_RAPPY, EnemyType::LOVE_RAPPY, EnemyType::EGG_RAPPY, EnemyType::HALLO_RAPPY, EnemyType::SAINT_RAPPY,
+        EnemyType::DIMENIAN, EnemyType::LA_DIMENIAN, EnemyType::SO_DIMENIAN,
+        EnemyType::POISON_LILY, EnemyType::NAR_LILY,
+        EnemyType::MONEST, EnemyType::MOTHMANT,
+        EnemyType::GRASS_ASSASSIN,
+        EnemyType::HILDEBEAR, EnemyType::HILDEBLUE,
+        EnemyType::DARK_BELRA,
+        EnemyType::PIG_RAY, EnemyType::BARBA_RAY,
+      }},
+      {"VR Spaceship", {0x03, 0x04, 0x0F}, {
+        EnemyType::SAVAGE_WOLF, EnemyType::BARBAROUS_WOLF,
+        EnemyType::GILLCHIC, EnemyType::DUBCHIC, EnemyType::DUBWITCH,
+        EnemyType::PAN_ARMS, EnemyType::HIDOOM, EnemyType::MIGIUM,
+        EnemyType::DELSABER,
+        EnemyType::GARANZ,
+        EnemyType::CHAOS_SORCERER, EnemyType::BEE_L, EnemyType::BEE_R,
+        EnemyType::GOL_DRAGON,
+      }},
+      {"Central Control Area", {0x05, 0x06, 0x07, 0x08, 0x09, 0x0C, 0x10}, {
+        EnemyType::MERILLIA, EnemyType::MERILTAS,
+        EnemyType::GEE,
+        EnemyType::UL_GIBBON, EnemyType::ZOL_GIBBON,
+        EnemyType::SINOW_BERILL, EnemyType::SINOW_SPIGELL,
+        EnemyType::GI_GUE,
+        EnemyType::GIBBLES,
+        EnemyType::MERICARAND, EnemyType::MERICAROL, EnemyType::MERICUS, EnemyType::MERIKLE,
+        EnemyType::GAL_GRYPHON,
+      }},
+      {"Seabed", {0x0A, 0x0B, 0x0D}, {
+        EnemyType::DOLMOLM, EnemyType::DOLMDARL,
+        EnemyType::SINOW_ZOA, EnemyType::SINOW_ZELE,
+        EnemyType::RECOBOX, EnemyType::RECON,
+        EnemyType::MORFOS,
+        EnemyType::DELDEPTH,
+        EnemyType::DELBITER,
+        EnemyType::GAEL_OR_GIEL, EnemyType::OLGA_FLOW_1, EnemyType::OLGA_FLOW_2,
+      }},
+      {"Control Tower", {0x11}, {
+        EnemyType::MERICARAND, EnemyType::MERICAROL, EnemyType::MERICUS, EnemyType::MERIKLE,
+        EnemyType::GIBBLES,
+        EnemyType::GI_GUE,
+        EnemyType::DELBITER,
+        EnemyType::ILL_GILL,
+        EnemyType::DEL_LILY,
+        EnemyType::EPSILON, EnemyType::EPSIGARD,
+      }},
+  }},
+  {Episode::EP4, {
+      {"Crater", {0x01, 0x02, 0x03, 0x04, 0x05}, {
+        EnemyType::SAND_RAPPY_CRATER, EnemyType::DEL_RAPPY_CRATER,
+        EnemyType::SATELLITE_LIZARD_CRATER,
+        EnemyType::YOWIE_CRATER,
+        EnemyType::BOOTA, EnemyType::ZE_BOOTA, EnemyType::BA_BOOTA,
+        EnemyType::ZU_CRATER, EnemyType::PAZUZU_CRATER,
+        EnemyType::ASTARK,
+        EnemyType::DORPHON, EnemyType::DORPHON_ECLAIR,
+      }},
+      {"Desert", {0x06, 0x07, 0x08, 0x09}, {
+        EnemyType::SAND_RAPPY_DESERT, EnemyType::DEL_RAPPY_DESERT,
+        EnemyType::SATELLITE_LIZARD_DESERT,
+        EnemyType::YOWIE_DESERT,
+        EnemyType::GORAN, EnemyType::PYRO_GORAN, EnemyType::GORAN_DETONATOR,
+        EnemyType::MERISSA_A, EnemyType::MERISSA_AA,
+        EnemyType::ZU_DESERT, EnemyType::PAZUZU_DESERT,
+        EnemyType::GIRTABLULU,
+        EnemyType::SAINT_MILION, EnemyType::SHAMBERTIN, EnemyType::KONDRIEU,
+      }},
+  }}};
+  // clang-format on
+
+  static const std::array<uint32_t, 10> bg_colors{
+      //   Vrd       Grn       Sky       Blu       Prp       Pnk       Red       Orn       Ylw       Wht
+      0x00A562, 0x76FE43, 0x59F9F9, 0x4488FF, 0xCC00FF, 0xFF87CB, 0xF70F0F, 0xF7830F, 0xF7F715, 0xFFFFFF};
+  static const std::array<uint32_t, 10> text_colors{
+      //   Vrd       Grn       Sky       Blu       Prp       Pnk       Red       Orn       Ylw       Wht
+      0xFFFFFF, 0x000000, 0x000000, 0xFFFFFF, 0xFFFFFF, 0x000000, 0xFFFFFF, 0x000000, 0x000000, 0x000000};
+
+  deque<string> blocks;
+  blocks.emplace_back(phosg::string_printf("\
+<html>\n\
+<head>\n\
+  <title>Drop charts for %s %s</title>\n\
+  <style type=\"text/css\">\n\
+    body {\n\
+      background-color: #222222;\n\
+      color: #EEEEEE;\n\
+    }\n\
+    table {\n\
+      border: 1px #222222;\n\
+      text-align: center;\n\
+      font-family: sans-serif;\n\
+      font-size: 14px;\n\
+    }\n\
+    td th {\n\
+      border: 1px #222222;\n\
+      text-align: center;\n\
+      padding: 4px;\n\
+    }\n\
+    th {\n\
+      font-size: 18px;\n\
+    }\n\
+    th.space {\n\
+      background-color: #222222;\n\
+      height: 20px;\n\
+    }\n",
+      name_for_episode(episode),
+      name_for_difficulty(difficulty)));
+  for (size_t z = 0; z < 10; z++) {
+    blocks.emplace_back(phosg::string_printf("\
+    .sec%zu {\n\
+      background-color: #%06" PRIX32 ";\n\
+      color: #%06" PRIX32 ";\n\
+    }\n",
+        z, bg_colors[z], text_colors[z]));
+  }
+  blocks.emplace_back("\
+  </style>\n\
+</head><body>\n");
+
+  blocks.emplace_back("<table>");
+  auto add_location_header = [&](const char* location_name) -> void {
+    blocks.emplace_back("<tr><th class=\"space\" colspan=\"11\" /></tr>");
+    blocks.emplace_back("<tr><th>");
+    blocks.emplace_back(location_name);
+    blocks.emplace_back("</th>");
+    for (size_t z = 0; z < 10; z++) {
+      blocks.emplace_back(phosg::string_printf("<th class=\"sec%zu\">%s</th>", z, name_for_section_id(z)));
+    }
+    blocks.emplace_back("</tr>");
+  };
+
+  auto add_specs_row = [&](const char* loc_name, const array<vector<ExpandedDrop>, 10>& specs_lists) -> void {
+    bool any_list_nonempty = false;
+    for (const auto& specs_list : specs_lists) {
+      any_list_nonempty |= !specs_list.empty();
+    }
+    if (!any_list_nonempty) {
+      return;
+    }
+
+    blocks.emplace_back(phosg::string_printf("<tr><td class=\"loc\">%s</td>", loc_name));
+    for (uint8_t section_id = 0; section_id < 10; section_id++) {
+      blocks.emplace_back(phosg::string_printf("<td class=\"sec%hhu\">", section_id));
+      vector<string> tokens;
+      for (const auto& spec : specs_lists[section_id]) {
+        auto frac = phosg::reduce_fraction<uint64_t>(spec.probability, 0x100000000);
+
+        ItemData example_item = spec.data;
+        if (example_item.can_be_encoded_in_rel_rare_table()) {
+          if (example_item.data1[0] == 2) {
+            example_item.data1[1] = 0x00;
+            example_item.assign_mag_stats(ItemMagStats());
+          } else if (example_item.data1[0] == 3) {
+            example_item.set_tool_item_amount(ItemData::StackLimits::DEFAULT_STACK_LIMITS_V3_V4, 1);
+          }
+        }
+
+        tokens.emplace_back(name_index->describe_item(example_item, false, true));
+        float denom = static_cast<float>(frac.second) / static_cast<double>(frac.first);
+        string denom_token = (floor(denom) == denom)
+            ? phosg::string_printf("1 / %g", denom)
+            : phosg::string_printf("1 / %.02f", denom);
+        tokens.emplace_back(phosg::string_printf(
+            "<span class=\"rate\" title=\"True rate: %" PRIu64 " / %" PRIu64 "\">%s</span>",
+            frac.first, frac.second, denom_token.c_str()));
+      }
+      if (!blocks.empty()) {
+        blocks.emplace_back(phosg::join(tokens, "<br />"));
+      }
+      blocks.emplace_back("</td>");
+    }
+    blocks.emplace_back("</tr>");
+  };
+
+  const auto& zone_types = zone_types_for_episode.at(episode);
+  for (const auto& zone_type : zone_types) {
+    add_location_header(zone_type.name);
+    for (EnemyType type : zone_type.types) {
+      uint8_t rt_index = type_definition_for_enemy(type).rt_index;
+      if (rt_index == 0xFF) {
+        continue;
+      }
+      array<vector<ExpandedDrop>, 10> specs_lists;
+      for (uint8_t section_id = 0; section_id < 10; section_id++) {
+        specs_lists[section_id] = this->get_enemy_specs(mode, episode, difficulty, section_id, rt_index);
+      }
+      add_specs_row(phosg::name_for_enum(type), specs_lists);
+    }
+    for (uint8_t floor : zone_type.floors) {
+      array<vector<ExpandedDrop>, 10> specs_lists;
+      for (uint8_t section_id = 0; section_id < 10; section_id++) {
+        specs_lists[section_id] = this->get_box_specs(mode, episode, difficulty, section_id, floor);
+      }
+      auto loc_name = phosg::string_printf("%s (box)", name_for_floor(episode, floor));
+      add_specs_row(loc_name.c_str(), specs_lists);
+    }
+  }
+  blocks.emplace_back("</table></body></html>");
+
+  return phosg::join(blocks, "");
+}
+
 phosg::JSON RareItemSet::json(shared_ptr<const ItemNameIndex> name_index) const {
   auto modes_dict = phosg::JSON::dict();
   static const array<GameMode, 4> modes = {GameMode::NORMAL, GameMode::BATTLE, GameMode::CHALLENGE, GameMode::SOLO};
@@ -455,7 +712,7 @@ phosg::JSON RareItemSet::json(shared_ptr<const ItemNameIndex> name_index) const
                 spec_json.emplace_back(name_index->describe_item(spec.data));
               }
               for (const auto& enemy_type : enemy_types) {
-                if (enemy_type_valid_for_episode(episode, enemy_type)) {
+                if (type_definition_for_enemy(enemy_type).valid_in_episode(episode)) {
                   phosg::JSON this_spec_json = spec_json;
                   collection_dict.emplace(phosg::name_for_enum(enemy_type), phosg::JSON::list()).first->second->emplace_back(std::move(this_spec_json));
                 }

src/RareItemSet.hh

@@ -37,6 +37,11 @@ public:
 
   std::string serialize_afs(bool is_v1) const;
   std::string serialize_gsl(bool big_endian) const;
+  std::string serialize_html(
+      GameMode mode,
+      Episode episode,
+      uint8_t difficulty,
+      std::shared_ptr<const ItemNameIndex> name_index = nullptr) const;
   phosg::JSON json(std::shared_ptr<const ItemNameIndex> name_index = nullptr) const;
 
   void multiply_all_rates(double factor);

src/ReceiveSubcommands.cc

@@ -2808,7 +2808,7 @@ DropReconcileResult reconcile_drop_request_with_map(
       res.ene_st = map->enemy_state_for_index(version, cmd.floor, cmd.entity_index);
       EnemyType type = res.ene_st->type(version, episode, event);
       log.info("Drop check for E-%03zX %s", res.ene_st->e_id, phosg::name_for_enum(type));
-      res.effective_rt_index = rare_table_index_for_enemy_type(type);
+      res.effective_rt_index = type_definition_for_enemy(type).rt_index;
       // rt_indexes in Episode 4 don't match those sent in the command; we just
       // ignore what the client sends.
       if ((episode != Episode::EP4) && (cmd.rt_index != res.effective_rt_index)) {
@@ -3106,7 +3106,7 @@ static void on_set_quest_flag(shared_ptr<Client> c, uint8_t command, uint8_t fla
             {
                 {0x60, 0x06, 0x0000},
                 static_cast<uint8_t>(c->floor),
-                rare_table_index_for_enemy_type(boss_enemy_type),
+                type_definition_for_enemy(boss_enemy_type).rt_index,
                 enemy_index == 0xFFFF ? 0x0B4F : enemy_index,
                 pos,
                 2,
@@ -3758,7 +3758,7 @@ static uint32_t base_exp_for_enemy_type(
   for (const auto& episode : episode_order) {
     try {
       const auto& bp_table = bp_index->get_table(is_solo, episode);
-      uint32_t bp_index = battle_param_index_for_enemy_type(episode, enemy_type);
+      uint32_t bp_index = type_definition_for_enemy(enemy_type).bp_index;
       return bp_table.stats[difficulty][bp_index].experience;
     } catch (const out_of_range&) {
     }

src/SaveFileFormats.cc

@@ -1161,7 +1161,7 @@ PSOBBCharacterFile::operator PSOXBCharacterFileCharacter() const {
   return ret;
 }
 
-LoadedPSOCHARFile load_psochar(const string& filename, bool load_system) {
+PSOCHARFile::LoadSharedResult PSOCHARFile::load_shared(const string& filename, bool load_system) {
   auto f = phosg::fopen_unique(filename, "rb");
   auto header = phosg::freadx<PSOCommandHeaderBB>(f.get());
   if (header.size != 0x399C) {
@@ -1175,7 +1175,7 @@ LoadedPSOCHARFile load_psochar(const string& filename, bool load_system) {
   }
   static_assert(sizeof(PSOBBCharacterFile) + sizeof(PSOBBBaseSystemFile) + sizeof(PSOBBTeamMembership) == 0x3994, ".psochar size is incorrect");
 
-  LoadedPSOCHARFile ret;
+  LoadSharedResult ret;
   ret.character_file = make_shared<PSOBBCharacterFile>(phosg::freadx<PSOBBCharacterFile>(f.get()));
   if (load_system) {
     ret.system_file = make_shared<PSOBBBaseSystemFile>(phosg::freadx<PSOBBBaseSystemFile>(f.get()));
@@ -1183,7 +1183,7 @@ LoadedPSOCHARFile load_psochar(const string& filename, bool load_system) {
   return ret;
 }
 
-void save_psochar(
+void PSOCHARFile::save(
     const std::string& filename,
     std::shared_ptr<const PSOBBBaseSystemFile> system,
     std::shared_ptr<const PSOBBCharacterFile> character) {

src/SaveFileFormats.hh

@@ -749,8 +749,9 @@ struct PSOXBCharacterFileCharacter {
 
 struct PSOBBCharacterFile {
   // Most fields have the same meanings as in PSOGCCharacterFile::Character.
-  // This is the character data used by the server for all game versions, and
+  // This is part of the .psochar file format, but it is not the first member
-  // is also the format used in .psochar files.
+  // of that structure, so add 8 to all the offsets here if you're working with
+  // a .psochar file. See PSOCHARFile below for the full file format.
 
   /* 0000 */ PlayerInventory inventory;
   /* 034C */ PlayerDispDataBB disp;
@@ -838,17 +839,28 @@ struct PSOBBCharacterFile {
   void recompute_stats(std::shared_ptr<const LevelTable> level_table);
 } __packed_ws__(PSOBBCharacterFile, 0x2EA4);
 
-struct LoadedPSOCHARFile {
+struct PSOCHARFile {
-  std::shared_ptr<PSOBBBaseSystemFile> system_file; // Null if load_system is false
+  // This is the format of .psochar files used by newserv and Ephinea (and
-  std::shared_ptr<PSOBBCharacterFile> character_file; // Never null
+  // perhaps other servers as well). newserv doesn't actually use this
-  // Team membership is present in the file, but ignored by newserv
+  // structure in its logic, so it's here primarily for documentation.
-};
 
-LoadedPSOCHARFile load_psochar(const std::string& filename, bool load_system);
+  /* 0000 */ PSOCommandHeaderBB header; // command = 0x00E7, size = 0x399C, flag = 0
-void save_psochar(
+  /* 0008 */ PSOBBCharacterFile character;
-    const std::string& filename,
+  /* 2EAC */ PSOBBBaseSystemFile system;
-    std::shared_ptr<const PSOBBBaseSystemFile> system,
+  /* 3164 */ PSOBBTeamMembership team_membership;
-    std::shared_ptr<const PSOBBCharacterFile> character);
+  /* 399C */
+
+  struct LoadSharedResult {
+    std::shared_ptr<PSOBBCharacterFile> character_file; // Never null
+    std::shared_ptr<PSOBBBaseSystemFile> system_file; // Null if load_system is false
+    // Team membership is present in the file, but newserv ignores it
+  };
+  static LoadSharedResult load_shared(const std::string& filename, bool load_system);
+  static void save(
+      const std::string& filename,
+      std::shared_ptr<const PSOBBBaseSystemFile> system,
+      std::shared_ptr<const PSOBBCharacterFile> character);
+} __packed_ws__(PSOCHARFile, 0x399C);
 
 ////////////////////////////////////////////////////////////////////////////////
 // Guild Card files