reimplement Episode 3 map categories

README.md

@@ -355,7 +355,7 @@ There are multiple PSO quest formats out there; newserv supports all of them. It
 4. *Episode 3 quests don't go in the system/quests directory. See the [Episode 3 section](#episode-3-features) section below.*
 5. *Quest source can be assembled into a .bin or .bind file with `newserv assemble-quest-script FILENAME.txt`. See system/quests/retrieval/q058-gc-e.bin.txt for an annotated example; this is the English GameCube version of Lost HEAT SWORD.*
 
-Episode 3 download quests consist only of a .bin file - there is no corresponding .dat file. Episode 3 download quest files may be named with the .mnm extension instead of .bin, since the format is the same as the standard map files (in system/ep3/). These files can be encoded in any of the formats described above, except .qst.
+Episode 3 download quests consist only of a .bin file - there is no corresponding .dat file. Episode 3 download quest files may be named with the .mnm extension instead of .bin, since the format is the same as the standard map files (in system/ep3/maps/). These files can be encoded in any of the formats described above, except .qst.
 
 When newserv indexes the quests during startup, it will warn (but not fail) if any quests are corrupt or in unrecognized formats.
 
@@ -452,9 +452,7 @@ Episode 3 state and game data is stored in the system/ep3 directory. The files i
 * card-text.mnr: Compressed card text archive. Generally only used for debugging.
 * card-text.mnrd: Decompressed card text archive; same format as TextCardE.bin. Generally only used for debugging.
 * com-decks.json: COM decks used in tournaments. The default decks in this file come from logs from Sega's servers, so the file doesn't include every COM deck Sega ever made - the rest are probably lost to time.
-* maps/: Online free battle and quest maps (.mnm/.bin/.mnmd/.bind files). newserv comes with the default online maps, as well as some fan-made variations and quests to help new players get up to speed.
+* maps/: Online free battle and quest maps (.mnm/.bin/.mnmd/.bind files). newserv comes with the default online maps, as well as some fan-made variations and quests to help new players get up to speed. Within the maps/ directory, each subdirectory is treated as a separate category and may be optionally downloadable or available at the battle setup counter. The category.json file in each subdirectory specifies the category's behavior; see system/ep3/maps/online/category.json for a documented example.
-* maps-download/: Download maps and quests (.mnm/.bin/.mnmd/.bind files). There are two subcategories by default (download maps and Trial Edition download maps), but you can add more by editing QuestCategories in config.json. Categories that have flag 0x40 (Ep3 download) set are indexed from this directory; all others are indexed from system/quests/. Files in maps-download/ subdirectories have the same format as those in the maps/ directory, but should be named like `e###-gc3-LANGUAGE.EXT` (similar to how non-Episode 3 quests are named in the system/quests/ directory). If you want a map to be available for online play and for downloading, the file must exist in both maps/ and in a maps-download/ subdirectory (a symbolic link is acceptable).
-* maps-offline/: Offline map files. These are all the offline quests and free battle maps from the client, including some debugging/test maps that were inaccessible during normal play. To make them playable online, put the files in the maps/ directory.
 * tournament-state.json: State of all active tournaments. This file is automatically written when any tournament changes state for any reason (e.g. a tournament is created/started/deleted or a match is resolved).
 
 There is no public editor for Episode 3 maps and quests, but the format is described fairly thoroughly in src/Episode3/DataIndexes.hh (see the MapDefinition structure). You'll need to use `newserv decompress-prs ...` to decompress a .bin or .mnm file before editing it, but you don't need to compress it again to use it - just put the .bind or .mnmd file in the maps directory and newserv will make it available.

src/Episode3/DataIndexes.cc

@@ -2670,8 +2670,9 @@ std::shared_ptr<const std::string> MapIndex::VersionedMap::trial_download() cons
   return this->download_data_trial;
 }
 
-MapIndex::Map::Map(shared_ptr<const VersionedMap> initial_version)
+MapIndex::Map::Map(shared_ptr<const VersionedMap> initial_version, uint8_t visibility_flags)
     : map_number(initial_version->map->map_number),
+      visibility_flags(visibility_flags),
       initial_version(initial_version) {
   size_t lang_index = static_cast<size_t>(this->initial_version->language);
   this->versions.resize(lang_index + 1);
@@ -2718,40 +2719,47 @@ shared_ptr<const MapIndex::VersionedMap> MapIndex::Map::version(Language languag
   throw logic_error("no map versions exist");
 }
 
-MapIndex::MapIndex(const string& directory) {
+MapIndex::Category::Category(uint32_t category_id, const phosg::JSON& json)
+    : category_id(category_id),
+      visibility_flags(json.get_int("VisibilityFlags")),
+      name(json.get_string("Name", "")),
+      description(json.get_string("Description", "")) {}
+
+MapIndex::MapIndex(const string& directory, bool raise_on_any_failure) {
   map<uint32_t, shared_ptr<Map>> mutable_maps;
-  for (const auto& item : std::filesystem::directory_iterator(directory)) {
+
-    string filename = item.path().filename().string();
+  auto try_add_map_file = [&](std::shared_ptr<Category> category, const std::string& file_path) -> void {
     try {
+      string filename = phosg::basename(file_path);
       string base_filename;
       string compressed_data;
       shared_ptr<MapDefinition> decompressed_data;
       if (filename.ends_with(".mnmd") || filename.ends_with(".bind")) {
-        decompressed_data = make_shared<MapDefinition>(phosg::load_object_file<MapDefinition>(directory + "/" + filename));
+        decompressed_data = make_shared<MapDefinition>(phosg::load_object_file<MapDefinition>(file_path));
         base_filename = filename.substr(0, filename.size() - 5);
       } else if (filename.ends_with(".mnm") || filename.ends_with(".bin")) {
-        compressed_data = phosg::load_file(directory + "/" + filename);
+        compressed_data = phosg::load_file(file_path);
         base_filename = filename.substr(0, filename.size() - 4);
       } else if (filename.ends_with(".bin.gci") || filename.ends_with(".mnm.gci")) {
-        compressed_data = decode_gci_data(phosg::load_file(directory + "/" + filename));
+        compressed_data = decode_gci_data(phosg::load_file(file_path));
         base_filename = filename.substr(0, filename.size() - 8);
       } else if (filename.ends_with(".gci")) {
-        compressed_data = decode_gci_data(phosg::load_file(directory + "/" + filename));
+        compressed_data = decode_gci_data(phosg::load_file(file_path));
         base_filename = filename.substr(0, filename.size() - 4);
       } else if (filename.ends_with(".bin.vms") || filename.ends_with(".mnm.vms")) {
-        compressed_data = decode_vms_data(phosg::load_file(directory + "/" + filename));
+        compressed_data = decode_vms_data(phosg::load_file(file_path));
         base_filename = filename.substr(0, filename.size() - 8);
       } else if (filename.ends_with(".vms")) {
-        compressed_data = decode_vms_data(phosg::load_file(directory + "/" + filename));
+        compressed_data = decode_vms_data(phosg::load_file(file_path));
         base_filename = filename.substr(0, filename.size() - 4);
       } else if (filename.ends_with(".bin.dlq") || filename.ends_with(".mnm.dlq")) {
-        compressed_data = decode_dlq_data(phosg::load_file(directory + "/" + filename));
+        compressed_data = decode_dlq_data(phosg::load_file(file_path));
         base_filename = filename.substr(0, filename.size() - 8);
       } else if (filename.ends_with(".dlq")) {
-        compressed_data = decode_dlq_data(phosg::load_file(directory + "/" + filename));
+        compressed_data = decode_dlq_data(phosg::load_file(file_path));
         base_filename = filename.substr(0, filename.size() - 4);
       } else {
-        continue; // Silently skip file
+        return; // Silently skip file
       }
 
       if (base_filename.size() < 2) {
@@ -2771,18 +2779,31 @@ MapIndex::MapIndex(const string& directory) {
         throw runtime_error("unknown map file format");
       }
 
+      uint8_t visibility_flags = category ? category->visibility_flags : 0x00;
+
       string name = vm->map->name.decode(vm->language);
       auto map_it = mutable_maps.find(vm->map->map_number);
       if (map_it == mutable_maps.end()) {
-        map_it = mutable_maps.emplace(vm->map->map_number, make_shared<Map>(vm)).first;
+        map_it = mutable_maps.emplace(vm->map->map_number, make_shared<Map>(vm, visibility_flags)).first;
         this->maps.emplace(vm->map->map_number, map_it->second);
-        static_game_data_log.debug_f("({}) Created Episode 3 map {:08X} {} ({}; {})",
+
+        string in_category_str;
+        if (category) {
+          in_category_str = std::format(" in category {}", category->name);
+          category->add_map(map_it->second);
+        }
+        static_game_data_log.debug_f("({}) Created Episode 3 map {:08X} {}{} ({}; {})",
             filename,
             vm->map->map_number,
             char_for_language(vm->language),
+            in_category_str,
             vm->map->is_quest() ? "quest" : "free",
             name);
       } else {
+        if (map_it->second->visibility_flags != visibility_flags) {
+          throw std::runtime_error(std::format("visibility flags {:02X} for added map {} do not match existing flags {}",
+              map_it->second->visibility_flags, file_path, visibility_flags));
+        }
         map_it->second->add_version(vm);
         static_game_data_log.debug_f("({}) Added Episode 3 map version {:08X} {} ({}; {})",
             filename,
@@ -2794,13 +2815,45 @@ MapIndex::MapIndex(const string& directory) {
       this->maps_by_name.emplace(vm->map->name.decode(vm->language), map_it->second);
 
     } catch (const exception& e) {
-      static_game_data_log.warning_f("Failed to index Episode 3 map {}: {}",
+      if (raise_on_any_failure) {
-          filename, e.what());
+        throw;
+      }
+      static_game_data_log.warning_f("Failed to index Episode 3 map {}: {}", file_path, e.what());
+    }
+  };
+
+  for (const auto& cat_item : std::filesystem::directory_iterator(directory)) {
+    string cat_dir_path = cat_item.path().string();
+
+    if (cat_item.is_directory()) {
+      shared_ptr<Category> category;
+      try {
+        string json_filename = std::format("{}/{}", cat_item.path().string(), "category.json");
+        auto category_json = phosg::JSON::parse(phosg::load_file(json_filename));
+        uint32_t category_id = this->categories.size() + 1;
+        auto category = make_shared<Category>(category_id, category_json);
+        this->categories.emplace(category_id, category);
+        static_game_data_log.debug_f("({}) Created Episode 3 map category {:08X} ({})",
+            cat_item.path().filename().string(), category_id, category->name);
+
+        for (const auto& map_item : std::filesystem::directory_iterator(cat_item)) {
+          try_add_map_file(category, map_item.path().string());
+        }
+
+      } catch (const exception& e) {
+        if (raise_on_any_failure) {
+          throw;
+        }
+        static_game_data_log.warning_f("Failed to index Episode 3 map category {}: {}", cat_item.path().string(), e.what());
+      }
+
+    } else {
+      try_add_map_file(nullptr, cat_dir_path);
     }
   }
 }
 
-const string& MapIndex::get_compressed_list(size_t num_players, Language language) const {
+const string& MapIndex::get_compressed_list(size_t num_players, Language language, bool is_trial) const {
   if (num_players == 0) {
     throw runtime_error("cannot generate map list for no players");
   }
@@ -2808,17 +2861,27 @@ const string& MapIndex::get_compressed_list(size_t num_players, Language languag
     throw logic_error("player count is too high in map list generation");
   }
 
+  auto& compressed_lists = is_trial ? this->compressed_map_lists_trial : this->compressed_map_lists_final;
+
   size_t lang_index = static_cast<size_t>(language);
-  if (lang_index >= this->compressed_map_lists.size()) {
+  if (lang_index >= compressed_lists.size()) {
-    this->compressed_map_lists.resize(lang_index + 1);
+    compressed_lists.resize(lang_index + 1);
   }
-  string& compressed_map_list = this->compressed_map_lists[lang_index].at(num_players - 1);
+  string& compressed_map_list = compressed_lists[lang_index].at(num_players - 1);
   if (compressed_map_list.empty()) {
     phosg::StringWriter entries_w;
     phosg::StringWriter strings_w;
 
+    auto vis_flag = is_trial
+        ? Episode3::MapIndex::VisibilityFlag::ONLINE_TRIAL
+        : Episode3::MapIndex::VisibilityFlag::ONLINE_FINAL;
+
     size_t num_maps = 0;
     for (const auto& map_it : this->maps) {
+      if (!map_it.second->check_visibility_flag(vis_flag)) {
+        continue;
+      }
+
       auto vm = map_it.second->version(language);
       size_t map_num_players = 0;
       for (size_t z = 0; z < 4; z++) {
@@ -2875,11 +2938,12 @@ const string& MapIndex::get_compressed_list(size_t num_players, Language languag
     compressed_w.write(prs.close());
     compressed_map_list = std::move(compressed_w.str());
     if (compressed_map_list.size() > 0x7BEC) {
-      throw runtime_error(std::format("compressed map list for {} players is too large (0x{:X} bytes)", num_players, compressed_map_list.size()));
+      throw runtime_error(std::format("compressed {} map list for {} players is too large (0x{:X} bytes)",
+          is_trial ? "trial" : "final", num_players, compressed_map_list.size()));
     }
     size_t decompressed_size = sizeof(header) + entries_w.size() + strings_w.size();
-    static_game_data_log.info_f("Generated Episode 3 compressed map list for {} player(s) ({} maps; 0x{:X} -> 0x{:X} bytes)",
+    static_game_data_log.info_f("Generated Episode 3 compressed {} map list for {} player(s) ({} maps; 0x{:X} -> 0x{:X} bytes)",
-        num_players, num_maps, decompressed_size, compressed_map_list.size());
+        is_trial ? "trial" : "final", num_players, num_maps, decompressed_size, compressed_map_list.size());
   }
   return compressed_map_list;
 }

src/Episode3/DataIndexes.hh

@@ -1587,7 +1587,12 @@ private:
 
 class MapIndex {
 public:
-  explicit MapIndex(const std::string& directory);
+  enum class VisibilityFlag : uint8_t {
+    ONLINE_TRIAL = 0x01,
+    ONLINE_FINAL = 0x02,
+    DOWNLOAD_TRIAL = 0x04,
+    DOWNLOAD_FINAL = 0x08,
+  };
 
   class VersionedMap {
   public:
@@ -1611,9 +1616,14 @@ public:
   class Map {
   public:
     uint32_t map_number;
+    uint8_t visibility_flags;
     std::shared_ptr<const VersionedMap> initial_version;
 
-    explicit Map(std::shared_ptr<const VersionedMap> initial_version);
+    Map(std::shared_ptr<const VersionedMap> initial_version, uint8_t visibility_flags);
+
+    inline bool check_visibility_flag(VisibilityFlag flag) const {
+      return (this->visibility_flags & static_cast<uint8_t>(flag));
+    }
 
     void add_version(std::shared_ptr<const VersionedMap> vm);
     bool has_version(Language language) const;
@@ -1626,24 +1636,76 @@ public:
     std::vector<std::shared_ptr<const VersionedMap>> versions;
   };
 
-  const std::string& get_compressed_list(size_t num_players, Language language) const;
+  class Category {
-  inline std::shared_ptr<const Map> get(uint32_t id) const {
+  public:
+    uint32_t category_id;
+    uint8_t visibility_flags;
+    std::string name;
+    std::string description;
+
+    Category(uint32_t category_id, const phosg::JSON& json);
+
+    inline bool check_visibility_flag(VisibilityFlag flag) const {
+      return (this->visibility_flags & static_cast<uint8_t>(flag));
+    }
+
+    inline void add_map(std::shared_ptr<const Map> map) {
+      this->maps.emplace(map->map_number, map);
+    }
+    inline const std::map<uint32_t, std::shared_ptr<const Map>>& all_maps() const {
+      return this->maps;
+    }
+
+  private:
+    std::map<uint32_t, std::shared_ptr<const Map>> maps;
+  };
+
+  explicit MapIndex(const std::string& directory, bool raise_on_any_failure = false);
+
+  const std::string& get_compressed_list(size_t num_players, Language language, bool is_trial) const;
+  inline std::shared_ptr<const Map> map_for_id(uint32_t id) const {
     return this->maps.at(id);
   }
-  inline std::shared_ptr<const Map> get(const std::string& name) const {
+  inline std::shared_ptr<const Map> map_for_name(const std::string& name) const {
     return this->maps_by_name.at(name);
   }
-  inline const std::map<uint32_t, std::shared_ptr<const Map>>& all() const {
+  inline const std::map<uint32_t, std::shared_ptr<const Map>>& all_maps() const {
     return this->maps;
   }
 
+  inline std::shared_ptr<const Category> category_for_id(uint32_t id) const {
+    return this->categories.at(id);
+  }
+  inline const std::map<uint32_t, std::shared_ptr<const Category>>& all_categories() const {
+    return this->categories;
+  }
+
 private:
-  // The compressed map lists are generated on demand from the maps map below
+  // The compressed map lists are generated on demand from the maps map below.
-  mutable std::vector<std::array<std::string, 4>> compressed_map_lists;
+  // THey are indexed as [language][num_players]
+  mutable std::vector<std::array<std::string, 4>> compressed_map_lists_trial;
+  mutable std::vector<std::array<std::string, 4>> compressed_map_lists_final;
+
+  std::map<uint32_t, std::shared_ptr<const Category>> categories;
   std::map<uint32_t, std::shared_ptr<const Map>> maps;
   std::unordered_map<std::string, std::shared_ptr<Map>> maps_by_name;
 };
 
+class MapCategoryIndex {
+public:
+  explicit MapCategoryIndex(const std::string& directory);
+
+  inline std::shared_ptr<const MapIndex> get(uint32_t id) const {
+    return this->indexes.at(id);
+  }
+  inline const std::map<uint32_t, std::shared_ptr<const MapIndex>>& all() const {
+    return this->indexes;
+  }
+
+private:
+  std::map<uint32_t, std::shared_ptr<const MapIndex>> indexes;
+};
+
 class COMDeckIndex {
 public:
   COMDeckIndex(const std::string& filename);

src/Episode3/Server.cc

@@ -2527,7 +2527,7 @@ void Server::handle_CAx40_map_list_request(shared_ptr<Client> sender_c, const st
 
   size_t num_players = l ? l->count_clients() : 1;
   Language language = sender_c ? sender_c->language() : Language::ENGLISH;
-  const auto& list_data = this->options.map_index->get_compressed_list(num_players, language);
+  const auto& list_data = this->options.map_index->get_compressed_list(num_players, language, this->options.is_nte());
 
   phosg::StringWriter w;
   uint32_t subcommand_size = (list_data.size() + sizeof(G_MapList_Ep3_6xB6x40) + 3) & (~3);
@@ -2596,7 +2596,7 @@ void Server::handle_CAx41_map_request(shared_ptr<Client>, const string& data) {
   const auto& cmd = check_size_t<G_MapDataRequest_Ep3_CAx41>(data);
   this->send_debug_command_received_message(cmd.header.subsubcommand, "MAP DATA");
   if (!this->options.tournament || (this->options.tournament->get_map()->map_number == cmd.map_number)) {
-    this->last_chosen_map = this->options.map_index->get(cmd.map_number);
+    this->last_chosen_map = this->options.map_index->map_for_id(cmd.map_number);
     this->send_6xB6x41_to_all_clients();
   }
 }

src/Episode3/Tournament.cc

@@ -357,7 +357,7 @@ void Tournament::init() {
   bool is_registration_complete;
   if (!this->source_json.is_null()) {
     this->name = this->source_json.get_string("name");
-    this->map = this->map_index->get(this->source_json.get_int("map_number"));
+    this->map = this->map_index->map_for_id(this->source_json.get_int("map_number"));
     this->rules = Rules(this->source_json.at("rules"));
     this->flags = this->source_json.get_int("flags", 0x02);
     if (this->source_json.get_bool("is_2v2", false)) {

src/Main.cc

@@ -2827,9 +2827,9 @@ Action a_show_ep3_maps(
       s->load_ep3_cards();
       s->load_ep3_maps();
 
-      const auto& map_ids = s->ep3_map_index->all();
+      const auto& all_maps = s->ep3_map_index->all_maps();
-      phosg::log_info_f("{} maps", map_ids.size());
+      phosg::log_info_f("{} maps", all_maps.size());
-      for (const auto& [map_number, map] : map_ids) {
+      for (const auto& [map_number, map] : all_maps) {
         const auto& vms = map->all_versions();
         for (size_t lang_index = 0; lang_index < vms.size(); lang_index++) {
           if (!vms[lang_index]) {
@@ -3042,6 +3042,15 @@ Action a_check_quests(
       phosg::fwrite_fmt(stdout, "All quests indexed\n");
     });
 
+Action a_check_ep3_maps(
+    "check-ep3-maps", nullptr,
+    +[](phosg::Arguments& args) {
+      config_log.info_f("Collecting Episode 3 data");
+      auto s = make_shared<ServerState>(get_config_filename(args));
+      s->is_debug = true;
+      s->load_ep3_maps(true);
+    });
+
 Action a_check_client_functions(
     "check-client-functions", nullptr,
     +[](phosg::Arguments&) {

src/Menu.hh

@@ -24,7 +24,7 @@ constexpr uint32_t QUEST_EP2 = 0x55020255;
 constexpr uint32_t QUEST_EP3 = 0x55030355;
 // See the decsription of the A2 command in CommandFormats.hh for why these
 // menu IDs don't fit the rest of the pattern.
-constexpr uint32_t QUEST_CATEGORIES_EP1 = 0x01000001;
+constexpr uint32_t QUEST_CATEGORIES_EP1_EP3_EP4 = 0x01000001;
 constexpr uint32_t QUEST_CATEGORIES_EP2 = 0x02000002;
 constexpr uint32_t PROXY_DESTINATIONS = 0x77000077;
 constexpr uint32_t PROGRAMS = 0x88000088;

src/ReceiveCommands.cc

@@ -2152,7 +2152,7 @@ static asio::awaitable<void> on_09(shared_ptr<Client> c, Channel::Message& msg)
   auto s = c->require_server_state();
 
   switch (cmd.menu_id) {
-    case MenuID::QUEST_CATEGORIES_EP1:
+    case MenuID::QUEST_CATEGORIES_EP1_EP3_EP4:
     case MenuID::QUEST_CATEGORIES_EP2:
       // Don't send anything here. The quest filter menu already has short
       // descriptions included with the entries, which the client shows in the
@@ -2179,8 +2179,12 @@ static asio::awaitable<void> on_09(shared_ptr<Client> c, Channel::Message& msg)
       break;
     }
     case MenuID::QUEST_EP3: {
-      auto map = s->ep3_download_map_index->get(cmd.item_id);
+      auto vis_flag = (c->version() == Version::GC_EP3_NTE)
-      if (!map) {
+          ? Episode3::MapIndex::VisibilityFlag::ONLINE_TRIAL
+          : Episode3::MapIndex::VisibilityFlag::ONLINE_FINAL;
+
+      auto map = s->ep3_map_index->map_for_id(cmd.item_id);
+      if (!map || !map->check_visibility_flag(vis_flag)) {
         send_quest_info(c, "$C4Map does not exist.", 0x00, true);
       } else {
         auto vm = map->version(c->language());
@@ -2557,11 +2561,7 @@ static asio::awaitable<void> on_10_main_menu(shared_ptr<Client> c, uint32_t item
       break;
 
     case MainMenuItemID::DOWNLOAD_QUESTS: {
-      if (is_ep3(c->version())) {
+      send_quest_categories_menu(c, QuestMenuType::DOWNLOAD, Episode::NONE);
-        send_ep3_download_quest_menu(c);
-      } else {
-        send_quest_categories_menu(c, QuestMenuType::DOWNLOAD, Episode::NONE);
-      }
       break;
     }
 
@@ -2811,27 +2811,32 @@ static asio::awaitable<void> on_10_game_menu(shared_ptr<Client> c, uint32_t item
 }
 
 static asio::awaitable<void> on_10_quest_categories(shared_ptr<Client> c, uint32_t item_id) {
-  // Episode 3 doesn't have this menu
   if (is_ep3(c->version())) {
-    throw runtime_error("Episode 3 client made selection on quest categories menu");
+    auto s = c->require_server_state();
-  }
+    if (!s->ep3_map_index) {
+      send_lobby_message_box(c, "$C7Quests are not available.");
+      co_return;
+    }
+    send_ep3_download_quest_menu(c, item_id);
 
-  auto s = c->require_server_state();
+  } else {
-  if (!s->quest_index) {
+    auto s = c->require_server_state();
-    send_lobby_message_box(c, "$C7Quests are not available.");
+    if (!s->quest_index) {
-    co_return;
+      send_lobby_message_box(c, "$C7Quests are not available.");
-  }
+      co_return;
+    }
 
-  shared_ptr<Lobby> l = c->lobby.lock();
+    shared_ptr<Lobby> l = c->lobby.lock();
-  Episode episode = l ? l->episode : Episode::NONE;
+    Episode episode = l ? l->episode : Episode::NONE;
-  uint16_t version_flags = (1 << static_cast<size_t>(c->version())) | (l ? l->quest_version_flags() : 0);
+    uint16_t version_flags = (1 << static_cast<size_t>(c->version())) | (l ? l->quest_version_flags() : 0);
-  QuestIndex::IncludeCondition include_condition = nullptr;
+    QuestIndex::IncludeCondition include_condition = nullptr;
-  if (l && !c->login->account->check_flag(Account::Flag::DISABLE_QUEST_REQUIREMENTS)) {
+    if (l && !c->login->account->check_flag(Account::Flag::DISABLE_QUEST_REQUIREMENTS)) {
-    include_condition = l->quest_include_condition();
+      include_condition = l->quest_include_condition();
-  }
+    }
 
-  const auto& quests = s->quest_index->filter(episode, version_flags, item_id, include_condition);
+    const auto& quests = s->quest_index->filter(episode, version_flags, item_id, include_condition);
-  send_quest_menu(c, quests, !l);
+    send_quest_menu(c, quests, !l);
+  }
 }
 
 static asio::awaitable<void> on_10_quest_menu(shared_ptr<Client> c, uint32_t item_id) {
@@ -2898,7 +2903,16 @@ static asio::awaitable<void> on_10_ep3_download_quest_menu(shared_ptr<Client> c,
   if (c->lobby.lock()) {
     throw runtime_error("Episode 3 quests can only be downloaded when client is not in a lobby");
   }
-  auto map = s->ep3_download_map_index->get(item_id);
+
+  auto map = s->ep3_map_index->map_for_id(item_id);
+
+  auto vis_flag = (c->version() == Version::GC_EP3_NTE)
+      ? Episode3::MapIndex::VisibilityFlag::ONLINE_TRIAL
+      : Episode3::MapIndex::VisibilityFlag::ONLINE_FINAL;
+  if (!map->check_visibility_flag(vis_flag)) {
+    throw runtime_error("map is not visible to this client");
+  }
+
   auto vm = map->version(c->language());
   auto name = vm->map->name.decode(vm->language);
   string filename = std::format("m{:06}p_{:c}.bin", map->map_number, tolower(char_for_language(vm->language)));
@@ -3055,7 +3069,7 @@ static asio::awaitable<void> on_10(shared_ptr<Client> c, Channel::Message& msg)
     case MenuID::GAME:
       co_await on_10_game_menu(c, base_cmd.item_id, std::move(password));
       break;
-    case MenuID::QUEST_CATEGORIES_EP1:
+    case MenuID::QUEST_CATEGORIES_EP1_EP3_EP4:
     case MenuID::QUEST_CATEGORIES_EP2:
       co_await on_10_quest_categories(c, base_cmd.item_id);
       break;

src/SendCommands.cc

@@ -1674,20 +1674,6 @@ void send_quest_menu_bb(
   send_command_vt(c, is_download_menu ? 0xA4 : 0xA2, entries.size(), entries);
 }
 
-void send_ep3_download_quest_menu(shared_ptr<Client> c) {
-  auto s = c->require_server_state();
-  vector<S_QuestMenuEntry_DC_GC_A2_A4> entries;
-  for (const auto& it : s->ep3_download_map_index->all()) {
-    auto vm = it.second->version(c->language());
-    auto& e = entries.emplace_back();
-    e.menu_id = MenuID::QUEST_EP3;
-    e.item_id = it.first; // map_number
-    e.name.encode(vm->map->name.decode(vm->language), c->language());
-    e.short_description.encode(add_color(vm->map->location_name.decode(vm->language)), c->language());
-  }
-  send_command_vt(c, 0xA4, entries.size(), entries);
-}
-
 template <typename EntryT>
 void send_quest_categories_menu_t(shared_ptr<Client> c, QuestMenuType menu_type, Episode episode) {
   QuestIndex::IncludeCondition include_condition = nullptr;
@@ -1706,7 +1692,7 @@ void send_quest_categories_menu_t(shared_ptr<Client> c, QuestMenuType menu_type,
   auto s = c->require_server_state();
   for (const auto& cat : s->quest_index->categories(menu_type, episode, version_flags, include_condition)) {
     auto& e = entries.emplace_back();
-    e.menu_id = cat->use_ep2_icon() ? MenuID::QUEST_CATEGORIES_EP2 : MenuID::QUEST_CATEGORIES_EP1;
+    e.menu_id = cat->use_ep2_icon() ? MenuID::QUEST_CATEGORIES_EP2 : MenuID::QUEST_CATEGORIES_EP1_EP3_EP4;
     e.item_id = cat->category_id;
     e.name.encode(cat->name, c->language());
     e.short_description.encode(add_color(cat->description), c->language());
@@ -1716,6 +1702,58 @@ void send_quest_categories_menu_t(shared_ptr<Client> c, QuestMenuType menu_type,
   send_command_vt(c, is_download_menu ? 0xA4 : 0xA2, entries.size(), entries);
 }
 
+void send_ep3_download_quest_categories_menu(shared_ptr<Client> c) {
+  if (c->lobby.lock()) {
+    throw std::runtime_error("cannot send Ep3 download quest menu to client in a lobby");
+  }
+
+  auto vis_flag = (c->version() == Version::GC_EP3_NTE)
+      ? Episode3::MapIndex::VisibilityFlag::DOWNLOAD_TRIAL
+      : Episode3::MapIndex::VisibilityFlag::DOWNLOAD_FINAL;
+
+  vector<S_QuestMenuEntry_DC_GC_A2_A4> entries;
+  auto s = c->require_server_state();
+  for (const auto& [_, cat] : s->ep3_map_index->all_categories()) {
+    if (cat->check_visibility_flag(vis_flag)) {
+      auto& e = entries.emplace_back();
+      e.menu_id = MenuID::QUEST_CATEGORIES_EP1_EP3_EP4;
+      e.item_id = cat->category_id;
+      e.name.encode(cat->name, c->language());
+      e.short_description.encode(add_color(cat->description), c->language());
+    }
+  }
+
+  send_command_vt(c, 0xA4, entries.size(), entries);
+}
+
+void send_ep3_download_quest_menu(shared_ptr<Client> c, uint32_t category_id) {
+  if (c->lobby.lock()) {
+    throw std::runtime_error("cannot send Ep3 download quest menu to client in a lobby");
+  }
+
+  auto vis_flag = (c->version() == Version::GC_EP3_NTE)
+      ? Episode3::MapIndex::VisibilityFlag::ONLINE_TRIAL
+      : Episode3::MapIndex::VisibilityFlag::ONLINE_FINAL;
+
+  auto s = c->require_server_state();
+  auto category = s->ep3_map_index->category_for_id(category_id);
+  if (!category->check_visibility_flag(vis_flag)) {
+    throw std::runtime_error("category is not visible to this client");
+  }
+
+  vector<S_QuestMenuEntry_DC_GC_A2_A4> entries;
+  for (const auto& [map_number, map] : category->all_maps()) {
+    auto vm = map->version(c->language());
+    auto& e = entries.emplace_back();
+    e.menu_id = MenuID::QUEST_EP3;
+    e.item_id = map_number;
+    e.name.encode(vm->map->name.decode(vm->language), c->language());
+    e.short_description.encode(add_color(vm->map->location_name.decode(vm->language)), c->language());
+  }
+
+  send_command_vt(c, 0xA4, entries.size(), entries);
+}
+
 void send_quest_menu(
     shared_ptr<Client> c,
     const vector<pair<QuestIndex::IncludeState, shared_ptr<const Quest>>>& quests,
@@ -1731,10 +1769,11 @@ void send_quest_menu(
     case Version::DC_V2:
     case Version::GC_NTE:
     case Version::GC_V3:
-    case Version::GC_EP3_NTE:
-    case Version::GC_EP3:
       send_quest_menu_t<S_QuestMenuEntry_DC_GC_A2_A4>(c, quests, is_download_menu);
       break;
+    case Version::GC_EP3_NTE:
+    case Version::GC_EP3:
+      throw std::logic_error("Episode 3 clients cannot receive a non-download quest menu");
     case Version::XB_V3:
       send_quest_menu_t<S_QuestMenuEntry_XB_A2_A4>(c, quests, is_download_menu);
       break;
@@ -1758,9 +1797,14 @@ void send_quest_categories_menu(shared_ptr<Client> c, QuestMenuType menu_type, E
     case Version::DC_V2:
     case Version::GC_NTE:
     case Version::GC_V3:
+      send_quest_categories_menu_t<S_QuestMenuEntry_DC_GC_A2_A4>(c, menu_type, episode);
+      break;
     case Version::GC_EP3_NTE:
     case Version::GC_EP3:
-      send_quest_categories_menu_t<S_QuestMenuEntry_DC_GC_A2_A4>(c, menu_type, episode);
+      if (menu_type != QuestMenuType::DOWNLOAD) {
+        throw std::runtime_error("Episode 3 clients cannot receive a non-download quest menu");
+      }
+      send_ep3_download_quest_categories_menu(c);
       break;
     case Version::XB_V3:
       send_quest_categories_menu_t<S_QuestMenuEntry_XB_A2_A4>(c, menu_type, episode);

src/SendCommands.hh

@@ -312,7 +312,8 @@ void send_quest_menu(
     std::shared_ptr<Client> c,
     const std::vector<std::pair<QuestIndex::IncludeState, std::shared_ptr<const Quest>>>& quests,
     bool is_download_menu);
-void send_ep3_download_quest_menu(std::shared_ptr<Client> c);
+void send_ep3_download_quest_categories_menu(std::shared_ptr<Client> c);
+void send_ep3_download_quest_menu(std::shared_ptr<Client> c, uint32_t category_id);
 void send_quest_categories_menu(std::shared_ptr<Client> c, QuestMenuType menu_type, Episode episode);
 void send_lobby_list(std::shared_ptr<Client> c);
 

src/ServerState.cc

@@ -2146,11 +2146,9 @@ void ServerState::load_ep3_cards() {
   this->ep3_com_deck_index = make_shared<Episode3::COMDeckIndex>("system/ep3/com-decks.json");
 }
 
-void ServerState::load_ep3_maps() {
+void ServerState::load_ep3_maps(bool raise_on_any_failure) {
   config_log.info_f("Collecting Episode 3 maps");
-  this->ep3_map_index = make_shared<Episode3::MapIndex>("system/ep3/maps");
+  this->ep3_map_index = make_shared<Episode3::MapIndex>("system/ep3/maps", raise_on_any_failure);
-  config_log.info_f("Collecting Episode 3 download maps");
-  this->ep3_download_map_index = make_shared<Episode3::MapIndex>("system/ep3/maps-download");
 }
 
 void ServerState::load_ep3_tournament_state() {

src/ServerState.hh

@@ -183,7 +183,6 @@ struct ServerState : public std::enable_shared_from_this<ServerState> {
   std::shared_ptr<const Episode3::CardIndex> ep3_card_index;
   std::shared_ptr<const Episode3::CardIndex> ep3_card_index_trial;
   std::shared_ptr<const Episode3::MapIndex> ep3_map_index;
-  std::shared_ptr<const Episode3::MapIndex> ep3_download_map_index;
   std::shared_ptr<const Episode3::COMDeckIndex> ep3_com_deck_index;
   std::shared_ptr<const G_SetEXResultValues_Ep3_6xB4x4B> ep3_default_ex_values;
   std::shared_ptr<const G_SetEXResultValues_Ep3_6xB4x4B> ep3_tournament_ex_values;
@@ -438,7 +437,7 @@ struct ServerState : public std::enable_shared_from_this<ServerState> {
   void load_set_data_tables();
   void load_word_select_table();
   void load_ep3_cards();
-  void load_ep3_maps();
+  void load_ep3_maps(bool raise_on_any_failure = false);
   void load_ep3_tournament_state();
   void load_quest_index(bool raise_on_any_failure = false);
   void compile_functions(bool raise_on_any_failure = false);

src/ShellCommands.cc

@@ -703,7 +703,7 @@ ShellCommand c_create_tournament(
     +[](ShellCommand::Args& args) -> asio::awaitable<deque<string>> {
       string name = get_quoted_string(args.args);
       string map_name = get_quoted_string(args.args);
-      auto map = args.s->ep3_map_index->get(map_name);
+      auto map = args.s->ep3_map_index->map_for_name(map_name);
       uint32_t num_teams = stoul(get_quoted_string(args.args), nullptr, 0);
       Episode3::Rules rules;
       rules.set_defaults();