User blog:Crazy teitoku/Guide to netgame assets ripping via Kancolle

Currently this guide is for Google Chrome users and would give a bit of a bonus to those who uses KC3 Kai. However in the case of you using some other viewer/browser/tool you may be able to find a more appropriate guide. For such information, please refer to the tab "Further reading". In theory it is possible replicate the set up mentioned here according to your needs without too much of a hassle other than just consulting manuals where appropriate. Preface=

I have seen discussions on how to extract data from kancolle servers here and there with loose bits and pieces of information which seems like it lacks some form of unification. Gradually over time however these bits and pieces of information becomes harder to find, along with at least two other limiting factors such as either the users are unable to find the resources themselves on how to do it and/or the way wikia searches work is very archaic to say at best.

The methods I shall discuss from here on in maybe a repeat or reiteration of what others may have said. Therefore please do not make a rant about how you have read the steps before and that it was a waste of blog entry blah blah blah. The purpose of this blog entry was to really collate all the bits and pieces of information into almost a "one-stop shop" for virtually anyone interested.

For now, I intend to only cover the extraction of data from SWF files or better known as Shockwave files. These files usually contain sprites, key animations and other information. However there are exceptions such as cases like these whereby the topic is about a game, SWF is basically a container for various game assets and probably a good way to organise how such assets are to be stored within a filesystem as opposed to having several hundreds and thousands of small files. The method could also be easily applied for extracting images, audio or virtually any other data that may be of an interest as once the fundamentals are explained the rest is just a matter of guess work.


 * -|Technique=


 * -|Server layout=

Overview
Example: http://125.6.189.215/kcs/mainD2.swf Note: You can actually hover the majority of the URL above and click on the links to see more information Fundamentals of how the servers are laid out are as follows:

Server IP addresses
The list of server IP addresses that can be found on Servers page tells us the amount of servers that hosts such Kancolle data. In the very least the fundamentals of data mining requires knowledge of these IP addresses so that one can obtain game assets irrespective if the corresponding servers are "on" or not. There are at times certain servers during maintenance receives new game data first compared to others and therefore the need to discover which server gets the preferrential treatment would be beneficial but not a necessity. For every other case, it is possible to obtain game assets from other game servers within the list of servers even though one does not "belong" to the said server.

Visible versus invisible assets
There are two forms of files that are on the server as one can observe from collection of various game assets. There are files that are visible to the public and there are those that aren't visible to the public. There is not much details about whatever files that are not visible to the public because of the way the servers itself are configured - that is they will either throw forbidden or not found as opposed to showing the listing of a directory (or folder). What we do not know is the backend program that is responsible for telling the outcome of RNG to the player for instance as that program is hidden. What we do know are as follows.

For game assets:
 * SWF - Used mainly as a container for storing varied multimedia with the exception of a few files such as Core.swf which is used to contain API calls. Is also used to interact with the program.
 * MP3 - Used mainly to store voice lines. For various BGM or certain sound elements, they are stored as SWF.
 * PNG - Used mainly to store images such as furniture or various other static images. For furniture that features animations or for newer Shop icons/assets, these are stored as SWF instead.
 * JS - Used mainly to instruct what SWF are to be displayed as well as various other elements on the web page. These however, does not contain anything to do with how the game is to function.
 * CSS - Used mainly to describe the style of the page.

For interaction with the game:
 * (JSON) - For virtually any file that does not contain a filename extension are typically plaintext files that starts with something like, "svdata=" and the details in the {} curly brackets are JSON data.

A majority of the files that the game requests the browser to fetch the assets has an extra parameter, "?version=". This however is not a way to obtain older copies of the said file from the server but is used internally by the game developers to keep track of the version they are up to for each of those files. It appears that there seems to be some form of concurrent versioning system going on in the backend or something but from the public eye these older versions of the files are not accessible. back to top

Filesystem hierarchy
Information shown with uppercase endings denotes the description and potential purpose for specified directory or folder. These are as of June 2016.

. └── kcs ├── Contains SWF assets that are usually loaded up when the game loads. ├── gadget │   ├── js   │   │   ├──Various JavaScript assets for handling either payments or server availability. │   └── script │   │   ├──Suspected to only contain aesthetic fixes. ├── kcscontents │   ├── css │   │   ├── These files only describe how the elements are to be laid out for Kancolle stuff. Possibly only used for non-flash related content i.e. banners on DMM page, etc.   │   ├── dictionary │   │   └── js   │   │   │   ├── Contains Dictionary.js which is probably used for glossary on the DMM page itself. │   ├── image │   │   ├── These are the buttons on the DMM page that you can easily see and click. │   ├── information │   │   └── image │   │   │   ├── Images for use on the DMM page. │   ├── news │   │   │   ├── Contains style.jss which is used to define how the rules are meant to be displayed on the DMM page. │   ├── play │   │   ├── image │   │   │   ├── Contains images on how to play the game. Not necessarily a game asset. │   │   └── js   │   │   │   ├── Contains Javascripts on how the page is to interact with the user/player. │   ├── script │   │   │   ├── Contains Javascripts on various aesthetics. │   └── twitter │      └── image │   │   │   ├── Contains a placeholder image for developer tweets to appear on the DMM page ├── resources │   ├── bgm_p │   │   │   ├── Contains various in-game BackGround Music (BGM) but does not contain all BGM. Instead common ones are usually placed here. │   ├── image │   │   ├── furniture │   │   │   ├── chest │   │   │   ├── desk │   │   │   ├── floor │   │   │   ├── object │   │   │   ├── wall │   │   │   └── window │   │   │   ├── These are all furniture related items and are either in PNG format or SWF format. │   │   ├── slotitem │   │   │   ├── airunit_banner │   │   │   ├── airunit_fairy │   │   │   ├── airunit_name │   │   │   ├── btxt_flat │   │   │   ├── card │   │   │   ├── item_character │   │   │   ├── item_on │   │   │   ├── item_up │   │   │   ├── remodal │   │   │   └── statustop_item │   │   │   ├── These contain various Computer Graphics (CG) such as the equipments, equipment fairies, cut-in banners, etc. PNG format. │   │   ├── useitem │   │   │   ├── card │   │   │   ├── icon │   │   │   └── icon_p │   │   │   ├── These contain various "use" items such as furniture coin boxes, furniture fairies, etc. If they are not in PNG they are in SWF for newer ones. │   │   └── world │   │   │   ├── These are server banners. PNG format. │   └── swf │      │    ├── Various in-game resources appears in here. All in SWF format. │      ├── map │   │   │   ├── Contains maps of the "world" in SWF format. │      └── ships │   │   │   ├── Contains the CG for the ships in SWF format. ├── scenes │      ├── Contains various assets in SWF format. └── sound ├── Voice lines for ships └── titlecall ├── Voice lines for introduction - Game start screen back to top

Exceptions
back to top
 * Of all the servers listed under here, there is actually one more other server that is not noted. However the said server does not actually host players but instead host other game details such as the outcome of the ranking rewards for the past (up to two) months, the various event banners, etc.
 * Not all the hierarchy will be accessible to the player as some of them may also be hosted on a particular server (see former bullet point) or that it does not really belong as a game asset but rather as some asset to display the information on Kancolle from the DMM page.
 * If using game cache dumps (as listed in the next tab), the contents may not be the same as the directory structure as listed above but the principle is virtually the same.


 * -|Caveats=


 * -|Further reading=

Thanks to Qunow for informing the existance of these services as links so that if (for whatever reason) you want to avoid ripping data from KC servers directly and/or you want to have basically a layout of KC server's filesystem with the majority of the contents being made available without the need to rip those assets by hand. The Chinese has offered ways to allow you to do just that - of course at the expense of knowing some Chinese.
 * -|Alternative method=

Currently only two sources of groups are known to offering these "game cache dumps";
 * ShimakazeGo, and
 * ACGPower

Ironically, the said same two groups also offers game cache dumps of various other select few DMM games;

Baidu file hosting service is pretty unstable outside of China as it is known to forcibly disconnect users from their servers at random intervals and/or to force captcha. Thanks to Horadrimus, I personally recommend you to try a download manager called JDownloader which can potentially make life easier.

{{#tag:tabber|Notice= Game cache dumps of other (DMM) "netgame" are also hosted on the Baidu file hosting service from both ShimakazeGo as well as ACGPro are as follows:
 * -|Misc.=
 * These links are Baidu links. You may wish to refer to the information provided under the "Alternative method" tab for some tips/info.
 * I will do my best to update these as I go along but the same disclaimer applies,
 * I will not be responsible for your loss of any sort when obtaining these game cache data.
 * I will also not be updating these links as frequently due to the way the uploaders imposed certain filenaming structures that is beyond my knowledge as well as the depth.

{{!}}-{{!}}Other game assets/cache links= {{#tag:tabber|From ShimakazeGo= {{{!}} style="width:100%;" class="toptextcells" {{!}} style="float:left; text-align:left; valign:top;" {{!}} Last snapshot (included in here for personal reasons only): ShimakazeGo_1.6.3.1.7z ShimakazeGo_ShiroproRe_cache_20160720.7z ShimakazeGo_1.6.3.0.7z ShimakazeGo_1.6.2.9.7z ShimakazeGo_1.6.2.9.7z Shadowverse ShimakazeGo_1.6.2.8.7z Shadowverse ShimakazeGo_KanColle_cache_CacheDifference_Last20160715.7z ShimakazeGo_KanColle_cache_20160715.7z ShimakazeGo_1.6.2.6.7z ShimakazeGo_1.6.2.6.7z KanColle_Hack_Live2D_ALL_2016-07-12.7z KanColle_Hack_Live2D_UPDATE_2016-07-12_金刚级改二_Верный.7z KanColle_Hack_Live2D_UPDATE_2016-07-11_金刚级改二_Верный.7z ShimakazeGo_1.6.2.5.7z Shadowverse ShimakazeGo_KamiHime_cache_20160702.7z.002 ShimakazeGo_KamiHime_cache_20160702.7z.001 ShimakazeGo_FlowerKnightGirls_cache_20160701.7z.002 ShimakazeGo_FlowerKnightGirls_cache_20160701.7z.001 {{!}} style="background:#bce2e8; valign:top; width:40%;" {{!}} {{{!}} style= "float:right;" class="mw-collapsible" {{!}} Table of Contents {{!}}- {{!}} style="text-align:left" {{!}} Touken Ranbu {{!}}- {{!}} style="text-align:left" {{!}} Touken Ranbu Android App {{!}}- {{!}} style="text-align:left" {{!}} Kamihime Project {{!}}- {{!}} style="text-align:left" {{!}} Shiro Pro:RE {{!}}- {{!}} style="text-align:left" {{!}} Kanpani Girls {{!}}- {{!}} style="text-align:left" {{!}} Flower Knight Girls {{!}}- {{!}} style="text-align:left" {{!}} Shinken!! {{!}}- {{!}} style="text-align:left" {{!}} To-Love Ru {{!}}- {{!}} style="text-align:left" {{!}} Ultra Adventures Go Go Osawari Island {{!}}- {{!}} style="text-align:left" {{!}} Monster Hunter Mezeporta Reclamation {{!}}- {{!}} style="text-align:left" {{!}} Shiro Project {{!}}- {{!}} style="text-align:left" {{!}} Criminal Girls {{!}}- {{!}} style="text-align:left" {{!}} Gran Blue Fantasy {{!}}- {{!}} style="text-align:left" {{!}} Sengoku Providence {{!}}} {{!}}}

Touken Ranbu
back to top

Touken Ranbu Android App
back to top

Kamihime Project
back to top

Shiro Pro:RE
back to top

Kanpani Girls
back to top

Flower Knight Girls
back to top

Shinken!!
back to top

To Love-Ru
back to top

Ultra Adventures Go Go Osawari Island
back to top

Monster Hunter Mezeporta Reclamation
back to top

Shiro Project
back to top

Criminal Girls
back to top

Gran Blue Fantasy
back to top

Sengoku Providence
back to top


 * -|From ACGPro=

Ultra Adventures Go Go Osawari Island
back to top

Kanpani Girls
back to top

Kanpani Girls Android App
back to top

Monster Hunter Mezeporta Reclamation
back to top

Over Legend Endless Tower
back to top

Touken Ranbu
back to top

Touken Ranbu Android App
back to top

Flower Knight Girls
back to top

To Love-Ru
back to top

Aigis
back to top

Gineita
back to top

X-Overd
back to top

Sengoku BASARA X
back to top

Shiro Project
back to top

Kamihime Project
back to top

OZ Chrono Chronicle
back to top

Shinken!!
back to top }} }}