From e8bd100f599b03d30224e6da12a33274579804f4 Mon Sep 17 00:00:00 2001 From: Josh Freeman Date: Wed, 13 Dec 2017 11:22:21 +0000 Subject: [PATCH 01/12] Update sqpack_index_files.txt --- research/sqpack_index_files.txt | 6 ------ 1 file changed, 6 deletions(-) diff --git a/research/sqpack_index_files.txt b/research/sqpack_index_files.txt index 7a5ef3c5f..92144aa3c 100644 --- a/research/sqpack_index_files.txt +++ b/research/sqpack_index_files.txt @@ -1,9 +1,3 @@ -# SqPack Index Files - -Big thanks to: http://ffxivexplorer.fragmenterworks.com/research.php - ---- - #### INDEX FILE RESEARCH #### By Ioncannon Current research on SqPack Index files From cb5a5e46454c39284109c7b79a84cb9d85484e0b Mon Sep 17 00:00:00 2001 From: Josh Freeman Date: Wed, 13 Dec 2017 11:22:49 +0000 Subject: [PATCH 02/12] Update sqpack_dat_files.txt --- research/sqpack_dat_files.txt | 90 ++++++++++++++++++++++------------- 1 file changed, 58 insertions(+), 32 deletions(-) diff --git a/research/sqpack_dat_files.txt b/research/sqpack_dat_files.txt index 7a5ef3c5f..186063a37 100644 --- a/research/sqpack_dat_files.txt +++ b/research/sqpack_dat_files.txt @@ -1,49 +1,75 @@ -# SqPack Index Files +#### DAT FILE RESEARCH #### By Ioncannon -Big thanks to: http://ffxivexplorer.fragmenterworks.com/research.php +Current research on SqPack Dat files ---- - -#### INDEX FILE RESEARCH #### By Ioncannon - -Current research on SqPack Index files +Last Updated: 12/14/2014 ===SQPACK HEADER=== (0x400 in size) 0x000: Signature Int32; "SqPack", followed by 0's (12 bytes) 0x00c: Header Length Int32; -0x010: ~~Unknown~~ Int32; Unknown but repeated in other header +0x010: ~~Unknown~~ Int32; Unknown but repeated in other header 0x014: SqPack Type Int32; Type 0: SQDB, Type 1: Data, Type 2: Index 0x018: ~~Unknown~~ A lot of 0s, but 0xFFFF at 0x20 -0x3c0: SHA-1 of Header 20B; SHA-1 of bytes 0x000-0x3BF, starts 64bytes before end of header +0x3c0: SHA-1 of Header 20B; SHA-1 of bytes 0x000-0x3BF, starts 64bytes before end of header ~~~~~~~~Padding~~~~~~~~~ Padding of 0's till [Header Length] -===SEGMENT HEADER=== (starts after SQPACK HEADER) +===DATA HEADER=== (0x400 in size) 0x000: Header Length Int32; +0x004: NULL +0x008: ~~Unknown~~ Int32; Static Value of 0x10 +0x00c: Data Size Int32; From end of this header (usually 0x800) to EOF. Divided by 0x08. +0x010: Spanned DAT Int32; 0x01 = .dat0, 0x02 = .dat1 or .dat2, etc +0x014: NULL Int32; +0x018: Max File Size Int32; Always 0x77359400 or 2GB. Interestingly if a file is > 2GB, a dat1 is formed. In 13 dat, is 20MB? +0x01c: NULL Int32; +0x020: SHA1 of Data 20B; From end of this header (usually 0x800) to EOF +0x3c0: SHA1 of Header 20B; Starts 64bytes before end of header +~~~~~~~~~~Padding~~~~~~~ Padding of 0's till [Header Length] -Each Segment follows this (starting right after header length): -0x000: Unknown/Num Dats Int32; For segment 2 it's the number of dats for this archive (dat0, dat1, etc), for others unknown. -0x004: Segment Offset Int32; Offset to the segment -0x008: Segment Size Int32; How large a segment is -0x00c: SHA-1 of Segment 20B; Hash of the segment... [Segment Offset] to [Segment Offset] + [Segment Size] +-----DATA IS BELOW THIS LINE------ !!!Use Index to find entries!!! -Notes: --Segment 1 is usually files, Segment 2/3 is unknown, Segment 4 is folders. --Segments may not exist, but their position is still treated as if they did (benchmark has no segment 3 but you still need to skip it's bytes). --Each segment is followed by a padding of 0x28 0's, except the first one, which has 4 extra 0s. +~~~~~~~~~~~~~~ -===FILE SEGMENTS=== (each one is at each folder's [Files Offset], in segment 1) (Each is 16 bytes padded) -0x000: File ID1 Hash Int32; Hash to the file name -0x004: File ID2 Hash Int32; Hash to the file path -0x008: File Data Offset Int32; Multiply by 0x08, points to compressed data in .dat file -0x00b: Padding Padded to a total segment entry size of 16 bytes. +===DATA ENTRY HEADER=== (Minimum size is 0x80) +0x000: Header Length Int32; +0x004: Content Type Int32; 0x01 - Empty Placeholder, 0x02 - Binary, 0x03 - Model, 0x04 - Texture +0x008: Uncompressed Size Int32; +0x00c: ~~Unknown~~ Int32; +0x010: Block Buffer Size Int32; Buffer size need to read largest block +0x014: Num Blocks Int32; -===FOLDER SEGMENTS=== (seen in segment 3, points to files in segment 1) (Each is 16 bytes padded) -0x000: FOLDER ID Hash Int32; Hash to the folder name -0x004: Files Offset Int32; Offset to file list in segment 1. -0x008: Total Files Size Int32; Total size of all file segments for this folder. To find # files, divide by 0x10 (16). -0x00b: Padding Padded to a total segment entry size of 16 bytes. +===TYPE-2 BLOCK TABLE=== (Size of Num blocks) +0x000: Offset Int32; From end of this header (Check [File Entry Offset] + [Header Length]) +0x004: Block Size Short; Total Block Size, from beginning of header to end of padding. +0x006: Decompressed Data Size Short; -NOTES: +===TYPE-3 BLOCK TABLE=== (Size of Num blocks) --Check the final byte of the offset. If Offset & 0x000F == 2, then the offset is pointing to dat1 rather dat0. Also remember -to subtract the 0x2 out. Theoretically, 0x4 would mean dat2 but unknown as of now. +0x000: Unknown Int32; Always 0x05000001 +0x004: Frame Uncompressed Chunk 44 Bytes; Each Block Follows Pattern: 5 Ints, 12b of NULL, 3 Ints. Size when uncompressed, but rounded +0x030: Frame Size Chunk 44 Bytes; Size of each frame (can be more than 1 data block). +0x05C: Frame Offset Chunk 44 Bytes; Offset to start of frame. +0x088: Block Size Indexes Short; Indexes into the next table below??? +0x0D0: Block Size Table Num Blocks*Short; Size of each data block below + +===TYPE-4 BLOCK TABLE=== (Size of Num blocks) +0x000: Frame Offset Int32; Starting block for this frame. +0x004: Frame Size Int32; This is the total size of the whole frame. +0x008: Unknown Int32; +0x00C: Frame Blocksize Offset Int32; Offset starting from the end of the block table, to the first size. +0x010: Frame Blocksize Count Int32; How many blocks this frame contains. +0x000: Frame Block Size Short; The size of the block in this frame. + +~~~~~~~~~~~~~~ + +~~~EXTRACTED FILE HEADER MAY BE HERE, PREPEND TO EXTRACT FILE~~~ + +-----Zlib compressed data starts here------- !!!Used Block table to find each block!!! + +===BLOCK HEADER=== +0x000: Header Size Int32; Seems to be always 0x10bytes +0x004: NULL Int32; +0x008: Compressed Length Int32; If this is 32000, IT'S NOT COMPRESSED. Use decompressed length to read the data in and just append +0x00c: Decompressed Length Int32; Will be max 16kb. + +0x010: Compressed Data ; Size will be [Compressed Length] From 430bff2e337de29f33acd79c98d08c47fa331fdf Mon Sep 17 00:00:00 2001 From: Josh Freeman Date: Wed, 13 Dec 2017 11:24:02 +0000 Subject: [PATCH 03/12] Rename sqpack_dat_files.txt to explorer_sqpack_dat_files.txt --- research/{sqpack_dat_files.txt => explorer_sqpack_dat_files.txt} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename research/{sqpack_dat_files.txt => explorer_sqpack_dat_files.txt} (100%) diff --git a/research/sqpack_dat_files.txt b/research/explorer_sqpack_dat_files.txt similarity index 100% rename from research/sqpack_dat_files.txt rename to research/explorer_sqpack_dat_files.txt From c7ce9a2a2c8e8464afb5139154b4d4999e45bf7f Mon Sep 17 00:00:00 2001 From: Josh Freeman Date: Wed, 13 Dec 2017 11:24:10 +0000 Subject: [PATCH 04/12] Rename sqpack_index_files.txt to explorer_sqpack_index_files.txt --- .../{sqpack_index_files.txt => explorer_sqpack_index_files.txt} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename research/{sqpack_index_files.txt => explorer_sqpack_index_files.txt} (100%) diff --git a/research/sqpack_index_files.txt b/research/explorer_sqpack_index_files.txt similarity index 100% rename from research/sqpack_index_files.txt rename to research/explorer_sqpack_index_files.txt From f89338d461e2d44ad75b0f48ffa25a35b21ecb06 Mon Sep 17 00:00:00 2001 From: Josh Freeman Date: Wed, 13 Dec 2017 11:24:45 +0000 Subject: [PATCH 05/12] Create explorer_exhf_files --- research/explorer_exhf_files | 46 ++++++++++++++++++++++++++++++++++++ 1 file changed, 46 insertions(+) create mode 100644 research/explorer_exhf_files diff --git a/research/explorer_exhf_files b/research/explorer_exhf_files new file mode 100644 index 000000000..eb9a63486 --- /dev/null +++ b/research/explorer_exhf_files @@ -0,0 +1,46 @@ +#### EXHF RESEARCH #### By Ioncannon + +Updated: 7/14/2015 + +Current research on EXHF files + +!!!NOTE: FILE IS IN BIG ENDIAN FORM!!! + +===EXHF Header=== +0x000 Signature Int32; EXHF +0x004 Version Short; Always 0x03 +0x006 Size of Dataset Chunk Short; How large the data set chunk will be. +0x008 Number of Datasets Short; Size of Dataset Definition table; +0x00A Number of Pages Short; The size of the Page table below. +0x00C Number of Lang Codes Short; Size of the language table at the end of the file. +0x00E Unknown Short; +0x010 Unknown Int32; Always 0x010000. +0x014 Number of entries Int32; How many total entries are in this file (spanning multiple EXDFs) +0x018 Padding ; Padded to 0x1F + +===Dataset Definition Table=== (Size of Number of Datasets) +0x000 Data Type Short; Most likely the data type. 2 - Byte, 3 - Byte, 4 - Short, 19 - String +0x002 Data Offset Short; Where in the data chunk this byte is (offset is after 01 and the chunk size variables). + +===Page Table=== (Size of Number of Pages) +0x000 Page Entry Int32; Page Entry Name. See Items Page entry names. +0x004 Page Entry Size Int32; Number of entries in a page. For example Item_0.exd has 500 entries... 0x1F4, with item_500.exd being the next file. + +===Languge Table==== (Size of Number of Lang Codes) + +0x000 Language Code Short; + +The language codes are: + +0x0 - n/a (No language) +0x1 - ja (Japanese) +0x2 - en (English) +0x3 - de (German) +0x4 - fr (French) +0x5 - chs (Chinese - Singapore) +0x6 - cht (Chinese - Traditional) +0x7 - ko (Korean) + +-----------------------NOTES--------------------- + +-The items.exd files span multiple pages. Start at items_0, then items__500, then items__1000, etc. From 1aa1ca204f0019fa79929fe03ac6a0cfea6f112f Mon Sep 17 00:00:00 2001 From: Josh Freeman Date: Wed, 13 Dec 2017 11:25:14 +0000 Subject: [PATCH 06/12] Create explorer_exdf_files --- research/explorer_exdf_files | 70 ++++++++++++++++++++++++++++++++++++ 1 file changed, 70 insertions(+) create mode 100644 research/explorer_exdf_files diff --git a/research/explorer_exdf_files b/research/explorer_exdf_files new file mode 100644 index 000000000..a7e660897 --- /dev/null +++ b/research/explorer_exdf_files @@ -0,0 +1,70 @@ +#### EXDF RESEARCH #### By Ioncannon + +Current research on EXDF files + +!!!NOTE: FILE IS IN BIG ENDIAN FORM!!! + +===HEADER=== (Header is 32 bytes, padded with 0s) +0x000: Signature Int32; "EXDF" +0x004: Version Short; Seems to be always 0x0002 +0x006: Unknown Short; +0x008: Offset Table Size Int32; Size of the offset table (below) +0x00C: Data Section Size Int32; Size of the string data section +0x00F: Padding ;Up to 0x1F + +===OFFSET TABLE=== (starts 0x1F(32) bytes from 0x00) +0x000: Index Int32; These continue on from page to page, not starting at 0 in every file. +0x004: Offset Int32; Points to a string segment in the data section + +===DATA SECTION=== (starts after [Offset Table] or [Offset Table Size] * 0x08 + 0x1F) THIS IS MOST LIKELY DEFINED IN EXH file. +0x000: Size of chunk Int32; Starting after this header +0x004: Unknown Short; +0x006: Data Chunk + String Chunk ;After Data Chunk, look in EXH for size of data chunk to skip + +~~~~~DEFINED IN EXH FILE~~~~~~~~~~~~ + +------Data Types----- + +>=0x19: bitflags * + +0xb: 4 ints packed in 64 bits +0x9: float +0x7: uint +0x6: int +0x5: ushort +0x4: short +0x3: ubyte +0x2: byte +0x1: bool +0x0: string + +*If greater or equal to 0x19, subtract datatype by 0x19. This value is the position of the bit in the data +at the given offset. If bit == 1, true else false. + +------Variables------ + +Special variables can be within strings. The follow the format: + +0x00: Start Marker, always 0x02 +0x01: Type +0x02: Size of payload including End Marker +0x03: Payload +0x--: End marker, always 0x03 + +Known Types: + +0x13: Color Change +0x20: Value Amount? +0x27: Name Start, (0x01 0x01 0x01 0x01) +0x27: Name End, (0xCF 0x01 0x01 0x01) +0x28: Icon? +0x2E: Autotranslate + + +Examples: + +: 02 2C 0D FF 07 02 29 03 EB 02 03 FF 02 20 02 03 (File 0x67A9C0A, 0x1A79) +: 02 2C 0D FF 07 02 29 03 EB 02 03 FF 02 20 03 03 + 02 28 0A FF 06 41 64 64 6F 6E 0A 01 03 + 02 20 03 E8 03 03 From 4ad292c63ea74c1cb9a41a92c1146ce70f823507 Mon Sep 17 00:00:00 2001 From: Josh Freeman Date: Wed, 13 Dec 2017 11:26:05 +0000 Subject: [PATCH 07/12] Create explorer_scd_files --- research/explorer_scd_files | 157 ++++++++++++++++++++++++++++++++++++ 1 file changed, 157 insertions(+) create mode 100644 research/explorer_scd_files diff --git a/research/explorer_scd_files b/research/explorer_scd_files new file mode 100644 index 000000000..1f838640f --- /dev/null +++ b/research/explorer_scd_files @@ -0,0 +1,157 @@ +#### SCD FILE RESEARCH #### By Ioncannon + +!Some information was based on VGMStream! + +Current research on SCD music/sound files. Sorry it's a mess. + +Structure (based on 0c0000.win32.index): + +===SEDB SSCF HEADER=== +0x00 Signature Int64; SEDBSSCF +0x08 Version Int32; For FFXIV should be 3; +0x0C ~~Unknown~~ Short; Seems to be always 0x0400 +0x0E Header Size~~ Short; Points to the address where the next header is (also size of this header). Seems to be always 0x30. +0x10 File Size Int32; Total File size +~~~~~~~~Padding~~~~~~~~ Padded to fill 48 bytes (0x2F + +===OFFSETS HEADER=== +0x00 Size of Offset Table 0 Short; +0x02 Size of Sound Entry Offset Table Short; This is also the number of sounds in this file. +0x04 Size of Offset Table 2 Short; +0x06 ~~Unknown~~ Short; In one file: 0x0001, in another 0x270F +0x08 Offset Table 0 Offset Int32; +0x0C Sound Entry Offset Table Offset Int32; +0x10 Offset to Offset Table 2 Int32; +0x14 ~~Unknown/Null~~~ Int32; +0x18 Offset To ??? Int32; + +--Table 0: 4 * [Size of Table 1] +--Table 1: 4 * [Size of Table 2] +--Table 2: 4 * [Size of Table 3] ;Offsets to Sound Entries +--Table 3: 4 * [Size of Table 1] + +--Data 1 [Pointed from Table 3] +--Data 2 [Pointed from Table 0] +--Data 3 [Pointed from Table 1] + 2027296 +---------------MUSIC IS HERE---------------------------- + +===SOUND ENTRY HEADER=== (32 byte header) +0x00 Music File Length Int32; +0x04 Num Channels Int32; 0x01: Mono, 0x02: Stereo, FFXIV is usually 0x02 channels +0x08 Frequency Int32; FFXIV is usually 44100HZ (AC440000) +0x0C Data Type Int32; 0x0C: MS-ADPCM, 0x06: OGG. FFXIV seems to use OGG for music, MS-ADPCM for sound. +0x10 Loop Start Int32; In Bytes. Calculation: (filesize/amount of samples)*sample +0x14 Loop End Int32; Ditto, if you wanted to loop a whole song, set this to the file's size. +0x18 First Frame Pos Int32; First OggS after (possibly, not always) encrypted header. +0x1C Aux Chunk Count Short; Number of Aux Chunks + +===AUX CHUNKS=== +-----MARK---- +0x00 ID Int32; ASCII MARK Id +0x04 Chunk Size Int32; Size of this chunk, from start to end. +0x08 Mark Table ;Chunk Size / 4 entries. + +===IF MS-ADPCM=== +WAVEFORMATEX See: http://msdn.microsoft.com/en-us/library/windows/desktop/dd390970%28v=vs.85%29.aspx +ADPCMCOEFSET Int32; * 7 + +===IF OGG SEEK TABLE HEADER=== (32 bytes) +0x00 Encode Type Short; Seems to tell what encoding is being used. 02,20 is XOR. 03,20 is Heavensward one. +0x02 Encode Byte Byte; If encoding is 02,20, XOR the header with this byte +0x04 +0x08 +0x0c +0x10 Size of Seek Table Int32; How many seek table entries there are. +0x14 Vorbis Header Size Int32; Size of the Ogg Vorbis header (for decoding purpose). +0x18 +~~~~~~~~~Padding~~~~~~~~ Padded to fill 32bytes + +===SEEK TABLE=== (Size is 4 * [Num Seek Table]) +0x00 Seek Entry Int32; An offset from the Vorbis header to some Ogg page. + +===VORBIS HEADER== (Size is [Vorbis Header Size]) +-A standard Ogg Vorbis header. Check the encode byte in the seek table header to see if this header is encoded. If it is, +XOR all header bytes with the encode byte. +-Many songs have a LOOPSTART and LOOPEND comment tag defining where music should begin and end after the first loop. + +===DATA=== +Ogg Vorbis data is here + +--------------Duplicate Header----------------------------- +It seems that the first header is 3 chunks in size, each 0x120 bytes long. It's the Vorbis +Indentification, Comment, and Setup headers. This is repeated after. + +-------------Heavensward Encoding-------------------------- +Square Enix had encrypted the Heavensward music using a strange algorithm. Before decoding, you need the XOR table which can be found in the .exe if you search for: + +3A 32 32 32 03 7E 12 F7 B2 E2 A2 67 32 32 22 32 + +The XOR table is 0xFF byte long. + +Example code to decode (note dataLength is [Music File Length] in the entry header): + +private void xorDecodeFromTable(byte[] dataFile, int dataLength) { + int byte1 = dataLength & 0xFF & 0x7F; + int byte2 = byte1 & 0x3F; + for (int i = 0; i < dataFile.length; i++) + { + int xorByte = XORTABLE[(byte2 + i) & 0xFF]; + xorByte &= 0xFF; + xorByte ^= (dataFile[i]&0xFF); + xorByte ^= byte1; + dataFile[i] = (byte) xorByte; + } +} + +--------------To Inject OGG------------------------------- + +-Change total file size +-Change single file size +-Change First Frame Position value +-Change Header length value +-Add data to end + +--------------Putting MS-ADPCM into WAV-------------------- +-Write "RIFF" +-Write 36 + [Size of first frame to end of file] +-Write "WAVE" +-Write "fmt " +-Write 0x10 +-Write wave header +-Write "data" +-Write [Size of first frame to end of file] +-Write data from [Size of first frame to end of file] + +--------------Ogg Vorbis format info for understanding the encrypted header----------------- + +===OGG PAGE=== +0x00 Signature Int32; Value is OggS +0x04 Version Byte; Seems to be 0x00 for FFXIV +0x05 Type Byte; Type of header: 0x01: Continuation, 0x02: Beginning, 0x03: End +0x06 Granule Position Int64; +0x0E Serial Number Int32; Seems to be 0x00 for FFXIV +0x12 Sequence Number Int32; +0x16 Checksum Int32; +0x1E Num Page Segs Byte; Seems to be 0x01 for FFXIV +0x1F Segment table Byte; Each segment is a byte long, * 0x1E's value. However, FFXIV's music only has 1 segment it seems, thus 1 byte size. + +===VORBIS PAGE=== +0x00 Header Type Byte; Type of vorbis header: 0x01: ID Header, 0x03: Comment Header, 0x05: Setup Header. If has leading 0, audio header. +0x01 Signature 6Bytes; VORBIS + +--If ID Header-- +1 [vorbis_version] = read 32 bits as unsigned integer +2 [audio_channels] = read 8 bit integer as unsigned +3 [audio_sample_rate] = read 32 bits as unsigned integer +4 [bitrate_maximum] = read 32 bits as signed integer +5 [bitrate_nominal] = read 32 bits as signed integer +6 [bitrate_minimum] = read 32 bits as signed integer +7 [blocksize_0] = 2 exponent (read 4 bits as unsigned integer) +8 [blocksize_1] = 2 exponent (read 4 bits as unsigned integer) +9 [framing_flag] = read one bit + +--If Comment Header--- +1 [vendor length] = read an unsigned integer of 32 bits +2 [vendor string] = size of vendor length; Seems to be always �Xiph.Org libVorbis I 20040717� with a varying year. +3 [fields] = read an unsigned integer of 32 bits; FFXIV only uses the LoopStart and LoopEnd fields From cc879b6e5ed01e77e436eb87eddbef9dd58dac1f Mon Sep 17 00:00:00 2001 From: Josh Freeman Date: Wed, 13 Dec 2017 11:26:37 +0000 Subject: [PATCH 08/12] Create explorer_macro_dat_file --- research/explorer_macro_dat_file | 36 ++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+) create mode 100644 research/explorer_macro_dat_file diff --git a/research/explorer_macro_dat_file b/research/explorer_macro_dat_file new file mode 100644 index 000000000..b31855838 --- /dev/null +++ b/research/explorer_macro_dat_file @@ -0,0 +1,36 @@ +MACRO DAT FILE By Ioncannon + +0x00: Unknown +0x02: Unknown +0x04: Size of file minux 0x20? +0x08: Size of macro book from 0x10. + +After a 0x11 byte header, the macro entries begin. All macro entries +are XOR encoded with 0x73. + +Each macro entry follows this format: + +-Title +-Icon +-Key +-Line List + +Each section begins with a marker (T, I, K, L), followed by two bytes (little endian) telling the size, and then the data. +Data seems to be always null terminated. For example: + +A title: T 06 00 Greet\0 +A icon: I 08 00 001024B\0 +A key: K 04 00 005\0 +A line: L 0A 00 Well met!\0 + +Each macro entry contains 1 title, 1 icon, 1 key, and 15 lines. All three of these entries must exist even if not used, +with data size being 0x01 for Titles and Lines, containing an empty string ("\0"). + +-Max line size is 0xB5. +-Max lines is 15. +-99 macros must exist, regardless if used (values are empty as above). +-Icons are hex values to the icon table. There is a exh file called "macroicon.exh" which contains all usable icons. The key MUST equal the row number (hex) of the icon. This is how SE fixed the +issue of allowing icons outside this set to be used. + +After 99 macros, the data is padded with 0s to 0x46000. This is most likely to allow the max amount of data to be filled +for all macros, lines, and titles. From 402f7f4e58f4fa056ce678ab145215b99a41e6f7 Mon Sep 17 00:00:00 2001 From: Josh Freeman Date: Wed, 13 Dec 2017 11:31:57 +0000 Subject: [PATCH 09/12] Rename map scales.txt to map_scales.md --- research/{map scales.txt => map_scales.md} | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) rename research/{map scales.txt => map_scales.md} (79%) diff --git a/research/map scales.txt b/research/map_scales.md similarity index 79% rename from research/map scales.txt rename to research/map_scales.md index 8d3ea1fbb..0e9292e41 100644 --- a/research/map scales.txt +++ b/research/map_scales.md @@ -3,4 +3,4 @@ 58 = 300 = The Tam-Tara Deepcroft = 14.6 359 = 400 = The Navel = 8.6 733 = 800 = Mizzenmast Inn = 6.0 -2200 = 95 = Coerthas Western Highlands = 44 \ No newline at end of file +2200 = 95 = Coerthas Western Highlands = 44 From 762ce1afe20288468fba8902a652405d78f3f5e9 Mon Sep 17 00:00:00 2001 From: Josh Freeman Date: Wed, 13 Dec 2017 11:32:07 +0000 Subject: [PATCH 10/12] Update map_scales.md --- research/map_scales.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/research/map_scales.md b/research/map_scales.md index 0e9292e41..8586c0ccd 100644 --- a/research/map_scales.md +++ b/research/map_scales.md @@ -1,6 +1,11 @@ 31 = 100 = Lower La Noscea = 41.9 + 52 = 200 = New Gridania = 21.4 + 58 = 300 = The Tam-Tara Deepcroft = 14.6 + 359 = 400 = The Navel = 8.6 + 733 = 800 = Mizzenmast Inn = 6.0 + 2200 = 95 = Coerthas Western Highlands = 44 From 49905ddbfd52c526189fe70e796be476e23084f4 Mon Sep 17 00:00:00 2001 From: Josh Freeman Date: Wed, 13 Dec 2017 11:33:05 +0000 Subject: [PATCH 11/12] Create README.md --- research/README.md | 67 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) create mode 100644 research/README.md diff --git a/research/README.md b/research/README.md new file mode 100644 index 000000000..2eb45c7af --- /dev/null +++ b/research/README.md @@ -0,0 +1,67 @@ +# Research + +This folder contains various research on the FFXIV database structure, file structure and various formulas and the craziness that comes with FFXIV :) + +### Achievements + +- Requirement Types: https://github.com/viion/ffxiv-datamining/blob/master/research/achievements_requirement_type.md + +### Collectables + +- https://github.com/viion/ffxiv-datamining/blob/master/research/collectable_reward_info.txt + +### Emotes + +- https://github.com/viion/ffxiv-datamining/blob/master/research/emotes.md + +### DevNPCs + +This really needs maintaining, there is a bunch of NPC's which are "dev" specific, you do not find them in game + +- https://github.com/viion/ffxiv-datamining/blob/master/research/devnpcs.csv + + +### F.A.T.Es + +- EXP: https://github.com/viion/ffxiv-datamining/blob/master/research/fate_exp.md +- Levels: https://github.com/viion/ffxiv-datamining/blob/master/research/fate_levels.md + +### Icons + +- https://github.com/viion/ffxiv-datamining/blob/master/research/icon_paths.md + +### Items + +- Stats: https://github.com/viion/ffxiv-datamining/blob/master/research/item_stats.md + +### Leves + +- Leve to NPC: https://github.com/viion/ffxiv-datamining/blob/master/research/leve_to_npc.md + +### Maps + +- Scales: https://github.com/viion/ffxiv-datamining/blob/master/research/map_scales.md +- Coordinate conversion: https://github.com/viion/ffxiv-datamining/blob/master/research/map_coordinates.md + +### Quests + +- EXP Formula: https://github.com/viion/ffxiv-datamining/blob/master/research/quest_exp_reward.md + +### Weather + +- https://github.com/viion/ffxiv-datamining/blob/master/research/weather.md + + +--- + + +# File structure research + +**Huge thank you to FFXIV Explorer Ioncannon for providing these** + +- EXDF: https://github.com/viion/ffxiv-datamining/blob/master/research/explorer_exdf_files +- EXHF: https://github.com/viion/ffxiv-datamining/blob/master/research/explorer_exhf_files +- Macro Dat: https://github.com/viion/ffxiv-datamining/blob/master/research/explorer_macro_dat_file +- SCD: https://github.com/viion/ffxiv-datamining/blob/master/research/explorer_scd_files +- SqPack Dat: https://github.com/viion/ffxiv-datamining/blob/master/research/explorer_sqpack_dat_files.txt +- SqPack Index: https://github.com/viion/ffxiv-datamining/blob/master/research/explorer_sqpack_index_files.txt From 09d9d7722c9f44c3aba7ce464224a3e71b87d8a1 Mon Sep 17 00:00:00 2001 From: Josh Freeman Date: Wed, 13 Dec 2017 11:37:30 +0000 Subject: [PATCH 12/12] Update README.md --- README.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/README.md b/README.md index 800731886..80b82ceb8 100644 --- a/README.md +++ b/README.md @@ -8,6 +8,17 @@ Huge thanks to the community team members involved: - [Clorifex](https://www.reddit.com/user/Clorifex) from [GarlandTools](http://www.garlandtools.org/db/) - [Hezkezl](https://www.reddit.com/user/Hezkezl) for being awesome +## Updating CSV files: + +- Download and open **SaintCoinach.cmd** +- Run: `rawexd` +- Copy all the contents in the folder: `/rawexd/**` to the `/csv` folder +- Push a PR + +The idea is to maintain an easy diffing view of what has changed during a patch + +It would also be very useful to keep a history of `ex.json` files from SaintCoinach for each patch as the Korean and Chinese versions are on different patches than the Official live client. + ## Helping out