Wind Waker Hacking Thread

[xdaniel]

Sage of Mirrors: I really need to get back to Wind Viewer one of these days. There's already some changes not in the released build (somewhat better TP rendering support, IIRC), and I do have to add support for at least some of those chunks, yeah...

 

Lord Ned: Good job with the model, also nice to see someone else interested in WW hacking Also, don't mind about having decompiled WV, I wanted to release the source anyway, I just didn't get to it yet.

[Lord Ned]

I began work on deciphering some of the Unknown sections from the .dzb format. I took my test file into photoshop to map out what sections the data looks like/corresponds to.

 

I'm currently looking into Unknown2, or the two brown stripes. Each brown stripe is 19bytes long, and has an near-exact repeated pattern. At this point, I had no idea the composition of the brown stripe, other than it was mostly filled with FF, which seems pretty invalid as far as using it for numbers. I opened up a second collision format, and found the same repeating pattern in the other file, but the pattern was slightly different:

 

test_0.dzb

01 01 FF FF 00 00 FF FF FF FF FF FF FF FF FF FF FF FF FF FF
01 01 FF FF 00 [b]01[/b] FF FF FF FF FF FF FF FF FF FF FF FF FF FF

test_1.dzb (A more complicated map)

01 00 FF FF 01 01 00 02 00 03 FF FF FF FF FF FF FF FF FF FF
01 [b]01[/b] 00 00 00 00 FF FF FF FF FF FF FF FF FF FF FF FF FF FF
01 01 00 00 00 01 FF FF FF FF FF FF FF FF FF FF FF FF FF FF

Of notable difference is the first map uses

"xx xx FF FF xx xx" at the start, but the second map uses

"xx xx FF FF xx xx xx xx", and

"xx xx xx xx xx xx xx".

 

I'm curious as to what datatype is stored here, and why it defaults to FF if it's not in use. My guess is it would be a short (U16) due to the consistent pairs of two bytes.

If anyone has any thoughts, let me know. I'm just figuring this stuff out as I go

 

What's also curious is the Unknown1 (short green strip) is only 4 bytes long which doesn't give you a lot to work with, unless it's an extra data field for another member (Eg; has the same number of entries as another unknown)

[Sage of Mirrors]

Lord Ned, when you edited K_test02, did you have to rotate the model at all to get it to render in the right position? Or does OBJ2BDL convert the model as-is, with no rotation? I'm asking because I want to make a custom test room to test out OBJ2BDL. Also, what was the scale that you edited the model in? The dumped models from BMDView2 are huge when imported into Blender, so I want to see if I actually have to make custom maps that big.

[Secant]

It's possible one reason the map is so huge upon importing is that the model units used to export the original map are a really small unit (say, millimeters when the map is scaled for meters, as an example). If you have an option to specify units when importing it, try selecting a larger unit size, and that should render a more workable size.

[Lord Ned]

I didn't modify the scale of the map when I importex/exported from max, I took the default units. I forget the exact units but the extents of the map is a pretty even number (like -1500, 1500 exactly).

 

No significant luck on the collision format. I discovered two bits of information to add to xDan's documentation.

 

struct Face_t
{
U16 verts[3];
U16 Unknown1;
U16 Unknown2;
};

What I figured out is that Unknown1 and Unknown2 are indexes into the "Type" structure. It is how properties (like water) are assigned to each triangle. What I don't understand is why there are two indexes. Both appear ot be indexes into the Type structure, and I would guess there are a minimum of two type structures. However, it gets more confusing: A Face_t never has the same index for Unknown1 and Unknown2, which makes me think that these Type structures need to go in pairs. However, the K_Test2 level has 5 embedded types, which means either they're not in pairs or someone reuses something.

 

I'll give the reason for why I think both Uknown1 and Unknown2 are indexes into the Type section after I talk about my research into the Type section.

 

struct Type_t
{
	//Offset from start of file. This contains Romanji names for objects, things like
	//"floor" "wall" "water" "surfacePoly" etc. My guess is these were the original
	//names in Nintendo's editor and can be reused to re-create them in the OBJ.
	U32		nameOffset;

	//This value is 1,1,1 for every Type_t on some maps, or ~94% of them on others
	//When the map does have some non-1,1,1 the values normally range between 0.5 and 2
	Vector3	Unknown1; //Unknown1 might be a scale, but doesn't make a lot of sense.

	U32		Unknown2; //Value such as 0 (common), 16384 55550 4282438625 2145253536 5102 3269
	U32		Unknown3; //65535 (most common), 1073807359

	//This value is 0,0,0 for every Type_t on some maps, or ~98% of them on others
	//When the map does have some non-0,0,0 the values range between -5000 and 5000
	Vector3 Unknown4;

	U32		Unknown5; //Can be something small like 2-5 or 6551-6557, or 131082-131085
	U32		Unknown6; //Either 429467295 458751 524287 589853 655359 983039
	U32		Unknown7; //Values like: 65535 0 655361 1048578 46661950 etc.

	//Possibly flags field? Water = 256, 'Default' is 0
	U32		surfaceType;
};

There are a lot of unknowns in this field, but I have some reasonable defaults for some of them. Unknown1 is 1,1,1 almost all of the time, Unknown2 is 0 most often, Unknown 3 is 65535 a lot of the time and Unknown4 is 0,0,0 most of the time.

 

However, Unknown5-7 have values that jump all over the place, there's a fairly set number of values it can be but there's no common one it's set to. (Is this indicative of another flags field? I'm not sure). They might define if it's grass or rock which the game could use to pick particle effects.

 

What I do know is that the last U32, now named surfaceType is a flags field for the surface material. If it's set to "0", then it's solid, default. If it's set to 256 (1 << 8) then the surface is water. I imagine there's other bitshifts for Ice, lava, etc. I haven't checked yet, but only the triangles that are water point to Type_t structures that have the last value set to 256.

 

Going back to the Face_t's Unknown1/2, a typical triangle might have the values: "{1, 2}", "{3, 4}", etc. However it can also be something like "{2, 5}" as is the case of my water triangles. What I did notice is that on triangles that are water, both Type values that the Unknown1/2 reference both have surfaceType set to 256.

 

Other then that, I haven't made a lot of progress on the format. There' still three unknown sections, one which is 4 bytes per-entry, one that is 19 bytes per-entry, and one that is 16 bytes per entry. I have zero information, and zero idea as to the composition or reasoning behind values for these entries.

 

When I get the time I will attempt a crude data injection, taking an exsting maps collision file, inserting my new vertexes and face geometry (pointing to the first and second Type structure for example), and then updating the remaining offsets in a hope the game can load it.

 

Edit:

The other little bits of interesting information I figured out today (Thanks to Sage_of_Mirror's prompting), is that regardless of collision model (we found the smallest one which appears as two triangles in WindViewer), it has a minimum of:

20 Vertexes

18 Faces

2 Unknown1 sections.

2 Unknown2 sections.

1 Unknown3 section.

5 Type sections.

 

The 5 type sections might be that Nintendo's tool packs in some defaults. Inspecting the 18 faces of that model, every single triangle had Unknown1 set to 0, and Unknown2 was either 2, 3 or 4. The index "1" was not used. This just adds onto the confusing-ness of this format. :/

[Lord Ned]

2am in the morning. Sage of Mirrors has randomly found me on an IRC channel I was idling in. He begins to help research with me. Write tool for Sage of Mirrors to simplify discovering offsets. He eventually turns his detective skills to the "Unknown1_t" section of the format. We battle back and forth on it, trying to get ideas about what it is.

 

Sage of Mirrors modifies the data in a way that crashes the game. Finds an exception printed out in the log left there by TWW's devs. It reads:

"35:58:186 SrcHLEHLE_OS.cpp:52 N[OSREPORT]: 802c77cc->802c779c| Failed assertion 0 <= id && id < pm_bgd->m_ti_num in "c_bg_w.h" on line 775"

Cryptic. But with some Romanji to English, a flicker of hope appears. "pm_bgd"... nothing translatable, even removin the pm_ (pointer member). "m_ti_num". Remove m_ and _num. ti appears to read "Ground".

 

The format stores the ground triangles seperatly, the unknown data is simply indexes into the triangle array!

We've cracked what the Unknown1 section does!

 

 

I get excited and make the OBJ writer export only the ground triangles. Import it into 3ds max. Get:

 

Sit back in shock. We were so right, we had the name matched and it made sense. But it's not right.

Examin data again. Sage of Mirrors notices pattern. Indexes incriment by between 2-8 each time. Struggle. Drown in programs. Think about triangle strips. Look for pattern. Struggle some more. Attempt. Attempt again. And again.

 

(Yes those url's are in the right order)

 

Sage of Mirrors goes to bed. It's late. We're tired. Look at pattern some more.Show to other people. No one can find a real pattern. A friend says "There is a pattern. Draw based on the deltas". Debate what this means. Realize all the deltas added up equal the original number of faces. Change code. Get:

 

It's the original triangle data! Except... We already had that! So what gives?

The best I can tell is the "Unknown1" section is a series of triangle groups that split the map up into small chunks. Small chunks are fast to run physics tests again.

Unknown1 is data used for optimization. 2 unknown lumps remain, and one partial lump. Navmeshes? Partially documented lump for player collision and hit effects?

 

That's the last 6 hours in one post. I hope someone gets something out of it. Onwards, to another day of debugging!

[Conker]

You guys are doing such great work!

 

Keep it up.

[Secant]

This is the kind of innovative thinking I love to see. I'm really looking forward to seeing your work develop further on this, I'd love to be able to bring my 3D modeling skills into Wind Waker.

[Okami Amaterasu]

It's one of the test rooms in the game.

[Sage of Mirrors]

Lord Ned and I have documented Unknown3 of the .dzb files, which turned out to be surface properties.

 

They're always in this format:

 

07 FF xx FF 00 yy yy FF FF FF FF 00 00 00 00 00

Where:

 

x = Walking sound effect/Item interaction (what happens when you hit stuff with the sword, arrows, ect.)

y = Surface type (Solid ground, lava, ice, ect.)

 

Lord Ned and I believe that each face contains its component vertices and two indexes. One of these indexes seems to point to a group in Type_t (the last section of the file), while the other points to a surface property.

 

 

Now, we're stuck. The last completely unknown section, Unknown2, makes absolutely NO sense. For an example, take the first few entries from the .dzb for Minihyo, the minidungeon inside Ice Ring Isle:

 

01 00 FF FF 00 01 00 08 00 1E 00 1F 00 2D 00 36 00 3E 00 57
01 00 00 00 FF FF 00 02 FF FF 00 03 00 04 00 05 00 06 00 07
01 01 00 01 00 00 FF FF FF FF FF FF FF FF FF FF FF FF FF FF
01 01 00 01 00 01 FF FF FF FF FF FF FF FF FF FF FF FF FF FF
01 01 00 01 00 02 FF FF FF FF FF FF FF FF FF FF FF FF FF FF
01 01 00 01 00 03 FF FF FF FF FF FF FF FF FF FF FF FF FF FF
01 01 00 01 00 04 FF FF FF FF FF FF FF FF FF FF FF FF FF FF
01 01 00 01 00 05 FF FF FF FF FF FF FF FF FF FF FF FF FF FF
01 00 00 00 00 09 00 10 00 11 00 12 00 13 00 14 00 15 00 1D
01 00 00 08 FF FF 00 0A 00 0B 00 0C 00 0D 00 0E FF FF 00 0F
01 01 00 09 00 06 FF FF FF FF FF FF FF FF FF FF FF FF FF FF
01 01 00 09 00 07 FF FF FF FF FF FF FF FF FF FF FF FF FF FF
01 01 00 09 00 08 FF FF FF FF FF FF FF FF FF FF FF FF FF FF
01 01 00 09 00 09 FF FF FF FF FF FF FF FF FF FF FF FF FF FF
01 01 00 09 00 0A FF FF FF FF FF FF FF FF FF FF FF FF FF FF

Here's what we do know:

• Entries that begin with 01 00 have valid data from byte 5 to the end, while those that start with 01 01 only have valid data in the 5th and 6th bytes
• Changing the data causes the game to behave oddly, such as triggering autojump at the top of a very shallow slope and causing water to force the room to reload as if Link fell into a bottomless pit
• Forcing the engine to read 1/2 the number of actual entries (by changing the count in the header) causes various things, such as the water, some walls and the dragon statue in Minihyo to become nonsolid
• The game refuses to load the room if the entry count drops below a certain point

It's a mystery as to what this does right now. Based on the findings of our poking around, the section is important to collision. Perhaps it defines groups within the Type_t groups? As I understand it, .bmd and .bdl files have something similar - packets of primitives organized into batches.

 

Type_t is also a mystery. It's very sensitive - most of the time, changing the values for a section either breaks that group's collision or causes the room to hang. In one case, giving one group, "Zou," (which appears to be the dragon statue in Minihyo) the values for "Water_00" caused the water collision to become solid. What caused this is unknown.

 

Right now we're stumped. Can any of you guys help us figure either of these out?

[Lord Ned]

Also to add is that those two are the last sections before we can create our own custom collisions from scratch, theoretically with 100% support. It's composed of 5 sections:

Header

Vertexes

Faces

SpatialQueryOptimization (see post #281. Our own name for a lack of anything better.

Unknown2 (see post #286.

SurfaceProperty (see #286 again)

Type (see #286).

 

Both Unknown2 and Type are incomplete, and until we figure out their exact purpose we'll be unable to create custom collision files, because modifying either of these sections makes the game quite unstable.

[Sage of Mirrors]

I think I may be onto something with Unknown2!

 

Look at this entry, from Minihyo:

 

01 00 FF FF 00 01 00 08 00 1E 00 1F 00 2D 00 36 00 3E 00 57

I wanted to see what would happen if I changed 01 00 to 01 01, and I did. It didn't make the room hang - it only made the walls (not the tops) of the ice slides nonsolid. This was interesting me, so I poked around a bit in the geometry sections of the file. Faces didn't seem to do anything, so I set my sights on vertices.

 

As it turns out, vertices 01 and 08 are parts of the slide walls! This might mean that Unknown2 defines wall surfaces, and/or climbable ledges by vertex. I still need some validation, but I can't really get that until Lord Ned can log back on. We might have finally figured out what Unknown2 does.

[xdaniel]

Latest Wind Viewer + source code: http://magicstone.de/dzd/random/WWActorEdit_full.rar

 

Maybe not all that different from the last release actually, but it did have some changes - slightly improved TP support is one thing I remember. Also honestly, I don't know when (if?) I'll get back to this, so that's another reason for me to release the source now, hoping that someone else can improve upon it and such.

[haddockd]

xDan,

 

I dub the "The Beginner". We will need to name "A Closer" at some point so we can finish up all your awesome projects

[Sage of Mirrors]

Got some info on some of the chunks in cutscene files.

 

JSND controls sound effects and music. This is the header:

 

xx xx xx xx 4A 53 4E 44 00 00 00 zz yy yy yy yy yy yy yy yy

Where:

 

X = Size of JSND chunk

Z = Unknown

Y = ASCII ID of chunk ("SE" and "sound" are common names here. I don't think it matters.)

 

The actual entries are surprisingly regular, and are 0x18 bytes long.

 

02 00 00 xx 80 00 00 10 00 04 07 99 yy 00 zz zz 00 04 05 C2 00 00 00 00

Where:

 

X = Delay before sound is played

Y = Type of sound: 00 for a sound effect, 80 for a sequence file, and C0 for a stream

Z = Index of desired sound effect/sequence file/stream

 

 

JMSG is similar, and deals with text.

 

02 00 00 xx 80 00 00 08 00 04 08 59 yy yy yy yy

Where:

 

X = Delay before displaying string

Y = Entry number of desired string

 

There's usually a second portion to JMSG, called "control." This section deals with what the cutscene should do when text is displayed - namely, if it should wait until the text box is closed to continue the scene. Unfortunately, it's not that simple, and the only thing I've really got is the entry length and the variable:

 

02 00 00 xx 04 00 00 01

Where:

 

X = The variable. What this actually does isn't quite clear. Changing it to a high value causes some of the strings to fill one text box and overflow it while the cutscene plays without stopping, while a lower value tends to stop the text box from displaying at all. My theory at the moment is that this value is related to something in the Jstudio chunk, which I haven't gotten around to digging into yet.

 

 

JPTC deals with particle effects. It's actually pretty simple. Here's a sample entry:

 

00 00 00 74 4A 50 54 43 00 00 00 xx xx xx xx xx xx xx xx xx xx 00 00 00 02 00 bb bb 80 00 00 34 00 07 06 18 yy yy yy yy yy yy 00 00 00 0B 06 38 zz zz zz zz zz zz zz zz zz zz 00 00 00 04 06 42 00 00 00 01 00 04 08 99 aa aa aa aa 00 04 05 C2 00 00 00 00 02 00 00 5E 80 00 00 10 00 04 04 02 00 00 00 00 00 04 05 E2 00 00 00 00 02 00 00 08 00 00 00 00

Where:

 

X = Unique ASCII ID of the entry

Y = Actor within STB file associated with particle effect

Z = Bone in actor to attach the effect to

A = Particle effect ID

B = Delay before effect is rendered

 

I was able to attach a particle effect to Grandma by simply changing Y to "Ba1" and Z to "neck:"

 

 

There are probably more variables in there, such as what particle bank to call from.

[kyrin64]

this i getting really good can't wait to see what other hacking things you discover about

this game

[Sage of Mirrors]

Nothing new to report in the past few months. I've still been documenting the STB files, but I'm having trouble with them. I can't figure out how the engine assigns JACT chunks to the models in the archive. I originally thought that it had to do with the order both the models and the JACT chunks are in the RARC/STB file (As in the first model gets the first JACT, the second model the second JACT, ect.), but after several attempts to put the King of Hyrule into the title screen's STB file, I'm starting to think that that's not the case. Could anybody lend me a hand in looking at these files?

 

Also, another thing I've been doing is trying to trace where the Wind Waker direction/notes are stored. However, I haven't had much luck yet. I've looked at the code in the .dol, but since I can't read PPC that well, I couldn't find anything. Having custom Wind Waker songs is a must. I'd appreciate some help in this area, too.

 

 

Ha! I did it!

 

 

What this is is the King of Hyrule's model and a wait animation inserted into the archive of the title cutscene, Demo51. I found out that model assignment is actually a command in the JACT section, 04 07 39. While I was able to get this working, I'm still not entirely sure how it works... I know 04 07 39 assigns a JACT to a model, but the source of the argument for that command that decides which model to use isn't clear. What I had to do to make the King work was cycle through values starting at 00 to find the ones that the game wouldn't error on. I still have to look into it, but I believe it has to do with the files' entry in the RARC header - that is, the chunk for those files that determines their size and location in the archive. 

[Airikita]

How would I dump the files to read them? I just have the rom file..

[xdaniel]

It looks like someone knows how event_list.dat and a few other things work: http://jhwork.net/?p=741

 

Airikita: Dolphin has a filesystem viewer; right-click the game in the emulator's browser and select "Properties", then "Filesystem". From there, you can extract the files on the disc and do whatever you want with them.

[daxgr]

http://rghost.net/private/45489172/3f50b85d5a36165ca1fec48b33df718e

 

Hi, I am REALLY new to all this and just tried out Wind Viewer yesterday and noticed it doesn't save Yaz0 files, so I added it from the C++ native code of yaz0enc(credit to this dude http://www.amnoid.de/gc/).
Basically you check the Enable Recompression tooltip prior to loading yaz0 files, and then you click save changes as usual. I am willing to do grunt work for gcn game editing utilities so feel free to contact me.
Credits to xdaniel and the dude @http://www.amnoid.de/gc

[JayFromNeptune]

Hey guys! Just saying thanks for all that you are doing. It's really helping me get a feel for this.

[Antidote]

Got some info on some of the chunks in cutscene files.JSND controls sound effects and music. This is the header: 

xx xx xx xx 4A 53 4E 44 00 00 00 zz yy yy yy yy yy yy yy yy

Where:X = Size of JSND chunkZ = UnknownY = ASCII ID of chunk ("SE" and "sound" are common names here. I don't think it matters.)The actual entries are surprisingly regular, and are 0x18 bytes long.

02 00 00 xx 80 00 00 10 00 04 07 99 yy 00 zz zz 00 04 05 C2 00 00 00 00

Where:X = Delay before sound is playedY = Type of sound: 00 for a sound effect, 80 for a sequence file, and C0 for a streamZ = Index of desired sound effect/sequence file/streamJMSG is similar, and deals with text.

02 00 00 xx 80 00 00 08 00 04 08 59 yy yy yy yy

Where:X = Delay before displaying stringY = Entry number of desired stringThere's usually a second portion to JMSG, called "control." This section deals with what the cutscene should do when text is displayed - namely, if it should wait until the text box is closed to continue the scene. Unfortunately, it's not that simple, and the only thing I've really got is the entry length and the variable:

02 00 00 xx 04 00 00 01

Where:X = The variable. What this actually does isn't quite clear. Changing it to a high value causes some of the strings to fill one text box and overflow it while the cutscene plays without stopping, while a lower value tends to stop the text box from displaying at all. My theory at the moment is that this value is related to something in the Jstudio chunk, which I haven't gotten around to digging into yet.JPTC deals with particle effects. It's actually pretty simple. Here's a sample entry:

00 00 00 74 4A 50 54 43 00 00 00 xx xx xx xx xx xx xx xx xx xx 00 00 00 02 00 bb bb 80 00 00 34 00 07 06 18 yy yy yy yy yy yy 00 00 00 0B 06 38 zz zz zz zz zz zz zz zz zz zz 00 00 00 04 06 42 00 00 00 01 00 04 08 99 aa aa aa aa 00 04 05 C2 00 00 00 00 02 00 00 5E 80 00 00 10 00 04 04 02 00 00 00 00 00 04 05 E2 00 00 00 00 02 00 00 08 00 00 00 00

Where:X = Unique ASCII ID of the entryY = Actor within STB file associated with particle effectZ = Bone in actor to attach the effect toA = Particle effect IDB = Delay before effect is renderedI was able to attach a particle effect to Grandma by simply changing Y to "Ba1" and Z to "neck:"--snip--There are probably more variables in there, such as what particle bank to call from.

 

 

Thanks sage, this should help me with my own STB research.

Also if you still want to research it, please let me know and I'll work with you on it.

Also x is the size of the name, in order to read it in properly it's to the nearest 4 byte region eg: char soundName[(nameLen + 3) & ~3];.

[StarLight]

How do you change objects

[Antidote]

That largely depends on what type of objects you're wanting to modify, and what type of data.

[Abahbob]

Me and Phofe have been working on some additions to Wind Viewer, the source is available at https://github.com/pho/WindViewer. Support for TGOB (the same as ACTR), TRES, and MULT editing has been added, as well as RARC Packing. We haven't implemented adding new objects or Yaz0 compression yet, but they are planned. I've recently been working on my own little build which has the ability to move rooms (actors, model, collision, and all) so I can create a cool ROM hack, but it's horribly coded so far and I still have some things to work out.

 

In my attempts to get my stuff working, I've figured out some new things about some of the file chunks. Not sure if it's all new, but it's not on spinout182.

 

Some changes to that on spinout:

 

MULT

Format:

xxxxxxxx zzzzzzzz aaaa rr cc

Where:

x = Translation on X-axis (single-precision float)

z = Translation on Z-axis (single-precision float)

a = Rotation on X-axis

r = Room number, matches number in filenames

b = Unknown

 

TRES

Format:

xxxxxxxxxxxxxxxx FF yyyy zzzzzzzz aaaaaaaa bbbbbbbb cccc rrrr dd

Where:

x = The word 'takara' in ASCII, followed by a number that differs from entry to entry (there are other words used here; needs investigation)

F = Padding

y = Chest type (common wooden, Big Key, ect)

z = Location on X-axis

a = Location on y-axis

b = Location on z-axis

c = Unknown

r = Rotation

d = Chest contents (IE Rupees or the Hookshot)

 

Regarding .dzb files, byte 0x33 of the Type_t struct often contains the room number. When I tried replacing a used room of a dungeon with an unused room, the entire room would unload unless I changed that byte in all the structs (not sure which one was the important one, but it seemed to do the job).

 

Hopefully some of this information was new, I have pretty high hopes for Wind Waker hacking. One thing I'm interested in that I haven't seen is the locations of loading zones. Sure SCLS is an exit listing, but I haven't come across any coordinates. That would be really cool and allow for some new possibilities.