Introduction to Hexing
Ok. For whatever reason, you've got some model that you want to use for something it wasn't meant for. Maybe you want a server side player model. Maybe you want to turn that G3 into its own SMOD custom weapon. Whatever it might be, you might think you need to go through the whole process of re-compiling a model and it's quite possible that you don't.
Things you can do with a hex editor:
1) Change what the model is called and where it is located
2) Change what textures a model looks for and where
3) Disable or alter actions/events in a model
4) Force a model to use another ragdoll without console errors
5) Make a model use alternate external animations
it must already use them and I won't demonstrate this because you'll be able to figure it out by the end.
If you need to do any of that, then you can hex the model. No need to fire up that finicky decompiler.
Required and Recommended Programs
Before we jump into this, you're going to need a hex editor and you'll probably want a better way to rename stuff than hitting F2. Manually renaming 7 files at a time gets old very quickly.
You need a hex editor. Notepad is not a hex editor. Some people use Notepad++ as a hex editor, but I don't really recommend it as I've never tried it that way and refuse to bother. For this tutorial I'll be using the freeware hex editor XVI32. You may use any remotely similar hex editor, but this is what I'll be working with.
You do not need this utility, but I highly suggest installing Oscar's File Renamer. A typical model is actually composed of 7 files and renaming them all manually is a pain in the ass. I will demonstrate and describe using this program to make your life easier in generic server-side hexing.
The Source SDK:
For the sake of making your life easier and knowing just what is what, I would recommend installing the Source SDK so that you have the model viewer available. You don't need it, but it's helpful.
I also recommend having this installed so that you can have access to all of the base models and textures that you have installed.
Chapter 1: Generic Hexing : Making a model into a server-side, non-replacement model (or vice versa)
If you want to make a model replace or not replace another, you need to start here. For making server side models, this is the first but not always final step.
The first step is to get a model that you wish to hex. For the sake of this tutorial, let's convert FxDarkloki's beta Terrorist into a server side model. Then for kicks, we'll convert it back into a normal replacement player model using the old school technique of consistency dodging.
Step 1) Prepare the model for working on
First thing's first. Download the file above. While that's downloading, prepare a new folder that you can work in. For simplicity, I'll use a directory named work that I've created on my desktop.
For the sake of this tutorial, don't install the model like you normally would. Install only the materials folder to your CSS directory. Once you do that, browse the zip file until you get to the models/player directory. Extract all of the t_phoenix files to your work directory. (Visual Example)
Step 2) Run the hex editor and prepare to hex
Leave the work folder open and run the hex editor. Drag and drop t_phoenix.mdl onto the hex editor (or open it manually if you insist). You will then see this. Make sure right now that you are in overwrite mode. Your hex editor will either say insert or overwrite, usually at the bottom. If you are in insert mode, push the insert key. Insert mode should never be used for hexing.
For this chapter, we are only interested in the defined location and name of the model. This is where, at the very top, it reads "player/t_phoenix.mdl". This means that the Source engine is reading this model as t_phoenix.mdl, it will look for all associated files with t_phoenix (remember this), and it is searching in root/models/player (where root is the game's directory). You do not define the models directory because it is implied. A model defined with only it's file name, such as alyx.mdl, will be located in root/models. This is how many HL2 NPCs, the drivable vehicles, and the weapons in HL:S are defined.
The large amount of blank space after the definition can be used freely, but you can also make a defined name and location smaller as well. I will demonstrate both.
Step 3a) Alter the model's definition (lengthen)
Highlight the first t in player/t_phoenix.mdl and simply start typing. For this tutorial, let's make this model have a new directory structure. Change t_phoenix.mdl to old/phoenix_beta.mdl. The whole definition will now read player/old/phoenix_beta.mdl. This changes the model's definition to now be the model phoenix_beta.mdl located in root/models/player/old/.
Save the model, and it is officially hexed.
See Image A, part A for visual example.
Step 3a) Alter the model's definition (shorten)
Keep the definition above, but now let's put the model is the base models directory instead.
Highlight the first p in player/old/phoenix_beta.mdl and type beta_phoenix.mdl. Now push tab to select the number side of the hex editor and type 00 until the remaining characters are overwritten in 0s. You have now hexed the model so that it is defined as beta_phoenix.mdl and it resides in the root/models directory.
Save your work, and you're ready for the next step.
See Image A, part B for visual example.
Step 4) Rename and move all associated files
You are done with the hex editor for now, so you can close it.
Assuming you have installed Oscar's File Renamer, right click the t_phoenix.mdl file in the work directory and select Renamer.
Assuming you don't have the same stupid problem I had, you should see this. If not, navigate to your work directory manually using the tree in the upper-left corner until you do.
Hit Ctrl+H to bring up replace mode and tell it to replace t_phoenix with beta_phoenix. (Visual Example) Hit replace all a couple of times, and you should eventually see this. Hit ok and close replacer. You should then be presented with this.
If you skipped using Replacer and did it manually, it should still look like that. Else you did it wrong.
Step 5) Install the model
Congratulations. You've hexed a model. Copy all of the files from your work directory to your CSS models directory. Had we kept the long version, you would to have gone to root/models/player/, created a folder named old, and pasted the files in that directory.
The model is now ready to be used as a server side player model.
Step 6) Convert that model into a replacement model
If you remember correctly, the model's definition tells Source engine where to look for a model's associated files. Using this, you can exploit the engine into allowing inconsistent models as player replacements.
To do this, simply copy your beta_phoenix.mdl file to the player directory Now, rename the file to t_phoenix.mdl. CS:S will be looking for root/models/player/t_phoenix.mdl, however you have just given it a model with a completely different definition. Because of this, Source engine looks for the files you have already generated in the model directory instead of the usual t_phoenix files.
This technique can be used to replace an entire team with one model and severely shorten load times...assuming the models all call the same textures.
Chapter 2: Altering Material Cues : Making a model use different materials
Sometimes, you want a model to use different textures than it has defined. You can use this technique to make ghetto rigged USP or M4A1 world models from the old format world models where there was no unsilenced version. In said example, you combine hexing with the VMT trick to make a model with no displayed silencer while leaving the silenced world model intact. I'm going to demonstrate another use for this technique by showing you how to make a copy of a CS:S model that uses different textures than its clone. You can use this technique in conjunction with the first to also generate a server side CS:S model.
For this tutorial, start with the previously generated model.
Step 1) Prepare the model for hexing
Open root/models/player/ if you aren't still there. Copy the t_phoenix.mdl you previously generated, paste it, and rename the copy t_guerilla.mdl. You now have a fully functional guerilla replacement. How about that?
Step 2) Understanding the model
Open the hex editor and drag and drop t_guerilla.mdl onto it. Now scroll to the very bottom until you see this.
For this tutorial, we are only interested in the two last strings. The final string of the entire .mdl file is always the directory from which the model looks for its texture. models/player/t_desert translates into root/materials/models/players/t_desert. The materials directory is implied just like the models directory was implied. All strings preceding the directory location and after the last external animation cue* refer to the VMTs that are used by the model. The VTFs are later defined in the VMTs. In our example model, only one material is cued: t_desert.vmt.
*Not all models have external animation cues (the strings ending in .mdl). In that case, the last item before the textures will be a bone. Refer to the defined material directory if you are confused.
Step 3) Actually hex the model
For the sake of simplicity, I will only demonstrate changing the VMT that is cued. There's not really a lot of skins for the Phoenix Faction player, anyways.
Highlight the t in t_desert for the VMT cue. Let's change it to something easy, like stupid. Since stupid is shorter than t_desert, you need to hit tab and fill the last two characters with 00s. See Image B for visual example.
Keep in mind, that due to the limitations of hexing, you cannot use a VMT or folder name that exceeds the length of the original. If you need to do that, you will have to recompile the model. Typically, this isn't necessary for something like this because the VMT can be modified to make up this limitation, anyways.
Save your work. Your guerrilla replacement will now look for stupid.vmt.
Step 4) Make sure the materials actually exist
Navigate to root/materials/models/player/t_desert/ and find t_desert.vmt. Since your guerrilla requires stupid.vmt and you only have t_desert.vmt, copy t_desert.vmt, paste it, and rename the copy stupid.vmt.
If you want, for the sake of education, do the same thing with t_desert.vtf. Then edit stupid.vmt in notepad so that models/player/t_desert/t_desert becomes models/player/t_desert/stupid. This step is only for educational purposes.
Had we changed the directory, we would had to have copied the new files to the new defined directory and modified the VMT to reflect a new directory. This is typically redundant, however.
And congratulations, you're done with Chapter 2. Using the two techniques demonstrated thus far, you can generate (or revert) server side player models and then some.
Chapter 3: Altering Events and Actions : Changing and disabling ejection, muzzle flash, and more
Sometimes, a model just does something absolutely stupid and you need it to stop. Depending on what it might be and how it's defined, you may be able to hex the model to fix it. Keep in mind that hexing is not always a possible solution. If you want to add an action, you will need to recompile the model.
For the first portion of this tutorial, I'll demonstrate hexing actions on HL2 models. A good model to work with is this Beretta 92FS. For whatever reason, Modderfreak has decided that the animation for firing with an empty clip is actually for the last round in the clip, thus it has ejection and muzzle flash. Unacceptable. Download the model, extract the v_pistol files from Dr. Zoidburg's animations to your work directory, and let's fix it. (Visual Example)
Step 1) Prepare the file for hexing
Drag v_pistol.mdl into your hex editor. Scroll to the very bottom and look for ACT_VM_PRIMARYFIRE. This is the first action with a muzzle flash. Now you see recoil1, recoil2, recoil3, and dryfire. This means that the fifth muzzle flash reference should be for the dryfire animation that isn't supposed to even have one.
A hint right off the bat, this model has two muzzle flashes per shot. Don't ask me why.
Ok. Go back to the top, and search for MUZZLE (yes, all caps). As I hinted, you'll two references of PISTOL MUZZLE very close. These belong to the same animation, so look for references 9 and 10. Highlight the first P in the 9th PISTOL MUZZLE reference and take a moment to look at what's going on.
Step 2) Understanding what the hell you're doing
Ok. This is a HL2 format weapon. This is not the same as DOD:S or CS:S. So let's investigate and understand what's going on. You can learn a lot from here. See image C for detailed information on what matters.
Keep in mind, there will be references to what frame these events occur, but since it's usually 0 for weapons, the bit is indistinguishable.
Step 3a) Actually hex it (remove the flash)
Now, we want to disable the muzzle flash. If we wanted to make it different, we could simply alter it, but we don't want to do that. We want to disable it. Fortunately, Source has no flash to default to in the event of an incorrect reference.
Feel free to use whatever witty words you wish to replace the two references. (Visual Example)
Step 3b) Actually hex it (remove shell)
Now, using image C as a reference, hit tab, highlight 7117, and change it to 0000. This will disable the ejection event.
Save your work.
Step 4) Install
If you wish, you may install the Beretta as normal. Then copy your new v_pistol.mdl and paste it in root/models/weapons/ so that it replaces the one from the zip file.
Congratulations. You've now edited events in a HL2 format weapon. Let's look at modifying a CS:S weapon real quick.
This Type 79 model for MP5 is really neat, but it ejects 7.62mm shells. Why is beyond me. 7.62x25mm is closer to 9mm than anything else available in CS:S. Download it, extract v_smg_mp5.mdl to your work directory just like before, and let's fix it.
Step Whatever) Hex the ejection
CS:S weapons do not use the event 7117h for ejecting brass. The use a much more obvious event which you'll see very shortly.
Open v_smg_mp5.mdl in the hex editor and search for eject. The first result will give you a string called EjectBrass_762Nato. What the following numbers do isn't so important. What is important is that the ejection event has to be followed by them and they have a space character in between.
Ok. Change all events of "EjectBrass_762Nato 2 150" to "EjectBrass_9mm 2 150" and zero out the last four characters like you have before. The spaces are actual spaces made with the space bar. They cannot be 00s or the line will be considered finished.
Save your work and install the Type 79 if you wish.
Chapter 4: Checksum Editing : Forcing models to use physmodels not compiled for them and then some
For some reason, you might want to use a collision model from another model. Maybe you have a model that doesn't have one or something. Who knows? You may require additional hex editing for props and non-ragdolls, but for two models with skeletons that are remotely similar, this will do that job.
For this tutorial, let's transplant Dr. Kleiner's ragdoll onto the Lost Coast Fisherman.
Step 1) Procure the necessary files
Using GCFScape, retrieve the follwing files:
Kleiner.mdl and Kleiner.phy from source models.gcf
Fisherman.mdl from lostcoast content.gcf
Step 2) Use the power of comparison to understand the checksum
Open Kleiner.mdl in the hex editor. The four characters that precede Kleiner.mdl are the checksum. In this case, 9420E3D7. Now open Kleiner.phy and find them there. See them? Good. Don't forget where they are. Copy Kleiner.phy, paste it, and rename the copy fisherman.phy.
Now open fisherman.mdl and look for its checksum. You'll see that it's 228896EB.
Step 3) Get it done
Open your new fisherman.phy in the hex editor and find where the checksum was. You remember, right? Change it to reflect the checksum in fisherman.mdl by selecting the number side of the program and typing in the new checksum. Now save it.
See image D for visual example.
Step 4) Install
Install fisherman.phy in it's proper location of root/models/lostcoast/fisherman/. It's fairly useless since you have to be able to kill him (and in way tht renders him an instant ragdoll, at that!), but that's the jist.