How to Insert OpCodes Manually

I’ve came across this issue when I try to create the VHD (Microsoft’s Virtual Disk) editing utility. The purpose of this application is to be able to perform same editing process, i.e. to remove or inject some files into the existing VHD file.

I require this utility to fix the broken VHD files so that it will shorten the required time to rebuild the VHD from scratch (i.e. creates empty VHD and re-installs the OS into the VHD).

At the research phase, I’ve found that the 7-zip open source utility can read this format and also the specification is available from Microsoft’s site. The 7-zip utility can be used to edit certain file format, but upon trying to remove some file inside VHD structure, it gives the error message:


So, I think it would be nice if I also have the edit functionality of the VHD file, either as a kind of plug-in for 7-zip, or just creates the stand alone utility specifically for this format.

After downloading the source code, and perform some exploration, I’ve found the source file called VhdHandler.cpp which should handle the reading and processing the file structures inside VHD file. But trying to read the logic offline without actually seeing it in action proves to be a near-impossible task.

With the usual drawback from almost every open source applications, i.e. the lack of compiled public PDB symbols, I have to perform tedious task of compiling the existing source. Just for the purpose of obtaining the debug symbols, and it is also just the preliminary steps in studying the algorithm.

I just wondering when the open source community revolutionized their ways to improve the above simple aspect. Currently the open source for me is to try getting things in the hard ways, either in the documentation and also troubleshooting/studying aspect of the application.

Because lamenting will not improve the situation, let’s see what happen when I try to compile this source:


The cause of this error happens when Microsoft Macro Assembler 8 tries to expand the macro command. Let’s what’s get expanded when it encounter the error:


Viewing the result text file we get:


The aesdec instruction is clearly exist in the intel instruction set, but somehow, Macro Assembler 8 failed to understand this. Thus, by trying to compile the source, I arrived at yet another issue, and this is a big one.

I try to compile it with higher version of MASM (i.e. for example, using Visual Studio 2008), and the problem is still there.

I try to search into the forum provided from 7-zip website, there are many compiler errors issue but after an hour of forum searching seems I don’t find any compiler error related to the “aesenc” opcode.

I guess this troubling module has some relation with encryption and decryption of archive which is actually not getting used for the study. So, It would be nice just to make it pass to compiler phase and perform some edit with the actual opcode when it is required.

The idea is to substitute the aesenc command with equivalent command with same parameter requirement and the command is recognized by the compiler. After successfull compilation, the dummy opcode is replaced by the real one by consulting the proper opcode from Intel Software Developer’s Manual.

I decided to replace aesenc opcode with movdqa and place the marker:


So, that I can later locate the substituted opcode and perform proper opcode replacement using byte level editor such as winhex.

The above statement will be compiled as:


Then it will be a routine job to replace the generated code with proper op codes and deletes the marker. The replacement process should also consider the inside of macro expansion. In the above case, the first movdqa instruction should not be replaced.


2 Responses to “How to Insert OpCodes Manually”

  1. Says:

    Your personal post, “How to Insert OpCodes Manually Welcome to the Corner of Excellence” was in fact worth writing a comment on!

    Only wished to state u did a good job. Thanks for your effort ,

  2. led Says:

    It’s genuinely very difficult in this active life to listen news on Television, therefore
    I only use the web for that purpose, and take the newest news.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: