|
Reasons for FAQ
[Miz]
This FAQ is intended to give *basic* background information on the topic of executable packers and how to manually unpack and rebuild such files.
It will also serve as an introduction to the much more detailed unpacking
project which will be created shortly.
It came about after several requests for information on +Sandman's Newbie
messageboard. The topic is so broad the posts were often fragmented. As a result,
a suggestion was made to centralize many of the questions people had, and to create
a project that will offer real world examples.
Hopefully it will help to de-mystify this area of reversing, as well as stimulate
some interest in associated areas such as API hooking, decompilation, virii,
anti-anti-Sice measures, encryption, etc.
This 'open' FAQ will be updated as frequently as possible and readers are actively
encouraged to submit queries and/or contribute answers/ideas/knowledge.
|
|
How can we ask questions and/or contribute to this project?
Email: muproject@hotmail.com
|
|
How do we know when this FAQ/Project has been updated?
[Miz]
Once the full project is underway there will probably be some sort of mail-list
or dedicated messageboard. In the meantime, please check for news on +Sandman's Newbie Forum. Details will be posted there when more information is available.
|
|
I have completed 'PartX' of the mini-projects where do I post the tutorial?
[Miz]
Please do NOT publicly release your tutorials.
Please send them here (see the address above). They will be published along with
this project as it gets updated. Once they are there (obviously), send them wherever
you like.
This is to stop the mini-projects becoming a race and spoiling things for others.
Please read the 'Intro.txt' for more information.
Thanks.
|
Why would we want to unpack an executable?
[Miz]
There are several reasons, but IMO the main one is so that you remain informed about
exactly what a specific program is doing. Deliberately hiding a programs code and
hiding which DLL's and API functions it uses should set off warning lights with people
concerned with even basic privacy and security.
In a Win9x environment any program has access to anything on your machine. Anything.
I have no problem with the actual compression programs themselves, they provide a
useful service and are excellent study material in their own right. My problem is with
programs that can potentially abuse the side-effects of executable compression to
hinder a target's examination and (if necessary) reversing. 'Baloney, you just want to patch the pants off it!' - purely a side effect, honest. Anyway, if a program's author has nothing to hide then he surely would not mind us checking for ourselves ;) By unpacking an executable down to the point that we can remove all traces of the original packer, repairing all the relevant PE data, we end up being able to dead-list code, use API monitors, make patches etc.etc.
We unpack so we can see.
|
|
I've seen unpack utilities for BrandX, why can't I simply use them?
[Miz]
Obviously you can if you want(!). But what if no unpacker yet exists for BrandZ?
What do you do then? As mentioned previously, an increasing number of programs
are being packed, and newer generations are coming out all the time. Learning how
to do it manually will help you in such times.
On top of that, it is (IMO) a very interesting area for study, and touches on many
other interesting areas. I've seen some hostile discussions on the importance of
understanding/ignoring the PE format. Maybe studying such things as manual
unpacking, virri, API hooking, etc, may make people understand it's relevance more,
and give some more insight into what the system is up to in the background.
|
|
What sort of utilities are available to pack/encrypt a program?
[Miz]
Not so long ago the tools available were quite limited and often flaky.
Lately they have become much more sophisticated, becoming commercial ventures,
and as such, have been adopted by other developers.
Why is it that developers only seem to trust things they have to pay for? Strange!
The number of programs released with packed executables is on the increase, and is
likely to keep increasing.
Programs such as Petite, Neolite, Shrinker, ASPack, etc, are widely used schemes,
having been around for some time and consequently being more refined than others.
Check out those programs. Many have evaluation periods. Reading their documentation
gives a good background to the topic. Also check out the more basic ones from tools
sites. Sometimes you find gems, like ones that come with source code or a 'how it
was done' doc.
|
I'm experienced at reversing, but this area is relatively new to me.
Where's a good place to start?
[Miz]
Get hold of as much information as possible. Check the Further Reading section below.
Read, Read, then Re-read; particularly documentation like MattP's. Then, get hold of a small, simple executable - one you are familiar with. Pack it with an uncomplicated packer (suggest ASPack) and make it your aim to reverse it to being as close a copy of the original as you possibly can. Make it an ongoing project while you continue to search and read.
If you want to dive in straight away then check out Torn@do's packed crackme and
its accompanying unpacking doc (included in the 'Library/OldStuff' directory).
It skirts around the huge 'import' problem (that will be covered in much more detail
as part of the main project), but it may be of some help. Please bear in mind the
'doc' was never intended to be a tutorial as such. Much more information will be
provided here.
Alternatively, (and the way I learnt), after reading as much info as your brain can
handle, write your own PE 'modifiers'. My first attempt was a modifier that reversed
the order of all the bytes in the code section and swapped them back at runtime.
Pointless, I know ;) But it didn't involve delving in the PE format that much, and
its surprising how much such a trivial task can teach you. From there you can move to
more involved tasks like packing the code section. From there packing the import
section. etc.etc.
Sounds scary? Not with the resources available. I will keep referring to the excellent work that Stone did long ago in this area. I'm sure others have contributed equally as much, but for me Stone's work was a revelation. There's a simple pe-encryptor with source and docs on his site that can work as an great reference.
|
|
Blimey! All I want to do is patch a single byte in the uncompressed exe, do I really have to do all this?
[Miz]
If that's *all* you want to do then you're in the wrong place.
But to answer you more properly.....No.
Leave the executable packed but divert the final 'jump' (to the original entry point)
to some of your own code that can then apply the patch to the now unpacked code/data.
For you to be able to do this properly however, you will need some knowledge of
caving or section construction; you'll need to write relocatable code that
references the patch address in a OS friendly way (i.e. uses the ACTUAL base address,
not the PREFERRED base address), etc.
This project, FAQ, and the references in Further Reading will give you
the knowledge to do these things, and hopefully much more besides.
|
|
Is packing a program like zipping a proggy? [Jeff]
[Miz]
No.
In terms of compression they may use similar algorithms, but the difference is the way
in which the compression is applied.,
- An Archiver (like zip) just compresses the entire file as a binary image into a compressed binary image. It makes no decisions as to the type of file being
packed, it will treat an .exe just like a .zzz, and makes no changes to the image
other than compressing it as a big bag of bits.
- An Executable packer has to make many more decisions about what it can and can not pack, working to restrictions imposed by the OS. For it to work it will need to modify the actual format of the executable, thereby creating the myth that they are not 100% reversible.
|
|
Is packing that simple and quick? [Jeff]
[Miz]
In terms of use, yes.
In terms of programming, no.
As mentioned above, packing executables involves many more details than simply compressing a chunk of raw data.
|
|
What is the purpose behind packing up a proggy? [Jeff]
[Miz]
Allegedly, just to make it smaller - 40-60% of original size is about normal.
However, the things that get our attention are claims such as:
- "'BrandX'...adding security against reverse engineering..."
- "'BrandY'...protects code from prying eyes, disassemblers..."
- "'BrandZ'...packed files can never be unpacked, please keep a backup..."
|
|
Who would use such a packed proggy and why? [Jeff]
[Miz]
Either someone who wants smaller files, or someone who is fooled by the false
security promises.
The suspicious (I know...;)) will be more concerned with the
'why' than the 'who'.
|
|
Are the changes to the proggy always the same? [Jeff]
[Miz]
No. (If they were then it would all be rather dull - LOL). The beauty/bane of this topic is that it is constantly changing. With every generation of packers/encryptors/virri, etc, you will find something new.
Again the OS dictates some limitations, but other than that a packer/virus/encryptor
can be as creative as its creator ;) It is because of this that this FAQ and the resulting full project will try to avoid direct descriptions of particular versions of packers. Instead we hope it will provide enough background knowledge for people to be able to cope with new 'generations' of packers as they emerge.
|
|
HOW do we KNOW when we dl a proggy that it is packed? [Jeff]
[Miz]
You probably won't be able to tell just by running it.
They usually claim to run totally transparent to the user. You can check it manually
(by looking at the PE structure or examining the code) or by using some Exe checker such as GetTyp.
NOTE: Exe checkers will only know about packers they have been programmed to know about.
They normally just check for signatures (byte strings) in the same way that virus checkers do.
If you rely on an exe checker and it says a program is not packed then be sure to check it manually. It may be a latest revision of an existing packer which is not recognised yet.
|
|
What, if any, tell tale signs tell us this proggy may be packed? [Jeff]
[Miz]
Without inspecting the code itself, the biggest clues are additional sections,
particularly if the entry point goes there. Sometimes additional sections are even
named by the packer to the packers name (!) Also, taking a quick look with a hex editor around suspect areas may also uncover strings such as 'BrandX Packer v1.04 by UmBongo!', etc. Other clues are the lack of a readable import section (if that has been compressed) and subtle changes to Section Characteristics (eg '.TEXT' section becomes writable, etc).
|
|
Does something happen in SoftIce or Win32Dasm to give us a clue? [Jeff]
[Miz]
Disassemblers such as W32dasm may display garbage, nothing at all, or sometimes
even crash. This is because they rely completely on the integrity of the PE structures. Softice is the initial tool of choice. We will use it, along with tools like ProcDump, to reverse an exe to the point that we can dead-list it again with something like W32dasm.
|
|
What tools would we need in our unpacking arsenal? [Jeff]
[Miz]
Useful tools to get familiar with are:
- Softice (say no more)
- A PE viewer (like PEBrowsePro)
- ProcDump (Awesome, awesome, tool)
- A 'temp-dumper' like +Quines SoftDump is very useful for cut and pasting blocks of memory.
- An Exe checker (eg GetTyp) may help in identification, although this step is not necessary.
We will be using these tools in more detail in the main project, so if they're new to you, it may be handy to get familiar with their functions now.
Check any decent tools site and you will find dozens of utils for studying and
modifying pe-files. Take a look at as many as you can, sometimes they even include
source. Anything you can get your hands on may be of help.
Also, (it should go without saying), try writing your own. Check out Stone's site for some great starting points. The pe-encryptor he presents is a great foundation to this topic. Start with simple things and gradually work your way up.
|
Packers/encryptors/Virri, etc, seem to be mentioned together quite often.
Are they so similar?
[Miz]
The programming concepts are very similar. Like a virus an unpacker typically bolts itself on to a host executable. Hooking itself in as the first thing to run, doing 'something' (in this case unpacking the host), before allowing the host to continue.
Like a virus, unpackers often have to be resourceful in how they initialise. They may
use similar steps to find API addresses, memory etc. Unpackers often decrypt part of their unpacking code, like exe-encryptors, to make examination more difficult. They may also employ similar anti-debugging measures. Packers/Encryptors/Virri quite often have to use fully relocatable code, and as such, use many of the same tricks (like the call [NextInstruction]/pop pair used to get the current eip).
Links to some virri descriptions are provided in Further Reading. Search for some more to see the similarities. Take a look at the cabanas one, hmmm ;)
|
|
What is a PE Header? [Jeff]
[Miz]
In general its the term used to describe the all the 'system' type information used
by the OS to work out the resources, memory (size and type of), import/export
information (eg DLL's required and API calls used), entry point, stack size, etc, of
an executable.
The important thing to grasp early on is that an executable is not just a binary
image of your code/data and nothing else. There are many pieces of additional system
information attached so that the OS knows exactly what resources your executable
will need.
It may be easier to think of an executable as a directory. In that directory are some files and some sub-folders containing more files. The structure of the whole directory tree will change from executable to executable. For example if your executable doesn't have any resources then the .RSRC section will be missing. If it contains debug information then a .DBG section may exist etc.
A simple scheme may be:
(Hint: The example here is actually Notepad.exe, so you can examine it yourself...)
DOS HEADER
FILE HEADER
OPTIONAL HEADERS
---IMPORT
---RESOURCE
---BASE RELOCATIONS
---IAT
SECTION HEADERS
---.TEXT
---.DATA
---.IDATA
-------IMPORT
-------IAT
---.RSRC
-------RESOURCE
---.RELOC
-------BASE RELOCATIONS
IMPORTS
----ADVAPI32.DLL
--------RegCloseKey
--------etc.etc.
----COMDLG32.DLL
--------ChooseFontA
--------etc.etc.
----GDI32.DLL
--------AbortDoc
--------etc.etc.
----KERNEL32.DLL
--------lclose
--------etc.etc.
----SHELL32.DLL
--------DragAcceptFiles
--------etc.etc.
----USER32.DLL
--------CharNextA
--------etc.etc.
RESOURCES
----ICON
--------1
--------etc.etc.
----MENU
--------1
--------etc.etc.
----DIALOG
--------12
--------etc.etc.
----STRING
--------1
--------etc.etc.
----ACCELERATOR
--------1
--------etc.etc.
----GROUPICON
--------1
--------etc.etc.
----VERSION
--------1
--------etc.etc.
(Still with us? ;))
If you're more familiar with older COM files you'll be amazed at the amount of information that is stored. It does indeed make unpacking more involved that before, however, it also makes certain areas (like API monitoring etc) very easy. More on that later.
If this structure is pretty new to you then I recommend you get hold of a decent PE viewer and get a feel for the structure of a PE file before delving deeper. Alot of tools exist, but my own favorite is PEBrowse Professional. There's also MattP's PEDUMP util, complete with source to study, see Further Reading below.
Look again at the above layout.
- Can you see now how things like APISpy utils could work? Hmmm....
- Can you see now how Resource grabbers could work? Hmmm.....
- You can also probably see how a 'theoretical' packer could work. Aahhh....
Note: 'Sections' themselves are covered in greater detail later on in this FAQ.
[From MattP's Windows Secrets]
"Like all other Microsoft executable file formats, the PE file has a collection of fields at a known (or easy-to-find) location that define what the rest of the file looks like. The PE header contains vital pieces of information such as the location and size of the code and data areas, what operating system the file is intended to be used with, and the initial stack size."
|
|
Can u tell at a glance, without tools, that the PE Header is screwy? [Jeff]
[Miz]
Thankfully, there's very little that can be 'screwy' without causing major complications. Remember, packers/encryptors/virii etc, are limited in the amount they can mess with header structures due to OS restrictions. For them to work on all versions of OS they have to remain fairly compliant. As many virii writers found out, even structure elements that are marked as 'UNUSED' are very often used and cause problems with some OS when written to with 'ID' bytes or whatever.....
Anyways, 'screwy' PE Headers are normally a result of incorrect unpacking - something
we will hopefully be avoiding ;)
|
What do the numbers we see in PE Header mean to us?
(eg Entry point, Image Base, Virtual size, Virtual Offset, etc. etc.)
[From MattP's Windows Secrets]
Image Base
When the linker creates an executable, it assumes that the file will be memory
mapped to a specific location in memory. That address is stored in this field.
Assuming a load address allows linker optimizations to take place. If the file
really is memory mapped to that address by the loader, the code doesn't need
any patching before it can be run. I'll talk more about this in the discussion of
the base relocations. In NT 3.1 executables, the default image base was
0x10000. For DLLs, the default was 0x400000. In Windows 95, the address
0x10000 can't be used to load 32-bit EXEs because it lies within a linear
address region that's shared by all processes. Therefore, in Windows NT 3.5,
Microsoft changed the default base address for Win32 Executables to
0x400000. Older programs that were linked assuming a base address of
0x10000 will take longer to load under Windows 95 because the loader needs
to apply the base relocations.
RVA (Relative Virtual Address) AKA Relative Offset
Many fields in PE files are specified in terms of RVAs. An RVA is simply the offset
of some item, relative to where the file is memory mapped to. For example, let's say
the Windows loader mapped a PE file into memory starting at address 0x400000 in the
virtual address space. If a certain table in the image starts at address 0x401464,
the table's RVA is 0x1464:
(virtual address 0x401464) - (base address 0x400000) = RVA 0x1464
Virtual Offset AKA Virtual Address
In EXEs, this field holds the RVA for where the loader should map the section
to. To calculate the real starting address of a given section in memory,
add the base address of the image to the section's Virtual Address stored in
this field. With Microsoft tools, the first section defaults to an RVA of
0xl000. In OBJs, this field is meaningless and is set to 0.
VirtualSize
This field has different meanings, depending on whether it occurs in an EXE
or an OBJ. In an EXE, it holds the virtual size of the code or data section.
This is the size before rounding up to the nearest file-alignment multiple.
The SizeOfRawData field later on in the structure holds this rounded-up
value. Interestingly, Borland's TLINK32 reverses the meaning of this field
and the SizeOfRawData field, and appears to be the correct linker. For OBJ
files, this field indicates the physical address of the section. The first section
starts at address 0. To find the physical address of the next section, add the
SizeOfRawData value to the physical address of the current section.
Entry Point AKA AddressOfEntry
The address where the image begins execution. This is an RVA, and usually
can be found in the .text section. This field is valid for both EXEs and DLLs.
[Miz]
Of course there are many more elements than these. We will explain the ones we
encounter as the projects progress, but get a full overview by reading other
references (like MattP or the official PE docs).
|
|
Is ProcDump the only way to fix a PE Header?
[Miz]
Not necessarily, most packers leave this up to the user.
As mentioned earlier, the '.TEXT' section is the general name for the section containing code. Early versions of packers and encryptors only modified this section as it was by far the easiest to do, and 'protected' the main code. More modern packers give the ability to pack/encrypt most section types including imports and resources. Doing this stops people using APISpy programs and Resource grabbers as well as complicating the whole unpack operation. Sometimes packers pack all but the first
Icon group so that your program still retains its own icons when in explorer etc.
Remember, by the time the first instruction of the executable is executed most of the usual headers have been processed by the OS. So if you choose to pack these as well (for example the import section) then the system will have no knowledge of it and so the unpacker has to build it up itself.
This is where most people get confused and the sub-topic of import/reloc, etc, rebuilding, will be covered in detail in another section.
|
|
What happens in the proggy when its packed? [Jeff]
[Miz]
That depends on the packer. However the OS determines some limitations.
Say you just wanted to pack the '.TEXT' (usually code) section.
(NOTE: You should always verify a sections contents by checking its Characteristics).
A *very* simplified flow could be:
- Pack and replace the original section.
- Make necessary changes to the 'SizeOfRawData' etc. for that section.
- Make necessary changes to the RawOffsets of other structures.
- Append new section containing decompression routine and any relevant data
(like which sections were packed and original entry point) to the executable.
- Add an additional section header to the Section Headers describing our new one.
- Change the number of sections in the main header.
- Change the entry point to our decompressor's entry point.
Like I said, a *very* simplified overview, but it should give you some idea what
is going on. Not all packers will create additional sections, for example. They may (if there's enough room) choose to cave an existing section or even extend the size of existing sections, techniques used by some virii.
|
|
What happens when our packed proggy is run?
[Miz]
As above, it depends on the packer. But a simple flow could be:
- Initialisation
The unpack code will probably need to get hold of some API function addresses.
Routines like GetModuleHandle, LoadLibrary, and GetProcAddress, are very commonly
used and so need to be found by the unpacker. Memory function addresses may also
be required if the unpack is not in-situ.
(Note: Some packers use their own import section and consequently do not need this 'find
addresses' stage).
What is meant by 'In-situ'?
Some packers unpack over the source data (section).
Others may unpack to a buffer and then copy the entire buffer down to the relevant
section. The In-situ method requires no external memory other than that already allocated
by the OS. The other method will require something like a VirtualAlloc/Free pair.
- Main unpack
Get next packed section and unpack it. If no more then goto stage 4, otherwise....
- Extra processing
If the section was simple (like a '.TEXT' code section) then goto 2.
If the code was more complex, like an import section or a reloc section then some
extra processing will need to be done. For import sections you will see the unpacker
filling the IAT manually by calling LoadLibrary (if not already loaded) for each DLL
listed and GetProcAddress for each API function used in the executable. Many will
also cover their tracks by destroying the original import data. This is very important.
- Calculate the Original entry point, using the pre-known offset and the ACTUAL
imagebase.
- 'Jump' to it.
|
|
So for a simple target, as described above, how would I go about unpacking it?
[Miz]
You don't - the unpacker unpacks for you ;)
Your task is really one of rebuilding, rather than unpacking.
What you would like is an unpacked image, as close as possible to the original, and
one that works with the OS correctly.
Here's an example you may be more familiar with:
- ProgX, when run, decrypts a small block of code, then runs it.
The standard way of tackling this is to let ProgX decrypt the block, copy it
to the relevant place and nop out the decrypt. That way, next time it runs,
it still works but is not decrypting anymore. The block is now stored as
plaintext (or plaincode ;)).
In principle, unpacking is exactly the same.
Now imagine a '.TEXT' section is packed. You know that depacker encounters it,
unpacks it and continues. *In principle* it's simply a case of letting the
unpacker unpack it, then stopping the unpacker unpacking it next time.
Why *in principle*?
Because there are many caveats, which we will see later,
but it's the concept that is important to grasp.
Think now about what wrapper-style packers (like Petite, Neolite, Shrinker,
ASPack etc.) can and can not do.
Specifically: at the time control is passed to the original executable's entry point,
all relevant code and data must be functionally identical to the original
(unpacked) exe. If this were not true then programs would obviously not run
correctly. At the point that the Original Entry Point (OEP) is called, the unpacker
has completed all it's tasks and waves goodbye.
'If this is the case, then (again, in principle) surely 'dumping' the executable's
image to disk at this point would be all that was needed, yes?'
Again, yes and no ;)
It is correct to assume that is all your code/data would need to be 100% correct at
this point but it is incorrect to assume that all the OS information stored in the
PE header would be correct. Remember, the windows loader uses this information
to allocate resources, process imports/exports, the loading and linking to
DLL's, etc, on your behalf. If this information is no longer correct then running
a program dumped in this way would cause a crash, probably at system level.
Boom, SIce pops up in Kernel. Grrr.
'Ah, that's just because the entry point is pointing to the wrong place, soon fix
that.....'.
No (patience grasshopper...) - remember, the entry point is just a *tiny* bit of the
information needed by the OS in order to correctly process an exe. We really need
to be sure we are fixing *all* the relevant information required by the OS. This
is the (assumed) difficult part of unpacking. For us to be confident in our ability
to restore executables then we really need to be familiar with the actions of
unpackers, very familiar with the structure and [and functionality of] the PE Header, as well as the actions of the Windows loader.
|
Ok, so what type of 'Sections' are there?
What exactly are they, what do they mean, and where do they come from?
[Miz]
Remember, the 'Section Headers' part of the PE Header refers to the *location* (offset)
within the main executable's image of the sections' data. They also detail the type,
size, characteristics (flags), etc, of that section. Remember, the actual data for
that section is stored elsewhere. If you are unfamiliar with structures and pointers
then think of these 'structures' as card indices in a library. If you want to find
the location in a book in a library you will look at the card indices. When you
find the correct card it will give more details as to the type of book, location, etc.
By reading the card you know where in the library to look. (ok, so maybe the standard
'letter/address/house' description would have been better ;))
Sections themselves can be thought of as chunks of distinct code, data, system
resources, user resources, etc.
'But when I program I have no knowledge of these!'
It depends on what type of assembler/compiler/linker you use. The data is orgainized
in this way because it is an OS requirement, not a programming one. It makes no
real difference to most programs WHERE the stuff is stored, but it very important
that everything is in the right place and correct for the OS.
'So if I wrote a simple program like a messagebox that says 'UmBongo!' then how
would that look when compiled/linked into and executable?'
Your code would be placed in it's own section. The string (data) 'UmBongo!'
would be placed in its own section. You would have and import section that described
what relevant DLL's and API calls you used (User32 and MessageBox), any icons that
were created would go in a resource section, etc.
Here's some brief descriptions of commonly encountered section types:
- .TEXT
Normally the name of the section that contains all the executable's code.
Normally? Well, like all sections here, it could be any name, but this is the convention.
Borland C++ put code in a more properly named 'CODE' section, although it's linker
did some more problematic things as well. Most however follow these conventions.
- .DATA
Speaks for itself really. This is where all the executable's data is stored.
*Stored* is an important word here. There are two types of data in terms of the
storage they require - initialised data, and you guessed it, uninitialised data.
Think of initialised data as things like strings (text), or a block of data to
decrypt; ie chunks of data with some predetermined value.
Think of uninitialised data as being variables (not predefined ones), empty
arrays, etc; ie blocks of data that will be filled by the executable with some
data at runtime, but at startup have no preset values.
'Why are these treated separately?'
Initialised data obviously needs room in the exe to be stored, whereas
uninitialised data does not. The Windows loader will MAKE the room for this data
when a file is loaded, but it requires no storage in the executable itself, other
than the section description. Hence the two types of data section.
- .BSS
The uninitialised data section (see above).
- .IDATA
Ah, this (along with reloc) is the painful one ;) [Deep breath.......]
The '.IDATA' section provides all the information the OS needs about what DLL's
and API calls were explicitly linked with the executable.
'explicitly linked?'
Yes - 99% of executables will use this method, although it is possible for an
executable to control it's own DLL loading via LoadLibrary calls in the main
code.
'So just by looking at this section, and without running the program itself,
I can figure out whether it uses winsock, mapi, even if it uses a messagebox?!'
Yes. That's how programs like QuickView/DLLShow etc. work, and more interestingly
how API spy programs can monitor API calls from executables. It is because of this that many packers choose to pack this section. Obviously by packing it you are 'hiding' it from such programs and (more importantly) us!
'So why is this 'painful' in terms of unpacking?'
Well, judging by the number of emails and posts, this is the thing that confuses
people the most. Understanding the concept of dumping, etc, comes quickly, but many
do not see the importance of correctly fixing this section. It is *vital*, in order for an
executable to work correctly all the time, in all enviroments, that it has a correct
import section.
'So why can't I just dump it, just before control passes to the original exe?'
Remember we said that for the original program to function correctly then all
it's code and data must be correct? Remember also that we said that an executable's
image contains much more information that just that? Remember that we said the
OS relies on this information to correctly provide the resources for an executable?
Also remember that by the time control passes to the original exe then all of it's
OS initialisation has already taken place?
'Well, if a packer has packed the import table then how on earth can the OS know what's
going on?'
The answer is it doesn't need to, *IF* the unpacker has done the work that
the OS would normally do for itself.
'What does the OS do with this section then?'
Take a look at an ascii-dump of any '.IDATA' section. You'll see strings; names of
DLL's and API functions. These are no use to an executable in that format. Somewhere,
somehow, these must be processed into a format more usable for the exe.
I'm going to divert for a bit, but you'll see why.......
Imagine your messagebox program again. When you do a call to 'MessageBox' the
compiler/linker does something that may at first seem very strange. If you look
at it in Softice you will see something like Call [USER32!MessageBox].
What is strange about that? Look more closely. This explanation from MattP's
book should clarify things. If not, keep re-reading. Its an important thing
to grasp.
[Extract From MattP's Windows Secrets]
"...I was surprised to find out that there was additional code in the ..text section
beyond what I created with the compiler or used from the runtime libraries. In a
PE file, when you call a function in another module (for example, GetMessage()
in USER32.DLL), the CALL instruction emitted by the compiler doesn't transfer
control directly to the function in the DLL. Instead, the call instruction transfers
control to a JMP DWORD PTR [XXXXXXXX] instruction that's also in the
..text section. The JMP instruction jumps to an address stored in a DWORD in
the .idata section. This .idata section DWORD contains the real address of the
operating system function entry point, as shown in Figure 8-4.
After contemplating this for awhile, I came to understand why calls to
DLLs are implemented this way. By funneling all calls to a given DLL function
through one location, there's no longer any need for the loader to patch
every instruction that calls a DLL. All the PE loader has to do is put the correct
address of the target function into the DWORD in the .idata section. No
CALL instructions need to be patched. This is markedly different from NE
files, where each segment contains a list of fixups that need to be applied to
the segment. If the segment calls a given DLL function 20 times, the loader
must copy the function's address into that segment 20 times. The downside
to the PE method is that you can't initialize a variable with the true address
of a DLL function."
....skips a few pages....
"....Visual C++ 2.0 (and onwards).....introduced a new twist to calling
imported functions.
If you look in the system header files from Visual C++ 2.0 (for example,
WINBASE.H), you'll see a difference from the Visual C++ 1.0 headers. In
Visual C++ 2.0, the operating system function prototypes in the system
DLLs include a __declspec(dllimport) as part of their definition. The
__declspec(dllimport) turns out to have quite a useful effect when calling
imported functions. When you call an imported function prototyped with
__ declspec(dllimport), the compiler doesn't generate a call to a JMP DWORD
PTR [XXXXXXXX] instruction elsewhere in the module. Instead, the compiler
generates the function call as CALL DWORD PTR [XXXXXXXX].
The [XXXXXXXX] address is in the .idata section. It's the same address
that would have been used had the old JMP DWORD PTR [XXXXXXXX]
form been used."
[Miz again...]
Ok, can you see now what the OS is doing with the '.IDATA' section before your
executable starts? Yup, it builds an indirect 'jump table', filling it with
the correct addresses of all the DLL functions used by an exe. The exe calls
these functions indirectly via this table of offsets. As you can see it means
that we don't have to fix hundreds of calls within the actual code section itself.
Readers familiar with other operating systems will understand how helpful
this really is!
'So for an unpacker to correctly initialise this table it must process the
names stored in the '.IDATA' section. How does it do this?'
API functions exist for this, they were mentioned previously, and are used by
programs that load and handle DLL's themselves. The more relevant calls are
GetModuleHandle, LoadLibrary, and GetProcAddress. Full docs on these functions
are supplied in separate .txt files with this package. Read them and see how
they would be applied to create the table.
'Alot of information to absorb, so where does that leave us in terms of unpacking
files and correctly restoring their PE Headers?'
As you can hopefully see by now, if we are intending to bypass (and remove!) the
unpacker completely then we really have to make sure that we have valid information
in the PE header so the OS can do all this for us again.
'How do we tackle this?'
There are many ways, and the way you choose will depend on how the unpacker handles
these things. Here's a simple example for a simple unpacker's scheme:
- Say 'BrandX' packer treats all sections the same when they are unpacking (not
uncommon...). When you step through the code, you would see the '.idata'
section being filled with the text we are expecting. This is nice! Later it
will process this section itself, generating the IAT, by using GetModuleHandle /
LoadLibrary / GetProcAddress calls. Then it may deliberately destroy the 'names'
in the '.idata' section. This is bad!
What we can do here is let it unpack the '.idata' section, but *importantly* stop
it from generating the IAT and wiping the contents of the original data. If we can
do this then we have a valid import table again, ready for the OS to process.
All we would need to do then is to alter the relevant 'Directory' settings in the
PE Header to point to the correct offset and size of this new (original!) import
section. Remember, we are doing all this so, eventually, we no longer have to
rely on ANY of the unpacker's code, and so can even remove the unpacker completely.
Virginity restoration.
This is a simple explanation of a simple scheme. It's the thing I 'glossed over'
in the original post on Torn@do's ASPack packed crackme. I'm sure you can
appreciate why now ;) The version of ASPack used made this easy to do, but not
particularly easy to explain (I didn't want to confuse people too much!).
Other packers, and I'm sure future versions of ASPack, will not make things this
simple so make sure you understand the information here - read as many other
resources as you can (See Further Reading). The concepts will always be
similar, but expect things to be get tricky soon ;)
Now, where were we.....oh yes, section descriptions....
- .RELOC
'Fixups', if you like.
Remember that 'ImageBase' was our PREFERRED loading address, but that the OS could,
in theory, put it anywhere it wants? (It is very rare for the OS to do this, but
it's the fact that it can that is important).
If a .RELOC section exists, then in unpackers, just like in the windows loader, you
will see the code checking the preferred and actual imagebases. If they are the
same then the info in the .RELOC section will be ignored, otherwise it will need
to make some 'fixups' to anything that assumed it would be located at the preferred
image base. The areas to 'fix' are stored in this (.RELOC) section.
'How does this affect us?'
Well, if we have (in effect) dumped the code/data and fixed the import issue, but
NOT fixed the reloc section then we have code, etc, that will ONLY work if it is
located at the imagebase when the dump occurred. Like the import section, we need
to fix this so that we have a valid one for the OS to process should it decide later
that it wants the imagebase to be something else. Here's something that explains a
real world example:
[Extract From MattP's Windows Secrets]
"Not having a correct .RELOC section....might cause the executable to not work on
other Win32 platforms. For example, let's say you built an EXE for NT and based the EXE at 0x10000.
If..(the reloc section was not there or invalid)....the EXE wouldn't run under
Windows 95, where the address 0x10000 isn't available (the minimum load address
in Windows 95 is 0x400000; that is, 4MB).
It's important to note that the JMP and CALL instructions generated by
a compiler use offsets relative to the instructions, rather than actual offsets
in the 32-bit fiat segment. If the image needs to be loaded somewhere other
than the location the linker assumed was a base address, these instructions
don't need to change, since they use relative addressing. As a result, there
are not as many relocations as you might think. Relocations are usually
needed only for instructions that use a 32-bit offset to some data.
For example, let's say you had the following global variable declarations:
int i;
int *ptr = &i;
If the linker assumed an image base of 0x10000, the address of the variable
'i' will end up containing something like 0x12004. At the memory used to hold
the pointer ptr, the linker will have written out 0x12004, since that's
the address of the variable 'i'. If the loader (for whatever reason) decided
to load the file at a base address of 0x70000, the address of 'i' would then
be 0x72004. However, the pre-initialized value of the ptr variable would then
be incorrect because i is now 0x60000 bytes higher in memory.
This is where the relocation information comes into play. The .reloc
section is a list of places in the image where the difference between the
linker-assumed load address and the actual load address needs to be taken
into account."
[Miz again....]
Ahhh! Something else to make sure we fix in the PE Header so that the OS can
process our executable correctly. The methods of identifying and fixing
..RELOC sections are almost identical to the way we should handle the '.IDATA'
section. We need to make sure we do this else we are not doing a correct
job of reversing the executable. We want something that works, not
something that works sometimes ;)
- .EDATA
The opposite of a .IDATA section. I=Import, E=Export.
Like the .IDATA section, the .EDATA contains a list of functions, only this time
they are the names of functions WITHIN the executable that are EXPORTED to other
modules. .EDATA sections are more frequent with DLL's (because they obviously
EXPORT the functions that the executable IMPORTs), but they can also (rarely)
occur within executables themselves. You will probably have noticed the
'Import Functions / Export Functions' controls within W32dasm. Load a few
executables into it and see. Then take alook at some dll's.....
If you want to read more on this (and things like export forwarding ;)) then
take alook at MattP's stuff. He goes into some detail there. For our purposes
(ie unpacking), it's just another section we'll have to deal with.
- .RSRC
Resources. Menus, Dialogs, Strings, Icons, Bitmaps, Accelerators, Versions etc.etc.
Everything a resource grabber grabs, and now you know how ;) Again, just like any other
section.
Note: If a program chose to pack this section then things like correct icons would
not be displayed in Desktop, explorer etc. For that reason many (if packing the .RSRC section) strip out the first icon group and store it unpacked elsewhere. That way the executable 'looks' correct when viewed by other programs via icons.
- .OTHERS ;)
There are many others (check out other documentation), but these are the main ones
we will be encountering.
As I'm sure you've seen above, the practice of manual unpacking is far more involved
that simply doing a 'dump'. Hopefully the above (brief!?) descriptions give some
further insight into the importance of fixing the PE Header so the OS doesn't go
ballistic when something is not correct.
|
It's been mentioned here that sections are unpacked over themselves.
How can this work?
Surely, if some sections are packed and others are not then some data would get
overwritten because the unpacked data must be bigger!
[Miz]
Another important concept to grasp.....
The disk image of the executable is not the same as its memory image.
Another slight digression now, but something you may wondered about before......
Imagine you wanted to patch some code in an exectable and have found the relevant
location using SIce (or whatever). It may give an address like 0x401234, and from
what you have learnt from the PE header, you know that the preferred ImageBase was
(say) 0x400000. If the disk image and the memory image were the same then the
offset in the executables disk image to patch would be
0x401234-0x400000 = 0x1234, right?
Wrong.
'So, what happens for these to be different?'
Well, the way the executables MEMORY image is layed out is specified by some
information within the PE Header. The OS uses this information to organize and
create the memory for the executable. It 'maps' the sections based on information
stored in the PE header.
Remember before, when we looked at the differences between initialised and
uninitialised data? Remember we said that for uninitialised data the PE Header
only specified how much 'room' it would need but required no actual storage in
the disk image? Lets look at how this was done. It's a simple example that
can be extended when looking at other section types.
Each section description within the section table has the following fields:
[..Extract from WinNT.h, remember to look here for the structure definitions....]
typedef struct _IMAGE_SECTION_HEADER {
BYTE Name[IMAGE_SIZEOF_SHORT_NAME];
union {
DWORD PhysicalAddress;
DWORD VirtualSize;
} Misc;
DWORD VirtualAddress;
DWORD SizeOfRawData;
DWORD PointerToRawData;
DWORD PointerToRelocations;
DWORD PointerToLinenumbers;
WORD NumberOfRelocations;
WORD NumberOfLinenumbers;
DWORD Characteristics;
} IMAGE_SECTION_HEADER, *PIMAGE_SECTION_HEADER;
[..Back again..]
The answer lies in VirtualSize, VirtualAddress, SizeOfRawData and PointerToRawData.
Now go and take a look at a few section descriptions using a PE viewer before continuing
to read the rest of this answer. Get a feel for the type of numbers in them and how
they differ. Look at some unpacked files' sections as well as packed ones. You should
immediately see some big differences.
Done that? Good. Then the following will make more sense....
VirtualAddress vs PointerToRawData:
- The PointerToRawData is the OFFSET from the *disk* image base (start of the file if that's
easier) to the beginning of that section's data.
- The VirtualAddress however is the OFFSET (RVA) that the OS will map that section to
in the MEMORY image, relative to the ACTUAL imagebase used.
(It should also be noted that both the PointerToRawData and VirtualAddress are multiples
of numbers dictated to by the OS - see MattP's for details on these - suffice to say
there are often (very!) useful 'holes' at the ends of sections that have been used to
pad the sections to whatever alignment size used.)
So, can you see how we could use this information to calculate the file offset for a
patch address (given it's address in the memory image) using the PE Header?
Heres an example:
Memory Address we want to patch = 0x401234
ImageBase used = 0x400000
RVA from imagebase therefore = 0x001234 (0x401234-0x400000)
In our imaginary PE File we also see the following for the .TEXT section
VirtualAddress = 0x001000 (say)
PointerToRawData = 0x000200 (say)
From that you can see that the offset in the exe to patch would be 0x434.
(0x1234 - 0x1000 = 0x234, so our data is 0x234 bytes 'into' this particular section,
and the DISK image of this section is 0x200, so our the diskimage address
would be 0x200+0x234 = 0x434!).
'This is all very interesting, but I don't see the relevance!'
It is very relevant if you are going to manually dump files and sections as we
will see later. But for now the important ones to understand are the differences
in the 'size' members - VirtualSize and SizeOfRawData.
As you have some knowledge now about the differences between RVAs and raw offsets,
VirtualSize and SizeOfRawData should be quite simple. VirtualSize is the size of the
block of memory the OS will allocate for this particular section and the SizeOfRawData
is the size that section takes up in the diskimage.
'Why would these be different?'
For a number of reasons. We already have seen one big one in how the OS will map
an uninitialised data section. In this case it will set SizeOfRawData to zero, and
set the VIRTUALSIZE to the size actually needed by the program.
'But how does it apply to packing/unpacking?'
Well, imagine the following scenario:
Unpacked Section size = 0x008000
Packed Section size = 0x004000 (Packed to half it's original size)
Given these numbers, how could we keep the disk image small (the whole point of
packing!) but also make sure the system prepared us enough room for the unpacked
data to go?
Read it again if the answer is not immediately obvious.
Got it? Cool.
Yup, simply set SizeOfRawData to the (aligned) PACKED size, and set the VirtualSize
to the size of the UNPACKED data.
'Is it really that simple?'
Well, these changes obviously have knockon effects for the other sections.
They can not be changed in isolation. If you change the sizes of addresses for a
section then all the others would need to have THEIR addresses/offsets moved up or
down to compensate. Remember that we said some packers packed in-situ and others used blocks of memory allocated by the unpacker? The reasons why they choose different methods is
normally determined by the compression algorithm used. But you should now be
able to see how an in-situ packer can work with out overwriting other sections.
|
How did u KNOW to change a C00000040 to a E00000020 (<-in your initial post?)
What are the significance of those numbers?
[Miz]
I was puzzled as to why the SIce loader would not break on entry.
Bearing in mind that no packing code could execute before the break meant that
the reason must be in the executable's PE header somewhere. The obvious section to look
at was the section that the break was to be applied and when I looked at the
characteristics they didn't tally with what I KNEW was there. A quick look at the
WinNT.h header file showed what bits needed to be set and after changing them SIce broke
at the entry point.
If you refer to the original post and the WinNT.h header the significance should be clear.
We are basically changing the section characteristics back to what they really represent.
For example, packers may change the system charateristics to make sections writable (because
they write the unpacked code back to the section's memory, as allocated by the OS) and to fool
SIce and disassemblers, etc.
I would have thought this would screw up some OS, particularly NT with it's stricter policy
on memory management, but it apparently works......most packers seem to have adopted it.
Anyway, by changing it back we may even be fixing bugs in the packers ;)
[From MattP's Windows Secrets]
"What most programmers call flags, the COFF/PE format refers to as characteristics.
This field is a set of flags that indicate the section's attributes (code/data, readable,
writeable, and so on). For a complete list of all possible section attributes, see the
IMAGE_SCN_XXX XXX #defines in WINNT.H.
Some of the more important flags.....
0x00000020 This section contains code. It is usually set
in conjunction with the executable flag (0x80000000).
0x00000040 This section contains initialized data. Almost all
sections except executable and the .bss section have
this flag set.
0x00000080 This section contains uninitialized data
(for example, the .bss section).
0x00000200 This section contains comments or some other type of
information. A typical use of this section is the
.drectve section emitted by the compiler, which contains
commands for the linker.
0x00000800 This section's contents shouldn't be put in the final
EXE file. This section is used by the compiler/assembler
lo pass information to the linker
0x02000000 This section can be discarded, since it's not needed by
the process once it's been loaded. The most common
discardable section is the base relocations section (.reloc).
0x10000000 This section is shareable. When used with a DLL,
the data in this section is shared among all processes
using the DLL. The default is for data sections to be
nonshared, meaning that each process using a DLL gets
its own separate copy of this section's data.
In more technical terms, a shared section tells the
memory manager to set the page mappings for this section
so that all processes using the DLL refer to the same
physical page in memory. To make a section shareable,
use the SHARED attribute at link time. For example:
LINK/SECTION:MYDATA, RWS ...
tells the linker that the section called MYDATA should be
readable, write able, and shared. By default, Borland C++
DLL data segments have the shared attribute.
0x20000000 This section is executable. This flag is usually set
whenever the Contains Code flag (0x00000020) is set.
0x40000000 This section is readable. This flag is almost always set
for sections in EXE flies.
0x80000000 The section is writeable. If this flag isn't set in an EXE's
section, the loader should mark the memory mapped pages
as read-only or execute-only.
Typical sections with this attribute are .data and .bss.
|
|
'I have applied the the things described here and in 'Further Reading' and have
successfully reversed a 'BrandX' packed file! I am indestructible!
I am 'master reverser', hear me roar!'
[Miz]
Congratulations sir/madam, you have now dipped your toe into this area. But I'm
afraid that is not all. Although the background information presented here should
stand you in good stead for life, many of the techniques will no doubt be out of date
within months, especially as more and more people tackle such targets. It's an
ongoing saga, and that is its real beauty. Very rarely are new ideas and approaches
offered up by targets on their own. For far too long have we been bored by
mind-numbingly tedious 'protections' against reversing. Countless keyboards must have
their 'B','P','X','G','E','T','D','L','I', and 'M' keys worn down to stumps.
Unpackers/encryptors etc have been adopted as 'protections' by many developers too
lazy to do their own. As such, when shown to have weaknesses, they will improve.
Remember, they are now commercial ventures, and it makes no commercial sense for them
to remain trivial to bypass. Expect new versions, new ideas and techniques.
More fun for us. At last, something interesting we can use to pass those long
lunchtimes and late night sessions that isn't all over in 3 seconds ;)
|
|
Your Questions/Answers/Comments/Corrections/Contributions
|
|
Further Reading
- The 'official' Microsoft PE format specification is included in this FAQ zip file.
Not a great deal of info and hardly any explanation, ie very 'microsoft'....
- Perhaps the best reference for PE info is Matt Pietrek's book Windows 95
Programming Secrets.
If you only ever read one book then read this, (ok, and Brave New World...).
There's an online version...search...just until your bookstore gets it back in
stock of course.......
http://www.tiac.net/users/mpietrek/
(Note: obviously the online version is not there(!), but an updated source disk is, with
a new improved PEDUMP etc.)
- Stone, as mentioned before, was the one who really opened all this up to me.
Fantastic resource. Lots of information, look at the stuff that may at first
seem unrelated to this topic....you'll be pleasantly suprised ;)
http://www.cracking.net
- Another great resource is Mammon_'s site. Everything you ever wanted to know
about decompilation but were afraid to ask. There's even a great breakdown of
the PE Format. (apologies mammon_, I missed this doc before!)
http://www.eccentrica.org/Mammon
- You should obviously be already armed with some decent API docs.....
- And, of course, +Fravia's....ah yes, *BIG* smile, our second home....;)
Check his messageboard for the latest site location and mirrors.
Where? Search....
As +Fravia points out, searching is the first tool. Seek and ye shall find.
There are many more interesting sites than these, but here are some quick
jumpoff points:
|