Reverse Engineering

Last modified: 2023-09-09

Malware Reverse Engineering

Analyze and get the knowledge of executables.


file ./sample

strings ./sample
# -i: Ignore case
# -A: Print N lines of trailing context
# -B: Print N lines of leading context
strings ./example | grep -i password -A 10 -B 10

# -B: signature
binwalk -B ./sample

# Dump hex
hexdump ./sample
# -C: canonical hex+ASCII display
# -n: length
hexdump -C -n 64 ./sample

# Dump hex
hd ./somefile
# -n: only length bytes of input (ex. display the first 28 bytes)
# -s: skip offset bytes from the beginning (ex. no display the first 568 bytes)
hd -n 28 -s 568
# only display the first 64 bytes
hd -n 64 example.bin

xxd ./sample
xxd ./sample | head

# Hex editor with GUI
ghex ./sample.bin

Security Properties

checksec --file=./sample
    Relocation Read-Only, which makes the global offset table (GOT) read-only.
  • Stack Canaries
    Tokens placed after a stack to detect a stack overflow.
  • NX
    Non-Executable. It prevents from shellcode.
  • RWX
    Read-Write-Executable. It's vulnerable to shellcode.
  • PIE
    Position Independent Executable. It loads the program dependencies into random locations.


# Get information
readelf -a ./sample

# Change MSB <=> LSB by editing binary number.
hexedit ./sample
(MSB) 7F 45 4C 46  02 02 01 ... <=> (LSB) 7F 45 4C 46  02 01 01 ...


# -d: disassemble
objdump -d example.obj

# -M: option
objdump -M intel -d example.obj

OLE Files

OLE is a mechanism that allows users to create and edit documents containing items or "objects" created by multiple applications.

  • Oledump

    It dumps the information of the OLE files. example.doc
    # -s: stream number to analyze
    # -d: dump -s 8 -d example.doc -s 9 -d example.doc

    Then decrypt the output using online tools like CyberChef.

  • Olevba

    Download the Oletools to use it.

    olevba example.docm

    Copy the above Visual Basic code, and access to OneCompiler.
    Select the programming language "Visual Basic".
    Paste the copied code to the editor, then click Run.


To check if a binary is compressed with packer such as UPX, dump the hex and extract text related to the packer as below.

# -B: Print before 20 lines from matched string.
# -A: Print after 20 lines from matched string.
hd ./sample | grep UPX -B 20 -A 20

xxd ./sample | grep UPX -B 20 -A 20

If found, we can decompress it.

upx -d ./sample 

Debugger, Decompiler


Cutter is a reverse engineering platform powered by Rizin.

cutter ./sample

Decompiler Explorer

We can also use Decompiler Explorer online tool.


ghidra ./sample
  • If you find “??” instructions in the analyzer of Ghidra, right-click on it and select “Disassemble”.
  • If you find “UD2 (Undefined Instruction)” instruction in the analyzer of Ghidra, replace them with “NOP” by right-clicking and selecting “patch instruction”.


It is used for decompiling .NET files.
AvaloniaILSpy is also available for Linux.

Run a Program

chmod 700 sample

If the program allow you to input some text, you can try first.

Format String Attack

"printf", a function in the C language, is vulnerable to display the internal values from inputs.

# the value of the pointer
> %p
# the value of the 10th pointer
> %10$p

Dynamic Analysis with GDB

GDB is a GNU debugger which is used for reverse engineering.

1. Start the debugger

Before starting, you need to change the permission for the executable so that the program can be executed.

chmod +x ./executable

Now statt gdb.

gdb ./executable

2. Investigation

Find the interesting function and get the address.

gdb> info function

3. Set breakpoints

When you find the interesting function, such as main, strcmp, etc., set the breakpoint at the address.

gdb> break *<address-of-the-function>
# e.g.
gdb> break *0x0000000000400580

By the way, you can delete them if you want.

gdb> info breakpoints

# Delete all breakpoints
gdb> delete

# Delete the specific breakpoint
gdb> delete <the-number-of-the-breakpoint>
# e.g.
gdb> delete 1
gdb> delete 2

4. Run the program

Since you set the breakpoint, the program will stop at the breakpoint.

gdb> run

To observe the information of the current position in the function, run the following command.

gdb> disassemble

# for the specific function
gdb> disassemble <function>

To proceed step by step.

gdb> stepi

# to proceed until the next function
gdb> step

To proceed until the next function.

gdb> continue

5. Examin values of registers

First off, check the address of registers.

gdb> info registers

Then examine values of registers.

  • String

    To get the value as string,

    gdb> x/s <address-of-the-register>
    # e.g.
    gdb> x/s 0x7fffffffdeb0
    # or you can specify register names directly
    gdb> x/s $rax
    gdb> x/s $rbx
    gdb> x/s $rcx
    gdb> x/s $rdx
  • Decimal

    To get the value as decimal,

    gdb> x/d <address-of-the-register>
    # e.g.
    gdb> x/d 0x7fffffffdeb0
    # or you can specify register names directly
    gdb> x/d $rax
    gdb> x/d $rbx
    gdb> x/d $rcx
    gdb> x/d $rdx

6. Quit the debugger

gdb> quit

Tips in GDB

Set addresses to pointers

gdb> set $eip = <address>
# e.g.
gdb> set $eip = 0x565561a9

Set values to registers

Coming soon.

Jump to the destination address

gdb> jump *0x0040096a