[FCSC 2020] Reversing an ELF VM - Keykoolol

Introduction

This challenge was part of the France Cybersecurity Challenge organized by the ANSSI organization.

Challenge Description

The goal is to reverse engineer an ELF binary which requests a username and a serial and then validates the provided inputs. We’re asked to create a valid serial generator.

Explanation

Initial discovery

Main function

Let’s start by getting some infos on the binary :

➜  keykoolol file keykoolol
keykoolol: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=1422aa3ad6edad4cc689ec6ed5d9fd4e6263cd72, stripped
➜  keykoolol ./keykoolol
[+] Username: Knowledge
[+] Serial:   IDontKnow... :/   
[!] Incorrect serial.

We have an ELF 64 binary, dynamically linked, without debug symbols and stripped. Let’s open it in Cutter and look at the main function :

Main Function Graph

As you can see, the main function is pretty understandable. Firstly, it gets username and serial input from the user :

User Input

Thestrcspn()function is used like anstrlen()but stopping when it matches a character given in the string as second argument (here the second argument is simply"\n"). The goal of this is to get the input length without the line feed (which is then replaced by a null byte).

The second part of the main is calling a function that I renamed :

int checkserial(void *code_ptr, int code_len, char *username,
  int username_len, char *serial, serial_len);

We’ll see howcode_ptrandcode_lenare used soon.

Main Serial Validation

As you can see, the last part is just getting the return value ofcheckserial(). If it’s0, then the serial is incorrect, else it is valid.

Checkserial function

This function is more complex. First of all, let’s see the global graph :

Checkserial Graph

Cutter isn’t rendering the complete graph. We can change that by setting an higher value foranal.depthoption.
However, this picture is telling us a lot. The program is looping in an “if forest”.

I didn’t paid much attention at the first block. I came back multiples time to get some information on certain registers / addresses. Here is the assembly :

First Block of Checkserial

There are 0x400 bytes of data copied fromcode_ptrto an address stored inr10andr11. The username and the serial are also copied at two offsets which are located some bytes after the code (r10 + 0x400). All of this is going to be used in the loop. We don’t need more information at the moment.

This is the beginning of the loop. We pass this block at each iteration :

Loop beginning checkserial

So we read 4 bytes atr10 + offset. The r10register is pointing to the data that have been copied at the beginning. By shifting right of 24 bits, the first byte out of the fourth is stored ineax.

Here are some blocks in the “if forest” :

Instruction Example

Thexorare pretty strange. However,eaxregister containing the first byte is compared to different values, and theneaxstoresrsi + 4. Below, we can see that at the bottom of the loop,eaxis put back inesi.

End of the loop

If you remember what we said,esiwas the offset of the 4 bytes we’re reading at each iteration.
At this point, it seems to be a VM. To be sure, we are going to find a better block in the loop.

Here’s the one ifeax == 1 :

Better Instruction

A lot of blocks follow the same pattern as this one. Theshrandandinstructions allow the program to extract some values like args out of the 4 bytes we analyzed. Therdiregister is pointing to a memory space. Bothecxandeaxregisters seems to be some offsets pointing to virtual registers of the vm. We can deduce that these register are 32 bits because of the4 times. By looking at the mask0xf, we can guess that we have 16 virtual registers.
So if we continue the analysis, this block take a value from a virtual register (whereecxis the index), get the value atr10 + RegValand put it into another register. So we may deduce thatr10is the base address andrdx(value of a virtual register) stores a virtual address.
This instruction seems to simulate aMOV Reg(eax), [Reg(ecx)]. I’ll let you confirm this by reading the code again.

After looking further and checking values with GDB we know a lot of things : * This is a VM. * The code is stored atr10 == r11. * Instructions : * Instructions are 4 bytes (fixed length) * The first byte is the opcode * The 3 remaining bytes are used as argument (virtual register, immutable, address) * Virtual registers : * We have 16 virtual registers * Each register is 32 bits * They’re stored atrdi + 4 * reg_id * Another memory space is used by the VM after the0x400bytes of code. For example, username is stored at0x410s (virtual address).

At this point, I hope that the xor instructions we saw earlier are useless… Anyway, we have everything to create a disassembler able to decode each instruction of the VM.

Finding every instruction

The first step is to do is understanding all instructions. To do this, I created a little python script using r2pipe. It’s printing all blocks corresponding to all opcodes of the VM. Here’s the script :

import r2pipe

r2 = r2pipe.open("./keykoolol")
r2.cmd('e anal.depth=10000')
r2.cmd('aaaa')

FIRST_BLOCK = 0x00000a79    # First block to check
LAST_CMP = 0xfe             # Last opcode


def print_block(address):
    r2.cmd('s %i' % address)
    block = r2.cmdj('pdbj')
    for i in block:
        print("\t%s" % i['disasm'])
    print("--------------------------------")


def main():
    r2.cmd('s %i' % FIRST_BLOCK)
    block = r2.cmdj('pdbj')
    while block[0]['ptr'] != LAST_CMP:
        print("Opcode : %s" % block[0]['opcode'].split(' ')[2])
        print("")
        r2.cmd('s %i' % block[1]['jump'])
        block = r2.cmdj('pdbj')
        print_block(block[1]['fail'])

if __name__ == "__main__":
    main()

And the result looks like this :

[...]
Opcode : 0xcd

  xor dword [0x00203074], 0xe46099e2
  lea eax, dword [rsi + 4]
  jmp 0xa73
--------------------------------
Opcode : 0xce

  xor dword [r8], 0x92201356
  lea eax, dword [rsi + 4]
  jmp 0xa73
--------------------------------
Opcode : 0xcf

  mov eax, ecx
  shr ecx, 0x10
  shr eax, 0x14
  and ecx, 0xf
  and eax, 0xf
  mov edx, dword [r8 + rcx*4]
  xor dword [r8 + rax*4], edx
  lea eax, dword [rsi + 4]
  jmp 0xa73
--------------------------------
Opcode : 0x12

  mov eax, ecx
  shr ecx, 0xc
  shr eax, 0x14
  movzx ecx, cl
  and eax, 0xf
  xor dword [r8 + rax*4], ecx
  lea eax, dword [rsi + 4]
  jmp 0xa73
[...]

From here, we can create a dictionary :

| Opcode | Instruction | |:------:|------------- |0x0 | MOV Reg, Reg | |0x1 | MOV Reg, [Reg] | |0x2 | MOV Reg, Value | |0x3 | MOV [Reg], Reg | |0x4 | MOV [Reg], [Reg] | |0x5 | MOV [Reg], Value | |0x1b | BSWAP Reg, [Reg] | |0x1c | BSWAP [Reg], Reg | |0x6 | CALL Value | |0x7 | CMP Reg, Reg | |0x8 | CMP Reg, Value | |0x9 | JE Value | |0xa | JNE Value | |0x14 | JB Value | |0x15 | JG Value | |0x18 | JUMP Value | |0xb | ADD Reg, Reg | |0xc | ADD Reg, Value | |0x16 | SUB Reg, Reg | |0x17 | SUB Reg, Value | |0xd | IMUL Reg, Reg | |0xe | IMUL Reg, Value | |0x19 | SHR Reg, Value | |0x1d | SHL Reg, Value | |0xf | INC Reg | |0x10 | MOD Reg, Reg | |0x11 | MOD Reg, Value | |0x12 | XOR Reg, Reg | |0x13 | XOR Reg, Value | |0x1a | GETEIP Reg | |0x1e | AESENC [Reg], [Reg] | |0xfe | RET | |0xff | END | |0x2a | xor dword [rdi], 0x480035e4 | |0xcf | xor dword [r8], 0x92201356 | |0xdb | xor dword [r8], 0xaca57ad | |0x23 | xor dword [0x0020305c], 0x8bb5b038 | |0x3f | xor dword [0x0020304c], 0xf5acad7d | |0x62 | xor dword [0x00203058], 0x7e0233a2 | |0xbc | xor dword [0x00203058], 0x66601391 | |0xc0 | xor dword [0x0020304c], 0x3a7ac323 | |0xc1 | xor dword [0x0020306c], 0xe2c7c3c3 | |0xc2 | xor dword [0x00203064], 0x93048f8b | |0xc3 | xor dword [0x00203070], 0x4110a870 | |0xc5 | xor dword [0x00203054], 0xb1653a57 | |0xc8 | xor dword [0x00203050], 0xd3bda74e | |0xc9 | xor dword [0x00203078], 0xd6632aca | |0xca | xor dword [0x00203068], 0xbc777df5 | |0xcc | xor dword [0x0020304c], 0x19f0505b | |0xce | xor dword [0x00203074], 0xe46099e2 | |0xd2 | xor dword [0x00203068], 0x2934e85a | |0xd4 | xor dword [0x00203064], 0x93da34fd | |0xd5 | xor dword [0x00203060], 0xfcb4cd4a | |0xd6 | xor dword [0x00203044], 0x71e85cfb | |0xd7 | xor dword [0x00203058], 0xf71a0cab | |0xd8 | xor dword [0x00203048], 0xd24eba88 | |0xd9 | xor dword [0x00203078], 0xbfb56256 | |0xda | xor dword [0x00203078], 0x2802f673 | |0xdc | xor dword [0x00203074], 0xd05cd042 | |0xdd | xor dword [0x00203054], 0xe4573279 | |0xdf | xor dword [0x0020305c], 0xb0f84472 | |0xf4 | xor dword [0x0020307c], 0x727c2426 |

Now we need two things : * The code to disassemble * The disassembler

Programming a disassembler

Let’s dump the code by breaking at the beginning of the loop. As I mentioned before, the code is stored atr10and it length is0x400.

gef➤  b *0x555555554a52
Breakpoint 1 at 0x555555554a52
gef➤  run
gef➤  pf -l 0x400 -c $r10
buf = [0x6e, 0x18, 0xb0, 0x17, 0xc9, 0xf5, 0xbf, 0x8, 0x74, 0x0, 0x0, 0xa, 0x37, 0x52, 0xa, 0x0, 0x98, 0x95, 0x1c, 0x0, 0x74, 0x3, 0x0, 0x6, 0x88, 0x1c, 0x0, 0x8, 0x74, 0x0, 0x0, 0xa, 0x3f, 0x9e, 0x8, 0x0, 0x56, 0x94, 0x1c, 0x0, 0xad, 0x6, 0x18, 0xc, 0xc6, 0xf, 0x20, 0x2, 0x88, 0x2, 0x0, 0x6, 0x89, 0x97, 0xc, 0x0, 0x7c, 0x2, 0x8, 0xc, 0xc9, 0x73, 0x1c, 0x0, 0x5b, 0x0, 0x19, 0xc, 0x7c, 0x0, 0x0, 0x6, 0xfa, 0x1b, 0xc, 0x0, 0xf7, 0x1, 0x10, 0x0, 0xa7, 0xf3, 0x1f, 0xc, 0x4b, 0x19, 0x10, 0xc, 0xfc, 0x0, 0x0, 0x6, 0x5a, 0x41, 0xc, 0x0, 0x9, 0x95, 0x1c, 0x0, 0x8e, 0x8, 0x18, 0xc, 0x28, 0xb, 0x26, 0x2, 0xe8, 0x2, 0x0, 0x6, 0x64, 0x34, 0x7b, 0xff, 0x5, 0xc, 0x0, 0x2, 0xaf, 0xb4, 0x68, 0xff, 0xde, 0x24, 0xf2, 0x1a, 0x5, 0x88, 0xf4, 0xc, 0xfd, 0x5c, 0xdd, 0x12, 0xc0, 0x49, 0xdf, 0x13, 0xb9, 0x82, 0xd0, 0x1d, 0x5a, 0x3a, 0xde, 0x13, 0xea, 0x8f, 0xd0, 0x1d, 0xc1, 0x2f, 0xdd, 0x13, 0x37, 0x86, 0xd0, 0x1d, 0xc0, 0x1f, 0xdc, 0x13, 0x2, 0xc4, 0xef, 0x1b, 0x64, 0x91, 0xed, 0x12, 0xa, 0x33, 0xfe, 0x1c, 0xdb, 0x8a, 0xe1, 0x1d, 0x40, 0x81, 0xe1, 0x19, 0x28, 0xfe, 0xe7, 0x8, 0xc8, 0x0, 0x0, 0x15, 0x1a, 0x46, 0xf0, 0xc, 0xa4, 0x0, 0x0, 0x18, 0xbe, 0xe2, 0xe2, 0xc3, 0xb8, 0x2b, 0xf2, 0xc1, 0x4, 0xa2, 0xf0, 0xc0, 0x29, 0xde, 0xf2, 0xcf, 0xc7, 0x18, 0xfd, 0xd2, 0xc1, 0x5b, 0xc0, 0xc2, 0xf5, 0x30, 0xe5, 0xce, 0x4c, 0xec, 0xe7, 0xc9, 0xc, 0xe3, 0xd2, 0xc8, 0xfc, 0xd7, 0xd9, 0xce, 0x8, 0xb2, 0xcf, 0xce, 0x38, 0xe3, 0xd2, 0xd9, 0x5e, 0x3d, 0x4c, 0x3f, 0x4e, 0x65, 0xff, 0x1a, 0xc0, 0x85, 0xf4, 0xc, 0xeb, 0x87, 0xdd, 0x12, 0xbf, 0x15, 0xda, 0x13, 0xf2, 0x82, 0xd0, 0x1d, 0xc3, 0x2c, 0xdb, 0x13, 0xbe, 0x80, 0xd0, 0x1d, 0xf0, 0x32, 0xdc, 0x13, 0x1c, 0x8d, 0xd0, 0x1d, 0x88, 0x49, 0xdd, 0x13, 0x6f, 0x26, 0xef, 0x1b, 0x54, 0x13, 0xed, 0x12, 0x1f, 0xad, 0xfe, 0x1c, 0xcc, 0x8d, 0xe1, 0x1d, 0xd8, 0x80, 0xe1, 0x19, 0x23, 0xf2, 0xe7, 0x8, 0x48, 0x1, 0x0, 0x15, 0x44, 0x44, 0xf0, 0xc, 0x24, 0x1, 0x0, 0x18, 0x11, 0x7, 0xa3, 0xd4, 0xab, 0xbd, 0xa5, 0xd8, 0x54, 0xf2, 0xb3, 0xd4, 0x54, 0xbb, 0x33, 0xd6, 0x44, 0xb0, 0x93, 0xd6, 0x66, 0x8d, 0x83, 0xd4, 0x7a, 0x9c, 0x86, 0xdf, 0xa5, 0x59, 0xf7, 0xd5, 0x3, 0xa0, 0x82, 0xd4, 0xd, 0xb4, 0x86, 0xdf, 0xd6, 0xf6, 0x80, 0xd7, 0x21, 0x2b, 0x96, 0xdb, 0x27, 0xb5, 0x92, 0xdc, 0x25, 0xb3, 0xc3, 0xdd, 0xfd, 0xb3, 0xc3, 0xcc, 0x75, 0x78, 0xf3, 0xd4, 0x97, 0xb8, 0xf6, 0xd8, 0x3a, 0xf3, 0x83, 0xd4, 0xc0, 0x13, 0x94, 0xd4, 0x9b, 0xf2, 0x90, 0xca, 0xd2, 0x81, 0xf3, 0xd4, 0x35, 0xbf, 0xf7, 0xd8, 0x39, 0x99, 0x85, 0xd4, 0x37, 0xb8, 0x82, 0xd8, 0xb9, 0xe4, 0x94, 0xd4, 0x51, 0xba, 0x96, 0xd8, 0x50, 0xf4, 0x90, 0xca, 0x9e, 0x13, 0xf3, 0xd4, 0x19, 0xbd, 0xf0, 0xd8, 0x51, 0x33, 0x85, 0xd4, 0xaa, 0xbe, 0x82, 0xd8, 0xdd, 0x87, 0x94, 0xd4, 0x27, 0xbe, 0x97, 0xd8, 0x53, 0xfc, 0x90, 0xca, 0x7b, 0xe7, 0xf3, 0xd4, 0x15, 0xb4, 0xf1, 0xd8, 0xb5, 0xe9, 0x83, 0xd4, 0x63, 0xb1, 0x80, 0xd8, 0x35, 0xd9, 0x94, 0xd4, 0xa6, 0xbc, 0x90, 0xd8, 0x20, 0xfc, 0x90, 0xca, 0xf1, 0xb8, 0xf3, 0xd4, 0xc0, 0xbd, 0xf2, 0xd8, 0x41, 0xb3, 0x85, 0xd4, 0xcb, 0x4c, 0x94, 0xd4, 0xd9, 0xb6, 0x91, 0xd8, 0xd1, 0xf0, 0x90, 0xca, 0x83, 0x17, 0xf2, 0xd4, 0xb7, 0x87, 0x85, 0xd4, 0xe6, 0x65, 0x94, 0xd4, 0x36, 0xb3, 0x92, 0xd8, 0xff, 0xf9, 0x90, 0xca, 0x31, 0x7c, 0x34, 0xdb, 0x6d, 0xb3, 0x31, 0xdc, 0x89, 0xb0, 0xc3, 0xdd, 0xf9, 0xb3, 0xc3, 0xcc, 0x83, 0xee, 0x5a, 0x2a, 0x34, 0xf0, 0xf, 0xf, 0x85, 0xeb, 0x2, 0xf, 0x7f, 0x82, 0xa, 0xf, 0x44, 0x7d, 0x0, 0xf, 0x9a, 0x88, 0x3, 0xf, 0x1a, 0xba, 0xf, 0xf, 0x8b, 0x89, 0xf, 0xf, 0xf4, 0x2d, 0x9, 0xf, 0x97, 0xa, 0x8, 0xf, 0x66, 0x55, 0xf, 0xf, 0xb3, 0x23, 0xc, 0xf, 0xfb, 0xd6, 0xb, 0xf, 0x33, 0x83, 0x9, 0xf, 0x3f, 0x94, 0xa, 0xf, 0xe3, 0xc1, 0x0, 0xf, 0x5f, 0xc8, 0x0, 0xf, 0x88, 0xd4, 0x7, 0xf, 0x23, 0x5c, 0x6, 0xf, 0x43, 0xde, 0xe, 0xf, 0x25, 0xfa, 0xaf, 0x62, 0x7c, 0x94, 0x2c, 0xcf, 0x8d, 0xa5, 0x9c, 0xbc, 0x67, 0x70, 0xde, 0xf4, 0xc6, 0x7e, 0x70, 0x1, 0xe2, 0xc, 0x70, 0x8, 0x98, 0x2, 0x0, 0xa, 0xe4, 0x2, 0x0, 0x18, 0xc5, 0xc, 0x60, 0x2, 0x9b, 0x85, 0x37, 0x0, 0x54, 0x65, 0x36, 0xb, 0x58, 0xd6, 0x30, 0xe, 0xe7, 0x50, 0x32, 0x13, 0x4b, 0xf6, 0x3f, 0x11, 0x9c, 0x4d, 0x46, 0x0, 0xa3, 0xa9, 0x42, 0xb, 0xa6, 0x6, 0x41, 0x11, 0x7e, 0x8a, 0x41, 0xb, 0xf5, 0xff, 0x54, 0x1, 0xe, 0x42, 0x53, 0x12, 0x65, 0x92, 0x45, 0x3, 0x57, 0xe2, 0x69, 0xf, 0xac, 0xe, 0x61, 0x8, 0x9c, 0x2, 0x0, 0xa, 0x7c, 0xca, 0x7, 0xf, 0x99, 0xf1, 0x25, 0xf, 0x88, 0x2, 0x0, 0x6, 0xfe, 0x32, 0x5b, 0xfe, 0xe8, 0x59, 0xfd, 0x1a, 0x2f, 0x83, 0xf4, 0xc, 0x27, 0xb8, 0xdd, 0x12, 0xb0, 0xa8, 0xda, 0x13, 0x36, 0x82, 0xd0, 0x1d, 0x6b, 0xbc, 0xdb, 0x13, 0xb5, 0x84, 0xd0, 0x1d, 0x28, 0xcb, 0xdc, 0x13, 0xde, 0x88, 0xd0, 0x1d, 0x39, 0xd2, 0xdd, 0x13, 0x37, 0x3d, 0xef, 0x1b, 0xe5, 0x59, 0xed, 0x12, 0xc, 0x7d, 0xfe, 0x1c, 0x5e, 0x89, 0xe1, 0x1d, 0x74, 0x8f, 0xe1, 0x19, 0xc9, 0xf6, 0xe7, 0x8, 0x34, 0x3, 0x0, 0x15, 0x16, 0x4a, 0xf0, 0xc, 0x10, 0x3, 0x0, 0x18, 0x9f, 0xbb, 0xfc, 0xdf, 0x30, 0x78, 0x8c, 0xdd, 0x32, 0xdc, 0x9d, 0xdd, 0xb8, 0xdd, 0x8f, 0xd6, 0xad, 0x2c, 0x9f, 0xd6, 0xda, 0x35, 0x88, 0xdc, 0x59, 0xb0, 0x99, 0xdc, 0x14, 0xfe, 0x89, 0xda, 0xc6, 0xb8, 0xcc, 0xd7, 0x81, 0x24, 0xfa, 0xd2, 0xf9, 0x6a, 0xfe, 0xda, 0x92, 0xb8, 0xcc, 0xd7, 0x56, 0xae, 0xcc, 0xdf, 0xda, 0xb8, 0xcc, 0xc5, 0xdd, 0xb1, 0xcc, 0xdf, 0xe5, 0x79, 0xc6, 0x23, 0x36, 0x1, 0x30, 0x2, 0x1, 0xc9, 0x20, 0x0, 0xa, 0x27, 0x23, 0xb, 0x5f, 0x56, 0x22, 0x1, 0x1c, 0x9, 0x20, 0x8, 0xf0, 0x3, 0x0, 0x9, 0xaf, 0x99, 0x23, 0x8, 0xa4, 0x3, 0x0, 0x15, 0x89, 0x9, 0x23, 0x8, 0xf8, 0x3, 0x0, 0x14, 0x7c, 0x8, 0x23, 0x17, 0xb8, 0x3, 0x0, 0x18, 0x68, 0x67, 0x26, 0x8, 0xf8, 0x3, 0x0, 0x15, 0x3e, 0x17, 0x26, 0x8, 0xf8, 0x3, 0x0, 0x14, 0xf9, 0x73, 0x25, 0x17, 0x3a, 0x47, 0x43, 0x0, 0x25, 0x22, 0x40, 0x11, 0xa4, 0x1c, 0x40, 0x8, 0xd4, 0x3, 0x0, 0x9, 0xb5, 0xc, 0x21, 0xe, 0x86, 0x93, 0x52, 0x0, 0xe8, 0x3, 0x0, 0x18, 0x29, 0xd6, 0x25, 0x12, 0x59, 0x80, 0x43, 0x0, 0x55, 0x19, 0x40, 0x19, 0xb6, 0x2, 0x41, 0xb, 0xc3, 0x39, 0x42, 0x3, 0xd7, 0x54, 0x35, 0xf, 0x78, 0x3, 0x0, 0x18, 0x1b, 0x13, 0x0, 0x2, 0xfc, 0x3, 0x0, 0x18, 0xdc, 0x0, 0x0, 0x2, 0xa0, 0xa1, 0x31, 0xfe]

To do that we’ll read four bytes of data and parse it like the VM using a dictionary which describe each instruction :

code = ['6e','18','b0','17','c9','f5','bf','08','74','00','00','0a','37','52','0a','00','98','95','1c','00','74','03','00','06','88','1c','00','08','74','00','00','0a','3f','9e','08','00','56','94','1c','00','ad','06','18','0c','c6','0f','20','02','88','02','00','06','89','97','0c','00','7c','02','08','0c','c9','73','1c','00','5b','00','19','0c','7c','00','00','06','fa','1b','0c','00','f7','01','10','00','a7','f3','1f','0c','4b','19','10','0c','fc','00','00','06','5a','41','0c','00','09','95','1c','00','8e','08','18','0c','28','0b','26','02','e8','02','00','06','64','34','7b','ff','05','0c','00','02','af','b4','68','ff','de','24','f2','1a','05','88','f4','0c','fd','5c','dd','12','c0','49','df','13','b9','82','d0','1d','5a','3a','de','13','ea','8f','d0','1d','c1','2f','dd','13','37','86','d0','1d','c0','1f','dc','13','02','c4','ef','1b','64','91','ed','12','0a','33','fe','1c','db','8a','e1','1d','40','81','e1','19','28','fe','e7','08','c8','00','00','15','1a','46','f0','0c','a4','00','00','18','be','e2','e2','c3','b8','2b','f2','c1','04','a2','f0','c0','29','de','f2','cf','c7','18','fd','d2','c1','5b','c0','c2','f5','30','e5','ce','4c','ec','e7','c9','0c','e3','d2','c8','fc','d7','d9','ce','08','b2','cf','ce','38','e3','d2','d9','5e','3d','4c','3f','4e','65','ff','1a','c0','85','f4','0c','eb','87','dd','12','bf','15','da','13','f2','82','d0','1d','c3','2c','db','13','be','80','d0','1d','f0','32','dc','13','1c','8d','d0','1d','88','49','dd','13','6f','26','ef','1b','54','13','ed','12','1f','ad','fe','1c','cc','8d','e1','1d','d8','80','e1','19','23','f2','e7','08','48','01','00','15','44','44','f0','0c','24','01','00','18','11','07','a3','d4','ab','bd','a5','d8','54','f2','b3','d4','54','bb','33','d6','44','b0','93','d6','66','8d','83','d4','7a','9c','86','df','a5','59','f7','d5','03','a0','82','d4','0d','b4','86','df','d6','f6','80','d7','21','2b','96','db','27','b5','92','dc','25','b3','c3','dd','fd','b3','c3','cc','75','78','f3','d4','97','b8','f6','d8','3a','f3','83','d4','c0','13','94','d4','9b','f2','90','ca','d2','81','f3','d4','35','bf','f7','d8','39','99','85','d4','37','b8','82','d8','b9','e4','94','d4','51','ba','96','d8','50','f4','90','ca','9e','13','f3','d4','19','bd','f0','d8','51','33','85','d4','aa','be','82','d8','dd','87','94','d4','27','be','97','d8','53','fc','90','ca','7b','e7','f3','d4','15','b4','f1','d8','b5','e9','83','d4','63','b1','80','d8','35','d9','94','d4','a6','bc','90','d8','20','fc','90','ca','f1','b8','f3','d4','c0','bd','f2','d8','41','b3','85','d4','cb','4c','94','d4','d9','b6','91','d8','d1','f0','90','ca','83','17','f2','d4','b7','87','85','d4','e6','65','94','d4','36','b3','92','d8','ff','f9','90','ca','31','7c','34','db','6d','b3','31','dc','89','b0','c3','dd','f9','b3','c3','cc','83','ee','5a','2a','34','f0','0f','0f','85','eb','02','0f','7f','82','0a','0f','44','7d','00','0f','9a','88','03','0f','1a','ba','0f','0f','8b','89','0f','0f','f4','2d','09','0f','97','0a','08','0f','66','55','0f','0f','b3','23','0c','0f','fb','d6','0b','0f','33','83','09','0f','3f','94','0a','0f','e3','c1','00','0f','5f','c8','00','0f','88','d4','07','0f','23','5c','06','0f','43','de','0e','0f','25','fa','af','62','7c','94','2c','cf','8d','a5','9c','bc','67','70','de','f4','c6','7e','70','01','e2','0c','70','08','98','02','00','0a','e4','02','00','18','c5','0c','60','02','9b','85','37','00','54','65','36','0b','58','d6','30','0e','e7','50','32','13','4b','f6','3f','11','9c','4d','46','00','a3','a9','42','0b','a6','06','41','11','7e','8a','41','0b','f5','ff','54','01','0e','42','53','12','65','92','45','03','57','e2','69','0f','ac','0e','61','08','9c','02','00','0a','7c','ca','07','0f','99','f1','25','0f','88','02','00','06','fe','32','5b','fe','e8','59','fd','1a','2f','83','f4','0c','27','b8','dd','12','b0','a8','da','13','36','82','d0','1d','6b','bc','db','13','b5','84','d0','1d','28','cb','dc','13','de','88','d0','1d','39','d2','dd','13','37','3d','ef','1b','e5','59','ed','12','0c','7d','fe','1c','5e','89','e1','1d','74','8f','e1','19','c9','f6','e7','08','34','03','00','15','16','4a','f0','0c','10','03','00','18','9f','bb','fc','df','30','78','8c','dd','32','dc','9d','dd','b8','dd','8f','d6','ad','2c','9f','d6','da','35','88','dc','59','b0','99','dc','14','fe','89','da','c6','b8','cc','d7','81','24','fa','d2','f9','6a','fe','da','92','b8','cc','d7','56','ae','cc','df','da','b8','cc','c5','dd','b1','cc','df','e5','79','c6','23','36','01','30','02','01','c9','20','00','0a','27','23','0b','5f','56','22','01','1c','09','20','08','f0','03','00','09','af','99','23','08','a4','03','00','15','89','09','23','08','f8','03','00','14','7c','08','23','17','b8','03','00','18','68','67','26','08','f8','03','00','15','3e','17','26','08','f8','03','00','14','f9','73','25','17','3a','47','43','00','25','22','40','11','a4','1c','40','08','d4','03','00','09','b5','0c','21','0e','86','93','52','00','e8','03','00','18','29','d6','25','12','59','80','43','00','55','19','40','19','b6','02','41','0b','c3','39','42','03','d7','54','35','0f','78','03','00','18','1b','13','00','02','fc','03','00','18','dc','00','00','02','a0','a1','31','fe']

formats = {
    0x0 : {
        "inst": "MOV R{}, R{}",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "register", "shift": 0x10, "mask": 0xf}
        ]
    },
    0x1 : {
        "inst": "MOV R{}, [R{}]",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "register", "shift": 0x10, "mask": 0xf}
        ]
    },
    0x2 : {
        "inst": "MOV R{}, {}",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "value", "shift": 0xc, "mask": 0xff}
        ]
    },
    0x3 : {
        "inst": "MOV [R{}], R{}",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "register", "shift": 0x10, "mask": 0xf}
        ]
    },
    0x4 : {
        "inst": "MOV [R{}], [R{}]",
        "args": [
            {"type": "register", "shift": 0x10, "mask": 0xf},
            {"type": "register", "shift": 0x14, "mask": 0xf}
        ]
    },
    0x5 : {
        "inst": "MOV [R{}], {}",
        "args": [
            {"type": "register", "shift": 0x10, "mask": 0xf},
            {"type": "value", "shift": 0x14, "mask": 0xf}
        ]
    },
    0x1b: {
        "inst": "BSWAP R{}, [R{}]",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "register", "shift": 0x10, "mask": 0xf}
        ]
    },
    0x1c: {
        "inst": "BSWAP [R{}], R{}",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "register", "shift": 0x10, "mask": 0xf}
        ]
    },
    0x6 : {
        "inst": "CALL {}",
        "args": [{"type": "value", "shift": 0, "mask": 0xffffff}]
    },
    0x7 : {
    "inst": "CMP R{}, R{}",
        "args": [
            {"type": "register", "shift": 0x10, "mask": 0xf},
            {"type": "register", "shift": 0x14, "mask": 0xf}
        ]
    },
    0x8 : {
        "inst": "CMP R{}, {}",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "value", "shift": 0xc, "mask": 0xff}
        ]
    },
    0x9 : {
        "inst": "JE {}",
        "args": [{"type": "value", "shift": 0, "mask": 0xffffff}]
    },
    0xa : {
        "inst": "JNE {}",
        "args": [{"type": "value", "shift": 0, "mask": 0xffffff}]
    },
    0x14: {
        "inst": "JS {}",
        "args": [{"type": "value", "shift": 0, "mask": 0xffffff}]
    },
    0x15: {
        "inst": "JG {}",
        "args": [{"type": "value", "shift": 0, "mask": 0xffffff}]
    },
    0x18: {
        "inst": "JUMP {}",
        "args": [{"type": "value", "shift": 0, "mask": 0xffffff}]
    },
    0xb : {
        "inst": "ADD R{}, R{}",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "register", "shift": 0x10, "mask": 0xf}
        ]
    },
    0xc : {
        "inst": "ADD R{}, {}",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "value", "shift": 0xc, "mask": 0xff}
        ]
    },
    0x16: {
        "inst": "SUB R{}, R{}",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "register", "shift": 0x10, "mask": 0xf}
        ]
    },
    0x17: {
        "inst": "SUB R{}, {}",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "value", "shift": 0xc, "mask": 0xff}
        ]
    },
    0xd : {
        "inst": "IMUL R{}, R{}",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "register", "shift": 0x10, "mask": 0xf}
        ]
    },
    0xe : {
        "inst": "IMUL R{}, {}",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "value", "shift": 0xc, "mask": 0xff}
        ]
    },
    0x19: {
        "inst": "SHR R{}, {}",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "value", "shift": 0xc, "mask": 0xff}
        ]
    },
    0x1d: {
        "inst": "SHL R{}, {}",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "value", "shift": 0xc, "mask": 0xff}
        ]
    },
    0xf : {
        "inst": "INC R{}",
        "args": [{"type": "register", "shift": 0x14, "mask": 0xf}]
    },
    0x10: {
        "inst": "MOD R{}, R{}",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "register", "shift": 0x10, "mask": 0xf}
        ]
    },
    0x11: {
        "inst": "MOD R{}, {}",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "value", "shift": 0xc, "mask": 0xff}
        ]
    },
    0x12: {
        "inst": "XOR R{}, R{}",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "register", "shift": 0x10, "mask": 0xf}
        ]
    },
    0x13: {
        "inst": "XOR R{}, {}",
        "args": [
            {"type": "register", "shift": 0x14, "mask": 0xf},
            {"type": "value", "shift": 0xc, "mask": 0xff}
        ]
    },
    0x1a: {
        "inst": "GETEIP R{}",
        "args": [{"type": "register", "shift": 0x14, "mask": 0xf}]
    },
    0x1e: {"inst": "AESENC [R{}], [R{}], [R{}]",
        "args": [
            {"type": "register", "shift": 0x10, "mask": 0xf},
            {"type": "register", "shift": 0xc, "mask": 0xf},
            {"type": "register", "shift": 0x14, "mask": 0xf}
        ]
    },
    0xfe: {"inst": "RET", "args": []},
    0xff: {"inst": "END", "args": []}
}

def print_inst(val, f):
    args = []
    for x in f['args']:
        v = hex((val >> x['shift']) & x['mask']) # We apply the same operations as if we were in the VM
        v = v if x['type'] == "value" else v[2:] # We don't want the 0x in a register (ex : R0xa)
        args.append(v)
    print(f['inst'].format(*args))

def main():
    for i in range(0, len(code), 4):
        inst = code[i:i + 4]
        opcode = int(inst[3], 16) # Bytes are store in little endian, so opcode is the last one
        val = int(''.join(inst[:3][::-1]), 16) # Bytes are stored in little endian, so we reverse the list
        if opcode in formats.keys(): # Check if it's a crazy xor...
            print_inst(val, formats[opcode])
        else:
            print("Oh noooo, we found a bad xor...")


if __name__ == "__main__":
    main()

Which output :

➜  keykoolol python disassembler.py
0x0     SUB Rb, 0x1
0x4     CMP Rb, 0xff
0x8     JNE 0x74
0xc     MOV R0, Ra
0x10    MOV R1, Rc
0x14    CALL 0x374
0x18    CMP R0, 0x1
0x1c    JNE 0x74
0x20    MOV R0, R8
0x24    MOV R1, Rc
0x28    ADD R1, 0x80
0x2c    MOV R2, 0x0
0x30    CALL 0x288
0x34    MOV R0, Rc
0x38    ADD R0, 0x80
0x3c    MOV R1, Rc
0x40    ADD R1, 0x90
0x44    CALL 0x7c
0x48    MOV R0, Rc
0x4c    MOV R1, R0
0x50    ADD R1, 0xff
0x54    ADD R1, 0x1
0x58    CALL 0xfc
0x5c    MOV R0, Rc
0x60    MOV R1, Rc
0x64    ADD R1, 0x80
0x68    MOV R2, 0x60
0x6c    CALL 0x2e8
0x70    END
0x74    MOV R0, 0x0
0x78    END
0x7c    GETEIP Rf
0x80    ADD Rf, 0x48
0x84    XOR Rd, Rd
0x88    XOR Rd, 0xf4
0x8c    SHL Rd, 0x8
0x90    XOR Rd, 0xe3
0x94    SHL Rd, 0x8
0x98    XOR Rd, 0xd2
0x9c    SHL Rd, 0x8
0xa0    XOR Rd, 0xc1
0xa4    BSWAP Re, [Rf]
0xa8    XOR Re, Rd
0xac    BSWAP [Rf], Re
0xb0    SHL Re, 0x18
0xb4    SHR Re, 0x18
0xb8    CMP Re, 0x7f
0xbc    JG 0xc8
0xc0    ADD Rf, 0x4
0xc4    JUMP 0xa4
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
0xfc    GETEIP Rf
0x100   ADD Rf, 0x48
0x104   XOR Rd, Rd
0x108   XOR Rd, 0xa1
0x10c   SHL Rd, 0x8
0x110   XOR Rd, 0xb2
0x114   SHL Rd, 0x8
0x118   XOR Rd, 0xc3
0x11c   SHL Rd, 0x8
0x120   XOR Rd, 0xd4
0x124   BSWAP Re, [Rf]
0x128   XOR Re, Rd
0x12c   BSWAP [Rf], Re
0x130   SHL Re, 0x18
0x134   SHR Re, 0x18
0x138   CMP Re, 0x7f
0x13c   JG 0x148
0x140   ADD Rf, 0x4
0x144   JUMP 0x124
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
[...]

Oh No Meme

The VM is patching its code at runtime in multiples functions using instructions such asxor dword [0x0020306c], 0xe2c7c3c3. If we suppose that every xor segment are only deobfuscating the function and not obfuscating other parts (everything is possible now…), then we can dump the code again at the end of the program.

So I applied the same method and put a breakpoint in theend instruction block. But it didn’t work because we’re taking the jump at 0x8 or 0x1c. So let’s reverse this part of the code.

The beginning seems to be the main function :

0x0     SUB Rb, 0x1     // Here Rb contains the serial length
0x4     CMP Rb, 0xff    // So the serial is composed 256 bytes
0x8     JNE 0x74

0xc     MOV R0, Ra
0x10    MOV R1, Rc
0x14    CALL 0x374      // This function check that the serial matches [0-9a-f]{256} (hexadecimal)
0x18    CMP R0, 0x1     // It also convert the hex in a real number (like atoi function) and store it at 0x530
0x1c    JNE 0x74

0x20    MOV R0, R8
0x24    MOV R1, Rc
0x28    ADD R1, 0x80
0x2c    MOV R2, 0x0
0x30    CALL 0x288

0x34    MOV R0, Rc
0x38    ADD R0, 0x80
0x3c    MOV R1, Rc
0x40    ADD R1, 0x90
0x44    CALL 0x7c

0x48    MOV R0, Rc
0x4c    MOV R1, R0
0x50    ADD R1, 0xff
0x54    ADD R1, 0x1
0x58    CALL 0xfc

0x5c    MOV R0, Rc
0x60    MOV R1, Rc
0x64    ADD R1, 0x80
0x68    MOV R2, 0x60
0x6c    CALL 0x2e8

0x70    END
0x74    MOV R0, 0x0
0x78    END

Here are more information about the function at 0x374 :

0x374   MOV R3, 0x0     // R3 is the index
0x378   MOV R2, R0
0x37c   ADD R2, R3
0x380   MOV R2, [R2]    // R2 = x containt each byte of the input serial
0x384   CMP R2, 0x0     // Iterate over each char of the string
0x388   JE 0x3f0
0x38c   CMP R2, 0x39
0x390   JG 0x3a4        // 1. If x > '9' go to the second if
0x394   CMP R2, 0x30
0x398   JS 0x3f8        // Else verify that x >= '0' (so '0' <= x <= '9'), stop if it's not the case
0x39c   SUB R2, 0x30
0x3a0   JUMP 0x3b8
0x3a4   CMP R2, 0x66    // 2. If x > 'f', then stop
0x3a8   JG 0x3f8
0x3ac   CMP R2, 0x61    // Else verify that x > 'a' (so 'a' <= x <= 'f')
0x3b0   JS 0x3f8
0x3b4   SUB R2, 0x57
0x3b8   MOV R4, R3      // Substract 0x57 or 0x30 depending on the value of x ([a-f] or [0-9])
0x3bc   MOD R4, 0x2
0x3c0   CMP R4, 0x1     // Then we check if the index is even or odd. The goal of the rest of the code is to transform
0x3c4   JE 0x3d4        // 2 hex char, for example 'ff' meaning 0x6666 in memory, in one byte 0xff in memory.
0x3c8   IMUL R2, 0x10   // So basiclly, if the index is odd, we left shift x of 4 bits (so we have 0xf0).
0x3cc   MOV R5, R2      // And if the index is even, we place x in the first 4 bits (-> 0xff) and store this byte in memory.
0x3d0   JUMP 0x3e8
0x3d4   XOR R2, R5
0x3d8   MOV R4, R3
0x3dc   SHR R4, 0x1
0x3e0   ADD R4, R1
0x3e4   MOV [R4], R2
0x3e8   INC R3
0x3ec   JUMP 0x378
0x3f0   MOV R0, 0x1
0x3f4   JUMP 0x3fc
0x3f8   MOV R0, 0x0
0x3fc   RET

So now, we can deobfuscate the code by entering a valid serial composed of 256 hexadecimal characters :

gef➤  b *0x55555555638c
Breakpoint 1 at 0x55555555638c
gef➤  run
Starting program: /mnt/hgfs/CTF/FCSC/keykoolol/keykoolol 
[+] Username: ABCD
[+] Serial:   aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
gef➤  pf -l 0x400 -c $r10
buf = [0x6e, 0x18, 0xb0, 0x17, 0xc9, 0xf5, 0xbf, 0x8, 0x74, 0x0, 0x0, 0xa, 0x37, 0x52, 0xa, 0x0, 0x98, 0x95, 0x1c, 0x0, 0x74, 0x3, 0x0, 0x6, 0x88, 0x1c, 0x0, 0x8, 0x74, 0x0, 0x0, 0xa, 0x3f, 0x9e, 0x8, 0x0, 0x56, 0x94, 0x1c, 0x0, 0xad, 0x6, 0x18, 0xc, 0xc6, 0xf, 0x20, 0x2, 0x88, 0x2, 0x0, 0x6, 0x89, 0x97, 0xc, 0x0, 0x7c, 0x2, 0x8, 0xc, 0xc9, 0x73, 0x1c, 0x0, 0x5b, 0x0, 0x19, 0xc, 0x7c, 0x0, 0x0, 0x6, 0xfa, 0x1b, 0xc, 0x0, 0xf7, 0x1, 0x10, 0x0, 0xa7, 0xf3, 0x1f, 0xc, 0x4b, 0x19, 0x10, 0xc, 0xfc, 0x0, 0x0, 0x6, 0x5a, 0x41, 0xc, 0x0, 0x9, 0x95, 0x1c, 0x0, 0x8e, 0x8, 0x18, 0xc, 0x28, 0xb, 0x26, 0x2, 0xe8, 0x2, 0x0, 0x6, 0x64, 0x34, 0x7b, 0xff, 0x5, 0xc, 0x0, 0x2, 0xaf, 0xb4, 0x68, 0xff, 0xde, 0x24, 0xf2, 0x1a, 0x5, 0x88, 0xf4, 0xc, 0xfd, 0x5c, 0xdd, 0x12, 0xc0, 0x49, 0xdf, 0x13, 0xb9, 0x82, 0xd0, 0x1d, 0x5a, 0x3a, 0xde, 0x13, 0xea, 0x8f, 0xd0, 0x1d, 0xc1, 0x2f, 0xdd, 0x13, 0x37, 0x86, 0xd0, 0x1d, 0xc0, 0x1f, 0xdc, 0x13, 0x2, 0xc4, 0xef, 0x1b, 0x64, 0x91, 0xed, 0x12, 0xa, 0x33, 0xfe, 0x1c, 0xdb, 0x8a, 0xe1, 0x1d, 0x40, 0x81, 0xe1, 0x19, 0x28, 0xfe, 0xe7, 0x8, 0xc8, 0x0, 0x0, 0x15, 0x1a, 0x46, 0xf0, 0xc, 0xa4, 0x0, 0x0, 0x18, 0x4a, 0x1, 0x30, 0x2, 0x4c, 0xc8, 0x20, 0x0, 0xf0, 0x41, 0x22, 0x1, 0xdd, 0x3d, 0x20, 0xe, 0x33, 0xfb, 0x2f, 0x13, 0x35, 0xb8, 0x12, 0x3, 0x1, 0xd3, 0x37, 0xf, 0xb8, 0xf, 0x35, 0x8, 0xf8, 0x0, 0x0, 0x9, 0x8, 0x34, 0xb, 0xf, 0xfc, 0x51, 0x1d, 0xf, 0xcc, 0x0, 0x0, 0x18, 0xaa, 0xde, 0x9e, 0xfe, 0x4e, 0x65, 0xff, 0x1a, 0xc0, 0x85, 0xf4, 0xc, 0xeb, 0x87, 0xdd, 0x12, 0xbf, 0x15, 0xda, 0x13, 0xf2, 0x82, 0xd0, 0x1d, 0xc3, 0x2c, 0xdb, 0x13, 0xbe, 0x80, 0xd0, 0x1d, 0xf0, 0x32, 0xdc, 0x13, 0x1c, 0x8d, 0xd0, 0x1d, 0x88, 0x49, 0xdd, 0x13, 0x6f, 0x26, 0xef, 0x1b, 0x54, 0x13, 0xed, 0x12, 0x1f, 0xad, 0xfe, 0x1c, 0xcc, 0x8d, 0xe1, 0x1d, 0xd8, 0x80, 0xe1, 0x19, 0x23, 0xf2, 0xe7, 0x8, 0x48, 0x1, 0x0, 0x15, 0x44, 0x44, 0xf0, 0xc, 0x24, 0x1, 0x0, 0x18, 0xb0, 0xb5, 0x60, 0x0, 0xa, 0xf, 0x66, 0xc, 0xf5, 0x40, 0x70, 0x0, 0xf5, 0x9, 0xf0, 0x2, 0xe5, 0x2, 0x50, 0x2, 0xc7, 0x3f, 0x40, 0x0, 0xdb, 0x2e, 0x45, 0xb, 0x4, 0xeb, 0x34, 0x1, 0xa2, 0x12, 0x41, 0x0, 0xac, 0x6, 0x45, 0xb, 0x77, 0x44, 0x43, 0x3, 0x80, 0x99, 0x55, 0xf, 0x86, 0x7, 0x51, 0x8, 0x84, 0x1, 0x0, 0x9, 0x5c, 0x1, 0x0, 0x18, 0xd4, 0xca, 0x30, 0x0, 0x36, 0xa, 0x35, 0xc, 0x9b, 0x41, 0x40, 0x0, 0x61, 0xa1, 0x57, 0x0, 0x3a, 0x40, 0x53, 0x1e, 0x73, 0x33, 0x30, 0x0, 0x94, 0xd, 0x34, 0xc, 0x98, 0x2b, 0x46, 0x0, 0x96, 0xa, 0x41, 0xc, 0x18, 0x56, 0x57, 0x0, 0xf0, 0x8, 0x55, 0xc, 0xf1, 0x46, 0x53, 0x1e, 0x3f, 0xa1, 0x30, 0x0, 0xb8, 0xf, 0x33, 0xc, 0xf0, 0x81, 0x46, 0x0, 0xb, 0xc, 0x41, 0xc, 0x7c, 0x35, 0x57, 0x0, 0x86, 0xc, 0x54, 0xc, 0xf2, 0x4e, 0x53, 0x1e, 0xda, 0x55, 0x30, 0x0, 0xb4, 0x6, 0x32, 0xc, 0x14, 0x5b, 0x40, 0x0, 0xc2, 0x3, 0x43, 0xc, 0x94, 0x6b, 0x57, 0x0, 0x7, 0xe, 0x53, 0xc, 0x81, 0x4e, 0x53, 0x1e, 0x50, 0xa, 0x30, 0x0, 0x61, 0xf, 0x31, 0xc, 0xe0, 0x1, 0x46, 0x0, 0x6a, 0xfe, 0x57, 0x0, 0x78, 0x4, 0x52, 0xc, 0x70, 0x42, 0x53, 0x1e, 0x22, 0xa5, 0x31, 0x0, 0x16, 0x35, 0x46, 0x0, 0x47, 0xd7, 0x57, 0x0, 0x97, 0x1, 0x51, 0xc, 0x5e, 0x4b, 0x53, 0x1e, 0x90, 0xce, 0xf7, 0xf, 0xcc, 0x1, 0xf2, 0x8, 0x28, 0x2, 0x0, 0x9, 0x58, 0x1, 0x0, 0x18, 0x22, 0x5c, 0x99, 0xfe, 0x34, 0xf0, 0xf, 0xf, 0x85, 0xeb, 0x2, 0xf, 0x7f, 0x82, 0xa, 0xf, 0x44, 0x7d, 0x0, 0xf, 0x9a, 0x88, 0x3, 0xf, 0x1a, 0xba, 0xf, 0xf, 0x8b, 0x89, 0xf, 0xf, 0xf4, 0x2d, 0x9, 0xf, 0x97, 0xa, 0x8, 0xf, 0x66, 0x55, 0xf, 0xf, 0xb3, 0x23, 0xc, 0xf, 0xfb, 0xd6, 0xb, 0xf, 0x33, 0x83, 0x9, 0xf, 0x3f, 0x94, 0xa, 0xf, 0xe3, 0xc1, 0x0, 0xf, 0x5f, 0xc8, 0x0, 0xf, 0x88, 0xd4, 0x7, 0xf, 0x23, 0x5c, 0x6, 0xf, 0x43, 0xde, 0xe, 0xf, 0x25, 0xfa, 0xaf, 0x62, 0x7c, 0x94, 0x2c, 0xcf, 0x8d, 0xa5, 0x9c, 0xbc, 0x67, 0x70, 0xde, 0xf4, 0xc6, 0x7e, 0x70, 0x1, 0xe2, 0xc, 0x70, 0x8, 0x98, 0x2, 0x0, 0xa, 0xe4, 0x2, 0x0, 0x18, 0xc5, 0xc, 0x60, 0x2, 0x9b, 0x85, 0x37, 0x0, 0x54, 0x65, 0x36, 0xb, 0x58, 0xd6, 0x30, 0xe, 0xe7, 0x50, 0x32, 0x13, 0x4b, 0xf6, 0x3f, 0x11, 0x9c, 0x4d, 0x46, 0x0, 0xa3, 0xa9, 0x42, 0xb, 0xa6, 0x6, 0x41, 0x11, 0x7e, 0x8a, 0x41, 0xb, 0xf5, 0xff, 0x54, 0x1, 0xe, 0x42, 0x53, 0x12, 0x65, 0x92, 0x45, 0x3, 0x57, 0xe2, 0x69, 0xf, 0xac, 0xe, 0x61, 0x8, 0x9c, 0x2, 0x0, 0xa, 0x7c, 0xca, 0x7, 0xf, 0x99, 0xf1, 0x25, 0xf, 0x88, 0x2, 0x0, 0x6, 0xfe, 0x32, 0x5b, 0xfe, 0xe8, 0x59, 0xfd, 0x1a, 0x2f, 0x83, 0xf4, 0xc, 0x27, 0xb8, 0xdd, 0x12, 0xb0, 0xa8, 0xda, 0x13, 0x36, 0x82, 0xd0, 0x1d, 0x6b, 0xbc, 0xdb, 0x13, 0xb5, 0x84, 0xd0, 0x1d, 0x28, 0xcb, 0xdc, 0x13, 0xde, 0x88, 0xd0, 0x1d, 0x39, 0xd2, 0xdd, 0x13, 0x37, 0x3d, 0xef, 0x1b, 0xe5, 0x59, 0xed, 0x12, 0xc, 0x7d, 0xfe, 0x1c, 0x5e, 0x89, 0xe1, 0x1d, 0x74, 0x8f, 0xe1, 0x19, 0xc9, 0xf6, 0xe7, 0x8, 0x34, 0x3, 0x0, 0x15, 0x16, 0x4a, 0xf0, 0xc, 0x10, 0x3, 0x0, 0x18, 0x35, 0x0, 0x30, 0x2, 0x9a, 0xc3, 0x40, 0x0, 0x98, 0x67, 0x51, 0x0, 0x12, 0x66, 0x43, 0xb, 0x7, 0x97, 0x53, 0xb, 0x70, 0x8e, 0x44, 0x1, 0xf3, 0xb, 0x55, 0x1, 0xbe, 0x45, 0x45, 0x7, 0x6c, 0x3, 0x0, 0xa, 0x2b, 0x9f, 0x36, 0xf, 0x53, 0xd1, 0x32, 0x7, 0x38, 0x3, 0x0, 0xa, 0xfc, 0x15, 0x0, 0x2, 0x70, 0x3, 0x0, 0x18, 0x77, 0xa, 0x0, 0x2, 0x4f, 0xc2, 0xa, 0xfe, 0x36, 0x1, 0x30, 0x2, 0x1, 0xc9, 0x20, 0x0, 0xa, 0x27, 0x23, 0xb, 0x5f, 0x56, 0x22, 0x1, 0x1c, 0x9, 0x20, 0x8, 0xf0, 0x3, 0x0, 0x9, 0xaf, 0x99, 0x23, 0x8, 0xa4, 0x3, 0x0, 0x15, 0x89, 0x9, 0x23, 0x8, 0xf8, 0x3, 0x0, 0x14, 0x7c, 0x8, 0x23, 0x17, 0xb8, 0x3, 0x0, 0x18, 0x68, 0x67, 0x26, 0x8, 0xf8, 0x3, 0x0, 0x15, 0x3e, 0x17, 0x26, 0x8, 0xf8, 0x3, 0x0, 0x14, 0xf9, 0x73, 0x25, 0x17, 0x3a, 0x47, 0x43, 0x0, 0x25, 0x22, 0x40, 0x11, 0xa4, 0x1c, 0x40, 0x8, 0xd4, 0x3, 0x0, 0x9, 0xb5, 0xc, 0x21, 0xe, 0x86, 0x93, 0x52, 0x0, 0xe8, 0x3, 0x0, 0x18, 0x29, 0xd6, 0x25, 0x12, 0x59, 0x80, 0x43, 0x0, 0x55, 0x19, 0x40, 0x19, 0xb6, 0x2, 0x41, 0xb, 0xc3, 0x39, 0x42, 0x3, 0xd7, 0x54, 0x35, 0xf, 0x78, 0x3, 0x0, 0x18, 0x1b, 0x13, 0x0, 0x2, 0xfc, 0x3, 0x0, 0x18, 0xdc, 0x0, 0x0, 0x2, 0xa0, 0xa1, 0x31, 0xfe]

Now we can run the disassembler again and get the code you’ll see below. In reality, this isn’t what happened. Indeed, I corrected and updated the disassembler multiples times before getting this output because I reversed incorrectly some instructions / mask / shift. Howerver, the upper code is the right one, you just have to replace the code variable by the new one.

Reversing the code inside the VM

So I put the code with some explanation. At this point, it’s useful to put some strategic breakpoints to see what is stored in R0, R1, Rc before each call to any function.

Main function :
0x0     SUB Rb, 0x1
0x4     CMP Rb, 0xff
0x8     JNE 0x74

0xc     MOV R0, Ra
0x10    MOV R1, Rc
0x14    CALL 0x374

0x18    CMP R0, 0x1
0x1c    JNE 0x74

0x20    MOV R0, R8
0x24    MOV R1, Rc
0x28    ADD R1, 0x80
0x2c    MOV R2, 0x0
0x30    CALL 0x288

0x34    MOV R0, Rc
0x38    ADD R0, 0x80
0x3c    MOV R1, Rc
0x40    ADD R1, 0x90
0x44    CALL 0x7c

0x48    MOV R0, Rc
0x4c    MOV R1, R0
0x50    ADD R1, 0xff
0x54    ADD R1, 0x1
0x58    CALL 0xfc

0x5c    MOV R0, Rc
0x60    MOV R1, Rc
0x64    ADD R1, 0x80
0x68    MOV R2, 0x60
0x6c    CALL 0x2e8

0x70    END
0x74    MOV R0, 0x0
0x78    END



Hash function:
0x7c    GETEIP Rf       // Start of deobfuscation block
0x80    ADD Rf, 0x48
0x84    XOR Rd, Rd
0x88    XOR Rd, 0xf4
0x8c    SHL Rd, 0x8
0x90    XOR Rd, 0xe3
0x94    SHL Rd, 0x8
0x98    XOR Rd, 0xd2
0x9c    SHL Rd, 0x8
0xa0    XOR Rd, 0xc1
0xa4    BSWAP Re, [Rf]
0xa8    XOR Re, Rd
0xac    BSWAP [Rf], Re
0xb0    SHL Re, 0x18
0xb4    SHR Re, 0x18
0xb8    CMP Re, 0x7f
0xbc    JG 0xc8
0xc0    ADD Rf, 0x4
0xc4    JUMP 0xa4       // End of deobfuscation block

0xc8    MOV R3, 0x0     // R3 stores an index
0xcc    MOV R2, R0
0xd0    MOV R2, [R2]    // R2 stores the char of username at index R3 -> will be 'x' in comments
0xd4    IMUL R2, 0x3    // r = x * 3
0xd8    XOR R2, 0xff    // r = (x * 3) ^ 0xff
0xdc    MOV [R1], R2    // r is stored at 0x5c0 + the index
0xe0    INC R3
0xe4    CMP R3, 0x50    // We do this for 0x50 bytes long
0xe8    JE 0xf8
0xec    INC R0
0xf0    INC R1
0xf4    JUMP 0xcc
0xf8    RET



Serial decrypt function :
0xfc    GETEIP Rf       // Start of deobfuscation block
0x100   ADD Rf, 0x48
0x104   XOR Rd, Rd
0x108   XOR Rd, 0xa1
0x10c   SHL Rd, 0x8
0x110   XOR Rd, 0xb2
0x114   SHL Rd, 0x8
0x118   XOR Rd, 0xc3
0x11c   SHL Rd, 0x8
0x120   XOR Rd, 0xd4
0x124   BSWAP Re, [Rf]
0x128   XOR Re, Rd
0x12c   BSWAP [Rf], Re
0x130   SHL Re, 0x18
0x134   SHR Re, 0x18
0x138   CMP Re, 0x7f
0x13c   JG 0x148
0x140   ADD Rf, 0x4
0x144   JUMP 0x124      // End of deobfuscation block

0x148   MOV R6, R0      // This block copies 16 bytes from 0x530 to 0x630 at each iteration
0x14c   ADD R6, 0x60
0x150   MOV R7, R0
0x154   MOV Rf, 0x0
0x158   MOV R5, 0x0
0x15c   MOV R4, R0
0x160   ADD R4, R5
0x164   MOV R3, [R4]
0x168   MOV R4, R1
0x16c   ADD R4, R5
0x170   MOV [R4], R3
0x174   INC R5
0x178   CMP R5, 0x10
0x17c   JE 0x184
0x180   JUMP 0x15c

0x184   MOV R3, R0      // One round of AES. R3, R4 and R5 are the addresses of the source, the key and the destination
0x188   ADD R3, 0x50    // All of them are 16 bytes long. The thing is the destination is sometines at the same offset of the key
0x18c   MOV R4, R0      // We will see that when we'll create the key generator
0x190   MOV R5, R7      // The following blocks do the same with different offset
0x194   AESENC [R3], [R4], [R5]

0x198   MOV R3, R0
0x19c   ADD R3, 0x40
0x1a0   MOV R4, R6
0x1a4   ADD R4, 0x10
0x1a8   MOV R5, R7
0x1ac   ADD R5, 0x50
0x1b0   AESENC [R3], [R4], [R5]

0x1b4   MOV R3, R0
0x1b8   ADD R3, 0x30
0x1bc   MOV R4, R6
0x1c0   ADD R4, 0x10
0x1c4   MOV R5, R7
0x1c8   ADD R5, 0x40
0x1cc   AESENC [R3], [R4], [R5]

0x1d0   MOV R3, R0
0x1d4   ADD R3, 0x20
0x1d8   MOV R4, R0
0x1dc   ADD R4, 0x30
0x1e0   MOV R5, R7
0x1e4   ADD R5, 0x30
0x1e8   AESENC [R3], [R4], [R5]

0x1ec   MOV R3, R0
0x1f0   ADD R3, 0x10
0x1f4   MOV R4, R6
0x1f8   MOV R5, R7
0x1fc   ADD R5, 0x20
0x200   AESENC [R3], [R4], [R5]

0x204   MOV R3, R1
0x208   MOV R4, R6
0x20c   MOV R5, R7
0x210   ADD R5, 0x10
0x214   AESENC [R3], [R4], [R5]

0x218   INC Rf
0x21c   CMP Rf, 0x20
0x220   JE 0x228
0x224   JUMP 0x158  // We repeat the copy and aes blocks 32 times

0x228   RET



0x22c   INC R0      // This block is useless because nothing jumps on it, so let's skip it
0x230   INC R0
0x234   INC R0
0x238   INC R0
0x23c   INC R0
0x240   INC R0
0x244   INC R0
0x248   INC R0
0x24c   INC R0
0x250   INC R0
0x254   INC R0
0x258   INC R0
0x25c   INC R0
0x260   INC R0
0x264   INC R0
0x268   INC R0
0x26c   INC R0
0x270   INC R0
0x274   INC R0
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...
Oh noooo, we found a bad xor...



Username derivation function :
0x288   MOV R7, [R0]    // This is a recursive function which iterate over username characters (check out 0x2e0)
0x28c   CMP R7, 0x0     // Some pseudo code is better than an explantion :
0x290   JNE 0x298       // for i in range(len(username)):
0x294   JUMP 0x2e4      //      for j in range(16):
0x298   MOV R6, 0x0     //          r = (((username[i] + j) * 0xd) ^ 0x25) % 0xff
0x29c   MOV R3, R7
0x2a0   ADD R3, R6
0x2a4   IMUL R3, 0xd
0x2a8   XOR R3, 0x25
0x2ac   MOD R3, 0xff
0x2b0   MOV R4, R6
0x2b4   ADD R4, R2
0x2b8   MOD R4, 0x10
0x2bc   ADD R4, R1
0x2c0   MOV R5, [R4]
0x2c4   XOR R5, R3
0x2c8   MOV [R4], R5
0x2cc   INC R6
0x2d0   CMP R6, 0x10
0x2d4   JNE 0x29c
0x2d8   INC R0
0x2dc   INC R2
0x2e0   CALL 0x288
0x2e4   RET



Comparison between decrypted serial and username derivation / hash function :
0x2e8   GETEIP Rf       // Start of deobfuscation block
0x2ec   ADD Rf, 0x48
0x2f0   XOR Rd, Rd
0x2f4   XOR Rd, 0xaa
0x2f8   SHL Rd, 0x8
0x2fc   XOR Rd, 0xbb
0x300   SHL Rd, 0x8
0x304   XOR Rd, 0xcc
0x308   SHL Rd, 0x8
0x30c   XOR Rd, 0xdd
0x310   BSWAP Re, [Rf]
0x314   XOR Re, Rd
0x318   BSWAP [Rf], Re
0x31c   SHL Re, 0x18
0x320   SHR Re, 0x18
0x324   CMP Re, 0x7f
0x328   JG 0x334
0x32c   ADD Rf, 0x4
0x330   JUMP 0x310      // End of deobfuscation block

0x334   MOV R3, 0x0     // This block is just iterating over the two computed values to check if everything is equal
0x338   MOV R4, R0      // Then it return 1 or 0 (which will then print valid serial or not in the main function)
0x33c   MOV R5, R1      // R3 is the index and R4 / R5 are the 2 pointers of the data being compared
0x340   ADD R4, R3      // Using GDB, we know that these 2 pointers are the derivated / hashed username and the encrypted serial
0x344   ADD R5, R3
0x348   MOV R4, [R4]
0x34c   MOV R5, [R5]
0x350   CMP R5, R4
0x354   JNE 0x36c
0x358   INC R3
0x35c   CMP R2, R3
0x360   JNE 0x338
0x364   MOV R0, 0x1
0x368   JUMP 0x370
0x36c   MOV R0, 0x0
0x370   RET


Check serial format and atoi function :
0x374   MOV R3, 0x0     // We previously analysed this function
0x378   MOV R2, R0
0x37c   ADD R2, R3
0x380   MOV R2, [R2]
0x384   CMP R2, 0x0
0x388   JE 0x3f0
0x38c   CMP R2, 0x39
0x390   JG 0x3a4
0x394   CMP R2, 0x30
0x398   JS 0x3f8
0x39c   SUB R2, 0x30
0x3a0   JUMP 0x3b8
0x3a4   CMP R2, 0x66
0x3a8   JG 0x3f8
0x3ac   CMP R2, 0x61
0x3b0   JS 0x3f8
0x3b4   SUB R2, 0x57
0x3b8   MOV R4, R3
0x3bc   MOD R4, 0x2
0x3c0   CMP R4, 0x1
0x3c4   JE 0x3d4
0x3c8   IMUL R2, 0x10
0x3cc   MOV R5, R2
0x3d0   JUMP 0x3e8
0x3d4   XOR R2, R5
0x3d8   MOV R4, R3
0x3dc   SHR R4, 0x1
0x3e0   ADD R4, R1
0x3e4   MOV [R4], R2
0x3e8   INC R3
0x3ec   JUMP 0x378
0x3f0   MOV R0, 0x1
0x3f4   JUMP 0x3fc
0x3f8   MOV R0, 0x0
0x3fc   RET

So here is a little summary : * The VM checks the serial input (256 hexadecimal characters) and convert it in memory bytes (like atoi) * The VM derivates the username * The VM computes a “hash” from the username derivate * The VM encrypts the serial using AESENC (one round of aes on 16 bytes) * The VM checks if the serial after encryption is equal to the username derivation and computed hash

At this point, we should be pretty confident.

Yes Meme

Creating the keygen

As the challenge description says, we should create a key generator able to compute a lot of serial asked by a service (via netcat) to get the flag.

So firstly, lets implement the username derivation and hashing function. I already gave you the pseudo code of both function.

Let’s just pythonyze that :

def username_derivation(name):
    data = [0] * 16
    for i in range(len(name)):
        for j in range(16):
            r = (((ord(name[i]) + j) * 0xd) ^ 0x25) % 0xff
            data[(i + j) % 16] ^= r
    return data


def custom_hash(derivated_username):
    data = derivated_username + [0] * 0x50
    for i in range(0x50):
        data[i + 16] = ((data[i] * 3) ^ 0xff) & 0xff
    return data

Last but not least, we need to reverse the encryption function. To create a valid serial, we can start from the derivated / hashed username and use AESDEC. Indeed, if we wantp = AESENC(x, key), then we just have to computex = AESDEC(p, key). We can use the key we want for this computation. Then we store in the data the source computed (x) and the key.
NB : Now we understand why we can generate multiples serial for each username.

However, we need to solve three issues : * Find a python implementation of AESENC instruction * So I put some breakpoints before and after an AESENC instruction and dumped the values * Then I tried a lot of libraries to get the same output * I found this working code : https://github.com/bozhu/AES-Python * The AESENC instructions sometimes write the output where the key is stored * Only applying instructions in a reverse order won’t be enough * In this case, we want to save the destination because it’ll be overwritten * The AESENC instructions sometimes uses a source which is used as a key for another AESENC * Only applying instructions in a reverse order won’t be enough * We should compute the source first from the destination we know using a random key * The we use this source as the key for the other AESENC

As you can probably see, this part isn’t simple to explain. You should take a look at the encryption function and try to start from the end and find the input you need to get the output you want.

Here’s my implementation for this function. I kept the virtual offsets to avoid mistakes :

def update_data(data, pos, s):
    for i in range(len(s)):
        data[pos + i] = s[i]
    return data


def get_block(data, pos):
    return data[pos:pos + 16]


def aesdec(dest, key):
    aes = crypto.AES()
    aes.key = key
    ciphertext = ''.join([chr(x) for x in dest])
    plaintext = aes.AESDEC(ciphertext, ''.join([chr(x) for x in key]))
    return [ord(i) for i in plaintext]


def aes_block(data, pos_src, pos_key, pos_dest, key, new_dest=None):
    dest = new_dest if new_dest != None else get_block(data, pos_dest)
    source = aesdec(dest, key)
    update_data(data, pos_src, source)
    update_data(data, pos_key, key)
    return source


def solve(name, key):
    data = custom_hash(username_derivation(name))
    data = [0] * 0x530 + data + [0] * 1000

    Rf = 0
    key = [ord(i) for i in key]
    while Rf != 0x20:
        dest1 = get_block(data, 0x530)
        special_key1 = aes_block(data, 0x630, 0x590, 0x540, key)
        aes_block(data, 0x540, 0x590, 0x550, key)
        dest2 = get_block(data, 0x560)
        special_key2 = aes_block(data, 0x560, 0x5a0, 0x570, key)
        aes_block(data, 0x550, 0x560, 0x560, special_key2, dest2)
        aes_block(data, 0x570, 0x5a0, 0x580, key)
        aes_block(data, 0x580, 0x530, 0x530, special_key1, dest1)

        Rf += 1

    return ''.join(['0x%02x'[2:] % i for i in data[0x530:0x530 + 128]])

So if we try to generate a serial for username “Knowledge”, we get :

➜  keykoolol python solve.py Knowledge
b6371850fe8903b70adf21f40b6a3fbc6a2fef27c4b27508845c8222c1aea865d0c01eb33d15d342433feb7906ce7598dfe480dd03605691733497094af87536358a18ba2acfd8c69faf2ba6786d1749ed3f1a3aa7ad3cc64552d98b9be16c473432343234323432343234323432343234323432343234323432343234323432

➜  keykoolol ./keykoolol   
[+] Username: Knowledge
[+] Serial:   b6371850fe8903b70adf21f40b6a3fbc6a2fef27c4b27508845c8222c1aea865d0c01eb33d15d342433feb7906ce7598dfe480dd03605691733497094af87536358a18ba2acfd8c69faf2ba6786d1749ed3f1a3aa7ad3cc64552d98b9be16c473432343234323432343234323432343234323432343234323432343234323432
[>] Valid serial!
[>] Now connect to the remote server and generate serials for the given usernames.

Getting the flag

Now this part is trivial. We just need to establish a communication with the server and generate two serials for some usernames. To do that, we’ll use two defaults keys : “4242424242424242” and “4242424242424243”

Here is the final script :

import aes as crypto
from netcat import Netcat


def username_derivation(name):
    data = [0] * 16
    for i in range(len(name)):
        for j in range(16):
            r = (((ord(name[i]) + j) * 0xd) ^ 0x25) % 0xff
            data[(i + j) % 16] ^= r
    return data


def custom_hash(derivated_username):
    data = derivated_username + [0] * 0x50
    for i in range(0x50):
        data[i + 16] = ((data[i] * 3) ^ 0xff) & 0xff
    return data


def update_data(data, pos, s):
    for i in range(len(s)):
        data[pos + i] = s[i]
    return data


def get_block(data, pos):
    return data[pos:pos + 16]


def aesdec(dest, key):
    aes = crypto.AES()
    aes.key = key
    ciphertext = ''.join([chr(x) for x in dest])
    plaintext = aes.AESDEC(ciphertext, ''.join([chr(x) for x in key]))
    return [ord(i) for i in plaintext]


def aes_block(data, pos_src, pos_key, pos_dest, key, new_dest=None):
    dest = new_dest if new_dest != None else get_block(data, pos_dest)
    source = aesdec(dest, key)
    update_data(data, pos_src, source)
    update_data(data, pos_key, key)
    return source


def solve(name, key):
    data = custom_hash(username_derivation(name))
    data = [0] * 0x530 + data + [0] * 1000

    Rf = 0
    key = [ord(i) for i in key]
    while Rf != 0x20:
        dest1 = get_block(data, 0x530)
        special_key1 = aes_block(data, 0x630, 0x590, 0x540, key)
        aes_block(data, 0x540, 0x590, 0x550, key)
        dest2 = get_block(data, 0x560)
        special_key2 = aes_block(data, 0x560, 0x5a0, 0x570, key)
        aes_block(data, 0x550, 0x560, 0x560, special_key2, dest2)
        aes_block(data, 0x570, 0x5a0, 0x580, key)
        aes_block(data, 0x580, 0x530, 0x530, special_key1, dest1)

        Rf += 1

    return ''.join(['0x%02x'[2:] % i for i in data[0x530:0x530 + 128]])


def get_flag():
    nc = Netcat('challenges2.france-cybersecurity-challenge.fr', 3000)
    a = ""
    while "FCSC" not in a:
        a = nc.read(3000)
        if a == ">>> ":
            a = nc.read(3000)
        name = a.split(': ')[1].split('\n')[0]
        r1 = solve(name, "4242424242424242")
        r2 = solve(name, "4242424242424243")
        print(a + r1)
        nc.write(r1 + '\n')
        print(nc.read_until(">>> ") + r2)
        nc.write(r2 + '\n')
    print("Flag !!!! %s" % a)


if __name__ == "__main__":
    get_flag()

And here is the output :

Give me two valid serials for username: Joseph Jones
>>> 4a8ddaae0531a6b400a6a3611584a9d367d64c583bfd2f043c4ef932aa5044fbab28266129231967c7c2135d4ad0fb60404e31acd46aedf05fb1955bd73fd440b45c7307eb6ba38790748b777ed9fc1048096a3f256ea8ef7546db8ef410ef8d3432343234323432343234323432343234323432343234323432343234323432
>>> 69865ca8311a3df4f96aedd66dbe14b4963146422dd1d66a3380cd56427bdc420c2ea54c60946268345012db3ef866bfb790132d15260d60945cd47440d922af6cf6b73ec2f03cabeb800f0105003690be44c71f6f1444e7dee5370bc4179d493432343234323432343234323432343334323432343234323432343234323433
Give me two valid serials for username: Theresa Smith
>>> 46a2bca0111dddfbf51cbe9dc0854d1d1a994afb0cbbaffa10047b3e9db304bf2f235563aac39d6845d7b4bc9e5edda12204bfd4fc66e11009dcc796071070fecbb0e7ce8fffd029a73eadbcaf9e6b6707c22ba0d784a2b2ea089d54ec515fff3432343234323432343234323432343234323432343234323432343234323432
>>> 0ad9785b2fcea05775bbac87bf12f2eee04ce86b68bcd8f22570ce94108ebdaec2dcd158e572a649af044eb3fad5ce08e92ea0cc67543c70021e78e956d7de5e9a3867aed3c246fcb6470c97beb502751cbaba519f2a79db63c213c6a84021303432343234323432343234323432343334323432343234323432343234323433
Give me two valid serials for username: Candy Ferraraccio
>>> eaf6a2e371e7e9936c78e3db196b7697f233fe97ba6960e20195be12c4bd72acf6d367ca1000b103fd96b3925245692af1ba51ce3e194cae670a62b1de956c43bff84e67f2529203f0b4a756610455077b3c5a9b091b26bd6342b00c9937df643432343234323432343234323432343234323432343234323432343234323432
>>> d89d78167d55822cb49f306ff19c9ae9934bb718a6f6950e3aa72e4e1982e85270dac1faf3037eeba845eebe3deaf7a94b7adac3be443425a3d7d7fd1c998685df78204ca72f8cd3e362ee68fc04d7bf94e10a1ee160a6193b2f6ee5a4663e063432343234323432343234323432343334323432343234323432343234323433
[...]
Give me two valid serials for username: mJkD91JUwl8xOSySiArYUa
>>> 70adc849b53c6c9206f9e79adc454eea8160d273c6868f28edf9f632d449c8761093ebc6a7e1a0ad81ec1e11c43bb1acef39ea8b966d79260efd5ceb6fe0a25d3434a7a902b56b2f6a2944792f1fe94307ec028d9ae7d163cd20c0ae22e8e11c3432343234323432343234323432343234323432343234323432343234323432
>>> 04985cf0045deb4e13b2a1fd517afb5493f6684b7f35d57100f124ef20327edac2d962333f2892117c5ea579192d91c4c69289b526a13b856b0cf35fad72ee1122c69d7b999ed1acd6ab419f596036a0a2b3fb3b02f6aba8a14b4b42e5a26d433432343234323432343234323432343334323432343234323432343234323433
Give me two valid serials for username: bCBv1bMJfPhqx336klJ
>>> d2d0130da63b7ba8e51951a4a756520d779ad6ce26f676d292d1b69ab4dc310e5cceefcbefd398a6fbcd313c123aadade8452e01f2d43f66b49837e729bce132cc934b978dc551629a43bee6314754af31006817ad4022db263874e583b491c13432343234323432343234323432343234323432343234323432343234323432
>>> f41a55164bf1a15b7b9295189b614e5224916dcd2cc6642f78cb7d86e5918b6ecf71c6a550083e66b76967dc01b684bcbea10413af7c58a6572669afcf62998da28365daf4c2951c0845192d66437fe3149a50816b20c47dd3ffec473389ccce3432343234323432343234323432343334323432343234323432343234323433
Well done! Here is the flag: FCSC{38b1135bc705b2f1464da07f3052611a91f26a957647a24ceb9607646a19c2dc}

Won Meme

Conclusion

I learned a lot through this challenge which I found amazing !
ELF VM are a good way to train your debugger / script automation skills.
I hope that this write up will help you to reverse ELF VM challenges by giving you some methodology. But also understanding how opcodes, instructions and registers are simulated.