WRITEUP – DEFCAMP2015 – reversing 200

Reverse 200 shows a file r200:

r200: ELF 64-bit LSB  executable, x86-64, version 1 (SYSV), dynamically linked
(uses shared libs), for GNU/Linux 2.6.24,
BuildID[sha1]=22e68980e521b43c90688ed0693df78150b10211, stripped

When executing this file it also wants some password which is presumably the flag. After some file exploring the function where the password is validated is found and decompiled:

[...]
for ( i = 0; i <= 5; ++i )
  {
    v5 = qword_601080;
    v4 = 0;
    while ( v5 )
    {
      if ( *(_BYTE *)(v5 + 4) == *(_BYTE *)(i + password) )
      {
        v4 = *(_DWORD *)v5;
        break;
      }
      v5 = *(_QWORD *)(v5 + 8);
    }
    *((_DWORD *)&v6 + i) = v4;
  }
  for ( j = 0; j <= 5; ++j )
  {
    if ( *((_DWORD *)&v6 + j) != *(&v9 + j) )
      return 1LL;
  }
  return 0LL;
[...]

The for loop suggests, that the password has 6 chars. It iterates (among other things) through the password array. The pointer v5 points to a qword which points to some array which was generated in the beginning of the program.

The function then does the following: It looks up what v5 points to (the array) and adds 4 byte to it and reads this byte. It then compares this byte to the current char in the password. If the character matches, it stores the byte which v5 originally points to in v4 and then in v6 which is another array. It does this with every char in the password. After this another loop compares the array v6 with v9 which is a fixed array of 6 byte, {5 2 7 2 5 6}.

This procedure will be clearer if we look at the array in the memory which is adressed with v5:

01 00 00 00 6E 00 00 00  00 00 00 00 00 00 00 00

02 00 00 00 6F 00 00 00  10 50 58 02 00 00 00 00

03 00 00 00 70 00 00 00  30 50 58 02 00 00 00 00

04 00 00 00 71 00 00 00  50 50 58 02 00 00 00 00

05 00 00 00 72 00 00 00  70 50 58 02 00 00 00 00

06 00 00 00 73 00 00 00  90 50 58 02 00 00 00 00

07 00 00 00 74 00 00 00  B0 50 58 02 00 00 00 00

08 00 00 00 75 00 00 00  D0 50 58 02 00 00 00 00

09 00 00 00 76 00 00 00  F0 50 58 02 00 00 00 00

0A 00 00 00 77 00 00 00  10 51 58 02 00 00 00 00

In the password checker function v5 always points to one char in the first column of the memory dump (so like 01, 02, 03…). If you add 4 to this pointer it points to the fifth column (like 6E, 6F, 70…). It then generates an array of bytes of the first column with the password characters which are looked up in the fifth row.

So if the current password character is 6E, it appends 01 to this array. If it is 73, it appends 06 to the array. Then this generated array is compared with the hardcoded byte array {5 2 7 2 5 6}. So we just have to look in the memory dump, which character we need. {5 2 7 2 5 6} is then {72 6F 74 6F 72 73} which is in ASCII symbols the password: rotors.