emh's blog

DiceCTF HOPE - drive

drive is a virtual machine challenge I wrote for DiceCTF @ HOPE (challenges are here). This challenge involves writing bytecode for a small VM. It also involved figuring out how rust traits are implemented for objects. The source and solution can be found here.

The Virtual Machine

The VM in the challenge consisted of a state which looked like this:

pub struct State<'a> {
    pub ip: usize,
    pub mem: [u8; 4],
    pub program: Vec<u8>,
    pub devices: Vec<Box<dyn Device + 'a>>,

I decided on this design as I wanted to incorporate trait implementations within the reversing challenge. I was also trying to keep the virtual machine as simple as possible otherwise, so I only provided registers and devices.


The main functionality of the virtual machine were the devices. Each device could either send a byte or read in a byte, as seen by the trait.

pub trait Device {
    fn get_byte(&mut self) -> u8;
    fn send_byte(&mut self, byte: u8);

The devices could be used with the Send and Get opcodes. The Send opcode sent a byte from a register to a device based on its index. The Get opcode would retrieve a byte from a device and place it in a register.

There were four main devices attached to the virtual machine:

  • A reader
  • A writer
  • An arithmetic unit
  • A stack

The reader implemented get_byte to read from a Cursor, and the writer implemented send_byte to write to another Cursor. These were checked at the end to see if the program had the right output.

The stack was used for large memory. The get_byte acted as a pop and the send_byte acted as a push.

The arithmetic unit needed 3 bytes sent — the first value, the second value, and the operation.

impl Device for Arithmetic {
    fn get_byte(&mut self) -> u8 {
        match self.buffer[2] {
            0 => self.buffer[0].wrapping_mul(self.buffer[1]),
            1 => self.buffer[0].wrapping_add(self.buffer[1]),
            2 => self.buffer[0].wrapping_sub(self.buffer[1]),
            3 => self.buffer[0] ^ self.buffer[1],
            4 => self.buffer[0].wrapping_div(self.buffer[1]),
            5 => self.buffer[0] % self.buffer[1],
            _ => 0,
    fn send_byte(&mut self, byte: u8) {
        self.buffer[self.index] = byte;
        self.index += 1;
        self.index %= 3;


The drive program first loaded a program from input and ran it on a random input. It then checked if the output was the same as the encrypt function.

fn encrypt(block: [u8; 4]) -> [u8; 4] {
    let mut block = block;
    let mut b = block.clone();
    let i = block[0].wrapping_add(block[3]);
    b[1] ^= i;
    let i = block[1].wrapping_add(block[0]);
    b[2] ^= i;
    let i = block[2].wrapping_add(block[1]);
    b[3] ^= i;
    let i = block[3].wrapping_add(block[2]);
    b[0] ^= i;
    block = b;
    for _ in 0..block[3] {
        let mul = block[2].wrapping_mul(block[0]);
        let sum = mul.wrapping_add(block[1]);
        block[0] = sum % block[3];

Originally I only had add, sub and xor in the arithmetic unit. Therefore, to implement the encryption, users would have had to implement multiplication and the modulus function, however implementing the encryption with only those three operations is more annoying than interesting and not worth the effort (thank you asphyxia for playtesting :D).

My solve still has the implementation of mul and mod within the bytecode since I was too lazy to change it. My solution and comments are here.

Finally, I also originally thought I would make this a golfing challenge, however the rust rev was already quite difficult and I did not want to add another source of annoyance.