The RSoC is over

October is over and we delayed a bit the end of the RSoC in order to get everything done for the release, and it seems that little happened as planned:

The RSoC advertisement was a great opportunity to get new developers interested in contributing to the project, some of them even without joining the RSoC took some points that weren't requested and delivered them! That's pretty cool, because our two selected students disapeared during the summer.

Abandonned tasks

We're a bit sad that the sdbization task wasn't completed, since this would have been a huge improvement, both in term of performances and cool new features, like collaboration. Also, no one gave some love to the web interface, so it will still be an ugly PoC a bit longer.

Complete tasks

Most of the tasks weren't completed on the test cases and documentation sides, but we hope to get things done in time for the next release, the 0.9.9.

We'd like to congratulate our 3 volunteers for their hard work and dedication. They were supposed to be the 'free' team, but since our two official students left, we'd like to pay them as if they were selected: We managed to get $2100 in budget, so each of them will receive $700.

Structures support

Skia did great to implement structures support, a bit à la 010Editor. structure Now with r2 -nn it is possible to analyze the file header structs using the pf., pxa and other related commands.

$ r2 -nn /path/to/bin

FLIRT and YARA support

jfrankowski did a great RSoC, and improved yara support in radare2, and also added the FLIRT one! Currently, radare2 is only able to use existing signatures, but feel free to drop us a patch to build our own, using radare!

Yara3 support is almost there, but we prefer to release for a welltested yara2 version and push the upgrade in 0.9.9.

PDB support

inisider implemented a standalone library to handle PDB files, and integrated it into radare2. You can now analyse/debug PE with much more ease.

> .!rabin2 -rP test.pdb

ESIL

Nobody took that task from the RSoC, but pancake and condret raised the bar and managed to get a working implementation of ESIL, mainly tested on gameboy, brainfuck, x86 and mips. Also, pancake made a new search command to use esil expressions to perform complex conditional carving useful for forensics and data analysis.

Conclusion

We enjoyed holding this first RSoC. We made some mikstakes, and learned a lot of things. If we won't get selected for a GSoC once again, be sure that the second RSoC will be better than the first one.

We'd like to thank everyone who attended, participated, donated, supported, helped, advertised, improved and tested things. Without you, the RSoC wouldn't have been so productive!

Radare 0.9.8

Eight months ago, radare2 0.9.7 was released; today, we're happy to announce radare 0.9.8!

In details and numbers:

Downloads:

Since you surely can read the 2500 commits by yourself (or the detailed changelog), we're just going to highlight some cool new features and improvements:

New platforms!

r2-0.9.8 now builds for Android L, which enables two new architectures:

  • Android mips64
  • Android aarch64

android screenshot

Also, we are now supporting crosscompilation for iOS devices. No need to use jailbroken devices as main builders. The sys/ios-sdk.sh script supports the very latest XCode from Yosemite.

We are also happy to get r2 running on Haiku and it's great that it's now finally possible to compile it natively on Windows7 using mingw32. Also xvilka helped to catch and fix the remaining bits to get r2 compilable with Cygwin too.

Javascript is also another supported target platform to compile r2 to, SDB and Rax2 have been already pushed into the FirefoxOS marketplace.

New architectures

We're now supporting:

  • Java, thanks to dso. Ok, it's not new, but it was greatly enhanced!
  • cr16 and msp430, thanks to montekki
  • Nintendo DS, thank to aortega
  • tms320(c55x, c55x+, more is coming), thanks to milabs
  • m68k - fixed some segfaults from the imported disassembler and use it to reverse engineer some Street Fighter roms.
  • Spc700 - (supernes sound-chip)
  • Propeller
  • Every arch supported by capstone that we didn't already had, like SystemZ

Better Debugger

debugger

Also, we have reworked on w32 to make the native debugger work again and support hardware breakpoint registers on x86. (thanks Skuater!)

The tracepoints have been implemented, Now each breakpoint can be be toggled to become active/disabled or be used as a tracepoint or breakpoint.

Lot of work have been done to make the GDB remote work and support X86, MIPS and ARM for qemu and gdb-server targets.

Bear in mind that GDB is a totally fucked up protocol with lot of incompatibilities (yes, more than ELF), and each new target depends on several sub-features to be implemented or handled differently in the client side. defragger did a great work testing and implementing them, but there are still more targets to be supported.

SDB integration

SDB is our key-value database system, that we're integrating into radare2. This simplifies the code, and increases the performances in many areas.

SDB

You can play around with SDB using the k commands.

Capstone

We're now using capstone to disassemble several architectures. This simplifies maintainance on our side, and bring you complete support for several architectures and their extensions (hello AVX, FPU, ...).

If you are having disasm or analysis problems, switch to the old udis86 or GNU plugins to compare and send us your feedback. We'll fix them as soon as possible.

Quality

Thanks to coverity, we fixed a lot of bugs. We're also using ASAN, valgrind, Jenkins and a complete testsuite to improves r2 efficiency and reliability.

We also cleaned up lot of warnings and memory corruption issues by using different fuzzers (radamsa, melkor, nightmare, zzuf, …), to make the RBin plugins much more reliable on fucked up targets. We know there are still some corner cases to fix, and we will address them in the next bugfix release 0.9.9.

The R_IO layer have been heavily tested, reviewed and partly rewritten to be more stable (thanks condret!)

DWARF

Montekki spent some time to add support for DWARF! Now you can debug your favourite ELF binary (almost) without reading assembly!

ESIL

ESIL stands for Evaluable Strings Intermedate Language. It aims to describe a Forth-like representation for every opcode; the goal within radare being emulation, in order to improve the analysis.

With ESIL it is now possible to:

  • Debug Brainfuck applications
  • Partially emulate Gameboy, X86, ARM and MIPS.
  • Define software watchpoint conditions.
  • Spawn complex search expressions
  • Analyze code without messing with arch-specific and low level struct issues.

We will continue to enhance the ESIL support to emulate code in a better fashion, and we plan to use it for injecting code on child processes and more.

Exploitation

pop pop ret - Analyze an specific memory address with ai - Take references from registers and memory with drr/pxr (à la PEDA) - ROP gadget searcher (/R) - Mitigation detections (in i) - De Bruijn patterns - search.in eval variable can be used to specify where to search for patterns (for example, inside heaps or executable pages).

ASCII art graphs

Spawned by default when pressing V inside the Visual mode when placing the cursor on an already analized function. For callgraphs use VVV.

graph

Graphviz, JSON and HTML5 interactive graphs are still supported, but not used by default.

In addition, some basic debugging can be done inside the ascii graph graphs.

The visual mode has been enhanced to better handle cursor, enable feedback mode (useful for screencasts), navigating references with x and X keys, and support the mouse wheel for scrolling (which can be disabled with the scr.wheel eval variable).

RSoC

Since we've been rejected from the GSoC, we've launched our own Summer of Code. Which turned to be kinda profitable from the point of view of new contributors, feedback, testing and getting new features ready for the masses. Congratulations to everyone who attended and participated!

We will soon release another post explaining all the details of the finalized RSoC.

Structures support

Skia did great to implement structures support, a bit à la 010Editor. structure

Now with r2 -nn it is possible to analyze the file header structs using the pf., pxa and other related commands.

$ r2 -nn /path/to/bin

FLIRT and YARA support

jfrankowski did a great RSoC, and improved yara support in radare2, and also added the FLIRT one! Currently, radare2 is only able to use existing signatures, but feel free to drop us a patch to build our own, using radare!

Yara3 support is almost there, but we prefer to release for a welltested yara2 version and push the upgrade in 0.9.9.

PDB support

inisider implemented a standalone library to handle PDB files, and integrated it into radare2. You can now analyse/debug PE with much more ease.

> .!rabin2 -rP test.pdb

Integration and bindings

The IDA to r2 Python script has been also updated.

Duktape have been included to support inline Javascript scripting, which turns to enable portable programs that can run in NodeJS, the r2 shell or the web browser without modifications.

We have also integrated support for GZip files which can be now slurped or loaded without depending on external programs (as we already did with ZIP).

Documentation and PR

Forensics

libmagic support has been enhanced to the point that we can already replace some features of binwalk to analyze router firmware images.

Also, several commands for searching with asteroids have been added, in order to make data analysis much better.

Also, commands like rabin2 -zzz allow to dump all strings from a file without storing the results in memory which works great for dumping harddisks or memory snapshots.

R2Pipe and JSON

Since this release you now can use a very fast kind of bindings - r2pipe, which is actually piping output of the commands inside the chosen language wrappers instead of directly calling C function from the library. Also it is very easy to use, because if you know radare2 commands.

A lot of commands spit JSON when typing a j at the end of the command. This is very useful for exporting information from r2 or processing it from the webui.

In addition a bunch of new bindings have been created under the name of r2pipe which use r2 with -0 flag to send commands and receive output asyncrunously.

https://www.npmjs.org/package/r2pipe

That solution tends to be far more realiable than depending on SWIG, faster than using FFI and easier to make it distributable or work remotely.

Bonus

Since we're a bunch of nice people, we also implemented:

  • An ascii version of 2048
  • A lot of colourschemes
  • Help is colourized
  • Greatly enhanced analysis
  • Nearly all commands can now output JSON

Screenshots!

colourscheme A new colourscheme

pxa Coloured hexdump

Gameby patching Because patching Gameboy ROM with radare2 is fun!

Demangling Prototype of demangling

sonare First screenshot of sonare, a Qt-based disassembly viewer based on radare2.

Extending r2 with new plugins

One of the key features behind r2 is how easily it can be extended with new libraries or plugins. In this blopost, we'll see the steps to add a new plugin in radare2.

Let's say we want to add a new plugin for r_asm because we are working with binaries of an architecture not supported by r2. Of course, adding a new plugin for another lib would be mostly the same.

From now to the end of the document, we will name this new plugin as "myarch".

Please note that you can keep your plugin outside the radare2 main tree, like the ones in radare2-extras.

The first thing we have to do is give a look to the r_asm header, located at libr/include/r_asm.h, and find out which callbacks we need to implement. They will be always defined under a structure named r_libname_plugin_t or RLibnamePlugin. In the case of r_asm, look closely at RAsmPlugin:

typedef struct r_asm_plugin_t {  
    [...]
    int (*init)(void *user);
    int (*fini)(void *user);
    int (*disassemble)(RAsm *a, RAsmOp *op, ut8 *buf, ut64 len);
    int (*assemble)(RAsm *a, RAsmOp *op, const char *buf);
    RAsmModifyCallback modify;
    int (*set_subarch)(RAsm *a, const char *buf);
} RAsmPlugin;

We can implement all the callbacks or only those needed for our purposes. In this case we only want to code the disassemble function.

Then, we will create a new file named asm_myarch.c under libr/asm/p/. The typical skeleton woud be:

static int disassemble(RAsm *a, RAsmOp *op, ut8 *buf, ut64 len) {  
    /* TODO: Implement disassemble code here,
     * give a look to the other plugins */
}

RAsmPlugin r_asm_plugin_myarch = {  
    .name = "MyArch",
    .desc = "disassembly plugin for MyArch",
    .arch = "myarch",
    .bits = (int[]){ 32, 64, 0 }, /* supported wordsizes */
    .init = NULL,
    .fini = NULL,
    .disassemble = &disassemble,
    .modify = NULL,
    .assemble = NULL,
};

#ifndef CORELIB
struct r_lib_struct_t radare_plugin = {  
    .type = R_LIB_TYPE_ASM,
    .data = &r_asm_plugin_myarch
};
#endif

Be careful with the name convention to avoid collisions with other existing plugins. Don't forget that you have lots of examples inside libr/*/p/ on how plugins are implemented.

If we need more files for the plugin to work, e.g. a backend, we will create a folder under libr/asm/arch/ called like the plugin arch (myarch) and copy all the files to libr/asm/arch/myarch/.

The next step is to add a new "extern" in the header file, so we can compile it as static plugin if we want to. In libr/include/r_asm.h:

[...]
extern RAsmPlugin r_asm_plugin_x86;  
extern RAsmPlugin r_asm_plugin_x86_olly;  
extern RAsmPlugin r_asm_plugin_x86_nasm;  
[...]
extern RAsmPlugin r_asm_plugin_myarch;  

Obviously, we need to tell the build system how to compile our code, this is done adding a new file called libr/asm/p/myarch.mk, which would seem like this:

OBJ_MYARCH=asm_myarch.o  
# myarch backend
OBJ_MYARCH+=../arch/myarch/udis86/file1.o  
OBJ_MYARCH+=../arch/myarch/udis86/file2.o  
[...]

STATIC_OBJ+=${OBJ_MYARCH}  
TARGET_MYARCH=asm_myarch.${EXT_SO}

ALL_TARGETS+=${TARGET_MYARCH}

${TARGET_MYARCH}: ${OBJ_MYARCH}
    ${CC} ${LDFLAGS} ${CFLAGS} -o ${TARGET_MYARCH} ${OBJ_MYARCH}

And appending it to the list of compiled plugins in libr/asm/p/Makefile:

[...]
ARCHS+=myarch.mk  
[...]

Finally, we only need to define if it must be compiled as static or shared, which is done editing plugins.def.cfg, in the source root. In this case, we will compile it as static, so we add a new line following the other asm plugins:

STATIC="asm.java  
asm.arm  
[...]
asm.x86  
asm.myarch  
[...]

Ok, We have just created our first r2 plugin. Now, exec:

$ make mrproper && ./configure && make && make install

and enjoy! :)