Initially we wanted to write our own firmware from scratch for an ARM Cortex M3. Then we started playing with some open source AVR firmwares. Man, these 8-bit hackers have done some impressive work. Some of them even run on an old Atmega168 Arduino.
We tested FiveD_on_Arduino and grbl (acceleration branch) and also looked through the source code of Hydra-MMM and RepRap FiveD. The grbl coding style is by far the easiest to understand and FiveD_on_Arduino seemed to have some interesting acceleration code. In the end we only got grbl to perform smoothly (after updating to the super cutting edge code of Feb 3).
These are the settings we use for grbl:
$0 = 32.808 (steps/mm x)
$1 = 32.808 (steps/mm y)
$2 = 32.808 (steps/mm z)
$3 = 30 (microseconds step pulse)
$4 = 20000.0 (mm/min default feed rate)
$5 = 40000.0 (mm/min default seek rate)
$6 = 0.100 (mm/arc segment)
$7 = 0 (step port invert mask. binary = 0)
$8 = 2000.0 (acceleration in mm/sec^2)
$9 = 25.0 (max instant cornering speed change in delta mm/min)
We also cranked up this to 80:
#define ACCELERATION_TICKS_PER_SECOND 80L