I started building my own H.264 encoder of an inspiration from my work at Tampere University of Technology where I've been working with H.264 Motion Estimator.
I've used the book E. G. Richardson, Iain (2003). H.264 and MPEG-4 Video Compression: Video Coding for Next-generation Multimedia. Chichester: John Wiley & Sons Ltd.. as a reference as well as the H.264 recommendation.

Encoder features:
  CAVLC 06/27/2009
  "Core" 4x4 transform 06/28/2009
  Hadamard 4x4/2x2 transform 06/28/2009
  Quantisation 06/28/2009
  4x4 Intra Prediction 07/01/2009



Source:
 Tools etc
  types.h- download- highlighted
  common.h- download- highlighted
  tables.c- download- highlighted
  tables.h- download- highlighted
  bitstream.c- download- highlighted
  bitstream.h- download- highlighted
  psnr.c- download- highlighted
  psnr.h- download- highlighted
 CALVC
  cavlc.c- download- highlighted
  cavlc.h- download- highlighted
 Transform
  transform.c- download- highlighted
  transform.h- download- highlighted
 Quantisation
  quant.c- download- highlighted
  quant.h- download- highlighted
 Intra coding
  intra.c- download- highlighted
  intra.h- download- highlighted


Examples


CAVLC (Show)


Complete transform (Show)


Transformed foreman QP 0-50:


NO INTRA PREDICTIONS USED!

CAVLC byte value is just 4x4 blocks coded with CAVLC, first value is with nC=0 and second value is with predicted nC as in the recommendation.
QP: 0 PSNR: 66.08 CAVLC: 97512 / 89269 bytesQP 10 PSNR: 45.63 CAVLC: 72910 / 66383 bytes
QP 20 PSNR: 35.63 CAVLC: 53567 / 50512 bytesQP 30 PSNR: 31.88 CAVLC: 50381 / 49146 bytes
QP 40 PSNR: 27.32 CAVLC: 19664 / 19114 bytesQP 50 PSNR: 20.45 CAVLC: 7776 / 7775 bytes


Transformed foreman 2 QP 0-50:


WITH FORCED 4x4 INTRA PREDICTIONS!
4 bits added to each 4x4 block for mode cost
QP: 0 PSNR: 66.03 CAVLC: 91528 bytesQP 10 PSNR: 41.15 CAVLC: 58439 bytes
QP 20 PSNR: 35.20 CAVLC: 23794 bytesQP 30 PSNR: 31.45 CAVLC: 9072 bytes
QP 40 PSNR: 28.52 CAVLC: 4780 bytesQP 50 PSNR: 23.73 CAVLC: 4064 bytes