Very short and sweet TODO list after a fresh install. I have taken some of this from various guides on the net.
# dpkg-reconfigure dash
# apt-get install localepurge
Install vim-nox (I do not use X):
# apt-get install vim-nox
Install openntpd (purge ntpdate):
# dpkg -P ntpdate
# apt-get install openntpd
Time to secure the installation a bit... rather than repeat, please follow the links below and pick'n'mix your hardening options.
Stricter Defaults (Ubuntu docs)
Secure Ubuntu (rehash of the stricter defaults guide but with tools suggestions)
Harden Ubuntu Guide (rehash of the stricter defaults guide plus some other suggestions)
Ubuntu Server Guide (Lucid)