Oh, sorry. I forgot to mention where to add that back in.
Those are done through the box_headers file in the template area. You would change each word to the image. So, for example, you would change Categories to:
<img src="templates/default/btn_shop.gif" border="0">
In the end, your box_headers page should look like this:
<?php
define('WHATS_NEW_HEADER', '<center>What\'s New</center>');
define('SPECIALS_HEADER', '<center>Specials</center>');
define('SHOPPING_CART_HEADER', '<center><img src="templates/default/btn_cart.gif" border="0"></center>');
define('TELL_A_FRIEND_HEADER', '<center>Tell A Friend</center>');
define('SEARCH_HEADER', '<center><img src="templates/default/btn_search.gif" border="0"></center>');
define('REVIEWS_HEADER', '<center>Reviews</center>');
define('PRODUCT_NOTIFICATIONS_HEADER', '<center>Product Notifications</center>');
define('ORDER_HISTORY_HEADER', '<center>Order History</center>');
define('MANUFACTURERS_HEADER', '<center>Manufacturers</center>');
define('MANUFACTURER_INFO_HEADER', '<center>Manufacturer Info</center>');
define('LINKS_HEADER', '<center>Links</center>');
define('LANGUAGES_NEW_HEADER', '<center>Languages</center>');
define('INFORMATION_HEADER', '<center><img src="templates/default/btn_details.gif" border="0"></center>');
define('GOOGLE_ADSENSE_HEADER', '<center>Google Adsense</center>');
define('CURRENT_AUCTIONS_BOX_HEADER', '<center>Current Auctions</center>');
define('CURRENCIES_HEADER', '<center>Currencies</center>');
define('CATEGORIES_HEADER', '<center><img src="templates/default/btn_shop.gif" border="0"></center>');
define('BEST_SELLERS_HEADER', '<center>Best Sellers</center>');
define('ACCOUNT_INFORMATION_HEADER', '<center>Account Information</center>');
define('NEW_PRODUCTS_HEADER', '<center>New Products</center>');
define('BLANK_HEADER', '');
?>