Lymph node metastasis is regarded as an indubitable prognostic factor for predicting disease recurrence and survival in patients with colorectal cancer. Lymph node status based on examination of a resected specimen is a key element of the current staging system and is also a crucial factor to determine use of adjuvant chemotherapy after surgical resection. However, the current tumor-node-metastasis (TNM) staging system only incorporates the number of metastatic lymph nodes in the N category. Numerous attempts have been made to supplement this simplified N staging including lymph node ratio, distribution of metastatic lymph nodes, tumor deposits, or extracapsular invasion. In addition, several attempts have been made to identify more specific prognostic factors in resected colorectal specimens than lymph node status. In this review, we will discuss controversies in lymph node staging and factors that may influence survival beyond lymph node status.