Background: It is unclear whether the number or distribution of lymph node metastases can provide a more accurate prognosis. The aim of this study was to evaluate the prognostic impact of inferior mesenteric artery (IMA) lymph node metastasis (LNM) in sigmoid colon and rectal cancer. Methods: We included 188 patients who underwent curative resection for stage III sigmoid colon and rectal cancer between January 2001 and December 2012. Patients were divided into two groups based on the presence of IMA LNM (LNM-positive vs. LNM-negative group). Clinicopathological characteristics, 3-year recurrence-free survival and 5-year overall survival rates, and recurrence patterns were compared between the two groups. Results: Of 188 patients, nine patients (4.79%) were in the LNM-positive group. After curative resection, 3-year recurrence-free survival and 5-year overall survival rates were significantly lower in the LNM-positive group compared to the LNM-negative group (44.44% vs. 69.98%, p = 0.016 for 3-year recurrence-free survival and 48.61% vs. 81.73%, p = 0.018 for 5-year overall survival). Multivariate analysis revealed that the presence of IMA LNM (p = 0.04), not the number of LNMs (p = 0.153), was an independent prognostic factor for recurrence-free survival. The paraaortic lymph node metastasis rate was significantly higher in the LNM-positive group (p = 0.0078). Conclusions: IMA LNM is an independent predictor of survival for stage III sigmoid colon and rectal cancer patients. Evaluation of IMA LNM enables accurate estimation of patient prognosis and enhances appropriate postoperative therapy.