To explore the model of using after neoadjuvant chemoradiotherapy (nCRT) imaging images of locally advanced rectal cancer (LARC) to predict whether pathological complete response (pCR) is achieved, thereby assisting physicians to develop a personalized plan.

Patients with LARC treated with nCRT at the Sixth Affiliated Hospital of Sun Yat-sen University from June 2013 to December 2018 were retrospectively included, and the treatment outcome was evaluated by total rectal mesenteric resection histopathology. The patients were divided into 2 data sets, Data A and Data B, according to the order of hospitalization time in a ratio of 1:2. Data A was used for semantic segmentation model training, and Data B was randomly divided into training and validation sets in the ratio of 7:3, which were used for pCR prediction model training and validation, respectively. The T2-weighted MRI images of Data A were collected, and the improved fully convolutional networks(FCN) model was used to semantically segment the tumor region, establish the semantic segmentation model and extract the image features in the final convolutional layer. The least absolute shrinkage and selection operator (LASSO) regression model was used to filter the extracted image features and construct a support vector machine (SVM) classifier that could predict the pCR state. The performance of the prediction model was trained on the basis of the Data B training set and further validated in the Data B validation set.

A total of 304 patients with LARC who met the inclusion and exclusion criteria were enrolled, 82 patients reached pCR after nCRT, while 222 patients did not reach pCR (non-pCR). Among them, 103 patients from June 2013 to November 2015 were in Data A and 201 patients from December 2015 to December 2018 were in Data B. In Data B, 140 patients were in the training set and 61 patients in the validation set. The improved FCN model had a Dice value of 0.79(95% CI: 0.65-0.81), a sensitivity of 80%(95% CI: 77%-83%), and a specificity of 72%(95% CI: 64%-85%). A total of 512 image features in the final convolutional layer were extracted by the semantic segmentation model, and 7 were retained after LASSO regression screening for pCR state prediction. The area under the curve of the prediction model was 0.65(95% CI: 0.61-0.71) in the Data B training set and 0.69(95% CI: 0.59-0.74) in the Data B validation set for predicting pCR.

The improved FCN model proposed in this study has high accuracy for semantic segmentation of MRI images. The prediction model constructed based on this method is feasible to predict pCR status of LARC patients after receiving nCRT treatment.

With the advance in digital pathology and artificial intelligence (AI)-powered approaches, necrosis is proposed as a marker of poor prognosis in colorectal cancer (CRC). However, most previous studies quantified necrosis merely as a tissue type and patch-level segmentation. Thus, it was worth exploring and validating the prognostic and predictive value of necrosis proportion with a pixel-level segmentation in large multicenter cohorts.

A semantic segmentation model was trained with 12 tissue types labeled by pathologists. Segmentation was performed using the U-net model with a subsequently derived necrosis tumor ratio (NTR). We proposed the NTR score (NTR-low or NTR-high) to evaluate the prognostic and predictive value of necrosis for disease-free survival (DFS) and overall survival (OS) in the development (N = 443) and validation cohorts (N = 333) using 75% as a threshold.

The 2-category NTR was an independent prognostic factor and NTR-low was associated with significant prolonged DFS (unadjusted HR for high vs. low 1.72 [95% CI 1.19–2.49] and 1.98 [1.22–3.23] in the development and validation cohorts). Similar trends were observed for OS. The prognostic value of NTR was maintained in the multivariate analysis for both cohorts. Furthermore, a stratified analysis showed that NTR-high was a high risk with adjuvant chemotherapy for OS in stage Ⅱ CRC (p = 0.047).

AI-based pixel-level quantified NTR has a stable prognostic value in CRC associated with unfavorable survival. Additionally, adjuvant chemotherapy provided survival benefits for patients with a high NTR score in stage Ⅱ CRC.