實例分割是一種在像素級別標註物體實例位置、分割並分類的任務，其可以提供目標對象的詳細訊息，並且已經應用於醫學分析。在醫療領域中，隨著病例的增加，新的資料集會不斷產生，因此在監督學習中，訓練完成的模型必須重新使用全部的資料來學習新資料並保留舊知識。然而，資料會隨著時間的推移而增加，使用所有數據重新訓練模型會消耗許多時間。另外，由於隱私問題，可能導致學習過的資料被移除而不能使用。為了解決這些問題，我們提出了iMRCNN，這是一種基於 cascade Mask R-CNN 的增量實例分割器，目標是可以透過增量學習在增加新資料但不訪問先前資料時進行分割，並在新舊資料上的預測都取得良好的表現。iMRCNN 利用輸出層級以及特徵層級知識蒸餾來解決由於缺乏先前資料而導致的災難性遺忘現象。除此之外，在增量實例分割中，可能會出現另一個稱為背景偏移的問題。背景偏移指的是背景包含之前已學習過或未來要學習類別的物體。我們透過偽標記處理背景偏移，偽標記會提供新模型先前資料集的信息以減輕錯誤背景的干擾。我們透過 SegPC-2021 資料集以及三個增量學習設定的實驗說明 iMRCNN 的有效性，以及與其他基礎方法相比取得了更好的結果。此外，我們提供消融實驗以了解 iMRCNN 中主要組件的相對重要性。

Instance segmentation is the task that localizes and distinguishes each object instance at a pixel level. Such a task provides detailed information about an object and has been applied to medical analysis. However, in the medical field, new datasets are continuously generated as cases increase. In a supervised learning paradigm, a trained model must be retrained on all the data in order to adapt to new datasets and preserve old knowledge. As the data increase over time, using all the data to retrain a model is time-comsuming. Furthermore, due to privacy concerns, previously acquired data are not always available. In order to address these issues, we present iMRCNN, an incremental instance segmenter based on the cascade Mask R-CNN, that aims to perform well on both old and new data by incrementally learning to segment when new data arrive but without access to previous data. iMRCNN leverages output-level and feature-level knowledge distillation to overcome catastrophic forgetting of the learned knowledge, a well-known phenomenon caused by the lack of previous data. Moreover, in incremental instance segmentation, a particular issue called background shift, where the background class contains objects in the old and unseen classes, could occur. iMRCNN deals with this issue by utilizing the old model to provide pseudo labels of the background class. Experiments on the SegPC-2021 dataset show that iMRCNN is effective and performs significantly better than the baseline approaches in three well-designed incremental learning settings. Ablation studies are also provided to understand the relative importance of the main components in iMRCNN.

論文審定書. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . i
摘要. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ii
Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . iii
Chapter 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Chapter 2 Related Work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5
2.1 Incremental Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Instance Segmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.3 Incremental Instance Segmentation Learning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9
Chapter 3 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
3.1 Formulation of Incremental Instance Segmentation . . . . . . . . . . . . . . . . . . . . . . . 13
3.2 Preliminaries: Cascade Mask R-CNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
3.3 iMRCNN: Incremental Cascade Mask R-CNN . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.3.1 Knowledge Distillation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
3.3.2 Pseudo Labeling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.3.3 Overall Learning Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
Chapter 4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.1 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
4.2 Main Results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29
4.3 Ablation Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
4.4 Qualitative Results and Failure Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
Chapter 5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

[1] Z. Cai and N. Vasconcelos, “Cascade R-CNN: High quality object detection and instance segmentation,” IEEE transactions on pattern analysis and machine intelligence, vol. 43, no. 5, pp. 1483–1498, 2021.
[2] E. Meijering, “Cell segmentation: 50 years down the road,” IEEE signal processing magazine, vol. 29, no. 5, pp. 140–145, 2012.
[3] J. C. Caicedo, J. Roth, A. Goodman, T. Becker, K. W. Karhohs, M. Broisin, C. Molnar, C. McQuin, S. Singh, F. J. Theis et al., “Evaluation of deep learning strategies for nucleus segmentation in fluorescence images,” Cytometry Part A, vol. 95, no. 9, pp. 952–965, 2019.
[4] P. Naylor, M. Laé, F. Reyal, and T. Walter, “Segmentation of nuclei in histopathology images by deep regression of the distance map,” IEEE transactions on medical imaging, vol. 38, no. 2, pp. 448–459, 2018.
[5] S. Gehlot, A. Gupta, and R. Gupta, “EDNFC-Net: Convolutional neural network with nested feature concatenation for nuclei-instance segmentation,” in Proceedings of the IEEE conference on Acoustics, Speech and Signal Processing, 2020, pp. 1389–1393.
[6] J. Dai, K. He, and J. Sun, “Instance-aware semantic segmentation via multi-task network cascades,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 3150–3158.
[7] P. H. Pinheiro, R. Collobert, and P. Dollár, “Learning to segment object candidates,” in NIPS, 2015.
[8] Y. Li, H. Qi, J. Dai, X. Ji, and Y. Wei, “Fully convolutional instance-aware semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2359–2367.
[9] A. Maracani, U. Michieli, M. Toldo, and P. Zanuttigh, “RECALL: Replay-based continual learning in semantic segmentation,” in Proceedings of the IEEE international conference on computer vision, 2021, pp. 7026–7035.
[10] J. Rajasegaran, M. Hayat, S. H. Khan, F. S. Khan, and L. Shao, “Random path selection for continual learning,” Advances in Neural Information Processing Systems, vol. 32, 2019.
[11] Y. Gu, C. Deng, and K. Wei, “Class-incremental instance segmentation via multiteacher networks,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 2, 2021, pp. 1478–1486.
[12] W. He, X. Wang, L. Wang, Y. Huang, Z. Yang, X. Yao, X. Zhao, L. Ju, L. Wu, L. Wu et al., “Incremental learning for exudate and hemorrhage segmentation on fundus images,” Information Fusion, vol. 73, pp. 157–164, 2021.
[13] F. Cermelli, A. Geraci, D. Fontanel, and B. Caputo, “Modeling missing annotations for incremental learning in object detection,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 3700–3710.
[14] S.-A. Rebuffi, A. Kolesnikov, G. Sperl, and C. H. Lampert, “iCaRL: Incremental classifier and representation learning,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 2001–2010.
[15] E. Belouadah and A. Popescu, “IL2M: Class incremental learning with dual memory,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019, pp. 583–592.
[16] S. Hou, X. Pan, C. C. Loy, Z. Wang, and D. Lin, “Learning a unified classifier incrementally via rebalancing,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 831–839.
[17] A. Douillard, Y. Chen, A. Dapogny, and M. Cord, “PLOP: Learning without forgetting for continual semantic segmentation,” in Proceedings of the IEEE international conference on computer vision, 2021, pp. 4040–4050.
[18] D. Abati, J. Tomczak, T. Blankevoort, S. Calderara, R. Cucchiara, and B. E. Bejnordi, “Conditional channel gated networks for task-aware continual learning,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2020, pp. 3931–3940.
[19] Y. Liu, Y. Su, A.-A. Liu, B. Schiele, and Q. Sun, “Mnemonics Training: Multi-class incremental learning without forgetting,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2020, pp. 12 245–12 254.
[20] Z. Li and D. Hoiem, “Learning without forgetting,” IEEE transactions on pattern analysis and machine intelligence, vol. 40, no. 12, pp. 2935–2947, 2017.
[21] A. Douillard, M. Cord, C. Ollion, T. Robert, and E. Valle, “PODNet: Pooled outputs distillation for small-tasks incremental learning,” in European conference on computer vision. Springer, 2020, pp. 86–102.
[22] L. Yu, B. Twardowski, X. Liu, L. Herranz, K. Wang, Y. Cheng, S. Jui, and J. v. d. Weijer, “Semantic drift compensation for class-incremental learning,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020, pp. 6982–6991.
[23] X. Gong, S. Chen, B. Zhang, and D. Doermann, “Style consistent image generation for nuclei instance segmentation,” in Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2021, pp. 3994–4003.
[24] L. Ke, M. Danelljan, X. Li, Y.-W. Tai, C.-K. Tang, and F. Yu, “Mask transfiner for high-quality instance segmentation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 4412–4421.
[25] S. Mohanapriya et al., “Instance segmentation for autonomous vehicle,” Turkish Journal of Computer and Mathematics Education (TURCOMAT), vol. 12, no. 9, pp. 565–570, 2021.
[26] J.-H. Witte, J. Gerberding, C. Melching, and J. M. Gómez, “Evaluation of deep learning instance segmentation models for pig precision livestock farming,” in Business Information Systems, 2021, pp. 209–220.
[27] S. Kong and C. C. Fowlkes, “Recurrent pixel embedding for instance grouping,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 9018–9028.
[28] S. Liu, J. Jia, S. Fidler, and R. Urtasun, “SGN: Sequential grouping networks for instance segmentation,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 3496–3504.
[29] L.-C. Chen, A. Hermans, G. Papandreou, F. Schroff, P. Wang, and H. Adam, “MaskLab: Instance segmentation by refining object detection with semantic and direction features,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4013–4022.
[30] W. Kuo, A. Angelova, J. Malik, and T.-Y. Lin, “ShapeMask: Learning to segment novel objects by refining shape priors,” in Proceedings of the IEEE international conference on computer vision, 2019, pp. 9207–9216.
[31] P. O. O Pinheiro, R. Collobert, and P. Dollár, “Learning to segment object candidates,” Advances in neural information processing systems, vol. 28, 2015.
[32] K. He, G. Gkioxari, P. Dollár, and R. Girshick, “Mask R-CNN,” in Proceedings of the IEEE international conference on computer vision, 2017, pp. 2961–2969.
[33] P. O. Pinheiro, T.-Y. Lin, R. Collobert, and P. Dollár, “Learning to refine object segments,” in European conference on computer vision. Springer, 2016, pp. 75–91.
[34] D. Pandey, P. Gupta, S. Bhattacharya, A. Sinha, and R. Agarwal, “Transformer assisted convolutional network for cell instance segmentation,” arXiv preprint arXiv:2110.02270, 2021.
[35] A. Gupta, R. Gupta, S. Gehlot, and S. Goswami, “SegPC-2021: Segmentation of multiple myeloma plasma cells in microscopic images,” 2021. [Online]. Available: https://dx.doi.org/10.21227/7np1-2q42
[36] D. A. Ganea, B. Boom, and R. Poppe, “Incremental few-shot instance segmentation,” in Proceedings of the IEEE international conference on computer vision, 2021, pp. 1185–1194.
[37] J. J. Lee, S. I. Lee, and H. Kim, “Continual learning for instance segmentation to mitigate catastrophic forgetting,” in 2021 18th International SoC Design Conference (ISOCC). IEEE, 2021, pp. 85–86.
[38] K. Nguyen and S. Todorovic, “iFS-RCNN: An incremental few-shot instance segmenter,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 7010–7019.
[39] A. Robins, “Catastrophic forgetting, rehearsal and pseudorehearsal,” Connection Science, vol. 7, no. 2, pp. 123–146, 1995.
[40] R. M. French, “Catastrophic forgetting in connectionist networks,” Trends in cognitive sciences, vol. 3, no. 4, pp. 128–135, 1999.
[41] F. Cermelli, M. Mancini, S. R. Bulo, E. Ricci, and B. Caputo, “Modeling the background for incremental learning in semantic segmentation,” in Proceedings of the IEEE international conference on computer vision, 2020, pp. 9233–9242.