[![](https://zhuanlan.zhihu.com/p/27722435)](javascript:void\(0\)) ![ZhiHu logo](../images/body-e1fa91dbdb48b8055184.png) [](https://www.zhihu.com) [关注](https://www.zhihu.com/signin?next=%2Ffollow)[推荐](https://www.zhihu.com/signin?next=%2F)[热榜](https://www.zhihu.com/signin?next=%2Fhot)[专栏](https://www.zhihu.com/signin?next=%2Fcolumn-square)[圈子](https://www.zhihu.com/signin?next=%2Fring-feeds)[AI WorksBeta ](https://www.zhihu.com/project-square)[故事](https://www.zhihu.com/fiore/h5/vip-web) ​ [直答](https://zhida.zhihu.com/) 切换模式 登录/注册 【简评】Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes 首发于[Segmentation Paper Reading](https://www.zhihu.com/column/semantic-segmentation) 切换模式 ![【简评】Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes](../images/000.img) # 【简评】Full-Resolution Residual Networks for Semantic Segmentation in Street Scenes [![余昌黔](../images/021.jpg)](https://www.zhihu.com/people/yu-chang-qian) [余昌黔](https://www.zhihu.com/people/yu-chang-qian) [​![](../images/body-ddf414f04793a667b2c1.png)](https://www.zhihu.com/question/48510028) 华中科技大学 控制科学与工程博士 [收录于 · Segmentation Paper Reading](https://www.zhihu.com/column/semantic-segmentation) 37 人赞同了该文章 本文提出了一种two-stream的网络结构来实现strong recognition和strong localization performance。 ## 网络结构 ![](../images/004.png) Two-Stream： * pooling stream（红线）: 所有的pooling操作均在于此，得到好的高层特征，提高网络[感受野](https://zhida.zhihu.com/search?content_id=3276440&content_type=Article&match_order=1&q=%E6%84%9F%E5%8F%97%E9%87%8E&zd_token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJ6aGlkYV9zZXJ2ZXIiLCJleHAiOjE3OTExMzk0MDUsInEiOiLmhJ_lj5fph44iLCJ6aGlkYV9zb3VyY2UiOiJlbnRpdHkiLCJjb250ZW50X2lkIjozMjc2NDQwLCJjb250ZW50X3R5cGUiOiJBcnRpY2xlIiwibWF0Y2hfb3JkZXIiOjEsInpkX3Rva2VuIjpudWxsfQ.cOFzn4Shc44SI4J5CYGKmQRHhJqjZeHZNNT4dnY0ous&zhida_source=entity)，同时对于图片small translation鲁棒一些 * [residual stream](https://zhida.zhihu.com/search?content_id=3276440&content_type=Article&match_order=1&q=residual+stream&zd_token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJ6aGlkYV9zZXJ2ZXIiLCJleHAiOjE3OTExMzk0MDUsInEiOiJyZXNpZHVhbCBzdHJlYW0iLCJ6aGlkYV9zb3VyY2UiOiJlbnRpdHkiLCJjb250ZW50X2lkIjozMjc2NDQwLCJjb250ZW50X3R5cGUiOiJBcnRpY2xlIiwibWF0Y2hfb3JkZXIiOjEsInpkX3Rva2VuIjpudWxsfQ.jsRGI77LU162vFeotJcQp-WS9aGJOtAGK3NP0X8komg&zhida_source=entity)（蓝线）: 不含任何pooling操作，一直保持full-resolution，从而保证好的定位能力 虽然看起来和普通的encoder-decoder网络结构比较类似，但是还是有所不同的。因为这个结构中存在residual stream，保持完整的resolution，而普通的encoder-decoder结构是会直接降低图像分辨率的，然后之后再逐渐升上去。 其中的RU就是不同的[Residual block](https://zhida.zhihu.com/search?content_id=3276440&content_type=Article&match_order=1&q=Residual+block&zd_token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJ6aGlkYV9zZXJ2ZXIiLCJleHAiOjE3OTExMzk0MDUsInEiOiJSZXNpZHVhbCBibG9jayIsInpoaWRhX3NvdXJjZSI6ImVudGl0eSIsImNvbnRlbnRfaWQiOjMyNzY0NDAsImNvbnRlbnRfdHlwZSI6IkFydGljbGUiLCJtYXRjaF9vcmRlciI6MSwiemRfdG9rZW4iOm51bGx9.iZOQbFzoySWG73_Ievc9khiwdKSY7uENKeYfSAOKKFk&zhida_source=entity)结构。其中FRRU的结构如下： ![](../images/005.png) z_{n-1}是residual input; y_{n-1}是pooling input 可以看到，pooling stream其实完全是为residual stream服务的，将residual stream上的输出降下来处理之后再升上去。 对于这个结构我们还可以这样理解，普通的[resnet](https://zhida.zhihu.com/search?content_id=3276440&content_type=Article&match_order=1&q=resnet&zd_token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJ6aGlkYV9zZXJ2ZXIiLCJleHAiOjE3OTExMzk0MDUsInEiOiJyZXNuZXQiLCJ6aGlkYV9zb3VyY2UiOiJlbnRpdHkiLCJjb250ZW50X2lkIjozMjc2NDQwLCJjb250ZW50X3R5cGUiOiJBcnRpY2xlIiwibWF0Y2hfb3JkZXIiOjEsInpkX3Rva2VuIjpudWxsfQ.lOsK672sMYf1-uugndrfMrTN0tohk0FK_PZWc5ehmpM&zhida_source=entity)中降采样的操作主要靠conv+stride来控制的，如下： ![](../images/006.png) 这样为了保证之后两条支路的[feature map](https://zhida.zhihu.com/search?content_id=3276440&content_type=Article&match_order=1&q=feature+map&zd_token=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJ6aGlkYV9zZXJ2ZXIiLCJleHAiOjE3OTExMzk0MDUsInEiOiJmZWF0dXJlIG1hcCIsInpoaWRhX3NvdXJjZSI6ImVudGl0eSIsImNvbnRlbnRfaWQiOjMyNzY0NDAsImNvbnRlbnRfdHlwZSI6IkFydGljbGUiLCJtYXRjaF9vcmRlciI6MSwiemRfdG9rZW4iOm51bGx9.SpTmriLqxMx4MTsc_8GA5BzP-VfZMV7Pks3vjxNwjww&zhida_source=entity)能够相加，shortcut上也进行了对应的尺寸调整。而现在FRRU中相当于保证了shortcut的绝对clean，然后将pool和unpool全放在了支路上进行。这样就实现了一条full-resolution的支路了。 FRRN具体网络如下： ![](../images/007.png) FRRN A和FRRN B主要是输入图片resolution不同。 ## Loss 文章中使用的是 bootstrapped cross-entropy loss 进行训练，即选择更难的像素进行训练。 l=-\frac{1}{K}\Sigma^{N}_{i=1}1[p_{i, y_i}