{
  "id": "54a9d3380f68",
  "url": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/",
  "source_excel_row": 1601,
  "source_url": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/",
  "is_paid_flag": "",
  "images": [
    {
      "status": "ok",
      "url": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/convolution-operation.png",
      "bytes": 21634,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/54a9d3380f68/images/body-0af902d15df97a679d1d.png",
      "src": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/convolution-operation.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/numerical_no_padding_no_strides.gif",
      "bytes": 54826,
      "reused": false,
      "content_type": "image/gif",
      "local_path": "attachments/54a9d3380f68/images/body-b14e80a69c065fcf923f.gif",
      "src": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/numerical_no_padding_no_strides.gif",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/numerical_padding_strides.gif",
      "bytes": 89960,
      "reused": false,
      "content_type": "image/gif",
      "local_path": "attachments/54a9d3380f68/images/body-2ff11bc219d2fddbb103.gif",
      "src": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/numerical_padding_strides.gif",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/alex_net_illustration.png",
      "bytes": 1470472,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/54a9d3380f68/images/body-299fa3d9ecac66743b06.png",
      "src": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/alex_net_illustration.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/residual-block.png",
      "bytes": 497605,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/54a9d3380f68/images/body-35effb374a6c3725fe88.png",
      "src": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/residual-block.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/DPM.png",
      "bytes": 788370,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/54a9d3380f68/images/body-c0e1f225ec0f222e8138.png",
      "src": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/DPM.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/DPM-matching.png",
      "bytes": 645042,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/54a9d3380f68/images/body-c3675d62fb7c6784228f.png",
      "src": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/DPM-matching.png",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/overfeat-training.png",
      "bytes": 31968,
      "reused": false,
      "content_type": "image/png",
      "local_path": "attachments/54a9d3380f68/images/body-069c442cc0ce9957e75d.png",
      "src": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/overfeat-training.png",
      "alt": "",
      "downloaded": true
    }
  ],
  "videos": [],
  "body_markdown": "Part 1\nof the “Object Detection for Dummies” series introduced: (1) the concept of image gradient vector and how HOG algorithm summarizes the information across all the gradient vectors in one image; (2) how the image segmentation algorithm works to detect regions that potentially contain objects; (3) how the Selective Search algorithm refines the outcomes of image segmentation for better region proposal.\nIn Part 2, we are about to find out more on the classic convolution neural network architectures for image classification. They lay the\nfoundation\nfor further progress on the deep learning models for object detection. Go check\nPart 3\nif you want to learn more on R-CNN and related models.\nLinks to all the posts in the series:\n[\nPart 1\n]\n[\nPart 2\n]\n[\nPart 3\n]\n[\nPart 4\n].\nCNN for Image Classification\n#\nCNN, short for “\nConvolutional Neural Network\n”, is the go-to solution for computer vision problems in the deep learning world. It was, to some extent,\ninspired\nby how human visual cortex system works.\nConvolution Operation\n#\nI strongly recommend this\nguide\nto convolution arithmetic, which provides a clean and solid explanation with tons of visualizations and examples. Here let’s focus on two-dimensional convolution as we are working with images in this post.\nIn short, convolution operation slides a predefined\nkernel\n(also called “filter”) on top of the input feature map (matrix of image pixels), multiplying and adding the values of the kernel and partial input features to generate the output. The values form an output matrix, as usually, the kernel is much smaller than the input image.\nAn illustration of applying a kernel on the input feature map to generate the output. (Image source:\nRiver Trail documentation\n)\nFigure 2 showcases two real examples of how to convolve a 3x3 kernel over a 5x5 2D matrix of numeric values to generate a 3x3 matrix. By controlling the padding size and the stride length, we can generate an output matrix of a certain size.\nTwo examples of 2D convolution operation: (top) no padding and 1x1 strides; (bottom) 1x1 border zeros padding and 2x2 strides. (Image source:\ndeeplearning.net\n)\nAlexNet (Krizhevsky et al, 2012)\n#\n5 convolution [+ optional max pooling] layers + 2 MLP layers + 1 LR layer\nUse data augmentation techniques to expand the training dataset, such as image translations, horizontal reflections, and patch extractions.\nThe architecture of AlexNet. (Image source:\nlink\n)\nVGG (Simonyan and Zisserman, 2014)\n#\nThe network is considered as “very deep” at its time; 19 layers\nThe architecture is extremely simplified with only 3x3 convolutional layers and 2x2 pooling layers. The stacking of small filters simulates a larger filter with fewer parameters.\nResNet (He et al., 2015)\n#\nThe network is indeed very deep; 152 layers of simple architecture.\nResidual Block\n: Some input of a certain layer can be passed to the component two layers later. Residual blocks are essential for keeping a deep network trainable and eventually work. Without residual blocks, the training loss of a plain network does not monotonically decrease as the number of layers increases due to\nvanishing and exploding gradients\n.\nAn illustration of the residual block of ResNet. In some way, we can say the design of residual blocks is inspired by V4 getting input directly from V1 in the human visual cortex system. (left image source:\nWang et al., 2017\n)\nEvaluation Metrics: mAP\n#\nA common evaluation metric used in many object recognition and detection tasks is “\nmAP\n”, short for “\nmean average precision\n”. It is a number from 0 to 100; higher value is better.\nCombine all detections from all test images to draw a precision-recall curve (PR curve) for each class; The “average precision” (AP) is the area under the PR curve.\nGiven that target objects are in different classes, we first compute AP separately for each class, and then average over classes.\nA detection is a true positive if it has\n“intersection over union” (IoU)\nwith a ground-truth box greater than some threshold (usually 0.5; if so, the metric is “\nmAP@0.5\n”)\nDeformable Parts Model\n#\nThe Deformable Parts Model (DPM) (\nFelzenszwalb et al., 2010\n) recognizes objects with a mixture graphical model (Markov random fields) of deformable parts. The model consists of three major components:\nA coarse\nroot filter\ndefines a detection window that approximately covers an entire object. A filter specifies weights for a region feature vector.\nMultiple\npart filters\nthat cover smaller parts of the object. Parts filters are learned at twice resolution of the root filter.\nA\nspatial model\nfor scoring the locations of part filters relative to the root.\nThe DPM model contains (a) a root filter, (b) multiple part filters at twice the resolution, and (c) a model for scoring the location and deformation of parts.\nThe quality of detecting an object is measured by the score of filters minus the deformation costs. The matching score $f$, in laymen’s terms, is:\n$$\nf(\\text{model}, x) = f(\\beta_\\text{root}, x) + \\sum_{\\beta_\\text{part} \\in \\text{part filters}} \\max_y [f(\\beta_\\text{part}, y) - \\text{cost}(\\beta_\\text{part}, x, y)]\n$$\nin which,\n$x$ is an image with a specified position and scale;\n$y$ is a sub region of $x$.\n$\\beta_\\text{root}$ is the root filter.\n$\\beta_\\text{part}$ is one part filter.\ncost() measures the penalty of the part deviating from its ideal location relative to the root.\nThe basic score model is the dot product between the filter $\\beta$ and the region feature vector $\\Phi(x)$: $f(\\beta, x) = \\beta \\cdot \\Phi(x)$. The feature set $\\Phi(x)$ can be defined by HOG or other similar algorithms.\nA root location with high score detects a region with high chances to contain an object, while the locations of the parts with high scores confirm a recognized object hypothesis. The paper adopted latent SVM to model the classifier.\nThe matching process by DPM. (Image source:\nFelzenszwalb et al., 2010\n)\nThe author later claimed that DPM and CNN models are not two distinct approaches to object recognition. Instead, a DPM model can be formulated as a CNN by unrolling the DPM inference algorithm and mapping each step to an equivalent CNN layer. (Check the details in\nGirshick et al., 2015\n!)\nOverfeat\n#\nOverfeat [\npaper\n][\ncode\n] is a pioneer model of integrating the object detection, localization and classification tasks all into one convolutional neural network. The main idea is to (i) do image classification at different locations on regions of multiple scales of the image in a sliding window fashion, and (ii) predict the bounding box locations with a regressor trained on top of the same convolution layers.\nThe Overfeat model architecture is very similar to\nAlexNet\n. It is trained as follows:\nThe training stages of the Overfeat model. (Image source:\nlink\n)\nTrain a CNN model (similar to AlexNet) on the image classification task.\nThen, we replace the top classifier layers by a regression network and train it to predict object bounding boxes at each spatial location and scale. The regressor is class-specific, each generated for one image class.\nInput: Images with classification and bounding box.\nOutput: $(x_\\text{left}, x_\\text{right}, y_\\text{top}, y_\\text{bottom})$, 4 values in total, representing the coordinates of the bounding box edges.\nLoss: The regressor is trained to minimize $l2$ norm between generated bounding box and the ground truth for each training example.\nAt the detection time,\nPerform classification at each location using the pretrained CNN model.\nPredict object bounding boxes on all classified regions generated by the classifier.\nMerge bounding boxes with sufficient overlap from localization and sufficient confidence of being the same object from the classifier.\nCited as:\n@article{weng2017detection2,\n  title   = \"Object Detection for Dummies Part 2: CNN, DPM and Overfeat\",\n  author  = \"Weng, Lilian\",\n  journal = \"lilianweng.github.io\",\n  year    = \"2017\",\n  url     = \"https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/\"\n}\nReference\n#\n[1] Vincent Dumoulin and Francesco Visin.\n“A guide to convolution arithmetic for deep learning.”\narXiv preprint arXiv:1603.07285 (2016).\n[2] Haohan Wang, Bhiksha Raj, and Eric P. Xing.\n“On the Origin of Deep Learning.”\narXiv preprint arXiv:1702.07800 (2017).\n[3] Pedro F. Felzenszwalb, Ross B. Girshick, David McAllester, and Deva Ramanan.\n“Object detection with discriminatively trained part-based models.”\nIEEE transactions on pattern analysis and machine intelligence 32, no. 9 (2010): 1627-1645.\n[4] Ross B. Girshick, Forrest Iandola, Trevor Darrell, and Jitendra Malik.\n“Deformable part models are convolutional neural networks.”\nIn Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), pp. 437-446. 2015.\n[5] Sermanet, Pierre, David Eigen, Xiang Zhang, Michaël Mathieu, Rob Fergus, and Yann LeCun.\n“OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks”\narXiv preprint arXiv:1312.6229 (2013).",
  "repair_version": 2,
  "repaired_at": "2026-10-14T11:44:42Z",
  "title": "Object Detection for Dummies Part 2: CNN, DPM and Overfeat | Lil'Log",
  "audio": [],
  "code_blocks": [
    {
      "file": "attachments/54a9d3380f68/code/body-000.txt",
      "chars": 276,
      "language": "text"
    }
  ],
  "papers": [
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/1603.07285.pdf",
      "bytes": 913752,
      "reused": true,
      "local_path": "papers/repair/b61fe9cb2654d748de68.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/1312.6229.pdf",
      "bytes": 937841,
      "reused": false,
      "content_type": "application/pdf",
      "local_path": "papers/repair/540d3c26a6305febc0e1.pdf"
    },
    {
      "status": "http_403",
      "url": "http://people.cs.uchicago.edu/~pff/papers/lsvm-pami.pdf",
      "retried_at": "2026-10-20T17:45:41Z"
    },
    {
      "status": "ok",
      "url": "https://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Girshick_Deformable_Part_Models_2015_CVPR_paper.pdf",
      "bytes": 832728,
      "reused": false,
      "content_type": "application/pdf",
      "local_path": "papers/repair/1743cfdd1a25ffae4bbb.pdf"
    },
    {
      "status": "http_202",
      "url": "https://pdfs.semanticscholar.org/f2c2/fbc35d0541571f54790851de9fcd1adde085.pdf",
      "retried_at": "2026-10-20T17:45:42Z"
    },
    {
      "status": "ok",
      "url": "http://vision.stanford.edu/teaching/cs231b_spring1415/slides/overfeat_eric.pdf",
      "bytes": 6298687,
      "reused": false,
      "content_type": "application/pdf",
      "local_path": "papers/repair/417cc548662a22985f80.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/1702.07800.pdf",
      "bytes": 5805767,
      "reused": false,
      "content_type": "application/pdf",
      "local_path": "papers/repair/bd21c9c5b15c3410f312.pdf"
    }
  ],
  "actual_video_files": [],
  "body_chars": 8988,
  "expected_body_images": 8,
  "discovery_status": "ok",
  "delivery_status": "partial",
  "missing_reason": "http_202；http_403",
  "raw_html": "attachments/54a9d3380f68/pages/source.html",
  "local_html": "attachments/54a9d3380f68/pages/article.html",
  "body_base_path": "attachments/54a9d3380f68/pages/",
  "body_markdown_path": "attachments/54a9d3380f68/pages/article.md",
  "links": [
    {
      "href": "https://lilianweng.github.io/posts/2017-10-29-object-recognition-part-1/",
      "label": "Part 1",
      "type": "a"
    },
    {
      "href": "https://lilianweng.github.io/posts/2017-12-31-object-recognition-part-3/",
      "label": "Part 3",
      "type": "a"
    },
    {
      "href": "https://lilianweng.github.io/posts/2017-12-15-object-recognition-part-2/",
      "label": "Part 2",
      "type": "a"
    },
    {
      "href": "https://lilianweng.github.io/posts/2018-12-27-object-recognition-part-4/",
      "label": "Part 4",
      "type": "a"
    },
    {
      "href": "https://lilianweng.github.io/posts/2017-06-21-overview/",
      "label": "inspired",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/pdf/1603.07285.pdf",
      "label": "guide",
      "type": "a"
    },
    {
      "href": "https://en.wikipedia.org/wiki/Kernel_(image_processing)",
      "label": "kernel",
      "type": "a"
    },
    {
      "href": "http://intellabs.github.io/RiverTrail/tutorial/",
      "label": "River Trail documentation",
      "type": "a"
    },
    {
      "href": "http://deeplearning.net/software/theano_versions/dev/tutorial/conv_arithmetic.html",
      "label": "deeplearning.net",
      "type": "a"
    },
    {
      "href": "http://vision03.csail.mit.edu/cnn_art/index.html",
      "label": "link",
      "type": "a"
    },
    {
      "href": "http://www.wildml.com/2015/10/recurrent-neural-networks-tutorial-part-3-backpropagation-through-time-and-vanishing-gradients/",
      "label": "vanishing and exploding gradients",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/pdf/1312.6229.pdf",
      "label": "Wang et al., 2017",
      "type": "a"
    },
    {
      "href": "http://people.cs.uchicago.edu/~pff/papers/lsvm-pami.pdf",
      "label": "Felzenszwalb et al., 2010",
      "type": "a"
    },
    {
      "href": "https://www.cv-foundation.org/openaccess/content_cvpr_2015/papers/Girshick_Deformable_Part_Models_2015_CVPR_paper.pdf",
      "label": "Girshick et al., 2015",
      "type": "a"
    },
    {
      "href": "https://pdfs.semanticscholar.org/f2c2/fbc35d0541571f54790851de9fcd1adde085.pdf",
      "label": "paper",
      "type": "a"
    },
    {
      "href": "https://github.com/sermanet/OverFeat",
      "label": "code",
      "type": "a"
    },
    {
      "href": "http://vision.stanford.edu/teaching/cs231b_spring1415/slides/overfeat_eric.pdf",
      "label": "link",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/pdf/1702.07800.pdf",
      "label": "“On the Origin of Deep Learning.”",
      "type": "a"
    }
  ],
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  }
}