{ "source_file": "0930", "source_platform": "Medium", "author": "Muhammad Ardi Putra", "title_listed": "FPN Paper Walkthrough: Leveraging the Internal Pyramid", "keywords": "", "domain": "图像处理", "content_type": "深度技术解析", "is_paid_flag": "否", "url": "https://towardsdatascience.com/fpn-paper-walkthrough-leveraging-the-internal-pyramid/", "publish_date": "2026-06-04", "corpus_stage": "", "notes": "", "id": "dcf4c8a3c08b", "collected_at": "2026-10-02T02:18:08", "title": "FPN Paper Walkthrough: Leveraging the Internal Pyramid | Towards Data Science", "body_markdown": "![Revisit consent button](../images/body-df29beacd7496220cc94.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\n# FPN Paper Walkthrough: Leveraging the Internal Pyramid\nUnderstanding how FPN allows deep learning models detecting small objects and how to implement it from scratch\n[Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\nJune 4, 202618 min read\n![](../images/000.jpg)\nPhoto by \nIn my previous article I was talking about YOLOv3 [1]. One of the factors that makes this YOLO version better than its predecessors is its ability in detecting small objects thanks to its FPN-like neck adopted by the model. Unfortunately, my explanation about FPN in that article was not quite thorough since I was focusing more on YOLOv3 itself. Thus, in this article I decided to write specifically about FPN from its original paper titled “ _Feature Pyramid Networks for Object Detection_ ” [2] so that you can get a better understanding of what it actually is and how it works. Not only that, here I will also demonstrate how to implement FPN from scratch and how to connect it with a CNN backbone and an RPN head.\n···\n## Backbone, Neck, and Head\nBefore we get into FPN, we first need to know that the structure of an object detection model is different from that of the classification model, in which the main difference lies in the very last layer. In a typical classification model, the last layer comprises a number of neurons where each of them corresponds to every single class available in the dataset. Or, in the case of binary classification, the output layer only consists of a single neuron, which is responsible to predict whether a sample belongs to class 0 or 1. This kind of output layer is actually not suitable for detection task since it also requires neurons dedicated for predicting the location and the size of an object in addition to its class.\nSo, in order for a model to be able to predict object location and size, we need to replace the output layer, i.e., the _classification head_ , with the so-called _detection head_. The remaining layers themselves (everything except the _head_) are commonly called _backbone_. Some models that use this structure are YOLOv1 and YOLOv2, where they use a stack of convolution layers as the _backbone_ and a specific _head_ for predicting object location and size within an image as well as its class.\nLearn this step by step with the interactive [Machine Learning](https://roadmap.sh/machine-learning) roadmap.\nOlder object detection models like YOLOv1 and YOLOv2 mentioned above only consist of a _backbone_ and a _head_. As time went on, researchers found that this structure is still not quite optimal, hence they finally came up with an idea by adding a new component called _neck_. As the name suggests, this is essentially something we place between the _backbone_ and the _head_. And FPN, which we are going to talk about in this article, is one of the earliest _necks_ proposed for object detection models. Look at the Figure 1 below to see the high-level architectural view of the older and modern object detection models.\n![](../images/001.png)\nFigure 1. The architecture of an object detection model in general [3].\nThe _backbone_ of a model is mainly responsible for performing feature extraction, while the _neck_ is useful for enhancing feature quality, and the _head_ is for making predictions. Based on this notion, we can say that by applying FPN, a network can potentially achieve better accuracy thanks to the feature enhancement mechanism performed by the _neck_.\n### The Evolution of Multi-Scale Detection Mechanism\nPreviously I mentioned that using _backbone_ and detection _head_ without _neck_ is not quite optimal. This is especially regarding its capability in detecting small objects. Now let’s take a look at the Figure 2 below. The first two YOLO versions I mentioned earlier utilize the structure in image (b), where the bounding box and the object class predictions are made solely on top of the feature map produced by the deepest layer in the backbone. This method is valid, but is effective only for large objects. The reason is pretty straightforward: as an image gets deeper into a network, the spatial dimension shrinks, and more importantly, the pixel information contained in the deeper feature maps becomes a representation of several neighboring pixels in the shallower ones, which causes the spatial information to blend. By doing this, feature maps from deeper layers got a large receptive field, allowing large objects to be detected and recognized easily. However, the degradation of spatial information as we get deeper prevents us from detecting small objects accurately because we do need a detailed pixel location in order to predict the exact coordinates of the objects.\nAdditionally, the receptive field size of a feature map is positively correlated with the amount of semantic information it contains. In the figure below, a feature map of high semantic information is indicated by a thick blue outline. This is essentially why the deepest feature map in (b) has the thickest outline.\n![](../images/body-e4168a51bb6e659fc555.png)\nFigure 2. Comparison of different feature pyramid architectures [2].\nThe most straightforward approach to allow a network to simultaneously detect large and small objects is by using _featurized image pyramid_ (a). This method is able to achieve high accuracy because we can make predictions from different image resolutions. What’s essentially done here is that we rescale our input image into multiple scales, perform feature extraction independently on each scale, and make predictions on the resulting feature maps. The smaller feature map is responsible to detect large objects, whereas the larger one is specialized to detect small objects thanks to its detailed spatial information. However, this method is computationally expensive since we need to process multiple raw images of different scales at once.\nAnother solution was proposed by the authors of SSD (Single Shot Multibox Detector), which in Figure 2 above is the one referred to as _pyramidal feature hierarchy_ (c). So, instead of feeding the network with the same image of different sizes, the authors of SSD attempted to use only the largest image and utilize the internal pyramidal structure of the CNN backbone to make predictions of varying scales. This approach allows the system to be computationally more feasible than option (a). Nevertheless, here we actually got a tradeoff like (b), where the feature map from the deeper layer contains a large amount of semantic information yet having minimal spatial information, while the feature map from the shallower layer has a lot of spatial information but it does not have that much semantic information. It is important to note that a detailed spatial information might not be quite important for large objects as we can just approximate the general shape of that object. However, both spatial and semantic information are necessary for detecting small objects because not only the detailed coordinates, but the model also needs to understand what’s actually inside the bounding box. So, while it is true that method (c) is indeed able to detect both large and small objects, but its ability in detecting the latter is not yet optimal.\nAnd here’s where FPN comes as a solution. If we take a look at image (d) in Figure 2, we can see that the predictions are made on top of the corresponding feature maps which are all semantically rich. This essentially allows objects of varying scales, including the smaller ones, to be detected accurately. We are going to talk about the details of how FPN enriches feature maps in the subsequent section.\n···\n## How FPN Works\nThe idea of FPN is to inject information from the deeper feature maps into the shallower ones, and by doing so we will have the shallower feature maps not only containing high spatial information but also high semantic information coming from the deeper part of the network. In theory, this should result in a better detection accuracy on small objects since the large feature maps are now enriched with a lot of semantic information. In order to achieve this, they introduce the so-called _top-down pathway_ and _lateral connections_. You can see the complete FPN architecture in Figure 3 below, which is essentially the detailed version of the one in Figure 2 (d).\n![](../images/body-76aa4d1ace7e6b83a894.png)\nFigure 3. The detailed FPN architecture [3].\nThe authors of this paper decided to use ResNet-50 and ResNet-101 as the backbone. Suppose we were to use the former, we would later have the _conv2_ , _conv3_ , _conv4_ , and _conv5_ layers repeated 3, 4, 6, and 3 times, respectively, as suggested by the architectural details of ResNet in Figure 4. _C2_ , _C3_ , _C4_ , and _C5_ , which are the tensors produced by the last layer of the corresponding stage, are going to be transferred to the _top-down pathway_ through _lateral connections_ , i.e., the arrows going out from the backbone.\n![](../images/body-6e5a7f00da4be75aea9a.png)\nFigure 4. The ResNet architecture [3, 4].\nThe _top-down pathway_ is used to transfer semantic information from deeper layers, whereas _lateral connections_ are used to preserve spatial information. We aggregate the two by performing element-wise summation, which the detailed process is given in Figure 5 below. For the tensors that come from the backbone (_C_), we first need to apply 1×1 conv to them. This convolution layer is responsible for adjusting the number of channels so that it matches with the tensor coming from the top-down pathway. The tensor from the top-down pathway itself (_M+1_) undergoes 2× nearest-neighbor upsampling. These processes are essentially done because we need both tensors to have the exact same dimension so that element-wise summation can be performed. As the summation is done, the resulting tensor is now referred to as _M_. This tensor has some aliasing effect due to the upsampling process we did earlier, hence we need to apply a 3×3 convolution to reduce that effect. Finally, we got the _P_ tensor, which is ready to be forwarded to the detection head.\n![](../images/body-cfb1a470dcca88dcafe2.png)\nFigure 5. How feature maps from lateral connection (C) and the top-down pathway (M+1) are aggregated [3].\nKeep in mind that all processes described in Figure 5 above only apply to _M2_ , _M3_ , and _M4_. Computing _M5_ is actually much simpler (see Figure 6), where the only thing we need to do is just to adjust the number of channels using 1×1 conv to make it uniform with the tensors in the other lateral connections. The _M5_ tensor itself does not need to be processed further with 3×3 conv because there is nothing to be smoothed out due to the absence of upsampling mechanism. And so, we can basically say that _P5_ is the exact same tensor as _M5_.\n![](../images/body-076dd85e523657447a62.png)\nFigure 6. The way to compute M5 and P5 is slightly different from that of M2-M4 and P2-P4 [3].\nAnd well I think that’s everything about the theory behind FPN. In the next section I am going to bring you into the lower-level view of the architecture by implementing it from scratch with PyTorch.\n···\n## FPN From Scratch\n### CNN Backbone\nSo as seen in the Codeblock 1 below, the very first thing we need to do in the code is to import the required modules.\npython\n\n```\n# Codeblock 1\nimport torch\nimport torch.nn as nn\n```\n\nSince the focus of this article is FPN, here I will use a dummy model for the backbone instead of using the actual ResNet in order to simplify things. But still, the layers in the code are named according to Figure 3 and 4: `conv1`, `conv2`, `conv3`, `conv4` and `conv5` as shown in Codeblock 2 below. The output tensor dimension of each stage is also set according to the original ResNet architecture. So, although this backbone is just a plain CNN-based model, you can think of this like a normal ResNet.\nNext, what we do inside the `forward()` method is to connect all the layers. If you take a closer look at the code, you will notice that each convolution layer is followed by a ReLU activation function and a maxpooling layer. The maxpooling layer itself is set to have the stride of 2, effectively halves the spatial dimension of the feature map. By repeating maxpooling layers multiple times, we will have our feature map gradually gets smaller as we get deeper into the network. This essentially creates a pyramidal structure within the CNN backbone which is leveraged by FPN to achieve high detection accuracy on varying object scales. In CNN, reducing spatial dimension like this is a standard practice to reduce computational complexity to compensate the increase of the number of channels.\nStill within the `forward()` method, don’t forget to clone the main tensor `x` as shown at the lines marked with `#(1)`, `#(2)`, and `#(3)`. The copied tensors, which are named `c2`, `c3`, and `c4`, will then be the return values of the `CNN` class alongside the feature map from the main flow (`c5`) (`#(4)`).\npython\n\n```\n# Codeblock 2\nclass CNN(nn.Module):\n def __init__(self):\n super().__init__()\n \n self.relu = nn.ReLU()\n self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)\n \n self.conv1 = nn.Conv2d(in_channels=3, out_channels=64, kernel_size=3, padding=1)\n self.conv2 = nn.Conv2d(in_channels=64, out_channels=256, kernel_size=3, padding=1)\n self.conv3 = nn.Conv2d(in_channels=256, out_channels=512, kernel_size=3, padding=1)\n self.conv4 = nn.Conv2d(in_channels=512, out_channels=1024, kernel_size=3, padding=1)\n self.conv5 = nn.Conv2d(in_channels=1024, out_channels=2048, kernel_size=3, padding=1)\n \n def forward(self, x):\n print(f'original\\t: {x.size()}\\n')\n \n x = self.relu(self.conv1(x))\n print(f'after conv1\\t: {x.size()}')\n \n x = self.maxpool(x)\n print(f'after maxpool\\t: {x.size()}\\n')\n \n x = self.relu(self.conv2(x))\n print(f'after conv2\\t: {x.size()}')\n \n x = self.maxpool(x)\n print(f'after maxpool\\t: {x.size()}\\n')\n \n c2 = x.clone() #(1)\n \n x = self.relu(self.conv3(x))\n print(f'after conv3\\t: {x.size()}')\n \n x = self.maxpool(x)\n print(f'after maxpool\\t: {x.size()}\\n')\n \n c3 = x.clone() #(2)\n \n x = self.relu(self.conv4(x))\n print(f'after conv4\\t: {x.size()}')\n \n x = self.maxpool(x)\n print(f'after maxpool\\t: {x.size()}\\n')\n \n c4 = x.clone() #(3)\n \n x = self.relu(self.conv5(x))\n print(f'after conv5\\t: {x.size()}')\n \n c5 = self.maxpool(x)\n print(f'after maxpool\\t: {c5.size()}\\n')\n \n return c2, c3, c4, c5 #(4)\n```\n\nAs the `CNN` class is done, we will now try to pass a dummy RGB image of size 224×224 through the network. This tensor dimension is chosen based on the input shape of the original ResNet.\npython\n\n```\n# Codeblock 3\ncnn = CNN()\n\nx = torch.randn(1, 3, 224, 224)\nout_cnn = cnn(x)\n```\n\nAnd below is what the output looks like. Here we can see that the number of channels after each conv layer matches exactly with the ResNet structure given in Figure 4. Not only that, it is also seen that the spatial dimension of our dummy tensor successfully halved after each stage thanks to the maxpooling layers. This essentially indicates that our simple CNN model really mimics the general structure of a ResNet model.\nmarkup\n\n```\n# Codeblock 3 Outputoriginal : torch.Size([1, 3, 224, 224])after conv1 : torch.Size([1, 64, 224, 224])after maxpool : torch.Size([1, 64, 112, 112])after conv2 : torch.Size([1, 256, 112, 112])after maxpool : torch.Size([1, 256, 56, 56])after conv3 : torch.Size([1, 512, 56, 56])after maxpool : torch.Size([1, 512, 28, 28])after conv4 : torch.Size([1, 1024, 28, 28])after maxpool : torch.Size([1, 1024, 14, 14])after conv5 : torch.Size([1, 2048, 14, 14])after maxpool : torch.Size([1, 2048, 7, 7])\n```\n\nWe can also check what the returned tensors look like by running the code below. You can see in the resulting output that the `c2` tensor has the shape of 256×56×56, `c3` is of shape 512×28×28, and so on. By the way, you can just ignore the number 1 in the 0th axis since it only indicates the number of samples we pass within a single batch.\npython\n\n```\n# Codeblock 4\nc2, c3, c4, c5 = out_cnn\n\nprint(c2.shape)\nprint(c3.shape)\nprint(c4.shape)\nprint(c5.shape)\n```\n\nmarkup\n\n```\n# Codeblock 4 Outputtorch.Size([1, 256, 56, 56])torch.Size([1, 512, 28, 28])torch.Size([1, 1024, 14, 14])torch.Size([1, 2048, 7, 7])\n```\n\n···\n### FPN Neck\nAs the CNN backbone is completed, now let’s move on to the FPN neck. In the Codeblock 5 below, we first initialize the upsample layer (`#(1)`) which we will use every time we want to double the spatial dimension of the _M_ tensor. Here I set the `mode` parameter to `nearest` as suggested in the paper, which is actually a very simple interpolation method, allowing the process to be fast. Take a look at Figure 7 to see what a nearest-neighbor interpolation looks like.\npython\n\n```\n# Codeblock 5\nclass FPN(nn.Module):\n def __init__(self):\n super().__init__()\n \n self.upsample = nn.Upsample(scale_factor=2, mode='nearest') #(1)\n \n self.lateral_c5 = nn.Conv2d(in_channels=2048, out_channels=256, kernel_size=1)\n self.lateral_c4 = nn.Conv2d(in_channels=1024, out_channels=256, kernel_size=1)\n self.lateral_c3 = nn.Conv2d(in_channels=512, out_channels=256, kernel_size=1)\n self.lateral_c2 = nn.Conv2d(in_channels=256, out_channels=256, kernel_size=1)\n \n self.smooth_m4 = nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, padding=1)\n self.smooth_m3 = nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, padding=1)\n self.smooth_m2 = nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, padding=1)\n \n def forward(self, c2, c3, c4, c5):\n m5 = self.lateral_c5(c5)\n p5 = m5\n \n m4 = self.upsample(m5) + self.lateral_c4(c4)\n p4 = self.smooth_m4(m4)\n \n m3 = self.upsample(m4) + self.lateral_c3(c3)\n p3 = self.smooth_m3(m3)\n \n m2 = self.upsample(m3) + self.lateral_c2(c2)\n p2 = self.smooth_m2(m2)\n \n return p2, p3, p4, p5\n```\n\n![](../images/body-fea0ee17592f14e8b54e.png)\nFigure 7. An example of a 2x upsampling process with nearest-neighbor interpolation method [3].\nIf you go back to Codeblock 4, you will see that the `c{5,4,3,2}`tensors returned by the backbone have different number of channels. This is basically the reason that we initialize the `lateral_c{5,4,3,2}`layers to process these tensors so that the resulting channel counts will be uniform. According to the paper, we need to set these convolution layers to produce 256 output channels, which is the reason why we use that number for the `out_channels`parameter. \nNext, based on the Figure 7, you can just imagine how pixelated the resulting feature maps are after being upsampled. Thus, we need to process the `m{4,3,2}`tensors further with the 3×3 conv layers which I refer to as `smooth_m{4,3,2}`. As all layers have been initialized, what we need to do next is to assemble them in the `forward()`method according to the structure I showed you earlier in Figure 3. \nIn addition to this, the paper also mentions that we don’t need to implement any nonlinearities within the FPN, which is the reason that all the convolution layers in the `FPN`class above are not followed with ReLU. Now in Codeblock 6 below I try to pass the _C_ tensors we obtained earlier through the FPN neck we just created. We can see in the output that the resulting tensors have different spatial resolutions. Later on, the `p2`tensor (the one that has 56×56 dimension) will be forwarded to a detection head to detect small objects, whereas `p5`(the 7×7 tensor) is going to be responsible for large objects.\npython\n\n```\n# Codeblock 6\nfpn = FPN()\n\nout_fpn = fpn(c2, c3, c4, c5)\np2, p3, p4, p5 = out_fpn\n\nprint(p2.shape)\nprint(p3.shape)\nprint(p4.shape)\nprint(p5.shape)\n```\n\nmarkup\n\n```\n# Codeblock 6 Outputtorch.Size([1, 256, 56, 56])torch.Size([1, 256, 28, 28])torch.Size([1, 256, 14, 14])torch.Size([1, 256, 7, 7])\n```\n\nHere have already completed the FPN part. Remember that FPN is just the _neck_ of a detection model, which essentially means that at this point we still haven’t got the bounding box prediction just yet. In order to actually obtain the prediction result, we need to connect a specific _head_ to the FPN, and in this case I will use the RPN (Region Proposal Network) head.\n···\n### RPN Head\nIn case you’re still not yet familiar with RPN, this is essentially the head of an object detection model used for creating bounding box, which was first proposed in the Faster R-CNN paper. Note that while in this demonstration we refer to the RPN as a head, keep in mind that it is actually not a complete detection head since it has no capability in performing classification on the detected objects.\nWe can see in the RPN architecture below that it utilizes the so-called _cls layer_ and _reg layer_ , which produce _objectness_ score and the bounding box coordinates, respectively. The objectness score tensor has the length of 2 _k_ , where _k_ is the number of predetermined anchor boxes and 2 is the probability of the corresponding anchor box being there. We can think of this like a binary classification treated with a one-hot representation (object/non-object). Meanwhile, the number 4 in the 4 _k_ length of the coordinates tensor simply correspond to the _xywh_ prediction.\n![](../images/body-95ca0e48450b435d8fa7.png)\nFigure 8. The structure of RPN [4].\nGoing back to our code implementation, in the Codeblock 7 below we initialize the `intermediate`, `cls`, and `reg` layers within the `__init__()` method of the RPN class. Note that the intermediate layer is the only one that uses 3×3 convolution, whereas both the `cls` and `reg` layers use 1×1 convs. Regarding the number of channels, the `intermediate` layer maps the input tensor into 256 channels, while the `cls` and `reg` map it into 2 _k_ and 4 _k_ , respectively. Finally, we can simply connect these layers within the `forward()` method.\npython\n\n```\n# Codeblock 7\nNUM_ANCHORS = 3\n\nclass RPN(nn.Module):\n def __init__(self):\n super().__init__()\n \n self.intermediate = nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, padding=1)\n \n self.cls = nn.Conv2d(in_channels=256, out_channels=NUM_ANCHORS*2, kernel_size=1)\n self.reg = nn.Conv2d(in_channels=256, out_channels=NUM_ANCHORS*4, kernel_size=1)\n \n def forward(self, x):\n x = self.intermediate(x)\n \n objectness_scores = self.cls(x)\n bbox_regressions = self.reg(x)\n \n return objectness_scores, bbox_regressions\n```\n\nNow let’s test if our RPN class works properly by running the Codeblock 8 below. Here I test it on the `p2` feature map we obtained from Codeblock 6.\npython\n\n```\n# Codeblock 8\nrpn = RPN()\n\np2_objectness, p2_bbox = rpn(p2)\n\nprint(p2_objectness.shape)\nprint(p2_bbox.shape)\n```\n\nBelow is what the resulting output looks like. You can see that `p2_objectness` is a tensor having the size of 6×56×56, indicating that every single pixel in the 56×56 spatial dimension contains 6 prediction values, where the first 2 values are for the first anchor box, the second 2 values are for the second anchor box, and the last 2 values are for the third one. The similar thing also applies to the `p2_bbox` tensor, which in this case it contains the _xywh_ values.\nmarkup\n\n```\n# Codeblock 8 Outputtorch.Size([1, 6, 56, 56])torch.Size([1, 12, 56, 56])\n```\n\n···\n### The Entire Detection Model\nIn Codeblock 9 below we are going to construct the entire detection model so that you can better understand how FPN works together with the other components. Here in the `__init__()` method I initialize the CNN bottleneck, FPN neck, and RPN head. In the `forward()` method, we first pass the image tensor into the CNN (`#(1)`). This backbone returns 4 tensors, which are ready to be connected to the FPN through lateral connections. Next, at line `#(2)` we feed all the _C_ tensors as the input of FPN, producing the _P_ tensors. Lastly, we use all the _P_ s as the input to the RPN (`#(3–4)`). Keep in mind that RPN shares its parameters across all detection heads, so we only need to initialize it once and use it for all feature map of different scales.\npython\n\n```\n# Codeblock 9\nclass DetectionModel(nn.Module):\n def __init__(self):\n super().__init__()\n \n self.cnn = CNN()\n self.fpn = FPN()\n self.rpn = RPN()\n \n def forward(self, x):\n \n c2, c3, c4, c5 = self.cnn(x) #(1)\n p2, p3, p4, p5 = self.fpn(c2, c3, c4, c5) #(2)\n \n p2_pred = self.rpn(p2) #(3)\n p3_pred = self.rpn(p3)\n p4_pred = self.rpn(p4)\n p5_pred = self.rpn(p5) #(4)\n \n return p2_pred, p3_pred, p4_pred, p5_pred\n```\n\nNow as the detection head is complete, we can test it with the Codeblock 10 below. Here I try to pass a dummy tensor of size 1×3×224×224, simulating a single RGB image of size 224×224 (`#(1)`). Next, we can just pass it through the `detection_model` (`#(2)`) and unpack the prediction results (`#(3–4)`).\npython\n\n```\n# Codeblock 10\ndetection_model = DetectionModel()\n\nx = torch.randn(1, 3, 224, 224) #(1)\np2_pred, p3_pred, p4_pred, p5_pred = detection_model(x) #(2)\n\np2_objectness, p2_bbox = p2_pred #(3)\np3_objectness, p3_bbox = p3_pred\np4_objectness, p4_bbox = p4_pred\np5_objectness, p5_bbox = p5_pred #(4)\n \nprint(p2_objectness.shape)\nprint(p3_objectness.shape)\nprint(p4_objectness.shape)\nprint(p5_objectness.shape)\nprint()\n\nprint(p2_bbox.shape)\nprint(p3_bbox.shape)\nprint(p4_bbox.shape)\nprint(p5_bbox.shape)\n```\n\nBelow is what the output looks like. You can see here that the resulting tensor dimensions are as intended, where the `objectness` and `bbox` tensors contain 6 and 12 values for each grid cell, respectively. So, I believe this implementation is correct and thus ready to be trained for object detection task.\nmarkup\n\n```\n# Codeblock 10 Outputtorch.Size([1, 6, 56, 56])torch.Size([1, 6, 28, 28])torch.Size([1, 6, 14, 14])torch.Size([1, 6, 7, 7])torch.Size([1, 12, 56, 56])torch.Size([1, 12, 28, 28])torch.Size([1, 12, 14, 14])torch.Size([1, 12, 7, 7])\n```\n\n···\n## Ending\nI think that’s pretty much all about the underlying theory and the from-scratch implementation of FPN. Here I challenge you to try implementing FPN on the real ResNet instead of a dummy CNN model like I demonstrated above. I actually got a separate article about ResNet, which you can check as a reference [5]. Or, it is also possible to use other models if you want, such as VGG, ResNeXt, ConvNeXt, etc since FPN can basically work on any CNN-based backbone model. Not only that, it would also be better if you can implement YOLO-style head as a replacement of RPN, which the examples can be seen in my previous articles given at references number [6] for YOLOv1, [7] for YOLOv2, and [1] for YOLOv3.\nPlease let me know if there are mistakes in my writing or in the code. Thanks for reading! By the way, you can find the code used in this article in my GitHub repo [8].\n···\n## References\n[1] [Muhammad Ardi](https://medium.com/u/9801a58700ac). YOLOv3 Paper Walkthrough: Even Better, but Not That Much. Medium. [Accessed June 1, 2026].\n[2] Tsung-Yi Lin. Feature Pyramid Networks for Object Detection. Arxiv. [Accessed September 9, 2025].\n[3] Image created originally by author.\n[4] Shaoqing Ren. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Arxiv. [Accessed September 9, 2025].\n[5] Muhammad Ardi. Paper Walkthrough: Residual Network (ResNet). Python in Plain English. [Accessed September 9, 2025].\n[6] [Muhammad Ardi](https://medium.com/u/9801a58700ac). YOLOv1 Paper Walkthrough: The Day YOLO First Saw the World. Medium. [Accessed June 1, 2026].\n[7] [Muhammad Ardi](https://medium.com/u/9801a58700ac). YOLOv2 & YOLO9000 Paper Walkthrough: Better, Faster, Stronger. Medium. [Accessed June 1, 2026].\n[8] MuhammadArdiPutra. FPN. GitHub. [Accessed September 9, 2025].\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](../images/body-59e1c17daab1fcb582c5.jpeg)\nWritten By\n## [Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\nDeep learning storyteller\n[See all from Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\n[Deep Dives](https://towardsdatascience.com/tag/deep-dives/)[Artificial Intelligence](https://towardsdatascience.com/tag/artificial-intelligence/)[Deep Learning](https://towardsdatascience.com/tag/deep-learning/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/)[Python](https://towardsdatascience.com/tag/python/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Ffpn-paper-walkthrough-leveraging-the-internal-pyramid%2F&title=FPN%20Paper%20Walkthrough%3A%20Leveraging%20the%20Internal%C2%A0Pyramid)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Ffpn-paper-walkthrough-leveraging-the-internal-pyramid%2F&title=FPN%20Paper%20Walkthrough%3A%20Leveraging%20the%20Internal%C2%A0Pyramid)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Ffpn-paper-walkthrough-leveraging-the-internal-pyramid%2F&text=FPN%20Paper%20Walkthrough%3A%20Leveraging%20the%20Internal%C2%A0Pyramid)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [![](../images/002.png)](https://towardsdatascience.com/dueling-double-deep-q-learning-using-tensorflow-2-x-7bbbcec06a2a/)\n## [Dueling Double Deep Q Learning using Tensorflow 2.x](https://towardsdatascience.com/dueling-double-deep-q-learning-using-tensorflow-2-x-7bbbcec06a2a/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nImplementing dueling double deep q learning with TensorFlow 2.x\n[Abhishek Suran](https://towardsdatascience.com/author/abhisheksuran/)\nJuly 10, 20202 min read\n * [![](../images/003.jpeg)](https://towardsdatascience.com/semantic-textual-similarity-with-bert-fc800656e7a3/)\n## [Semantic Textual Similarity with BERT](https://towardsdatascience.com/semantic-textual-similarity-with-bert-fc800656e7a3/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nHow to use BERT to calculate the semantic similarity between two texts\n[Ruben Winastwan](https://towardsdatascience.com/author/marcellusruben/)\nFebruary 15, 202311 min read\n * [![](../images/004.jpg)](https://towardsdatascience.com/stop-using-print-and-start-debugging-3cd11524c92a/)\n## [Stop using Print, and start Debugging](https://towardsdatascience.com/stop-using-print-and-start-debugging-3cd11524c92a/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nLearn how to debug your Python program with Visual Studio Code\n[Marcello Politi](https://towardsdatascience.com/author/marcellopoliti/)\nDecember 5, 20227 min read\n * [![](../images/005.png)](https://towardsdatascience.com/getting-started-with-sentiment-analysis-731531ec880d/)\n## [Getting Started With Sentiment Analysis](https://towardsdatascience.com/getting-started-with-sentiment-analysis-731531ec880d/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nNotes From Natural Lanaguage Processing Specialization Course1 Week1\n[Kurtis Pykes](https://towardsdatascience.com/author/kurtispykes/)\nOctober 17, 20208 min read\n * [![](../images/006.jpg)](https://towardsdatascience.com/5-visualizations-with-python-to-show-simultaneous-changes-in-geospatial-data-ddc2eaab9d78/)\n## [5 Visualizations with Python to Show Simultaneous Changes in Geospatial Data](https://towardsdatascience.com/5-visualizations-with-python-to-show-simultaneous-changes-in-geospatial-data-ddc2eaab9d78/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nIdeas to express simultaneous data value changes in multiple locations over time\n[Boriharn K](https://towardsdatascience.com/author/borih.k/)\nNovember 4, 20237 min read\n * [![](../images/007.png)](https://towardsdatascience.com/interpretable-machine-learning-using-shap-theory-and-applications-26c12f7a7f1a/)\n## [Interpretable Machine Learning using SHAP - theory and applications](https://towardsdatascience.com/interpretable-machine-learning-using-shap-theory-and-applications-26c12f7a7f1a/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nIntroduction\n[Khalil Zlaoui](https://towardsdatascience.com/author/khalilz/)\nOctober 1, 20218 min read\n * [![](../images/008.jpg)](https://towardsdatascience.com/tds-newsletter-how-to-build-robust-data-and-ai-systems/)\n## [TDS Newsletter: How to Build Robust Data and AI Systems](https://towardsdatascience.com/tds-newsletter-how-to-build-robust-data-and-ai-systems/)\n[The Variable](https://towardsdatascience.com/category/the-variable/)\nGain a high-level understanding of how the different pieces in your workflow come together\n[TDS Editors](https://towardsdatascience.com/author/towardsdatascience/)\nNovember 20, 20253 min read\n * [![](../images/009.jpeg)](https://towardsdatascience.com/build-a-simple-web-app-to-classify-us-50-dollar-100-dollar-bills-d4ef8298ae6/)\n## [Build a Simple Web App to Classify US 50 Dollar & 100 Dollar Bills](https://towardsdatascience.com/build-a-simple-web-app-to-classify-us-50-dollar-100-dollar-bills-d4ef8298ae6/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nFrom model to production using Jupyter notebook\n[ASM Saiful Islam](https://towardsdatascience.com/author/a-islam/)\nJanuary 25, 202112 min read\n * [![](../images/010.jpg)](https://towardsdatascience.com/uncertainty-in-deep-learning-experiments-with-bayesian-cnn-1ca37ddb6954/)\n## [Uncertainty in Deep Learning: Experiments with Bayesian CNN](https://towardsdatascience.com/uncertainty-in-deep-learning-experiments-with-bayesian-cnn-1ca37ddb6954/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nHow does KL-Weight affect the model? Writing a Bayesian CNN for chest X-Ray images.\n[Kaan Bıçakcı](https://towardsdatascience.com/author/frightera/)\nApril 12, 20225 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n#### Recommended Articles \nClose\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n", "metadata": { "title": "FPN Paper Walkthrough: Leveraging the Internal Pyramid | Towards Data Science", "description": "Understanding how FPN allows deep learning models detecting small objects and how to implement it from scratch", "keywords": null, "author": "Muhammad Ardi Putra", "og:locale": "en_US", "og:type": "article", "og:title": "FPN Paper Walkthrough: Leveraging the Internal Pyramid", "og:description": "Understanding how FPN allows deep learning models detecting small objects and how to implement it from scratch", "og:url": "https://towardsdatascience.com/fpn-paper-walkthrough-leveraging-the-internal-pyramid/", "og:site_name": "Towards Data Science", "og:image": "https://assets.insightmediagroup.io/media/wp-content/uploads/2026/06/0_FE3IgUisiKV5ZlqW.jpg", "og:image:width": "1350", "og:image:height": "900", "og:image:type": "image/jpeg", "twitter:card": "summary_large_image", "twitter:site": "@TDataScience", "twitter:creator": "@TDataScience", "twitter:label1": "Written by", "twitter:data1": "Muhammad Ardi Putra", "twitter:label2": "Est. reading time", "twitter:data2": "18 min read", "article:published_time": "2026-06-04T07:30:05.000Z", "article:modified_time": "2026-06-04T07:30:05.000Z", "article:author": "https://towardsdatascience.com/author/muhammad_ardi/" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 202, "error": "" }, "code_blocks": [ { "language": "txt", "file": "attachments/dcf4c8a3c08b/code/block_000.txt", "chars": 49 }, { "language": "txt", "file": "attachments/dcf4c8a3c08b/code/block_001.txt", "chars": 1817 }, { "language": "txt", "file": "attachments/dcf4c8a3c08b/code/block_002.txt", "chars": 76 }, { "language": "txt", "file": "attachments/dcf4c8a3c08b/code/block_003.txt", "chars": 511 }, { "language": "txt", "file": "attachments/dcf4c8a3c08b/code/block_004.txt", "chars": 104 }, { "language": "txt", "file": "attachments/dcf4c8a3c08b/code/block_005.txt", "chars": 133 }, { "language": "txt", "file": "attachments/dcf4c8a3c08b/code/block_006.txt", "chars": 1247 }, { "language": "txt", "file": "attachments/dcf4c8a3c08b/code/block_007.txt", "chars": 147 }, { "language": "txt", "file": "attachments/dcf4c8a3c08b/code/block_008.txt", "chars": 131 }, { "language": "txt", "file": "attachments/dcf4c8a3c08b/code/block_009.txt", "chars": 612 }, { "language": "txt", "file": "attachments/dcf4c8a3c08b/code/block_010.txt", "chars": 109 }, { "language": "txt", "file": "attachments/dcf4c8a3c08b/code/block_011.txt", "chars": 74 }, { "language": "txt", "file": "attachments/dcf4c8a3c08b/code/block_012.txt", "chars": 558 }, { "language": "txt", "file": "attachments/dcf4c8a3c08b/code/block_013.txt", "chars": 513 }, { "language": "txt", "file": "attachments/dcf4c8a3c08b/code/block_014.txt", "chars": 230 } ], "images": [ { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T18:10:14Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T18:10:14Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T18:10:14Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T18:10:15Z" } ], "audio": [], "videos": [ { "url": "https://www.youtube.com/c/TowardsDataScience", "kind": "channel_reference", "status": "reference_only" } ], "links": [ { "type": "internal", "href": "https://towardsdatascience.com/" }, { "type": "internal", "href": "https://towardsdatascience.com/latest/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/editors-pick/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/the-variable/" }, { "type": "internal", "href": "https://towardsdatascience.com/shipai/" }, { "type": "internal", "href": "https://towardsdatascience.com/submissions/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/artificial-intelligence/deep-learning/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/muhammad_ardi/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/deep-dives/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/artificial-intelligence/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/deep-learning/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/machine-learning/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/python/" }, { "type": "internal", "href": "https://towardsdatascience.com/dueling-double-deep-q-learning-using-tensorflow-2-x-7bbbcec06a2a/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/artificial-intelligence/machine-learning/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/abhisheksuran/" }, { "type": "internal", "href": "https://towardsdatascience.com/semantic-textual-similarity-with-bert-fc800656e7a3/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/marcellusruben/" }, { "type": "internal", "href": "https://towardsdatascience.com/stop-using-print-and-start-debugging-3cd11524c92a/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/marcellopoliti/" }, { "type": "external", "href": "https://write.insightmediagroup.io" }, { "type": "external", "href": "https://www.linkedin.com/company/towards-data-science/" }, { "type": "external", "href": "https://x.com/TDataScience" }, { "type": "external", "href": "https://roadmap.sh/machine-learning" }, { "type": "external", "href": "https://medium.com/u/9801a58700ac" }, { "type": "external", "href": "https://ai.gopubby.com/yolov3-paper-walkthrough-even-better-but-not-that-much-4dc6c0c1b42c" }, { "type": "external", "href": "https://arxiv.org/abs/1612.03144" }, { "type": "external", "href": "https://arxiv.org/abs/1506.01497" }, { "type": "external", "href": "https://medium.com/python-in-plain-english/paper-walkthrough-residual-network-resnet-62af58d1c521" }, { "type": "external", "href": "https://medium.com/ai-advances/yolov1-paper-walkthrough-the-day-yolo-first-saw-the-world-ccff8b60d84b" }, { "type": "external", "href": "https://ai.gopubby.com/yolov2-yolo9000-paper-walkthrough-better-faster-stronger-c9906e0438a3" }, { "type": "external", "href": "https://github.com/MuhammadArdiPutra/medium_articles/blob/main/FPN.ipynb" }, { "type": "external", "href": "https://www.facebook.com/sharer/sharer.php?title=FPN+Paper+Walkthrough%3A+Leveraging+the+Internal%C2%A0Pyramid&u=https%3A%2F%2Ftowardsdatascience.com%2Ffpn-paper-walkthrough-leveraging-the-internal-pyramid%2F" }, { "type": "external", "href": "https://www.linkedin.com/shareArticle?mini=true&title=FPN+Paper+Walkthrough%3A+Leveraging+the+Internal%C2%A0Pyramid&url=https%3A%2F%2Ftowardsdatascience.com%2Ffpn-paper-walkthrough-leveraging-the-internal-pyramid%2F" }, { "type": "external", "href": "https://x.com/share?text=FPN+Paper+Walkthrough%3A+Leveraging+the+Internal%C2%A0Pyramid&url=https%3A%2F%2Ftowardsdatascience.com%2Ffpn-paper-walkthrough-leveraging-the-internal-pyramid%2F" }, { "type": "external", "href": "https://www.youtube.com/c/TowardsDataScience" }, { "type": "external", "href": "https://www.threads.net/@towardsdatascience" }, { "type": "external", "href": "https://bsky.app/profile/towardsdatascience.com" }, { "type": "external", "href": "https://thenewstack.io" }, { "type": "external", "href": "https://roadmap.sh" } ], "repair_version": 2, "repaired_at": "2026-10-20T17:34:47Z", "source_url": "https://towardsdatascience.com/fpn-paper-walkthrough-leveraging-the-internal-pyramid/", "delivery_status": "unverified", "discovery_status": "http_403", "missing_reason": "源页面未重新核验:http_403；已按保留正文补采图片", "repair_error": "", "actual_video_files": [], "expected_saved_body_images": 4, "image_discovery": "saved_markdown", "source_body_markdown": "![Revisit consent button](https://cdn-cookieyes.com/assets/images/revisit.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](https://cdn-cookieyes.com/assets/images/close.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\n# FPN Paper Walkthrough: Leveraging the Internal Pyramid\nUnderstanding how FPN allows deep learning models detecting small objects and how to implement it from scratch\n[Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\nJune 4, 202618 min read\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/06/0_FE3IgUisiKV5ZlqW.jpg)\nPhoto by \nIn my previous article I was talking about YOLOv3 [1]. One of the factors that makes this YOLO version better than its predecessors is its ability in detecting small objects thanks to its FPN-like neck adopted by the model. Unfortunately, my explanation about FPN in that article was not quite thorough since I was focusing more on YOLOv3 itself. Thus, in this article I decided to write specifically about FPN from its original paper titled “ _Feature Pyramid Networks for Object Detection_ ” [2] so that you can get a better understanding of what it actually is and how it works. Not only that, here I will also demonstrate how to implement FPN from scratch and how to connect it with a CNN backbone and an RPN head.\n···\n## Backbone, Neck, and Head\nBefore we get into FPN, we first need to know that the structure of an object detection model is different from that of the classification model, in which the main difference lies in the very last layer. In a typical classification model, the last layer comprises a number of neurons where each of them corresponds to every single class available in the dataset. Or, in the case of binary classification, the output layer only consists of a single neuron, which is responsible to predict whether a sample belongs to class 0 or 1. This kind of output layer is actually not suitable for detection task since it also requires neurons dedicated for predicting the location and the size of an object in addition to its class.\nSo, in order for a model to be able to predict object location and size, we need to replace the output layer, i.e., the _classification head_ , with the so-called _detection head_. The remaining layers themselves (everything except the _head_) are commonly called _backbone_. Some models that use this structure are YOLOv1 and YOLOv2, where they use a stack of convolution layers as the _backbone_ and a specific _head_ for predicting object location and size within an image as well as its class.\nLearn this step by step with the interactive [Machine Learning](https://roadmap.sh/machine-learning) roadmap.\nOlder object detection models like YOLOv1 and YOLOv2 mentioned above only consist of a _backbone_ and a _head_. As time went on, researchers found that this structure is still not quite optimal, hence they finally came up with an idea by adding a new component called _neck_. As the name suggests, this is essentially something we place between the _backbone_ and the _head_. And FPN, which we are going to talk about in this article, is one of the earliest _necks_ proposed for object detection models. Look at the Figure 1 below to see the high-level architectural view of the older and modern object detection models.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/06/1Ue7IIX9NGp6gT0bqCavNdw.png)\nFigure 1. The architecture of an object detection model in general [3].\nThe _backbone_ of a model is mainly responsible for performing feature extraction, while the _neck_ is useful for enhancing feature quality, and the _head_ is for making predictions. Based on this notion, we can say that by applying FPN, a network can potentially achieve better accuracy thanks to the feature enhancement mechanism performed by the _neck_.\n### The Evolution of Multi-Scale Detection Mechanism\nPreviously I mentioned that using _backbone_ and detection _head_ without _neck_ is not quite optimal. This is especially regarding its capability in detecting small objects. Now let’s take a look at the Figure 2 below. The first two YOLO versions I mentioned earlier utilize the structure in image (b), where the bounding box and the object class predictions are made solely on top of the feature map produced by the deepest layer in the backbone. This method is valid, but is effective only for large objects. The reason is pretty straightforward: as an image gets deeper into a network, the spatial dimension shrinks, and more importantly, the pixel information contained in the deeper feature maps becomes a representation of several neighboring pixels in the shallower ones, which causes the spatial information to blend. By doing this, feature maps from deeper layers got a large receptive field, allowing large objects to be detected and recognized easily. However, the degradation of spatial information as we get deeper prevents us from detecting small objects accurately because we do need a detailed pixel location in order to predict the exact coordinates of the objects.\nAdditionally, the receptive field size of a feature map is positively correlated with the amount of semantic information it contains. In the figure below, a feature map of high semantic information is indicated by a thick blue outline. This is essentially why the deepest feature map in (b) has the thickest outline.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/06/1Z9vt5hdEstYxlcPOOapjLw.png)\nFigure 2. Comparison of different feature pyramid architectures [2].\nThe most straightforward approach to allow a network to simultaneously detect large and small objects is by using _featurized image pyramid_ (a). This method is able to achieve high accuracy because we can make predictions from different image resolutions. What’s essentially done here is that we rescale our input image into multiple scales, perform feature extraction independently on each scale, and make predictions on the resulting feature maps. The smaller feature map is responsible to detect large objects, whereas the larger one is specialized to detect small objects thanks to its detailed spatial information. However, this method is computationally expensive since we need to process multiple raw images of different scales at once.\nAnother solution was proposed by the authors of SSD (Single Shot Multibox Detector), which in Figure 2 above is the one referred to as _pyramidal feature hierarchy_ (c). So, instead of feeding the network with the same image of different sizes, the authors of SSD attempted to use only the largest image and utilize the internal pyramidal structure of the CNN backbone to make predictions of varying scales. This approach allows the system to be computationally more feasible than option (a). Nevertheless, here we actually got a tradeoff like (b), where the feature map from the deeper layer contains a large amount of semantic information yet having minimal spatial information, while the feature map from the shallower layer has a lot of spatial information but it does not have that much semantic information. It is important to note that a detailed spatial information might not be quite important for large objects as we can just approximate the general shape of that object. However, both spatial and semantic information are necessary for detecting small objects because not only the detailed coordinates, but the model also needs to understand what’s actually inside the bounding box. So, while it is true that method (c) is indeed able to detect both large and small objects, but its ability in detecting the latter is not yet optimal.\nAnd here’s where FPN comes as a solution. If we take a look at image (d) in Figure 2, we can see that the predictions are made on top of the corresponding feature maps which are all semantically rich. This essentially allows objects of varying scales, including the smaller ones, to be detected accurately. We are going to talk about the details of how FPN enriches feature maps in the subsequent section.\n···\n## How FPN Works\nThe idea of FPN is to inject information from the deeper feature maps into the shallower ones, and by doing so we will have the shallower feature maps not only containing high spatial information but also high semantic information coming from the deeper part of the network. In theory, this should result in a better detection accuracy on small objects since the large feature maps are now enriched with a lot of semantic information. In order to achieve this, they introduce the so-called _top-down pathway_ and _lateral connections_. You can see the complete FPN architecture in Figure 3 below, which is essentially the detailed version of the one in Figure 2 (d).\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/06/15WVSovhKKOE_ryS1FLTEXQ.png)\nFigure 3. The detailed FPN architecture [3].\nThe authors of this paper decided to use ResNet-50 and ResNet-101 as the backbone. Suppose we were to use the former, we would later have the _conv2_ , _conv3_ , _conv4_ , and _conv5_ layers repeated 3, 4, 6, and 3 times, respectively, as suggested by the architectural details of ResNet in Figure 4. _C2_ , _C3_ , _C4_ , and _C5_ , which are the tensors produced by the last layer of the corresponding stage, are going to be transferred to the _top-down pathway_ through _lateral connections_ , i.e., the arrows going out from the backbone.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/06/1odO7rwKryVZ8ieNrwTOwAA.png)\nFigure 4. The ResNet architecture [3, 4].\nThe _top-down pathway_ is used to transfer semantic information from deeper layers, whereas _lateral connections_ are used to preserve spatial information. We aggregate the two by performing element-wise summation, which the detailed process is given in Figure 5 below. For the tensors that come from the backbone (_C_), we first need to apply 1×1 conv to them. This convolution layer is responsible for adjusting the number of channels so that it matches with the tensor coming from the top-down pathway. The tensor from the top-down pathway itself (_M+1_) undergoes 2× nearest-neighbor upsampling. These processes are essentially done because we need both tensors to have the exact same dimension so that element-wise summation can be performed. As the summation is done, the resulting tensor is now referred to as _M_. This tensor has some aliasing effect due to the upsampling process we did earlier, hence we need to apply a 3×3 convolution to reduce that effect. Finally, we got the _P_ tensor, which is ready to be forwarded to the detection head.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/06/1AjYfapHMLWLMD0a0-XcKXw.png)\nFigure 5. How feature maps from lateral connection (C) and the top-down pathway (M+1) are aggregated [3].\nKeep in mind that all processes described in Figure 5 above only apply to _M2_ , _M3_ , and _M4_. Computing _M5_ is actually much simpler (see Figure 6), where the only thing we need to do is just to adjust the number of channels using 1×1 conv to make it uniform with the tensors in the other lateral connections. The _M5_ tensor itself does not need to be processed further with 3×3 conv because there is nothing to be smoothed out due to the absence of upsampling mechanism. And so, we can basically say that _P5_ is the exact same tensor as _M5_.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/06/1_vMhf75CpJfkJvwxtEc2RQ.png)\nFigure 6. The way to compute M5 and P5 is slightly different from that of M2-M4 and P2-P4 [3].\nAnd well I think that’s everything about the theory behind FPN. In the next section I am going to bring you into the lower-level view of the architecture by implementing it from scratch with PyTorch.\n···\n## FPN From Scratch\n### CNN Backbone\nSo as seen in the Codeblock 1 below, the very first thing we need to do in the code is to import the required modules.\npython\n\n```\n# Codeblock 1\nimport torch\nimport torch.nn as nn\n```\n\nSince the focus of this article is FPN, here I will use a dummy model for the backbone instead of using the actual ResNet in order to simplify things. But still, the layers in the code are named according to Figure 3 and 4: `conv1`, `conv2`, `conv3`, `conv4` and `conv5` as shown in Codeblock 2 below. The output tensor dimension of each stage is also set according to the original ResNet architecture. So, although this backbone is just a plain CNN-based model, you can think of this like a normal ResNet.\nNext, what we do inside the `forward()` method is to connect all the layers. If you take a closer look at the code, you will notice that each convolution layer is followed by a ReLU activation function and a maxpooling layer. The maxpooling layer itself is set to have the stride of 2, effectively halves the spatial dimension of the feature map. By repeating maxpooling layers multiple times, we will have our feature map gradually gets smaller as we get deeper into the network. This essentially creates a pyramidal structure within the CNN backbone which is leveraged by FPN to achieve high detection accuracy on varying object scales. In CNN, reducing spatial dimension like this is a standard practice to reduce computational complexity to compensate the increase of the number of channels.\nStill within the `forward()` method, don’t forget to clone the main tensor `x` as shown at the lines marked with `#(1)`, `#(2)`, and `#(3)`. The copied tensors, which are named `c2`, `c3`, and `c4`, will then be the return values of the `CNN` class alongside the feature map from the main flow (`c5`) (`#(4)`).\npython\n\n```\n# Codeblock 2\nclass CNN(nn.Module):\n def __init__(self):\n super().__init__()\n \n self.relu = nn.ReLU()\n self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, padding=1)\n \n self.conv1 = nn.Conv2d(in_channels=3, out_channels=64, kernel_size=3, padding=1)\n self.conv2 = nn.Conv2d(in_channels=64, out_channels=256, kernel_size=3, padding=1)\n self.conv3 = nn.Conv2d(in_channels=256, out_channels=512, kernel_size=3, padding=1)\n self.conv4 = nn.Conv2d(in_channels=512, out_channels=1024, kernel_size=3, padding=1)\n self.conv5 = nn.Conv2d(in_channels=1024, out_channels=2048, kernel_size=3, padding=1)\n \n def forward(self, x):\n print(f'original\\t: {x.size()}\\n')\n \n x = self.relu(self.conv1(x))\n print(f'after conv1\\t: {x.size()}')\n \n x = self.maxpool(x)\n print(f'after maxpool\\t: {x.size()}\\n')\n \n x = self.relu(self.conv2(x))\n print(f'after conv2\\t: {x.size()}')\n \n x = self.maxpool(x)\n print(f'after maxpool\\t: {x.size()}\\n')\n \n c2 = x.clone() #(1)\n \n x = self.relu(self.conv3(x))\n print(f'after conv3\\t: {x.size()}')\n \n x = self.maxpool(x)\n print(f'after maxpool\\t: {x.size()}\\n')\n \n c3 = x.clone() #(2)\n \n x = self.relu(self.conv4(x))\n print(f'after conv4\\t: {x.size()}')\n \n x = self.maxpool(x)\n print(f'after maxpool\\t: {x.size()}\\n')\n \n c4 = x.clone() #(3)\n \n x = self.relu(self.conv5(x))\n print(f'after conv5\\t: {x.size()}')\n \n c5 = self.maxpool(x)\n print(f'after maxpool\\t: {c5.size()}\\n')\n \n return c2, c3, c4, c5 #(4)\n```\n\nAs the `CNN` class is done, we will now try to pass a dummy RGB image of size 224×224 through the network. This tensor dimension is chosen based on the input shape of the original ResNet.\npython\n\n```\n# Codeblock 3\ncnn = CNN()\n\nx = torch.randn(1, 3, 224, 224)\nout_cnn = cnn(x)\n```\n\nAnd below is what the output looks like. Here we can see that the number of channels after each conv layer matches exactly with the ResNet structure given in Figure 4. Not only that, it is also seen that the spatial dimension of our dummy tensor successfully halved after each stage thanks to the maxpooling layers. This essentially indicates that our simple CNN model really mimics the general structure of a ResNet model.\nmarkup\n\n```\n# Codeblock 3 Outputoriginal : torch.Size([1, 3, 224, 224])after conv1 : torch.Size([1, 64, 224, 224])after maxpool : torch.Size([1, 64, 112, 112])after conv2 : torch.Size([1, 256, 112, 112])after maxpool : torch.Size([1, 256, 56, 56])after conv3 : torch.Size([1, 512, 56, 56])after maxpool : torch.Size([1, 512, 28, 28])after conv4 : torch.Size([1, 1024, 28, 28])after maxpool : torch.Size([1, 1024, 14, 14])after conv5 : torch.Size([1, 2048, 14, 14])after maxpool : torch.Size([1, 2048, 7, 7])\n```\n\nWe can also check what the returned tensors look like by running the code below. You can see in the resulting output that the `c2` tensor has the shape of 256×56×56, `c3` is of shape 512×28×28, and so on. By the way, you can just ignore the number 1 in the 0th axis since it only indicates the number of samples we pass within a single batch.\npython\n\n```\n# Codeblock 4\nc2, c3, c4, c5 = out_cnn\n\nprint(c2.shape)\nprint(c3.shape)\nprint(c4.shape)\nprint(c5.shape)\n```\n\nmarkup\n\n```\n# Codeblock 4 Outputtorch.Size([1, 256, 56, 56])torch.Size([1, 512, 28, 28])torch.Size([1, 1024, 14, 14])torch.Size([1, 2048, 7, 7])\n```\n\n···\n### FPN Neck\nAs the CNN backbone is completed, now let’s move on to the FPN neck. In the Codeblock 5 below, we first initialize the upsample layer (`#(1)`) which we will use every time we want to double the spatial dimension of the _M_ tensor. Here I set the `mode` parameter to `nearest` as suggested in the paper, which is actually a very simple interpolation method, allowing the process to be fast. Take a look at Figure 7 to see what a nearest-neighbor interpolation looks like.\npython\n\n```\n# Codeblock 5\nclass FPN(nn.Module):\n def __init__(self):\n super().__init__()\n \n self.upsample = nn.Upsample(scale_factor=2, mode='nearest') #(1)\n \n self.lateral_c5 = nn.Conv2d(in_channels=2048, out_channels=256, kernel_size=1)\n self.lateral_c4 = nn.Conv2d(in_channels=1024, out_channels=256, kernel_size=1)\n self.lateral_c3 = nn.Conv2d(in_channels=512, out_channels=256, kernel_size=1)\n self.lateral_c2 = nn.Conv2d(in_channels=256, out_channels=256, kernel_size=1)\n \n self.smooth_m4 = nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, padding=1)\n self.smooth_m3 = nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, padding=1)\n self.smooth_m2 = nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, padding=1)\n \n def forward(self, c2, c3, c4, c5):\n m5 = self.lateral_c5(c5)\n p5 = m5\n \n m4 = self.upsample(m5) + self.lateral_c4(c4)\n p4 = self.smooth_m4(m4)\n \n m3 = self.upsample(m4) + self.lateral_c3(c3)\n p3 = self.smooth_m3(m3)\n \n m2 = self.upsample(m3) + self.lateral_c2(c2)\n p2 = self.smooth_m2(m2)\n \n return p2, p3, p4, p5\n```\n\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/06/1sbz8gBOx6W-ei-1_bHk0mA.png)\nFigure 7. An example of a 2x upsampling process with nearest-neighbor interpolation method [3].\nIf you go back to Codeblock 4, you will see that the `c{5,4,3,2}`tensors returned by the backbone have different number of channels. This is basically the reason that we initialize the `lateral_c{5,4,3,2}`layers to process these tensors so that the resulting channel counts will be uniform. According to the paper, we need to set these convolution layers to produce 256 output channels, which is the reason why we use that number for the `out_channels`parameter. \nNext, based on the Figure 7, you can just imagine how pixelated the resulting feature maps are after being upsampled. Thus, we need to process the `m{4,3,2}`tensors further with the 3×3 conv layers which I refer to as `smooth_m{4,3,2}`. As all layers have been initialized, what we need to do next is to assemble them in the `forward()`method according to the structure I showed you earlier in Figure 3. \nIn addition to this, the paper also mentions that we don’t need to implement any nonlinearities within the FPN, which is the reason that all the convolution layers in the `FPN`class above are not followed with ReLU. Now in Codeblock 6 below I try to pass the _C_ tensors we obtained earlier through the FPN neck we just created. We can see in the output that the resulting tensors have different spatial resolutions. Later on, the `p2`tensor (the one that has 56×56 dimension) will be forwarded to a detection head to detect small objects, whereas `p5`(the 7×7 tensor) is going to be responsible for large objects.\npython\n\n```\n# Codeblock 6\nfpn = FPN()\n\nout_fpn = fpn(c2, c3, c4, c5)\np2, p3, p4, p5 = out_fpn\n\nprint(p2.shape)\nprint(p3.shape)\nprint(p4.shape)\nprint(p5.shape)\n```\n\nmarkup\n\n```\n# Codeblock 6 Outputtorch.Size([1, 256, 56, 56])torch.Size([1, 256, 28, 28])torch.Size([1, 256, 14, 14])torch.Size([1, 256, 7, 7])\n```\n\nHere have already completed the FPN part. Remember that FPN is just the _neck_ of a detection model, which essentially means that at this point we still haven’t got the bounding box prediction just yet. In order to actually obtain the prediction result, we need to connect a specific _head_ to the FPN, and in this case I will use the RPN (Region Proposal Network) head.\n···\n### RPN Head\nIn case you’re still not yet familiar with RPN, this is essentially the head of an object detection model used for creating bounding box, which was first proposed in the Faster R-CNN paper. Note that while in this demonstration we refer to the RPN as a head, keep in mind that it is actually not a complete detection head since it has no capability in performing classification on the detected objects.\nWe can see in the RPN architecture below that it utilizes the so-called _cls layer_ and _reg layer_ , which produce _objectness_ score and the bounding box coordinates, respectively. The objectness score tensor has the length of 2 _k_ , where _k_ is the number of predetermined anchor boxes and 2 is the probability of the corresponding anchor box being there. We can think of this like a binary classification treated with a one-hot representation (object/non-object). Meanwhile, the number 4 in the 4 _k_ length of the coordinates tensor simply correspond to the _xywh_ prediction.\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2026/06/15_a_l4M5Bd3DTMpS1xDsYQ.png)\nFigure 8. The structure of RPN [4].\nGoing back to our code implementation, in the Codeblock 7 below we initialize the `intermediate`, `cls`, and `reg` layers within the `__init__()` method of the RPN class. Note that the intermediate layer is the only one that uses 3×3 convolution, whereas both the `cls` and `reg` layers use 1×1 convs. Regarding the number of channels, the `intermediate` layer maps the input tensor into 256 channels, while the `cls` and `reg` map it into 2 _k_ and 4 _k_ , respectively. Finally, we can simply connect these layers within the `forward()` method.\npython\n\n```\n# Codeblock 7\nNUM_ANCHORS = 3\n\nclass RPN(nn.Module):\n def __init__(self):\n super().__init__()\n \n self.intermediate = nn.Conv2d(in_channels=256, out_channels=256, kernel_size=3, padding=1)\n \n self.cls = nn.Conv2d(in_channels=256, out_channels=NUM_ANCHORS*2, kernel_size=1)\n self.reg = nn.Conv2d(in_channels=256, out_channels=NUM_ANCHORS*4, kernel_size=1)\n \n def forward(self, x):\n x = self.intermediate(x)\n \n objectness_scores = self.cls(x)\n bbox_regressions = self.reg(x)\n \n return objectness_scores, bbox_regressions\n```\n\nNow let’s test if our RPN class works properly by running the Codeblock 8 below. Here I test it on the `p2` feature map we obtained from Codeblock 6.\npython\n\n```\n# Codeblock 8\nrpn = RPN()\n\np2_objectness, p2_bbox = rpn(p2)\n\nprint(p2_objectness.shape)\nprint(p2_bbox.shape)\n```\n\nBelow is what the resulting output looks like. You can see that `p2_objectness` is a tensor having the size of 6×56×56, indicating that every single pixel in the 56×56 spatial dimension contains 6 prediction values, where the first 2 values are for the first anchor box, the second 2 values are for the second anchor box, and the last 2 values are for the third one. The similar thing also applies to the `p2_bbox` tensor, which in this case it contains the _xywh_ values.\nmarkup\n\n```\n# Codeblock 8 Outputtorch.Size([1, 6, 56, 56])torch.Size([1, 12, 56, 56])\n```\n\n···\n### The Entire Detection Model\nIn Codeblock 9 below we are going to construct the entire detection model so that you can better understand how FPN works together with the other components. Here in the `__init__()` method I initialize the CNN bottleneck, FPN neck, and RPN head. In the `forward()` method, we first pass the image tensor into the CNN (`#(1)`). This backbone returns 4 tensors, which are ready to be connected to the FPN through lateral connections. Next, at line `#(2)` we feed all the _C_ tensors as the input of FPN, producing the _P_ tensors. Lastly, we use all the _P_ s as the input to the RPN (`#(3–4)`). Keep in mind that RPN shares its parameters across all detection heads, so we only need to initialize it once and use it for all feature map of different scales.\npython\n\n```\n# Codeblock 9\nclass DetectionModel(nn.Module):\n def __init__(self):\n super().__init__()\n \n self.cnn = CNN()\n self.fpn = FPN()\n self.rpn = RPN()\n \n def forward(self, x):\n \n c2, c3, c4, c5 = self.cnn(x) #(1)\n p2, p3, p4, p5 = self.fpn(c2, c3, c4, c5) #(2)\n \n p2_pred = self.rpn(p2) #(3)\n p3_pred = self.rpn(p3)\n p4_pred = self.rpn(p4)\n p5_pred = self.rpn(p5) #(4)\n \n return p2_pred, p3_pred, p4_pred, p5_pred\n```\n\nNow as the detection head is complete, we can test it with the Codeblock 10 below. Here I try to pass a dummy tensor of size 1×3×224×224, simulating a single RGB image of size 224×224 (`#(1)`). Next, we can just pass it through the `detection_model` (`#(2)`) and unpack the prediction results (`#(3–4)`).\npython\n\n```\n# Codeblock 10\ndetection_model = DetectionModel()\n\nx = torch.randn(1, 3, 224, 224) #(1)\np2_pred, p3_pred, p4_pred, p5_pred = detection_model(x) #(2)\n\np2_objectness, p2_bbox = p2_pred #(3)\np3_objectness, p3_bbox = p3_pred\np4_objectness, p4_bbox = p4_pred\np5_objectness, p5_bbox = p5_pred #(4)\n \nprint(p2_objectness.shape)\nprint(p3_objectness.shape)\nprint(p4_objectness.shape)\nprint(p5_objectness.shape)\nprint()\n\nprint(p2_bbox.shape)\nprint(p3_bbox.shape)\nprint(p4_bbox.shape)\nprint(p5_bbox.shape)\n```\n\nBelow is what the output looks like. You can see here that the resulting tensor dimensions are as intended, where the `objectness` and `bbox` tensors contain 6 and 12 values for each grid cell, respectively. So, I believe this implementation is correct and thus ready to be trained for object detection task.\nmarkup\n\n```\n# Codeblock 10 Outputtorch.Size([1, 6, 56, 56])torch.Size([1, 6, 28, 28])torch.Size([1, 6, 14, 14])torch.Size([1, 6, 7, 7])torch.Size([1, 12, 56, 56])torch.Size([1, 12, 28, 28])torch.Size([1, 12, 14, 14])torch.Size([1, 12, 7, 7])\n```\n\n···\n## Ending\nI think that’s pretty much all about the underlying theory and the from-scratch implementation of FPN. Here I challenge you to try implementing FPN on the real ResNet instead of a dummy CNN model like I demonstrated above. I actually got a separate article about ResNet, which you can check as a reference [5]. Or, it is also possible to use other models if you want, such as VGG, ResNeXt, ConvNeXt, etc since FPN can basically work on any CNN-based backbone model. Not only that, it would also be better if you can implement YOLO-style head as a replacement of RPN, which the examples can be seen in my previous articles given at references number [6] for YOLOv1, [7] for YOLOv2, and [1] for YOLOv3.\nPlease let me know if there are mistakes in my writing or in the code. Thanks for reading! By the way, you can find the code used in this article in my GitHub repo [8].\n···\n## References\n[1] [Muhammad Ardi](https://medium.com/u/9801a58700ac). YOLOv3 Paper Walkthrough: Even Better, but Not That Much. Medium. [Accessed June 1, 2026].\n[2] Tsung-Yi Lin. Feature Pyramid Networks for Object Detection. Arxiv. [Accessed September 9, 2025].\n[3] Image created originally by author.\n[4] Shaoqing Ren. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Arxiv. [Accessed September 9, 2025].\n[5] Muhammad Ardi. Paper Walkthrough: Residual Network (ResNet). Python in Plain English. [Accessed September 9, 2025].\n[6] [Muhammad Ardi](https://medium.com/u/9801a58700ac). YOLOv1 Paper Walkthrough: The Day YOLO First Saw the World. Medium. [Accessed June 1, 2026].\n[7] [Muhammad Ardi](https://medium.com/u/9801a58700ac). YOLOv2 & YOLO9000 Paper Walkthrough: Better, Faster, Stronger. Medium. [Accessed June 1, 2026].\n[8] MuhammadArdiPutra. FPN. GitHub. [Accessed September 9, 2025].\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](https://assets.insightmediagroup.io/media/1787657598287_taxhya.jpeg)\nWritten By\n## [Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\nDeep learning storyteller\n[See all from Muhammad Ardi Putra](https://towardsdatascience.com/author/muhammad_ardi/)\n[Deep Dives](https://towardsdatascience.com/tag/deep-dives/)[Artificial Intelligence](https://towardsdatascience.com/tag/artificial-intelligence/)[Deep Learning](https://towardsdatascience.com/tag/deep-learning/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/)[Python](https://towardsdatascience.com/tag/python/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Ffpn-paper-walkthrough-leveraging-the-internal-pyramid%2F&title=FPN%20Paper%20Walkthrough%3A%20Leveraging%20the%20Internal%C2%A0Pyramid)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Ffpn-paper-walkthrough-leveraging-the-internal-pyramid%2F&title=FPN%20Paper%20Walkthrough%3A%20Leveraging%20the%20Internal%C2%A0Pyramid)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Ffpn-paper-walkthrough-leveraging-the-internal-pyramid%2F&text=FPN%20Paper%20Walkthrough%3A%20Leveraging%20the%20Internal%C2%A0Pyramid)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/07/1n8UyR2HxQPudoBbZ6z4MjA.png)](https://towardsdatascience.com/dueling-double-deep-q-learning-using-tensorflow-2-x-7bbbcec06a2a/)\n## [Dueling Double Deep Q Learning using Tensorflow 2.x](https://towardsdatascience.com/dueling-double-deep-q-learning-using-tensorflow-2-x-7bbbcec06a2a/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nImplementing dueling double deep q learning with TensorFlow 2.x\n[Abhishek Suran](https://towardsdatascience.com/author/abhisheksuran/)\nJuly 10, 20202 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/02/1mmiW12YIIKHl2uPtQitWtg-scaled.jpeg)](https://towardsdatascience.com/semantic-textual-similarity-with-bert-fc800656e7a3/)\n## [Semantic Textual Similarity with BERT](https://towardsdatascience.com/semantic-textual-similarity-with-bert-fc800656e7a3/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nHow to use BERT to calculate the semantic similarity between two texts\n[Ruben Winastwan](https://towardsdatascience.com/author/marcellusruben/)\nFebruary 15, 202311 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/12/0jFNOIt1x1ZCpw-75-scaled.jpg)](https://towardsdatascience.com/stop-using-print-and-start-debugging-3cd11524c92a/)\n## [Stop using Print, and start Debugging](https://towardsdatascience.com/stop-using-print-and-start-debugging-3cd11524c92a/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nLearn how to debug your Python program with Visual Studio Code\n[Marcello Politi](https://towardsdatascience.com/author/marcellopoliti/)\nDecember 5, 20227 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/10/1pWlZS0JxEJKEKhfzV_DJdw.png)](https://towardsdatascience.com/getting-started-with-sentiment-analysis-731531ec880d/)\n## [Getting Started With Sentiment Analysis](https://towardsdatascience.com/getting-started-with-sentiment-analysis-731531ec880d/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nNotes From Natural Lanaguage Processing Specialization Course1 Week1\n[Kurtis Pykes](https://towardsdatascience.com/author/kurtispykes/)\nOctober 17, 20208 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/11/0hUQHvp59v1g0bWj3-scaled.jpg)](https://towardsdatascience.com/5-visualizations-with-python-to-show-simultaneous-changes-in-geospatial-data-ddc2eaab9d78/)\n## [5 Visualizations with Python to Show Simultaneous Changes in Geospatial Data](https://towardsdatascience.com/5-visualizations-with-python-to-show-simultaneous-changes-in-geospatial-data-ddc2eaab9d78/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nIdeas to express simultaneous data value changes in multiple locations over time\n[Boriharn K](https://towardsdatascience.com/author/borih.k/)\nNovember 4, 20237 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/10/124TAW0mKioebxg0xztC7tg.png)](https://towardsdatascience.com/interpretable-machine-learning-using-shap-theory-and-applications-26c12f7a7f1a/)\n## [Interpretable Machine Learning using SHAP - theory and applications](https://towardsdatascience.com/interpretable-machine-learning-using-shap-theory-and-applications-26c12f7a7f1a/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nIntroduction\n[Khalil Zlaoui](https://towardsdatascience.com/author/khalilz/)\nOctober 1, 20218 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/11/erik-mclean-KbYdYFvBNN8-unsplash.jpg)](https://towardsdatascience.com/tds-newsletter-how-to-build-robust-data-and-ai-systems/)\n## [TDS Newsletter: How to Build Robust Data and AI Systems](https://towardsdatascience.com/tds-newsletter-how-to-build-robust-data-and-ai-systems/)\n[The Variable](https://towardsdatascience.com/category/the-variable/)\nGain a high-level understanding of how the different pieces in your workflow come together\n[TDS Editors](https://towardsdatascience.com/author/towardsdatascience/)\nNovember 20, 20253 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/01/1JQ4WQRwNmeA5R7ERZuGikw-scaled.jpeg)](https://towardsdatascience.com/build-a-simple-web-app-to-classify-us-50-dollar-100-dollar-bills-d4ef8298ae6/)\n## [Build a Simple Web App to Classify US 50 Dollar & 100 Dollar Bills](https://towardsdatascience.com/build-a-simple-web-app-to-classify-us-50-dollar-100-dollar-bills-d4ef8298ae6/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nFrom model to production using Jupyter notebook\n[ASM Saiful Islam](https://towardsdatascience.com/author/a-islam/)\nJanuary 25, 202112 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/04/0cq8KX7pyXGPiqEzw-scaled.jpg)](https://towardsdatascience.com/uncertainty-in-deep-learning-experiments-with-bayesian-cnn-1ca37ddb6954/)\n## [Uncertainty in Deep Learning: Experiments with Bayesian CNN](https://towardsdatascience.com/uncertainty-in-deep-learning-experiments-with-bayesian-cnn-1ca37ddb6954/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nHow does KL-Weight affect the model? Writing a Bayesian CNN for chest X-Ray images.\n[Kaan Bıçakcı](https://towardsdatascience.com/author/frightera/)\nApril 12, 20225 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n#### Recommended Articles \nClose\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n", "body_base_path": "attachments/dcf4c8a3c08b/pages/", "body_markdown_path": "attachments/dcf4c8a3c08b/pages/archived-body.md", "local_html": "attachments/dcf4c8a3c08b/pages/archived-body.html", "body_source": "prior_capture" }