{
  "source_file": "0930",
  "source_platform": "技术平台",
  "author": "Roboflow Blog",
  "title_listed": "Grounding DINO: SOTA Zero-Shot Object Detection",
  "keywords": "Grounding DINO,零样本,检测",
  "domain": "图像处理",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://blog.roboflow.com/grounding-dino-zero-shot-object-detection/",
  "publish_date": "2023-03",
  "corpus_stage": "图像处理",
  "notes": "零样本目标检测",
  "id": "4cbb3c8c1724",
  "collected_at": "2026-10-02T18:08:30",
  "title": "Grounding DINO : SOTA Zero-Shot Object Detection | Roboflow",
  "body_markdown": "Grounding DINO : SOTA Zero-Shot Object Detection\nPiotr Skalski\n|\nPublished\nMar 30, 2023\n6\nmin read\nSUMMARY\nGrounding DINO is a zero-shot object detection model that identifies objects from a text prompt rather than a fixed class list, reaching 52.5 AP on the COCO zero-shot transfer benchmark and 63.0 AP after finetuning. Its architecture combines a transformer-based detector (DINO) with grounded language-image pretraining, enabling detection of novel objects and fine-grained descriptions that appear in no standard training dataset. The model is best suited for tasks where flexibility matters more than inference speed, such as automatic data annotation and building detection pipelines driven by natural language constraints.\nAs of March 2023, there is a new SOTA zero-shot object detection model -\nGrounding DINO\n. In this post, we will talk about the advantages of Grounding DINO, analyze the model architecture, and provide real prompt examples. To jump right to the model, use the Jupyter\nnotebook\nto start trying it yourself.\nLet's dive in!\n💡\nYou can train small models using Grounding DINO in a few lines of code and without labeling data using\nAutodistill\n, an open-source ecosystem for distilling large foundation models into smaller models trained on your data.\nCheck out our\nAutodistill guide\nfor more information, and our\nAutodistill Grounding DINO documentation\n.\nIntroduction\nMost\nobject detection models\nare trained to identify a narrow predetermined collection of classes. The main problem with this is the lack of flexibility. Every time you want to expand or change the set of recognizable objects, you have to collect data, label it, and train the model again. This — of course — is time-consuming and expensive.\nZero-shot detectors want to break this status quo by making it possible to detect new objects without re-training a model.\nAll you have to do is change the prompt\nand the model will detect the objects you describe.\nBelow we see two images visualizing predictions made with\nGrounding DINO\n— the new SOTA zero-shot object detection model.\nIn the case of the image on the left, we prompted the model to identify the class \"\nchair\n\" — a class belonging to the\nCOCO\ndataset. The model managed to detect all objects of this class without any problems.\nFor the image on the right, we tried to find the boundary of the model's knowledge and asked about the dog's tail. This object — of course — is not in any of the commonly used datasets, and yet it has been found.\n\"\nchair\n\" vs. \"\ndog's tail\n\" query\nGrounding DINO Performance\nGrounding DINO achieves a\n52.5 AP\non the COCO detection zero-shot transfer benchmark — without any training data from COCO. After finetuning with COCO data, Grounding DINO reaches\n63.0 AP\n. It sets a new record on the ODinW zero-shot benchmark with a mean of\n26.1 AP\n.\nGLIP T vs. Grounding DINO T speed and mAP comparison\nAdvantages of Grounding DINO\nZero-Shot Object Detection —\nGrounding DINO excels at\ndetecting objects even when they are not part of the predefined set of classes in the training data\n. This unique capability enables the model to adapt to novel objects and scenarios, making it highly versatile and applicable to various real-world tasks.\nReferring Expression Comprehension (REC)\n— Identifying and localizing a specific object or region within an image is based on a given textual description.\nIn other words, instead of detecting people and chairs in an image and then writing custom logic to determine whether a chair is occupied, prompt engineering can be used to ask the model to detect only those chairs where a person is sitting.\nThis requires the model to possess a deep understanding of both the language and the visual content, as well as the ability to associate words or phrases with corresponding visual elements.\nElimination of Hand-Designed Components like NMS —\nGrounding DINO\nsimplifies the object detection pipeline by removing the need for hand-designed components\n, such as\nNon-Maximum Suppression (NMS)\n. This streamlines the model architecture and training process while improving efficiency and performance.\nfigure from the official Grounding DINO paper\nGrounding DINO Architecture\nGrounding DINO aims to merge concepts found in the\nDINO\nand\nGLIP\npapers. DINO, a transformer-based detection method,\noffers state-of-the-art object detection performance\nand end-to-end optimization, eliminating the need for handcrafted modules like NMS (Non-Maximum Suppression).\nOn the other hand, GLIP focuses on\nphrase grounding\n. This task involves associating phrases or words from a given text with corresponding visual elements in an image or video, effectively linking textual descriptions to their respective visual representations.\nText backbone and Image backbone\n— Multiscale image features are extracted using an image backbone like Swin Transformer, and text features are extracted with a text backbone like BERT.\ntext backbone and Image backbone\nFeature enhancer\n— After vanilla image and text features are extracted, they are fed into a feature enhancer for cross-modality feature fusion. The feature enhancer includes multiple feature enhancer layers. Deformable self-attention is leveraged to enhance image features, and vanilla self-attention is utilized for text feature enhancers.\nfeature enhancer\nLanguage-guided query selection\n— To effectively leverage the input text for guiding object detection, a language-guided query selection module is designed to select features more relevant to the input text as decoder queries.\nCross-modality decoder\n— A cross-modality decoder is developed to combine image and text modality features. Each cross-modality query is fed into a self-attention layer, an image cross-attention layer to combine image features, a text cross-attention layer to combine text features, and an FFN layer in each cross-modality decoder layer. Each decoder layer has an extra text cross-attention layer compared with the DINO decoder layer, as text information needs to be injected into queries for better modality alignment.\nlanguage-guided query selection and cross-modality decoder\nTry Grounding DINO and Examples\nTo make it easier for you to experiment with the model, we have prepared a Jupyter\nnotebook\nthat you can test, among other places, on\nGoogle Colab\n.\nAs mentioned earlier, Grounding DINO allows precise detection with\nROC\n. Instead of asking about all the chairs in the image, we can narrow the search to only those chairs occupied by a person using the following prompt — \"\nchair with a man sitting on it\n\".\nWe can see that the model returns not only a narrowed set of chairs but also a detected \"\nman\n\" — an additional noun found in the prompt.\n\"\nchair\n\" vs. \"\nchair with a man sitting on it\n\" query\nAs with GLIP, the quality of the predictions may be improved through appropriate\nprompt engineering\n. In the case of the photo below, querying the model with a simple query — \"\nnapkin\n\" — we will not detect any objects.\nIn such a situation, it is helpful to provide more details describing the object we are looking for such as \"\nblue napkin\n\" and \"\nnapkin on the table\n\" to achieve the desired output.\n\"\nnapkin\n\" vs. \"\nblue napkin\n\" query\nSuggestions for Using Grounding DINO\nAs you can see in the table at the beginning of the blog post,\nGrounding DINO\nis faster than GLIP but still too slow to consider in real-time scenarios.\nRF-DETR\nis safe for now.\nHowever, the model can shine in tasks where flexibility and versatility are important. Grounding DINO can be successfully used for automatic data annotation or replacing heavy models with many classes.\nWith REC, Grounding DINO can also greatly simplify the image and video processing pipelines by enabling detection relationships to be analyzed using language constraints rather than complex and error-prone hand-made components.\nConclusion\nGrounding DINO sets a new standard in the world of object detection by offering a highly adaptable and flexible zero-shot detection model. The model's ability to identify objects outside its training set and understand both language and visual content allows it to excel in various real-world tasks.\nGrounding DINO is undoubtedly a breakthrough that will pave the way for more innovative applications in object detection and beyond.\nResources\n[1] DINO Paper —\nDETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection\n[2] GLIP Paper —\nGrounded Language-Image Pre-training\n[3] Grounding DINO Paper —\nGrounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection",
  "metadata": {
    "title": "Grounding DINO : SOTA Zero-Shot Object Detection | Roboflow",
    "description": "Most object detection models are trained to identify a narrow predetermined collection of classes. Zero-shot detectors like Grounding DINO want to break this status quo by making it possible to detect new objects without re-training a model.",
    "keywords": null,
    "author": null,
    "og:title": "Grounding DINO : SOTA Zero-Shot Object Detection",
    "og:description": "Most object detection models are trained to identify a narrow predetermined collection of classes. Zero-shot detectors like Grounding DINO want to break this status quo by making it possible to detect new objects without re-training a model.",
    "og:url": "https://roboflow.com/blog/grounding-dino-zero-shot-object-detection",
    "og:site_name": "Roboflow",
    "og:image": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/2024/04/image-1228.webp",
    "og:type": "article",
    "twitter:card": "summary_large_image",
    "twitter:title": "Grounding DINO : SOTA Zero-Shot Object Detection",
    "twitter:description": "Most object detection models are trained to identify a narrow predetermined collection of classes. Zero-shot detectors like Grounding DINO want to break this status quo by making it possible to detect new objects without re-training a model.",
    "twitter:image": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/2024/04/image-1228.webp",
    "article:published_time": "2023-03-30T23:11:01.000+00:00",
    "article:modified_time": "2026-07-31T03:11:49.000+00:00"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [],
  "images": [
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w3840/format/webp/2022/11/photo_2022-11-17-18.22.41.jpeg",
      "bytes": 15300,
      "reused": true,
      "local_path": "attachments/4cbb3c8c1724/images/000.jpeg",
      "src": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w3840/format/webp/2022/11/photo_2022-11-17-18.22.41.jpeg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w3840/2024/04/image-1228.webp",
      "bytes": 210376,
      "reused": true,
      "local_path": "attachments/4cbb3c8c1724/images/015.webp",
      "src": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w3840/2024/04/image-1228.webp",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://i.ytimg.com/vi/cMa77r3YrDk/maxresdefault.jpg",
      "bytes": 132985,
      "reused": true,
      "local_path": "attachments/4cbb3c8c1724/images/023.jpg",
      "src": "https://i.ytimg.com/vi/cMa77r3YrDk/maxresdefault.jpg",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w1000/2024/04/image-1233.webp",
      "bytes": 146732,
      "reused": true,
      "local_path": "attachments/4cbb3c8c1724/images/024.webp",
      "src": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w1000/2024/04/image-1233.webp",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w1000/2024/04/image-1237.webp",
      "bytes": 11668,
      "reused": true,
      "local_path": "attachments/4cbb3c8c1724/images/027.webp",
      "src": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w1000/2024/04/image-1237.webp",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w1000/2024/04/image-1239.webp",
      "bytes": 93594,
      "reused": true,
      "local_path": "attachments/4cbb3c8c1724/images/030.webp",
      "src": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w1000/2024/04/image-1239.webp",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w1000/2024/04/image-1244.webp",
      "bytes": 32542,
      "reused": true,
      "local_path": "attachments/4cbb3c8c1724/images/033.webp",
      "src": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w1000/2024/04/image-1244.webp",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w1000/2024/04/image-1246.webp",
      "bytes": 33160,
      "reused": true,
      "local_path": "attachments/4cbb3c8c1724/images/036.webp",
      "src": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w1000/2024/04/image-1246.webp",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w1000/2024/04/image-1251.webp",
      "bytes": 29792,
      "reused": true,
      "local_path": "attachments/4cbb3c8c1724/images/039.webp",
      "src": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w1000/2024/04/image-1251.webp",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w1000/2024/04/image-1255.webp",
      "bytes": 18482,
      "reused": false,
      "local_path": "attachments/4cbb3c8c1724/images/body-a03da3572950b205ae44.webp",
      "src": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w1000/2024/04/image-1255.webp",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w1000/2024/04/image-1259.webp",
      "bytes": 117066,
      "reused": false,
      "local_path": "attachments/4cbb3c8c1724/images/body-2e2dd655b8ba2c186a99.webp",
      "src": "https://storage.ghost.io/c/2c/8d/2c8d8c0d-1c15-4b6d-825e-02b78d61d40a/content/images/size/w1000/2024/04/image-1259.webp",
      "alt": "",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [
    {
      "url": "https://www.youtube.com/watch?v=gKTYMfwPo4M",
      "status": "ok",
      "local_path": "attachments/4cbb3c8c1724/videos/gKTYMfwPo4M.mp4",
      "bytes": 194194450,
      "valid": true,
      "height": 1080,
      "width": 1920,
      "duration": "625.081000"
    },
    {
      "url": "https://www.youtube.com/watch?v=cMa77r3YrDk",
      "src": "https://www.youtube.com/watch?v=cMa77r3YrDk",
      "status": "ok",
      "kind": "single_video",
      "local_path": "attachments/4cbb3c8c1724/videos/repair-cMa77r3YrDk.mp4",
      "bytes": 198618573,
      "valid": true,
      "height": 1080,
      "width": 1920,
      "duration": "847.881000",
      "reused": true,
      "reused_from": "xinxing-zimeiti-corpus/attachments/e40f797fe91d/videos/cMa77r3YrDk.mp4"
    }
  ],
  "links": [
    {
      "href": "https://roboflow.com/blog/author/skalskip",
      "label": "Piotr Skalski",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/pdf/2303.05499.pdf?ref=blog.roboflow.com",
      "label": "Grounding DINO",
      "type": "a"
    },
    {
      "href": "https://github.com/roboflow/notebooks/blob/main/notebooks/zero-shot-object-detection-with-grounding-dino.ipynb?ref=blog.roboflow.com",
      "label": "notebook",
      "type": "a"
    },
    {
      "href": "https://github.com/autodistill/autodistill?ref=blog.roboflow.com",
      "label": "Autodistill",
      "type": "a"
    },
    {
      "href": "https://www.youtube.com/watch?v=gKTYMfwPo4M&ref=blog.roboflow.com",
      "label": "Autodistill guide",
      "type": "a"
    },
    {
      "href": "https://docs.autodistill.com/target_models/yolo-nas/?ref=blog.roboflow.com",
      "label": "Autodistill Grounding DINO documentation",
      "type": "a"
    },
    {
      "href": "https://playground.roboflow.com/models/task/object-detection?ref=blog.roboflow.com",
      "label": "object detection models",
      "type": "a"
    },
    {
      "href": "https://www.youtube.com/watch?v=cMa77r3YrDk",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://blog.roboflow.com/coco-dataset/",
      "label": "COCO",
      "type": "a"
    },
    {
      "href": "https://blog.roboflow.com/how-to-code-non-maximum-suppression-nms-in-plain-numpy/",
      "label": "Non-Maximum Suppression (NMS)",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/pdf/2203.03605.pdf?ref=blog.roboflow.com",
      "label": "DINO",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/pdf/2112.03857.pdf?ref=blog.roboflow.com",
      "label": "GLIP",
      "type": "a"
    },
    {
      "href": "https://colab.research.google.com/github/roboflow-ai/notebooks/blob/main/notebooks/zero-shot-object-detection-with-grounding-dino.ipynb?ref=blog.roboflow.com",
      "label": "Google Colab",
      "type": "a"
    },
    {
      "href": "https://roboflow.com/model/groundingdino?ref=blog.roboflow.com",
      "label": "Grounding DINO",
      "type": "a"
    },
    {
      "href": "https://rfdetr.roboflow.com/latest/?ref=blog.roboflow.com",
      "label": "RF-DETR",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:54:53Z",
  "source_url": "https://blog.roboflow.com/grounding-dino-zero-shot-object-detection/",
  "papers": [
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2303.05499.pdf?ref=blog.roboflow.com",
      "bytes": 4329735,
      "reused": false,
      "local_path": "papers/repair/f01b49247c7ba0e8a57a.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2203.03605.pdf?ref=blog.roboflow.com",
      "bytes": 2331745,
      "reused": false,
      "local_path": "papers/repair/95ac92086e13827a77c3.pdf"
    },
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/2112.03857.pdf?ref=blog.roboflow.com",
      "bytes": 9170881,
      "reused": false,
      "local_path": "papers/repair/a9e78f62dc11ce3ff105.pdf"
    }
  ],
  "actual_video_files": [
    {
      "local_path": "attachments/4cbb3c8c1724/videos/gKTYMfwPo4M.mp4",
      "bytes": 194194450,
      "valid": true,
      "height": 1080,
      "width": 1920,
      "duration": "625.081000",
      "status": "ok"
    }
  ],
  "body_chars": 8538,
  "expected_body_images": 11,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/4cbb3c8c1724/pages/source.html",
  "local_html": "attachments/4cbb3c8c1724/pages/article.html",
  "body_base_path": "attachments/4cbb3c8c1724/pages/",
  "body_markdown_path": "attachments/4cbb3c8c1724/pages/article.md"
}