{
  "source_file": "0930",
  "source_platform": "技术平台",
  "author": "Adrian Rosebrock / PyImageSearch",
  "title_listed": "Semantic segmentation with OpenCV and deep learning",
  "keywords": "语义分割,OpenCV,深度学习",
  "domain": "图像处理",
  "content_type": "教程",
  "is_paid_flag": "否",
  "url": "https://pyimagesearch.com/2018/09/03/semantic-segmentation-with-opencv-and-deep-learning/",
  "publish_date": "2018-09",
  "corpus_stage": "预训练/SFT",
  "notes": "OpenCV语义分割",
  "id": "8381776d3ee0",
  "collected_at": "2026-10-02T17:21:35",
  "title": "Semantic segmentation with OpenCV and deep learning - PyImageSearch",
  "body_markdown": "Click here to download the source code to this post\nIn this tutorial, you will learn how to perform semantic segmentation using OpenCV, deep learning, and the ENet architecture. After reading today’s guide, you will be able to apply semantic segmentation to images and video using OpenCV.\nDeep learning has helped facilitate unprecedented accuracy in computer vision, including image classification, object detection, and now even\nsegmentation\n.\nTraditional segmentation involves partitioning an image into parts (Normalized Cuts, Graph Cuts, Grab Cuts, superpixels, etc.);\nhowever\n, the algorithm has no actual\nunderstanding\nof what these parts represent.\nSemantic segmentation\nalgorithms on the other hand attempt to:\nPartition\nthe image into meaningful parts\nWhile at the same time,\nassociate every pixel\nin an input image with a class label (i.e., person, road, car, bus, etc.)\nSemantic segmentation algorithms are super powerful and have many use cases, including\nself-driving cars\n— and in today’s post, I’ll be showing you how to apply semantic segmentation to\nroad-scene images/video\n!\nTo learn how to apply semantic segmentation using OpenCV and deep learning,\njust keep reading!\nLooking for the source code to this post?\nJump Right To The Downloads Section\nSemantic segmentation with OpenCV and deep learning\nIn the first part of today’s blog post, we will discuss the ENet deep learning architecture.\nFrom there, I’ll demonstrate how to use ENet to apply semantic segmentation to both images and video streams.\nAlong the way, I’ll be sharing example outputs from the segmentation so you can get a feel for what to expect when applying semantic segmentation to your own projects.\nThe ENet semantic segmentation architecture\nFigure 1:\nThe ENet deep learning semantic segmentation architecture. This figure is a combination of Table 1 and Figure 2 of\nPaszke et al.\nThe semantic segmentation architecture we’re using for this tutorial is ENet, which is based on Paszke et al.’s 2016 publication,\nENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation\n.\nOne of the primary benefits of ENet is that it’s fast — up to 18x faster and requiring 79x fewer parameters with similar or better accuracy than larger models. The model size itself is only\n3.2MB!\nA single forward pass on a CPU took 0.2 seconds on my machine — if I were to use a GPU this segmentation network could run even faster. Paszke et al. trained that dataset on\nThe Cityscapes Dataset\n, a semantic, instance-wise, dense pixel annotation of 20-30 classes (depending on which model you’re using).\nAs the name suggests, the Cityscapes dataset includes examples of images that can be used for urban scene understanding, including self- driving vehicles.\nThe particular model we’re using is trained on 20 classes, including:\nUnlabeled (i.e., background)\nRoad\nSidewalk\nBuilding\nWall\nFence\nPole\nTrafficLight\nTrafficSign\nVegetation\nTerrain\nSky\nPerson\nRider\nCar\nTruck\nBus\nTrain\nMotorcycle\nBicycle\nIn the rest of this blog post, you’ll learn how to apply semantic segmentation to extract a dense, pixel-wise map of each of these classes in both images and video streams.\nIf you’re interested in training your own ENet models for segmentation on your own custom datasets, be sure to refer to\nthis page\nwhere the authors have provided a tutorial on how to do so.\nProject structure\nToday’s project can be obtained from the\n“Downloads”\nsection of this blog post. Let’s take a look at our project structure using the\ntree\ncommand:\n$ tree --dirsfirst\n.\n├── enet-cityscapes\n│   ├── enet-classes.txt\n│   ├── enet-colors.txt\n│   └── enet-model.net\n├── images\n│   ├── example_01.png\n│   ├── example_02.jpg\n│   ├── example_03.jpg\n│   └── example_04.png\n├── videos\n│   ├── massachusetts.mp4\n│   └── toronto.mp4\n├── output\n├── segment.py\n└── segment_video.py\n\n4 directories, 11 files\nOur project has four directories:\nenet-cityscapes/\n: Contains our pre-trained deep learning model, classes list, and color labels to correspond with the classes.\nimages/\n: A selection of four sample images to test our image segmentation script.\nvideos/\n: Includes two sample videos for testing our deep learning segmentation video script.  Credits for these videos are listed in the\n“Video segmentation results”\nsection.\noutput/\n: For organizational purposes, I like to have my script save the processed videos to the\noutput\nfolder. I’m not including the output images/videos in the downloads as the file sizes are quite larger. You’ll need to use today’s code to generate them on your own.\nToday we’ll be reviewing two Python scripts:\nsegment.py\n: Performs deep learning semantic segmentation on a single image. We’ll walk through this script to learn how segmentation works and then test it on single images before moving on to video.\nsegment_video.py\n: As the name suggests, this script will perform semantic segmentation on video.\nSemantic segmentation in images with OpenCV\nLet’s go ahead and get started — open up the\nsegment.py\nfile and insert the following code:\n# import the necessary packages\nimport numpy as np\nimport argparse\nimport imutils\nimport time\nimport cv2\nWe begin by importing necessary packages.\nFor this script, I recommend\nOpenCV 3.4.1 or higher\n. You can follow one of my\ninstallation tutorials\n— just be sure to specify which version of OpenCV you want to download and install as you follow the steps.\nYou’ll also need to install my package of OpenCV convenience functions,\nimutils\n— just use pip to install the package:\n$ pip install --upgrade imutils\nIf you are using Python virtual environments don’t forget to use the\nworkon\ncommand before using\npip\nto install\nimutils\n!\nMoving on, let’s parse our command line arguments:\n# construct the argument parse and parse the arguments\nap = argparse.ArgumentParser()\nap.add_argument(\"-m\", \"--model\", required=True,\n\thelp=\"path to deep learning segmentation model\")\nap.add_argument(\"-c\", \"--classes\", required=True,\n\thelp=\"path to .txt file containing class labels\")\nap.add_argument(\"-i\", \"--image\", required=True,\n\thelp=\"path to input image\")\nap.add_argument(\"-l\", \"--colors\", type=str,\n\thelp=\"path to .txt file containing colors for labels\")\nap.add_argument(\"-w\", \"--width\", type=int, default=500,\n\thelp=\"desired width (in pixels) of input image\")\nargs = vars(ap.parse_args())\nThis script has five command line arguments, two of which are optional:\n--model\n: The path to our deep learning semantic segmentation model.\n--classes\n: The path to a text file containing class labels.\n--image\n: Our input image file path.\n--colors\n:\nOptional\npath to a colors text file. If no file is specified, random colors will be assigned to each class.\n--width\n:\nOptional\ndesired image width. By default the value is\n500\npixels.\nIf you aren’t familiar with the concept of\nargparse\nand command line arguments, definitely review\nthis blog post\nwhich covers command line arguments in-depth.\nLet’s handle our parsing our class labels files and colors next:\n# load the class label names\nCLASSES = open(args[\"classes\"]).read().strip().split(\"\\n\")\n\n# if a colors file was supplied, load it from disk\nif args[\"colors\"]:\n\tCOLORS = open(args[\"colors\"]).read().strip().split(\"\\n\")\n\tCOLORS = [np.array(c.split(\",\")).astype(\"int\") for c in COLORS]\n\tCOLORS = np.array(COLORS, dtype=\"uint8\")\n\n# otherwise, we need to randomly generate RGB colors for each class\n# label\nelse:\n\t# initialize a list of colors to represent each class label in\n\t# the mask (starting with 'black' for the background/unlabeled\n\t# regions)\n\tnp.random.seed(42)\n\tCOLORS = np.random.randint(0, 255, size=(len(CLASSES) - 1, 3),\n\t\tdtype=\"uint8\")\n\tCOLORS = np.vstack([[0, 0, 0], COLORS]).astype(\"uint8\")\nWe load our\nCLASSES\ninto memory from the supplied text file where the path is contained in the command line\nargs\ndictionary (\nLine 23\n).\nIf a pre-specified set of\nCOLORS\nfor each class label is provided in a text file (one per line), we load them into memory (\nLines 26-29\n). Otherwise, we randomly generate\nCOLORS\nfor each label (\nLines 33-40\n).\nFor testing purposes (and since we have 20 classes), let’s create a pretty color lookup legend using OpenCV drawing functions:\n# initialize the legend visualization\nlegend = np.zeros(((len(CLASSES) * 25) + 25, 300, 3), dtype=\"uint8\")\n\n# loop over the class names + colors\nfor (i, (className, color)) in enumerate(zip(CLASSES, COLORS)):\n\t# draw the class name + color on the legend\n\tcolor = [int(c) for c in color]\n\tcv2.putText(legend, className, (5, (i * 25) + 17),\n\t\tcv2.FONT_HERSHEY_SIMPLEX, 0.5, (0, 0, 255), 2)\n\tcv2.rectangle(legend, (100, (i * 25)), (300, (i * 25) + 25),\n\t\ttuple(color), -1)\nHere we generate a legend visualization so we can easily visually associate a class label with a color. The legend consists of the class label and a colored rectangle next to it. This is quickly created by creating a canvas (\nLine 43\n) and dynamically building the legend with a loop (\nLines 46-52\n). Drawing basics are covered in\nthis blog post\n.\nHere’s the result:\nFigure 2:\nOur deep learning semantic segmentation class color legend generated with OpenCV.\nThe deep learning segmentation heavy lifting takes place in the next block:\n# load our serialized model from disk\nprint(\"[INFO] loading model...\")\nnet = cv2.dnn.readNet(args[\"model\"])\n\n# load the input image, resize it, and construct a blob from it,\n# but keeping mind mind that the original input image dimensions\n# ENet was trained on was 1024x512\nimage = cv2.imread(args[\"image\"])\nimage = imutils.resize(image, width=args[\"width\"])\nblob = cv2.dnn.blobFromImage(image, 1 / 255.0, (1024, 512), 0,\n\tswapRB=True, crop=False)\n\n# perform a forward pass using the segmentation model\nnet.setInput(blob)\nstart = time.time()\noutput = net.forward()\nend = time.time()\n\n# show the amount of time inference took\nprint(\"[INFO] inference took {:.4f} seconds\".format(end - start))\nTo perform deep learning semantic segmentation of an image with Python and OpenCV, we:\nLoad the model (\nLine 56\n).\nConstruct a\nblob\n(\nLines 61-64\n).The ENet model we are using in this blog post was trained on input images with 1024×512 resolution — we’ll use the same here. You can learn more about\nhow OpenCV’s blobFromImage works\nhere.\nSet the\nblob\nas input to the network (\nLine 67\n) and perform a forward pass through the neural network (\nLine 69\n).\nI surrounded the forward pass statement with timestamps. The elapsed time is printed to the terminal on\nLine 73\n.\nOur work isn’t done yet — now it’s time to take steps to visualize our results. In the remaining lines of the script, we’ll be generating a color map to overlay on the original image. Each pixel has a corresponding class label index, enabling us to see the results of semantic segmentation on our screen visually.\nTo begin, we need to extract volume dimension information from our output, followed by calculating the class map and color mask:\n# infer the total number of classes along with the spatial dimensions\n# of the mask image via the shape of the output array\n(numClasses, height, width) = output.shape[1:4]\n\n# our output class ID map will be num_classes x height x width in\n# size, so we take the argmax to find the class label with the\n# largest probability for each and every (x, y)-coordinate in the\n# image\nclassMap = np.argmax(output[0], axis=0)\n\n# given the class ID map, we can map each of the class IDs to its\n# corresponding color\nmask = COLORS[classMap]\nWe determine the spatial dimensions of the\noutput\nvolume on\nLine 77\n.\nNext, let’s find the class label index with the largest probability for each and every\n(x, y)\n-coordinate of the output volume (\nLine 83\n). This is known now as our\nclassMap\nand contains a class index for each pixel.\nGiven the class ID indexes, we can use NumPy array indexing to “magically” (and not to mention,\nsuper efficiently\n) lookup the corresponding visualization color for each pixel (\nLine 87\n). Our color\nmask\nwill be overlayed transparently on the original image.\nLet’s finish the script:\n# resize the mask and class map such that its dimensions match the\n# original size of the input image (we're not using the class map\n# here for anything else but this is how you would resize it just in\n# case you wanted to extract specific pixels/classes)\nmask = cv2.resize(mask, (image.shape[1], image.shape[0]),\n\tinterpolation=cv2.INTER_NEAREST)\nclassMap = cv2.resize(classMap, (image.shape[1], image.shape[0]),\n\tinterpolation=cv2.INTER_NEAREST)\n\n# perform a weighted combination of the input image with the mask to\n# form an output visualization\noutput = ((0.4 * image) + (0.6 * mask)).astype(\"uint8\")\n\n# show the input and output images\ncv2.imshow(\"Legend\", legend)\ncv2.imshow(\"Input\", image)\ncv2.imshow(\"Output\", output)\ncv2.waitKey(0)\nWe resize the\nmask\nand\nclassMap\nsuch that they have the exact same dimensions as our input\nimage\n(\nLines 93-96\n). It is\ncritical\nthat we apply nearest neighbor interpolation rather than cubic, bicubic, etc. interpolation as we want to maintain the original class IDs/mask values.\nNow that sizing is correct, we create a “transparent color overlay” by overlaying the mask on our original image (\nLine 100\n). This enables us to easily visualize the output of the segmentation. More information on transparent overlays, and how to construct them,\ncan be found in this post.\nFinally, the\nlegend\nand original +\noutput\nimages are shown to the screen on\nLines 103-105\n.\nSingle-image segmentation results\nBe sure to grab the\n“Downloads”\nto this blog post before using the commands in this section. I’ve provided the model + associated files, images, and Python scripts in a zip file for your convenience.\nThe command line arguments that you supply in your terminal are important to replicate my results. Learn about\ncommand line arguments\nhere if you are new to them.\nWhen you’re ready, open up a terminal + navigate to the project, and execute the following command:\n$ python segment.py --model enet-cityscapes/enet-model.net \\\n\t--classes enet-cityscapes/enet-classes.txt \\\n\t--colors enet-cityscapes/enet-colors.txt \\\n\t--image images/example_01.png\n[INFO] loading model...\n[INFO] inference took 0.2100 seconds\nFigure 3:\nSemantic segmentation with OpenCV reveals a road, sidewalk, person, bycycle, traffic sign, and more!\nNotice how accurate the segmentation is — it clearly segments classes and accurately identifies the person and bicycle (a safety issue for self-driving cars). The road, sidewalk, cars, and even foliage are identified.\nLet’s try another example simply by changing the\n--image\ncommand line argument to be a different image:\n$ python segment.py --model enet-cityscapes/enet-model.net \\\n\t--classes enet-cityscapes/enet-classes.txt \\\n\t--colors enet-cityscapes/enet-colors.txt \\\n\t--image images/example_02.jpg\n[INFO] loading model...\n[INFO] inference took 0.1989 seconds\nFigure 4:\nPython and OpenCV are used to perform deep learning semantic segmentation of a city neighborhood road scene.\nThe result in\nFigure 4\ndemonstrates the accuracy and clarity of this semantic segmentation model. The cars, road, trees, and sky are clearly marked.\nHere’s another example:\n$ python segment.py --model enet-cityscapes/enet-model.net \\\n\t--classes enet-cityscapes/enet-classes.txt \\\n\t--colors enet-cityscapes/enet-colors.txt \\\n\t--image images/example_03.png\n[INFO] loading model...\n[INFO] inference took 0.1992 seconds\nFigure 5:\nIn this example of deep learning semantic segmentation with OpenCV, the road is misclassified as sidewalk, but this could be because people are walking in the road.\nThe above figure is a more complex scene, but ENet can still segment the people walking in front of the car. Unfortunately, the model incorrectly classifies the road as sidewalk, but could be due to the fact that people are walking on it.\nA final example:\n$ python segment.py --model enet-cityscapes/enet-model.net \\\n\t--classes enet-cityscapes/enet-classes.txt \\\n\t--colors enet-cityscapes/enet-colors.txt \\\n\t--image images/example_04.png\n[INFO] loading model...\n[INFO] inference took 0.1916 seconds\nFigure 6:\nThe ENet semantic segmentation neural network demonstrates how deep learning can effectively be used for self driving car applications. The road, sidewalks, cars, foliage, and other classes are clearly identified by the model and displayed with OpenCV.\nThe final image that we’ve sent through ENet shows how the model can clearly segment a truck from a car among other scene classes such as road, sidewalk, foliage, person, etc.\nImplementing semantic segmentation in video with OpenCV\nLet’s continue on and apply semantic segmentation to video. Semantic segmentation in video follows the same concept as on a single image — this time we’ll loop over all frames in a video stream and process each one. I recommend a GPU if you need to process frames in real-time.\nOpen up the\nsegment_video.py\nfile and insert the following code:\n# import the necessary packages\nimport numpy as np\nimport argparse\nimport imutils\nimport time\nimport cv2\n\n# construct the argument parse and parse the arguments\nap = argparse.ArgumentParser()\nap.add_argument(\"-m\", \"--model\", required=True,\n\thelp=\"path to deep learning segmentation model\")\nap.add_argument(\"-c\", \"--classes\", required=True,\n\thelp=\"path to .txt file containing class labels\")\nap.add_argument(\"-v\", \"--video\", required=True,\n\thelp=\"path to input video file\")\nap.add_argument(\"-o\", \"--output\", required=True,\n\thelp=\"path to output video file\")\nap.add_argument(\"-s\", \"--show\", type=int, default=1,\n\thelp=\"whether or not to display frame to screen\")\nap.add_argument(\"-l\", \"--colors\", type=str,\n\thelp=\"path to .txt file containing colors for labels\")\nap.add_argument(\"-w\", \"--width\", type=int, default=500,\n\thelp=\"desired width (in pixels) of input image\")\nargs = vars(ap.parse_args())\nHere we\nimport\nour required packages and parse command line arguments with\nargparse\n. Imports are the same as the previous script. With the exception of the following two command line arguments, the other five are the same as well:\n--video\n: The path to the input video file.\n--show\n: Whether or not to show the video on the screen while processing. You’ll achieve higher FPS throughput if you set this value to\n0\n.\nThe following lines load our classes and associated colors data (or generate random colors). These lines are\nidentical\nto the previous script:\n# load the class label names\nCLASSES = open(args[\"classes\"]).read().strip().split(\"\\n\")\n\n# if a colors file was supplied, load it from disk\nif args[\"colors\"]:\n\tCOLORS = open(args[\"colors\"]).read().strip().split(\"\\n\")\n\tCOLORS = [np.array(c.split(\",\")).astype(\"int\") for c in COLORS]\n\tCOLORS = np.array(COLORS, dtype=\"uint8\")\n\n# otherwise, we need to randomly generate RGB colors for each class\n# label\nelse:\n\t# initialize a list of colors to represent each class label in\n\t# the mask (starting with 'black' for the background/unlabeled\n\t# regions)\n\tnp.random.seed(42)\n\tCOLORS = np.random.randint(0, 255, size=(len(CLASSES) - 1, 3),\n\t\tdtype=\"uint8\")\n\tCOLORS = np.vstack([[0, 0, 0], COLORS]).astype(\"uint8\")\nAfter loading classes and associating a color with each class for visualization, we’ll load the model and initialize the video stream:\n# load our serialized model from disk\nprint(\"[INFO] loading model...\")\nnet = cv2.dnn.readNet(args[\"model\"])\n\n# initialize the video stream and pointer to output video file\nvs = cv2.VideoCapture(args[\"video\"])\nwriter = None\n\n# try to determine the total number of frames in the video file\ntry:\n\tprop =  cv2.cv.CV_CAP_PROP_FRAME_COUNT if imutils.is_cv2() \\\n\t\telse cv2.CAP_PROP_FRAME_COUNT\n\ttotal = int(vs.get(prop))\n\tprint(\"[INFO] {} total frames in video\".format(total))\n\n# an error occurred while trying to determine the total\n# number of frames in the video file\nexcept:\n\tprint(\"[INFO] could not determine # of frames in video\")\n\ttotal = -1\nOur model only needs to be loaded once on\nLine 48\n— we’ll use that same model to process each and every frame.\nFrom there we open a video stream pointer to input video file on and initialize our video writer object (\nLines 51 and 52\n).\nLines 55-59\nattempt to determine the\ntotal\nnumber of frames in the video, otherwise a message is printed indicating that the value could not be determined via\nLines 63 and 64\n. The\ntotal\nvalue will be used later to calculate the approximate runtime of this video processing script.\nLet’s begin looping over video frames:\n# loop over frames from the video file stream\nwhile True:\n\t# read the next frame from the file\n\t(grabbed, frame) = vs.read()\n\n\t# if the frame was not grabbed, then we have reached the end\n\t# of the stream\n\tif not grabbed:\n\t\tbreak\n\n\t# construct a blob from the frame and perform a forward pass\n\t# using the segmentation model\n\tframe = imutils.resize(frame, width=args[\"width\"])\n\tblob = cv2.dnn.blobFromImage(frame, 1 / 255.0, (1024, 512), 0,\n\t\tswapRB=True, crop=False)\n\tnet.setInput(blob)\n\tstart = time.time()\n\toutput = net.forward()\n\tend = time.time()\nOur\nwhile\nloop begins on\nLine 68\n.\nWe grab a\nframe\non\nLine 70\nand subsequently check that it is valid on\nLine 74\n. If it was\nnot grabbed\nproperly, we’ve likely reached the end of the video, so we\nbreak\nout of the frame processing loop (\nLine 75\n).\nThe next set of lines mimic what we accomplished previously with a single image, but this time we are operating on a video\nframe\n. Inference occurs here, so don’t overlook these steps where we:\nConstruct a\nblob\nfrom a resized\nframe\n(\nLines 79-81\n). The ENet model we are using in this blog post was trained on input images with 1024×512 resolution — we’ll use the same here. Learn about\nhow OpenCV’s blobFromImage works\nhere.\nSet the\nblob\nas input (\nLine 82\n) and perform a\nforward\npass through the neural network (\nLine 84\n).\nSegmentation inference is now complete, but we want to post process the data in order to visualize + output the results. The remainder of the loop handles this process over three code blocks:\n# infer the total number of classes along with the spatial\n\t# dimensions of the mask image via the shape of the output array\n\t(numClasses, height, width) = output.shape[1:4]\n\n\t# our output class ID map will be num_classes x height x width in\n\t# size, so we take the argmax to find the class label with the\n\t# largest probability for each and every (x, y)-coordinate in the\n\t# image\n\tclassMap = np.argmax(output[0], axis=0)\n\n\t# given the class ID map, we can map each of the class IDs to its\n\t# corresponding color\n\tmask = COLORS[classMap]\n\n\t# resize the mask such that its dimensions match the original size\n\t# of the input frame\n\tmask = cv2.resize(mask, (frame.shape[1], frame.shape[0]),\n\t\tinterpolation=cv2.INTER_NEAREST)\n\n\t# perform a weighted combination of the input frame with the mask\n\t# to form an output visualization\n\toutput = ((0.3 * frame) + (0.7 * mask)).astype(\"uint8\")\nJust as before:\nWe extract the spatial dimensions of the\noutput\nvolume on\nLine 89\n.\nGenerate our\nclassMap\nby finding the class label index with the largest probability for each and every pixel of the\noutput\nimage array (\nLine 95\n).\nCompute our color\nmask\nfrom the\nCOLORS\nassociated with each class label index in the\nclassMap\n(\nLine 99\n).\nResize the\nmask\nto match the\nframe\ndimensions (\nLines 103 and 104\n).\nAnd finally, overlay the mask on the frame transparently (\nLine 108\n).\nLet’s write the output frames to disk:\n# check if the video writer is None\n\tif writer is None:\n\t\t# initialize our video writer\n\t\tfourcc = cv2.VideoWriter_fourcc(*\"MJPG\")\n\t\twriter = cv2.VideoWriter(args[\"output\"], fourcc, 30,\n\t\t\t(output.shape[1], output.shape[0]), True)\n\n\t\t# some information on processing single frame\n\t\tif total > 0:\n\t\t\telap = (end - start)\n\t\t\tprint(\"[INFO] single frame took {:.4f} seconds\".format(elap))\n\t\t\tprint(\"[INFO] estimated total time: {:.4f}\".format(\n\t\t\t\telap * total))\n\n\t# write the output frame to disk\n\twriter.write(output)\nThe first time the loop runs, the\nwriter is None\n, so we need to instantiate it on\nLines 111-115\n. Learn more about\nwriting video to disk with OpenCV\n.\nUsing the\ntotal\nvideo frame count, we can estimate how long it will take to process the video (\nLines 118-122\n).\nFinally, we actually\nwrite\nthe\noutput\nto disk on\nLine 125\n.\nLet’s display the\nframe\n(if needed) and clean up:\n# check to see if we should display the output frame to our screen\n\tif args[\"show\"] > 0:\n\t\tcv2.imshow(\"Frame\", output)\n\t\tkey = cv2.waitKey(1) & 0xFF\n \n\t\t# if the `q` key was pressed, break from the loop\n\t\tif key == ord(\"q\"):\n\t\t\tbreak\n\n# release the file pointers\nprint(\"[INFO] cleaning up...\")\nwriter.release()\nvs.release()\nIn the last block, we check to see if we should display the output\nframe\nand take action accordingly (\nLines 128 and 129\n). While we’re showing the frames in a window on the screen, if “q” is pressed, we’ll “quit” the frame processing loop (\nLines 130-134\n). Finally we cleanup by releasing pointers.\nVideo segmentation results\nTo perform semantic segmentation in video, grab the\n“Downloads”\nfor this blog post.\nThen, open up a terminal and execute the following command:\n$ python segment_video.py --model enet-cityscapes/enet-model.net \\\n\t--classes enet-cityscapes/enet-classes.txt \\\n\t--colors enet-cityscapes/enet-colors.txt \\\n\t--video videos/massachusetts.mp4 \\\n\t--output output/massachusetts_output.avi\n[INFO] loading model...\n[INFO] 4235 total frames in video\n[INFO] single frame took 0.2491 seconds\n[INFO] estimated total time: 1077.3574\n[INFO] cleaning up...\nI’ve included a sample of my output below:\nCredits:\nThank you to Davis King from\ndlib\nfor putting together a dataset of front/rear views of vehicles. Davis included the videos in his dataset which I then used for this example. Thank you\nJ Utah\nand\nMassachusetts Dash Cam\nfor the example videos. Audio credit to\nBenSound\n.\nWhat if I want to train my own segmentation networks?\nAt this point, if you reviewed both scripts, you learned that deep learning semantic segmentation with a pretrained model is quite easy for both images and video. Python and OpenCV make the process straightforward for us, but don’t be fooled by the low line count of the scripts — there are\na ton\nof computations going on under the hood of the segmentation model.\nTraining a model isn’t as difficult as you’d imagine. If you would like to train your own segmentation networks on your own custom datasets, make sure you refer to\nthe following tutorial provided by the ENet authors\n.\nPlease note that I have not trained a network from scratch using ENet but I wanted to provide it in this post as (1) a matter of completeness and (2) just in case you may want to give it a try.\nKeep in mind though —\nlabeling image data requires a ton of time and resources\n. The ENet authors were able to train their model thanks to the hard work of the\nCityscapes\nteam who graciously have made their efforts available for learning and research.\nNote:\nThe Cityscapes data is for non-commercial use (i.e. academic, research, and learning). Only use the ENet model accordingly.\nWhat's next? We recommend\nPyImageSearch University\n.\nCourse information:\n120+ total classes • 115+ hours of on-demand code walkthrough videos • Last updated: October 2026\n★★★★★\n4.84 (128 Ratings) • 16,000+ Students Enrolled\nI strongly believe that if you had the right teacher you could\nmaster\ncomputer vision and deep learning.\nDo you think learning computer vision and deep learning has to be time-consuming, overwhelming, and complicated? Or has to involve complex mathematics and equations? Or requires a degree in computer science?\nThat’s\nnot\nthe case.\nAll you need to master computer vision and deep learning is for someone to explain things to you in\nsimple, intuitive\nterms.\nAnd that’s exactly what I do\n. My mission is to change education and how complex Artificial Intelligence topics are taught.\nIf you're serious about learning computer vision, your next stop should be PyImageSearch University, the most comprehensive computer vision, deep learning, and OpenCV course online today. Here you’ll learn how to\nsuccessfully\nand\nconfidently\napply computer vision to your work, research, and projects. Join me in computer vision mastery.\nInside PyImageSearch University you'll find:\n✓\n120+ courses\non essential computer vision, deep learning, and OpenCV topics\n✓\n94+ Certificates\nof Completion\n✓\n115+ hours\nof on-demand video\n✓\nBrand new courses released\nregularly\n, ensuring you can keep up with state-of-the-art techniques\n✓\nPre-configured Jupyter Notebooks in Google Colab\n✓ Run all code examples in your web browser — works on Windows, macOS, and Linux (no dev environment configuration required!)\n✓ Access to\ncentralized code repos for\nall\n540+ tutorials\non PyImageSearch\n✓\nEasy one-click downloads\nfor code, datasets, pre-trained models, etc.\n✓\nAccess\non mobile, laptop, desktop, etc.\nClick here to join PyImageSearch University\nSummary\nIn today’s blog post we learned how to apply semantic segmentation using OpenCV, deep learning, and the ENet architecture.\nUsing the pre-trained ENet model on the Cityscapes dataset, we were able to segment both images and video streams into 20 classes in the context of self-driving cars and road scene segmentation, including people (both walking and riding bicycles), vehicles (cars, trucks, buses, motorcycles, etc.), construction (building, walls, fences, etc.), as well as vegetation, terrain, and the ground itself.\nIf you enjoyed today’s blog post, be sure to share it!\nAnd to download the code to this guide, just\nenter your email address in the form below\n— I’ll be sure to notify you when new posts are published here on PyImageSearch as well.\nDownload the Source Code and FREE 17-page Resource Guide\nEnter your email address below to get a .zip of the code and a\nFREE 17-page Resource Guide on Computer Vision, OpenCV, and Deep Learning.\nInside you'll find my hand-picked tutorials, books, courses, and libraries to help you master CV and DL!\nDownload the code!\nWebsite",
  "metadata": {
    "title": "Semantic segmentation with OpenCV and deep learning - PyImageSearch",
    "description": "Learn how to perform semantic segmentation using OpenCV, deep learning, and Python. Utilize the ENet architecture to perform semantic segmentation in images and video using OpenCV.",
    "keywords": null,
    "author": "Adrian Rosebrock",
    "og:locale": "en_US",
    "og:type": "article",
    "og:title": "Semantic segmentation with OpenCV and deep learning - PyImageSearch",
    "og:description": "Learn how to perform semantic segmentation using OpenCV, deep learning, and Python. Utilize the ENet architecture to perform semantic segmentation in images and video using OpenCV.",
    "og:url": "https://pyimagesearch.com/2018/09/03/semantic-segmentation-with-opencv-and-deep-learning/",
    "og:site_name": "PyImageSearch",
    "og:image": "https://pyimagesearch.com/wp-content/uploads/2018/08/opencv_semantic_segmentation_result01.jpg",
    "og:image:width": "600",
    "og:image:height": "406",
    "og:image:type": "image/jpeg",
    "twitter:label1": "Written by",
    "twitter:data1": "Adrian Rosebrock",
    "twitter:label2": "Est. reading time",
    "twitter:data2": "21 minutes",
    "article:published_time": "2018-09-03T14:00:43+00:00",
    "article:modified_time": "2021-04-17T18:05:12+00:00"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/8381776d3ee0/code/body-000.txt",
      "chars": 345,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-001.txt",
      "chars": 105,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-002.txt",
      "chars": 32,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-003.txt",
      "chars": 597,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-004.txt",
      "chars": 705,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-005.txt",
      "chars": 470,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-006.txt",
      "chars": 691,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-007.txt",
      "chars": 529,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-008.txt",
      "chars": 741,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-009.txt",
      "chars": 243,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-010.txt",
      "chars": 243,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-011.txt",
      "chars": 243,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-012.txt",
      "chars": 243,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-013.txt",
      "chars": 896,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-014.txt",
      "chars": 705,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-015.txt",
      "chars": 642,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-016.txt",
      "chars": 552,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-017.txt",
      "chars": 885,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-018.txt",
      "chars": 517,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-019.txt",
      "chars": 325,
      "language": "text"
    },
    {
      "file": "attachments/8381776d3ee0/code/body-020.txt",
      "chars": 394,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "http_404",
      "url": "https://s3-us-west-2.amazonaws.com/static.pyimagesearch.com/opencv-semantic-segmentation/opencv_semantic_segmentation_animation.gif",
      "src": "https://s3-us-west-2.amazonaws.com/static.pyimagesearch.com/opencv-semantic-segmentation/opencv_semantic_segmentation_animation.gif",
      "alt": "",
      "downloaded": false,
      "retried_at": "2026-10-20T18:00:38Z"
    },
    {
      "status": "ok",
      "url": "https://b2633864.assetcdn.net/2633864/wp-content/uploads/2020/01/source-code-icon.png?lossy=2&strip=1&webp=1",
      "bytes": 884,
      "reused": false,
      "local_path": "attachments/8381776d3ee0/images/body-0f8ec41d2103feb4399d.png",
      "src": "https://b2633864.assetcdn.net/2633864/wp-content/uploads/2020/01/source-code-icon.png?lossy=2&strip=1&webp=1",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://b2633864.assetcdn.net/2633864/wp-content/uploads/2018/08/opencv_semantic_segmentation_arch.jpg?lossy=2&strip=1&webp=1",
      "bytes": 40250,
      "reused": true,
      "local_path": "attachments/8381776d3ee0/images/001.jpg",
      "src": "https://b2633864.assetcdn.net/2633864/wp-content/uploads/2018/08/opencv_semantic_segmentation_arch.jpg?lossy=2&strip=1&webp=1",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://b2633864.assetcdn.net/2633864/wp-content/uploads/2018/08/opencv_semantic_segmentation_legend.jpg?lossy=2&strip=1&webp=1",
      "bytes": 32861,
      "reused": false,
      "local_path": "attachments/8381776d3ee0/images/body-e9e8baf5e39e6e1138dd.jpg",
      "src": "https://b2633864.assetcdn.net/2633864/wp-content/uploads/2018/08/opencv_semantic_segmentation_legend.jpg?lossy=2&strip=1&webp=1",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://b2633864.assetcdn.net/2633864/wp-content/uploads/2018/08/opencv_semantic_segmentation_result01.jpg?lossy=2&strip=1&webp=1",
      "bytes": 36442,
      "reused": true,
      "local_path": "attachments/8381776d3ee0/images/005.jpg",
      "src": "https://b2633864.assetcdn.net/2633864/wp-content/uploads/2018/08/opencv_semantic_segmentation_result01.jpg?lossy=2&strip=1&webp=1",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://b2633864.assetcdn.net/2633864/wp-content/uploads/2018/08/opencv_semantic_segmentation_result02.jpg?lossy=2&strip=1&webp=1",
      "bytes": 35503,
      "reused": true,
      "local_path": "attachments/8381776d3ee0/images/007.jpg",
      "src": "https://b2633864.assetcdn.net/2633864/wp-content/uploads/2018/08/opencv_semantic_segmentation_result02.jpg?lossy=2&strip=1&webp=1",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://b2633864.assetcdn.net/2633864/wp-content/uploads/2018/08/opencv_semantic_segmentation_result03.jpg?lossy=2&strip=1&webp=1",
      "bytes": 39943,
      "reused": true,
      "local_path": "attachments/8381776d3ee0/images/009.jpg",
      "src": "https://b2633864.assetcdn.net/2633864/wp-content/uploads/2018/08/opencv_semantic_segmentation_result03.jpg?lossy=2&strip=1&webp=1",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://b2633864.assetcdn.net/2633864/wp-content/uploads/2018/08/opencv_semantic_segmentation_result04.jpg?lossy=2&strip=1&webp=1",
      "bytes": 34740,
      "reused": true,
      "local_path": "attachments/8381776d3ee0/images/011.jpg",
      "src": "https://b2633864.assetcdn.net/2633864/wp-content/uploads/2018/08/opencv_semantic_segmentation_result04.jpg?lossy=2&strip=1&webp=1",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://fast.wistia.com/embed/medias/kno0cmko2z/swatch",
      "bytes": 3143,
      "reused": false,
      "local_path": "attachments/8381776d3ee0/images/body-8ec9834d54100244f57f.img",
      "src": "https://fast.wistia.com/embed/medias/kno0cmko2z/swatch",
      "alt": "",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://b2633864.assetcdn.net/2633864/wp-content/uploads/2020/01/cta-source-guide-1.png?lossy=2&strip=1&webp=1",
      "bytes": 28126,
      "reused": false,
      "local_path": "attachments/8381776d3ee0/images/body-8cd5816cf73385d96fcd.png",
      "src": "https://b2633864.assetcdn.net/2633864/wp-content/uploads/2020/01/cta-source-guide-1.png?lossy=2&strip=1&webp=1",
      "alt": "",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [
    {
      "url": "https://www.youtube.com/watch?v=1iK1gMOVZF0",
      "src": "https://www.youtube.com/embed/1iK1gMOVZF0?feature=oembed",
      "status": "no_1080_or_720",
      "kind": "single_video",
      "error": "ERROR: [youtube] 1iK1gMOVZF0: Requested format is not available. Use --list-formats for a list of available formats\n",
      "title": "",
      "retried_at": "2026-10-20T18:00:43Z"
    },
    {
      "url": "https://www.youtube.com/watch?v=50Uf_T12OGY",
      "src": "https://www.youtube.com/watch?v=50Uf_T12OGY",
      "status": "ok",
      "kind": "single_video",
      "local_path": "attachments/8381776d3ee0/videos/repair-50Uf_T12OGY.mp4",
      "bytes": 273456487,
      "valid": true,
      "height": 1080,
      "width": 1920,
      "duration": "1269.121000",
      "reused": false
    },
    {
      "url": "https://www.youtube.com/watch?v=ihQdg0kiLR4",
      "src": "https://www.youtube.com/watch?v=ihQdg0kiLR4",
      "status": "source_unavailable",
      "kind": "single_video",
      "error": "ERROR: [youtube] ihQdg0kiLR4: Video unavailable\n",
      "title": "",
      "retried_at": "2026-10-20T18:00:39Z"
    }
  ],
  "links": [
    {
      "href": "https://www.youtube.com/embed/1iK1gMOVZF0?feature=oembed",
      "label": "",
      "type": "iframe"
    },
    {
      "href": "https://pyimagesearch.com/wp-content/uploads/2018/08/opencv_semantic_segmentation_arch.jpg",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://arxiv.org/abs/1606.02147",
      "label": "Paszke et al.",
      "type": "a"
    },
    {
      "href": "https://www.cityscapes-dataset.com/",
      "label": "The Cityscapes Dataset",
      "type": "a"
    },
    {
      "href": "https://github.com/TimoSaemann/ENet/tree/master/Tutorial",
      "label": "this page",
      "type": "a"
    },
    {
      "href": "https://pyimagesearch.com/opencv-tutorials-resources-guides/",
      "label": "installation tutorials",
      "type": "a"
    },
    {
      "href": "https://github.com/jrosebr1/imutils",
      "label": "imutils",
      "type": "a"
    },
    {
      "href": "https://pyimagesearch.com/2018/03/12/python-argparse-command-line-arguments/",
      "label": "this blog post",
      "type": "a"
    },
    {
      "href": "https://pyimagesearch.com/2018/07/19/opencv-tutorial-a-guide-to-learn-opencv/",
      "label": "this blog post",
      "type": "a"
    },
    {
      "href": "https://pyimagesearch.com/wp-content/uploads/2018/08/opencv_semantic_segmentation_legend.jpg",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://pyimagesearch.com/2017/11/06/deep-learning-opencvs-blobfromimage-works/",
      "label": "how OpenCV’s blobFromImage works",
      "type": "a"
    },
    {
      "href": "https://pyimagesearch.com/2016/03/07/transparent-overlays-with-opencv/",
      "label": "can be found in this post.",
      "type": "a"
    },
    {
      "href": "https://pyimagesearch.com/wp-content/uploads/2018/08/opencv_semantic_segmentation_result01.jpg",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://pyimagesearch.com/wp-content/uploads/2018/08/opencv_semantic_segmentation_result02.jpg",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://pyimagesearch.com/wp-content/uploads/2018/08/opencv_semantic_segmentation_result03.jpg",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://pyimagesearch.com/wp-content/uploads/2018/08/opencv_semantic_segmentation_result04.jpg",
      "label": "",
      "type": "a"
    },
    {
      "href": "https://pyimagesearch.com/2016/02/22/writing-to-video-with-opencv/",
      "label": "writing video to disk with OpenCV",
      "type": "a"
    },
    {
      "href": "http://dlib.net/",
      "label": "dlib",
      "type": "a"
    },
    {
      "href": "https://www.youtube.com/watch?v=50Uf_T12OGY",
      "label": "J Utah",
      "type": "a"
    },
    {
      "href": "https://www.youtube.com/watch?v=ihQdg0kiLR4",
      "label": "Massachusetts Dash Cam",
      "type": "a"
    },
    {
      "href": "https://www.bensound.com/royalty-free-music/track/downtown",
      "label": "BenSound",
      "type": "a"
    },
    {
      "href": "https://pyimagesearch.com/pyimagesearch-university/?utm_source=blogPost&utm_medium=bottomBanner&utm_campaign=What%27s%20next%3F%20I%20recommend",
      "label": "PyImageSearch University",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:54:09Z",
  "source_url": "https://pyimagesearch.com/2018/09/03/semantic-segmentation-with-opencv-and-deep-learning/",
  "papers": [
    {
      "status": "ok",
      "url": "https://arxiv.org/pdf/1606.02147",
      "bytes": 3081653,
      "reused": false,
      "local_path": "papers/repair/4bd65c3f1a2b622fb4f0.pdf"
    }
  ],
  "actual_video_files": [
    {
      "local_path": "attachments/8381776d3ee0/videos/kno0cmko2z.mp4",
      "bytes": 154424456,
      "valid": true,
      "height": 1080,
      "width": 1920,
      "duration": "231.500000",
      "status": "ok"
    }
  ],
  "body_chars": 29773,
  "expected_body_images": 10,
  "discovery_status": "ok",
  "delivery_status": "partial",
  "missing_reason": "http_404；no_1080_or_720；source_unavailable",
  "raw_html": "attachments/8381776d3ee0/pages/source.html",
  "local_html": "attachments/8381776d3ee0/pages/article.html",
  "body_base_path": "attachments/8381776d3ee0/pages/",
  "body_markdown_path": "attachments/8381776d3ee0/pages/article.md"
}