{
  "source_file": "0930",
  "source_platform": "Hashnode",
  "author": "Lakshmi Mahabaleshwara",
  "title_listed": "Why Your Deep Learning Model Isn't Learning: Diagnosing Data Problems in Medical Imaging",
  "keywords": "",
  "domain": "机器学习",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://freecodecamp.org/news/why-your-deep-learning-model-isn-t-learning-data-problems-in-medical-imaging",
  "publish_date": "2026-05-29",
  "corpus_stage": "",
  "notes": "",
  "id": "7703bb087e0c",
  "collected_at": "2026-10-02T14:12:02",
  "title": "Why Your Deep Learning Model Isn't Learning: Diagnosing Data Problems in Medical Imaging",
  "body_markdown": "I built a clean, well-structured deep learning pipeline using\nMONAI\n(Medical Open Network for AI) on a public abdominal ultrasound dataset.\nThe pipeline included:\nproper subject-grouped train/validation splits\nrobust preprocessing\ncarefully decoded segmentation masks\nsensible loss functions\nconsistent evaluation\nAnd the model still struggled to learn.\nThe interesting part isn't that the model underperformed. What mattered was the diagnosis: a series of simple checks that traced the problem back to the dataset, not the model.\nThose checks are useful far beyond medical imaging. They apply to almost any machine learning project.\nIf you're new to ML, this is a lesson worth carrying into every project:\nunderstand your data before you tune your model.\nI set out to build a medical image segmentation tutorial. I ended up learning a more valuable lesson: no amount of careful engineering can rescue a model from a dataset that can't support the task.\nBy the end of this article, you'll understand:\nHow to evaluate whether a dataset can actually support your task\nWhy \"the model isn't learning\" is often a data problem\nHow to rule out engineering bugs before blaming the data\nPractical diagnostics you can run in minutes\nWhy synthetic training data often struggles in real-world deployment\nWhen to stop tuning and walk away from a dataset\nThis is not a beginner introduction to deep learning – it assumes familiarity with concepts like UNet architectures and training loops. But the data-quality lessons apply broadly to many ML projects.\nWhat We'll Cover:\nThe Dataset\nStep 1: Rule Out the Pipeline Before Blaming the Data\nSubject-grouped splits\nDecoding masks correctly\nLoss design and class weighting\nStep 2: The Model Still Struggled\nStep 3: Interrogating the Dataset\nDiagnostic 1: What Does the Dataset Actually Contain?\nDiagnostic 2: Do Synthetic and Real Images Look Similar?\nDiagnostic 3: Can the gap be fixed by adding real data?\nStep 4: Knowing When to Stop\nA Practical Dataset Evaluation Checklist\nWhat I Would Try Next\nThe Bigger Lesson\nThe Dataset\nI used the\nUS Simulation & Segmentation dataset\n, a public collection of abdominal ultrasound images with organ segmentation labels from Kaggle.\nIt contains:\n926 synthetic ultrasound images\n— generated by a ray-casting simulator from CT scans, with full organ annotations\n617 real ultrasound images\n— from an actual ultrasound scanner\nLabels for 8 organs\n— liver, kidney, gallbladder, pancreas, spleen, bones, vessels, and adrenals\nAt first glance, the dataset looked ideal:\nthousands of images\nmultiple organ classes\nboth synthetic and real ultrasound data\nWhether it actually supported the task was a different question.\nStep 1: Rule Out the Pipeline Before Blaming the Data\nGround rule: you should always rule out the pipeline before blaming the data. A model failing on buggy code looks exactly like a model failing on bad data. The engineering needs to be trustworthy.\nSubject-Grouped Splits\nA common mistake in medical imaging is randomly splitting images into train and test sets.\nThat approach is problematic because many frames come from the same patient. Those frames share anatomy, scanner settings, and noise patterns.\nIf frames from the same patient appear in both the train and test sets, the model can partially memorize patient-specific patterns. Test scores look artificially good, even though the model may fail on truly unseen patients.\nThis is called\nsubject leakage\n.\nThe fix is to split by patient instead of by image:\nfrom sklearn.model_selection import GroupShuffleSplit\n\ndef assign_splits(manifest, val_fraction=0.15, seed=42):\n    train_data = manifest[manifest[\"orig_split\"] == \"train\"]\n    groups = train_data[\"subject_id\"].values\n\n    gss = GroupShuffleSplit(n_splits=1, test_size=val_fraction, random_state=seed)\n    train_idx, val_idx = next(gss.split(X=train_data, y=None, groups=groups))\n\n    train_subjects = set(train_data.iloc[train_idx][\"subject_id\"].unique())\n    val_subjects = set(train_data.iloc[val_idx][\"subject_id\"].unique())\n\n    # Crash loudly if leakage ever sneaks in\n    assert train_subjects.isdisjoint(val_subjects), \"Subject leak detected!\"\n    return train_subjects, val_subjects\nThat assertion matters.\nIf the split logic ever breaks, the pipeline fails loudly instead of silently producing misleading metrics.\nDecoding Masks Correctly\nThe dataset stores labels as color-coded masks. Each organ corresponds to a different RGB color.\nTraining requires converting those colors into integer class labels.\nA naïve implementation uses exact color matching, but resizing operations can slightly alter colors at mask boundaries.\nA more robust approach maps each pixel to its nearest palette color:\nimport numpy as np\n\nPALETTE = np.array([\n    [0, 0, 0],\n    [100, 0, 100],\n    [255, 255, 255],\n    [0, 255, 0],\n    [255, 255, 0],\n    [0, 0, 255],\n    [255, 0, 0],\n    [255, 0, 255],\n    [0, 255, 255],\n], dtype=np.int32)\n\ndef decode_mask(mask_rgb):\n    h, w = mask_rgb.shape[:2]\n    flat = mask_rgb.reshape(-1, 3).astype(np.int32)\n    d2 = (\n        (flat[:, None, :] - PALETTE[None, :, :]) ** 2\n    ).sum(-1)\n    classes = d2.argmin(axis=1).astype(np.uint8)\n    return classes.reshape(h, w)\nBefore training, it’s worth visually checking a few decoded masks against the original images. This catches issues like incorrect palettes, RGB/BGR channel swaps, or resizing artifacts that silently corrupt labels.\nThese bugs rarely throw errors. Instead, the model simply learns poorly. And “\ntrained on wrong labels\n” looks exactly like “\nthe model can’t learn the data.\n”\nVerifying masks early removes that uncertainty.\nLoss Design and Class Weighting\nFor training, I usd standard MONAI segmentation losses. The goal wasn’t to aggressively maximize performance, but to establish a stable and trustworthy baseline.\nThe training curves below show that the model optimized normally: the loss decreased consistently, and the validation dice stabilized rather than diverging. This helped rule out optimization instability as the primary cause of poor final performance.\nThree choices were deliberate:\nDice + Cross-Entropy combined:\nCross-entropy keeps learning stable early on – Dice directly rewards good region overlap. Together they balance each other.\ninclude_background=False\nfor binary segmentation:\nIn a single-organ task, background can be 85–90% of the pixels. Counting it in the loss drowns out the signal for the organ you actually care about, so it's better left out.\nClass weighting for multi-class segmentation:\nWith organs of very different sizes, an unweighted loss lets the model ignore the small, rare ones and still score well. Weighting rare-class mistakes more heavily pushes back against that.\nStep 2: The Model Still Struggled\nThe first experiment focused on liver segmentation — the simplest single-organ task in the dataset.\nTest set\nLiver Dice\nSynthetic test set\n~0.68\nReal ultrasound test set\n~0.48\nDice scores range from 0 (no overlap) to 1 (perfect overlap).\nQualitatively, the predictions often captured rough liver regions but failed at boundaries and consistency across real scans.\nEspecially important:\nthe model struggled even on synthetic in-domain data\nperformance dropped further on real ultrasound images\nAt this point, two explanations were possible:\nthe model or pipeline was flawed\nthe dataset itself was limiting performance\nBecause the engineering had been carefully validated, the second possibility became worth investigating seriously.\nThat's where the real lesson began.\nStep 3: Interrogating the Dataset\nRather than endlessly tuning the model, the productive move is to turn the diagnostic lens on the dataset.\nThree simple checks revealed the real problem. None required retraining or expensive experiments.\nDiagnostic 1: What Does the Dataset Actually Contain?\nThe first step was simply plotting the dataset composition.\n926 labeled synthetic images\n(the bulk of training data)\nOnly 60 labeled real images\n— less than 4% of the dataset\n557 unlabeled real images\n— real data exists, but without labels it can't be used for supervised training\nThis immediately changed the interpretation of the dataset.\nAlthough the dataset contains many real ultrasound scans, almost all labeled training data is synthetic.\nThe model is effectively trained on synthetic ultrasound and expected to generalize to real ultrasound.\nThat's a difficult transfer problem from the start.\nThe limitation is simple: the real images mostly don't have labels, so supervised training has very little real-world data to learn from.\nLesson:\nBefore training anything, chart the dataset composition. A headline image count can be misleading. \"1,500 images\" sounds large until you discover that only a tiny fraction are labeled examples from the target domain.\nDiagnostic 2: Do Synthetic and Real Images Look Similar?\nThe next question was whether the synthetic and real ultrasound images actually followed similar visual distributions.\nPlotting intensity histograms showed a clear mismatch.\nsynthetic images clustered heavily near darker intensities\nreal ultrasound images had broader mid-range intensity distributions\nThe synthetic simulator captured anatomical geometry reasonably well, but it didn't reproduce the texture and noise characteristics of real ultrasound:\nspeckle patterns\nintensity falloff\nscanner-specific artifacts\nThis is the classic\nsynthetic-to-real domain gap.\nThe model learned features tuned to synthetic images and then encountered a substantially different distribution during evaluation. Poor transfer performance became expected rather than surprising.\nLesson:\nWhenever training and deployment happen on different domains — synthetic → real, scanner A → scanner B, hospital A → hospital B — measure the distribution shift directly. Simple histogram comparisons can reveal major problems in minutes.\nDiagnostic 3: Can the gap be fixed by adding real data?\nThe obvious next idea was: why not include some real labeled data during training?\nBut before implementing that approach, it's worth checking how many distinct patients actually had labels.\nLabeled real images: 60\nDistinct subjects (labeled real): 4\n\nFrames per subject:\n  subject h: 26\n  subject a: 16\n  subject g: 10\n  subject b: 8\nOnly\nfour\npatients.\nThat result fundamentally changed the situation.\nProper medical imaging evaluation requires subject-grouped train/test splits. But with only four patients, any evaluation becomes statistically unstable.\nTraining on two or three patients and testing on one or two patients would produce highly unreliable metrics that depend heavily on which patient happened to be held out.\nAt that point, the dataset simply couldn't support trustworthy real-world evaluation.\nLesson:\nIn medical imaging, count subjects, not images. The true size of a dataset is bounded by the number of independent patients, not the number of files.\nStep 4: Knowing When to Stop\nAt this point, additional tuning no longer made sense.\nThe bottleneck was not the architecture, optimizer, or learning rate. The bottleneck was the dataset itself.\nThe pipeline was still valuable and reusable. But this particular dataset couldn't reliably support the intended segmentation task.\nThat distinction matters: sometimes a problem is difficult but solvable, and sometimes the data simply can't support the conclusion you want to draw.\nLearning to recognize the difference is an important ML skill.\nA Practical Dataset Evaluation Checklist\nBefore committing weeks to model development, these checks are worth running on any dataset:\nChart the dataset composition\n— labeled vs unlabeled, class distribution, domain distribution\nCount subjects, not images\n— independent patients matter more than frame count\nCheck class balance\n— rare classes are often ignored without weighting or sampling strategies\nCompare train and deployment distributions\n— especially for cross-domain problems\nVerify labels visually\n— catch preprocessing or annotation errors early\nLook for published baselines\n— low published performance may indicate dataset limitations\nThese checks take minutes and can save weeks of unnecessary tuning.\nWhat I Would Try Next\nImproving results would likely require better data rather than a larger model. The next steps I'd prioritize:\ncollecting more labeled real ultrasound scans, from more distinct patients\nimproving annotation consistency\nsemi-supervised learning to make use of the unlabeled real images\ndomain adaptation between synthetic and real ultrasound\nAll of these target the actual bottleneck: data quality and data diversity.\nThe Bigger Lesson\nIn machine learning, it's easy to focus most of our attention on architectures, hyperparameters, optimization tricks, and newer models.\nBut the dataset quietly defines the ceiling.\nA sophisticated model on weak data often disappoints, while a simpler model on strong data performs surprisingly well.\nThat was the real lesson from this project.\nThe most valuable skill wasn't building the pipeline. It was diagnosing why the model couldn't succeed and being willing to trust what the data was saying.\nThe workflow — checking dataset composition, counting subjects, comparing distributions, ruling out engineering bugs, and deciding when to stop — transfers to almost any ML project.\nIn many projects, better judgment about the data matters more than a better model.\nThe pipeline code and diagnostic notebooks are available at the\nMONAI\nUltrasound Working Group\nrepository\n. Questions, corrections, and improvements are always welcome.",
  "metadata": {
    "title": "Why Your Deep Learning Model Isn't Learning: Diagnosing Data Problems in Medical Imaging",
    "description": "I built a clean, well-structured deep learning pipeline using MONAI (Medical Open Network for AI) on a public abdominal ultrasound dataset. The pipeline included: proper subject-grouped train/validat",
    "keywords": null,
    "author": null,
    "og:site_name": "freeCodeCamp.org",
    "og:type": "article",
    "og:title": "Why Your Deep Learning Model Isn't Learning: Diagnosing Data Problems in Medical Imaging",
    "og:description": "I built a clean, well-structured deep learning pipeline using MONAI (Medical Open Network for AI) on a public abdominal ultrasound dataset. The pipeline included: proper subject-grouped train/validat",
    "og:url": "https://www.freecodecamp.org/news/why-your-deep-learning-model-isn-t-learning-data-problems-in-medical-imaging/",
    "og:image": "https://cdn.hashnode.com/uploads/covers/5e1e335a7a1d3fcc59028c64/36be814e-4189-4905-9470-1cb5860e7124.png",
    "og:image:width": "1920",
    "og:image:height": "1080",
    "twitter:card": "summary_large_image",
    "twitter:title": "Why Your Deep Learning Model Isn't Learning: Diagnosing Data Problems in Medical Imaging",
    "twitter:description": "I built a clean, well-structured deep learning pipeline using MONAI (Medical Open Network for AI) on a public abdominal ultrasound dataset. The pipeline included: proper subject-grouped train/validat",
    "twitter:url": "https://www.freecodecamp.org/news/why-your-deep-learning-model-isn-t-learning-data-problems-in-medical-imaging/",
    "twitter:image": "https://cdn.hashnode.com/uploads/covers/5e1e335a7a1d3fcc59028c64/36be814e-4189-4905-9470-1cb5860e7124.png",
    "twitter:label1": "Written by",
    "twitter:data1": "Lakshmi Mahabaleshwara",
    "twitter:label2": "Filed under",
    "twitter:data2": "Medical Imaging, Healthcare AI, Artificial Intelligence, Machine Learning, Dataanalysis, Deep Learning",
    "twitter:site": "@freecodecamp",
    "article:published_time": "2026-05-29T15:20:57.622Z",
    "article:modified_time": "2026-05-29T15:20:57.622Z",
    "article:tag": "Deep Learning",
    "article:publisher": "https://www.facebook.com/freecodecamp"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/7703bb087e0c/code/body-000.txt",
      "chars": 692,
      "language": "text"
    },
    {
      "file": "attachments/7703bb087e0c/code/body-001.txt",
      "chars": 494,
      "language": "text"
    },
    {
      "file": "attachments/7703bb087e0c/code/body-002.txt",
      "chars": 144,
      "language": "text"
    }
  ],
  "images": [
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/uploads/covers/69fd77e89f93a850a46d376f/841346d4-d3df-48a9-bc4d-31a5dd0d9bb0.png",
      "bytes": 99932,
      "reused": true,
      "local_path": "attachments/7703bb087e0c/images/003.png",
      "src": "https://cdn.hashnode.com/uploads/covers/69fd77e89f93a850a46d376f/841346d4-d3df-48a9-bc4d-31a5dd0d9bb0.png",
      "alt": "Two training curves from a MONAI liver segmentation experiment. The left plot shows training loss steadily decreasing across 50 epochs, while the right plot shows validation Dice scores stabilizing around 0.55–0.60 after initial fluctuations, indicating stable optimization but limited segmentation performance.",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/uploads/covers/69fd77e89f93a850a46d376f/d2855b12-b416-4a76-b743-971bf4389628.png",
      "bytes": 102208,
      "reused": true,
      "local_path": "attachments/7703bb087e0c/images/004.png",
      "src": "https://cdn.hashnode.com/uploads/covers/69fd77e89f93a850a46d376f/d2855b12-b416-4a76-b743-971bf4389628.png",
      "alt": "Bar chart showing the composition of the ultrasound segmentation dataset. The dataset contains 926 labeled synthetic ultrasound images, 60 labeled real ultrasound images, and 557 unlabeled real ultrasound images, for a total of 1,543 images. Labeled real data represents only 3.9% of the dataset.",
      "downloaded": true
    },
    {
      "status": "ok",
      "url": "https://cdn.hashnode.com/uploads/covers/69fd77e89f93a850a46d376f/baac5168-292e-45f8-ab9c-fd468dc63b46.png",
      "bytes": 102368,
      "reused": true,
      "local_path": "attachments/7703bb087e0c/images/005.png",
      "src": "https://cdn.hashnode.com/uploads/covers/69fd77e89f93a850a46d376f/baac5168-292e-45f8-ab9c-fd468dc63b46.png",
      "alt": "Histogram comparing pixel intensity distributions between synthetic and real ultrasound images. Synthetic images cluster heavily around lower intensity values, while real ultrasound images show a broader mid-range distribution. The figure also reports summary statistics including mean intensity, standard deviation, and percentile ranges for both datasets.",
      "downloaded": true
    }
  ],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://project-monai.github.io/",
      "label": "MONAI",
      "type": "a"
    },
    {
      "href": "https://www.kaggle.com/datasets/ignaciorlando/ussimandsegm",
      "label": "US Simulation & Segmentation dataset",
      "type": "a"
    },
    {
      "href": "https://github.com/lakshmi-mahabaleshwara/wg-ultrasound/tree/abdomen_simulation_segmentation/data_and_tutorials/abdomen_us_multiorgan_segmentation",
      "label": "MONAI",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:50:58Z",
  "source_url": "https://freecodecamp.org/news/why-your-deep-learning-model-isn-t-learning-data-problems-in-medical-imaging",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 13510,
  "expected_body_images": 3,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/7703bb087e0c/pages/source.html",
  "local_html": "attachments/7703bb087e0c/pages/article.html",
  "body_base_path": "attachments/7703bb087e0c/pages/",
  "body_markdown_path": "attachments/7703bb087e0c/pages/article.md"
}