{ "source_file": "0930", "source_platform": "Medium", "author": "Chaim Rand", "title_listed": "Capturing and Deploying PyTorch Models with torch.export", "keywords": "", "domain": "机器学习", "content_type": "深度技术解析", "is_paid_flag": "否", "url": "https://towardsdatascience.com/capturing-and-deploying-pytorch-models-with-torch-export/", "publish_date": "2025-08-19", "corpus_stage": "", "notes": "", "id": "cac89aaeead0", "collected_at": "2026-10-02T11:15:17", "title": "Capturing and Deploying PyTorch Models with torch.export | Towards Data Science", "body_markdown": "![Revisit consent button](../images/body-df29beacd7496220cc94.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\n# Capturing and Deploying PyTorch Models with torch.export\nA demonstration of PyTorch’s exciting new export feature on a HuggingFace model\n[Chaim Rand](https://towardsdatascience.com/author/chaimrand/)\nAugust 19, 202515 min read\n![](../images/000.webp)\nPhoto by [Alferio Njau](https://unsplash.com/@rionjau?utm_source=medium&utm_medium=referral) on [Unsplash](https://unsplash.com/)\nWhen we think of starting a new AI/ML model project, most of our attention tends to focus on the big problems, such as: creating and curating datasets, designing the best ML architecture, acquiring appropriately large GPU clusters for training, and building an inference solution that meets target quality-of-service (QOS) requirements. However, it’s often the small details that become our _Achilles’ heel_ , leading to unanticipated bugs and significant production delays.\nOne detail that’s often overlooked is the handoff of a trained model to the inference environment. While this handoff may seem trivial, it can easily become the source of a great deal of frustration. The training and inference environments are rarely identical, with differences varying from runtime libraries to hardware targets. To navigate these differences, the AI/ML model developer must ensure:\n 1. that the model definition, along with its trained weights, is loaded properly into the inference environment, and\n 2. that the model’s behavior does not change.\n\n\nThis post will focus on the first challenge — reliable restoration of the model definition and state in an inference environment. We will survey some of the legacy options and their shortcomings. We will then introduce the new [torch.export](https://docs.pytorch.org/docs/stable/export.html) API and demonstrate it in action on a toy model built with [HuggingFace’s](https://huggingface.co/) [transformers](https://huggingface.co/docs/transformers/en/index) library (version 4.54.0). For our experiments, we will use an [Amazon EC2 g5.xlarge](https://aws.amazon.com/ec2/instance-types/g5/) instance (containing an NVIDIA A10G GPU and 4 vCPUs) running a [PyTorch (2.7) Deep Learning AMI (DLAMI)](https://docs.aws.amazon.com/dlami/latest/devguide/aws-deep-learning-ami-gpu-pytorch-2.7-ubuntu-22-04.html).\nOne of the technologies underlying [torch.export](https://docs.pytorch.org/docs/stable/export.html) is [Torch Dynamo](https://docs.pytorch.org/docs/stable/torch.compiler_dynamo_overview.html), a key component of PyTorch’s graph compilation solution, [torch.compile](https://docs.pytorch.org/tutorials/intermediate/torch_compile_tutorial.html). In a [recent post](https://towardsdatascience.com/maximizing-ai-ml-model-performance-with-pytorch-compilation/) we demonstrated the power of [torch.compile](https://docs.pytorch.org/tutorials/intermediate/torch_compile_tutorial.html) and its importance in optimizing the runtime performance of AI/ML models. In some ways this post can be viewed as a sequel: we will revisit some of the key concepts — particularly graph-breaks, we will reuse the same toy model, and we will demonstrate the use of torch.compile for just-in-time (JIT) compilation in our inference environment. While we recommend reading the [previous post](https://towardsdatascience.com/maximizing-ai-ml-model-performance-with-pytorch-compilation/), it is not a prerequisite for this one.\nLearn this step by step with the interactive [AI Engineer](https://roadmap.sh/ai-engineer) roadmap.\n### Disclaimers:\nAs of the time of this writing (PyTorch 2.8.0), [torch.export](https://docs.pytorch.org/docs/stable/export.html) is a “prototype” feature. While the behaviors discussed in this post are likely to mostly remain, the API definitions could change. Be sure to align your use of [torch.export](https://docs.pytorch.org/docs/stable/export.html) to the latest API versions.\nThis post will cover only a subset of torch.export’s features and behaviors. It should not be viewed as an alternative to the official PyTorch [documentation](https://docs.pytorch.org/tutorials/intermediate/torch_export_tutorial.html). There are a number of pages covering the export feature, including an [introductory tutorial](https://docs.pytorch.org/tutorials/intermediate/torch_export_tutorial.html), an [overview of the programming model](https://docs.pytorch.org/docs/2.8/export.programming_model.html), and a [guide to solving export challenges](https://docs.pytorch.org/tutorials/recipes/torch_export_challenges_solutions.html).\nWhile we will demonstrate its functionality on a toy example, torch.export depends heavily on the model’s details and may exhibit very different behavior in your own project.\nThe code we will share is for demonstration purposes only and should not be relied on for correctness or optimality. Please do not interpret our choice of platform, framework, or any other tool or library as an endorsement for its use.\n## A Toy HuggingFace Model\nTo facilitate our discussion, we define a simple image-to-text generative model using the HuggingFace [transformers](https://huggingface.co/docs/transformers/en/index) library. For simplicity, we skip the training phase and assume that the default constructor returns a pretrained model.\npython\n\n```\nimport torch\n\nNUM_TOKENS = 1024\nMAX_SEQ_LEN = 256\nPAD_ID = 0\nSTART_ID = 1\nEND_ID = 2\n\n# Set up an image-to-text model.\ndef get_model():\n\n # import transformers utilities\n from transformers import (\n VisionEncoderDecoderModel,\n VisionEncoderDecoderConfig,\n AutoConfig\n )\n\n config = VisionEncoderDecoderConfig.from_encoder_decoder_configs(\n encoder_config=AutoConfig.for_model(\"vit\"), # vit encoder\n decoder_config=AutoConfig.for_model(\"gpt2\") # gpt2 decoder\n )\n config.decoder.vocab_size = NUM_TOKENS\n config.decoder.use_cache = False\n config.decoder_start_token_id = START_ID\n config.pad_token_id = PAD_ID\n config.eos_token_id = END_ID\n config.max_length = MAX_SEQ_LEN\n\n model = VisionEncoderDecoderModel(config=config)\n model.encoder.pooler = None # remove unused pooler\n model.eval() # prepare the model for evaluation\n return model\n```\n\nWe define an auto-regressive image-to-text generator that uses the encoder and decoder components of the model to produce a caption for the input image. For simplicity, we use a basic implementation, leaving out common optimization techniques, such as [ _KV caching_](https://huggingface.co/blog/not-lain/kv-caching).\npython\n\n```\n# generate the next token\ndef generate_token(decoder, encoder_hidden_states, sequence):\n outputs = decoder(\n sequence,\n encoder_hidden_states\n )\n logits = outputs[0][:, -1, :]\n return torch.argmax(logits, dim=-1, keepdim=True)\n\n# simple auto-regressive sequence generator\ndef image_to_text_generator(encoder, decoder, image):\n # run encoder\n encoder_hidden_states = encoder(image)[0]\n\n # initialize sequence\n generated_ids = torch.ones(\n (image.shape[0], 1),\n dtype=torch.long,\n device=image.device\n ) * START_ID\n\n for _ in range(MAX_SEQ_LEN):\n # generate next token\n next_token = generate_token(\n decoder,\n encoder_hidden_states,\n generated_ids\n )\n generated_ids = torch.cat([generated_ids, next_token], dim=-1)\n if (next_token == END_ID).all():\n break\n\n return generated_ids\n```\n\nThe following code block demonstrates the use of our generator on a batch of random input images.\npython\n\n```\nimport os, time, random, torch\n\ntorch.manual_seed(42)\nrandom.seed(42)\n\nBATCH_SIZE = 64\nEXPORT_PATH = '/tmp/export/'\n\ndef test_inference(model_path=EXPORT_PATH, mode=None, compile=False):\n device = 'cuda' if torch.cuda.is_available() else 'cpu'\n rnd_image = torch.randn(BATCH_SIZE, 3, 224, 224).to(device)\n encoder, decoder = load_model(model_path, mode)\n encoder = encoder.to(device)\n decoder = decoder.to(device)\n\n if compile:\n encoder = torch.compile(encoder, mode=\"reduce-overhead\")\n decoder = torch.compile(decoder, dynamic=True)\n # run a few warmup rounds\n for i in range(10):\n image_to_text_generator(encoder, decoder, random_image)\n\n t0 = time.perf_counter()\n \n # optionally enable mixed precision\n with torch.amp.autocast(device, dtype=torch.bfloat16, enabled=True):\n with torch.no_grad():\n caption = image_to_text_generator(encoder, decoder, rnd_image)\n\n total_time = time.perf_counter() - t0\n print(f'batched inference total time: {total_time}')\n```\n\nThe encoder and decoder models are loaded via a _load_model_ utility. We define an initial implementation and amend it later based on our choice of model capturing strategy.\nTo prepare our model for the export process, we define a pass-through wrapper class for the decoder. This wrapper ensures the model can be traced using positional (rather than keyword) arguments, which is a current requirement of torch.export.\npython\n\n```\nclass DecoderWrapper(torch.nn.Module):\n def __init__(self, decoder_model):\n super().__init__()\n self.decoder = decoder_model\n\n def forward(self, input_ids, encoder_hidden_states):\n return self.decoder(\n input_ids=input_ids,\n encoder_hidden_states=encoder_hidden_states,\n use_cache=False,\n output_attentions=False,\n output_hidden_states=False,\n return_dict=False\n )\n\ndef load_model(path=EXPORT_PATH, mode=None):\n model = get_model()\n encoder = model.encoder\n decoder = model.decoder\n return encoder, DecoderWrapper(decoder)\n```\n\nNow that we have defined our toy model, let’s explore different strategies for capturing and deploying it to an inference environment.\n## Model Capturing and Deployment Strategies\nIn this section, we’ll review two common methods for capturing and deploying an AI/ML model’s state: _weights-only capture_ and conversion to a serializable intermediate graph-representation using _TorchScript_.\n### Weights Only Capture\nThe first option is to use [torch.save](https://docs.pytorch.org/tutorials/beginner/saving_loading_models.html) to capture only the model weights, not the model definition. This requires you to explicitly redefine the PyTorch model in the inference environment. There are several ways to carryg over the code definition including: copy-pasting (which is very error-prone), pulling from a shared code repository, or using Python packages. If the model definition relies on specific Python package dependencies, you will need to make sure that those packages exist in the inference environment and that their versions match the training environment.\nThe following code block demonstrates capturing and loading the model weights for our toy model:\npython\n\n```\ndef capture_model(model, path=EXPORT_PATH):\n # weights only\n weights_path = os.path.join(EXPORT_PATH, \"weights.pth\")\n torch.save(model.state_dict(), weights_path)\n\ndef load_model(path=EXPORT_PATH, mode=None):\n if mode == 'weights':\n model = get_model()\n weights_path = os.path.join(path,\"weights.pth\")\n state_dict = torch.load(weights_path, map_location=\"cpu\")\n model.load_state_dict(state_dict)\n return model.encoder, DecoderWrapper(model.decoder)\n else:\n model = get_model()\n return model.encoder, DecoderWrapper(model.decoder)\n```\n\nOne advantage of this method is that it provides maximum flexibility for tuning the model’s configuration to the inference environment. For example, you can apply machine-specific optimizations that increase the throughput of the inference workload. This freedom is especially important for very large models that require advanced sharding techniques for model execution.\nHowever, this method assumes you can easily design and configure the inference environment as desired — which cannot be taken for granted. Some inference environments are extremely constrained — with limited control over the runtime libraries and package installations.\nThe separation between the model definition and the model weights also can be the source of all sorts of ugly bugs. Ensuring appropriate alignment between the source code and the model weights requires disciplined version management. A preferred approach would be to bundle the model definition and weights into a single archive.\n### TorchScript Variants\nFor many years, the primary method for capturing a PyTorch model along with its weights was [TorchScript](https://docs.pytorch.org/docs/stable/jit.html). And despite its recent deprecation, it remains widely popular. TorchScript encapsulates two different capturing solutions, [torch.jit.script](https://docs.pytorch.org/docs/stable/generated/torch.jit.script.html) and [torch.jit.trace](https://docs.pytorch.org/docs/stable/generated/torch.jit.trace.html), both of which convert PyTorch models into serializable graph representations. This graph can be loaded into another environment and run as a standalone PyTorch program, with minimal runtime dependencies.\nThe [scripting](https://docs.pytorch.org/docs/stable/generated/torch.jit.script.html) and [tracing](https://docs.pytorch.org/docs/stable/generated/torch.jit.trace.html) functionalities are complementary. [Scripting](https://docs.pytorch.org/docs/stable/generated/torch.jit.script.html) performs _ahead-of-time_ (AOT) static analysis of the source code, whereas [tracing](https://docs.pytorch.org/docs/stable/generated/torch.jit.trace.html) traces the actual execution of the model on a sample input. Scripting is able to capture more complexity in the graph, such as conditional control-flow and dynamic shapes. This is contrary to tracing that captures just the execution path and tensor shapes dictated by the input sample it runs on. On the other hand, torch.jit.trace supports more operations than torch.jit.script (e.g., see [here](https://docs.pytorch.org/docs/stable/jit_language_reference.html)). Often, scripting will succeed where tracing will fail and vice versa. Sometimes, some form of [combination of both](https://docs.pytorch.org/docs/stable/jit.html#mixing-tracing-and-scripting) methods is required.\nLet’s now attempt to convert our toy model to TorchScript. Since the sequence generator makes separate calls to the encoder (once) and the decoder (iteratively) separately, we’ll capture them as separate graphs. The encoder takes fixed-shape inputs and has no input-dependent conditional logic, so we can apply the more flexible tracing option. However, the sequence that we input to the decoder increases in size each time a token is generated, so we have no choice but to use the scripting option.\nHuggingFace supports TorchScript via a dedicated configuration. See the [HuggingFace documentation](https://huggingface.co/docs/transformers/en/torchscript) for more details.\npython\n\n```\nconfig.decoder.torchscript = True\nconfig.encoder.torchscript = True\n```\n\nIn the code block below, we extend our capture and loading utilities with TorchScript support. Note the inclusion of the [torch.jit.freeze](https://docs.pytorch.org/docs/stable/generated/torch.jit.freeze.html) optimization during capture and the use the [torch.jit.optimize_for_inference](https://docs.pytorch.org/docs/stable/generated/torch.jit.optimize_for_inference.html) optimization on the target device.\npython\n\n```\ndef capture_model(model, path=EXPORT_PATH):\n # weights only\n weights_path = os.path.join(EXPORT_PATH, \"weights.pth\")\n torch.save(model.state_dict(), weights_path)\n\n encoder = model.encoder\n decoder = DecoderWrapper(model.decoder)\n\n # torchscript encoder using trace\n example = torch.randn(1, 3, 224, 224)\n encoder_jit = torch.jit.trace(encoder, example)\n # optionally apply jit.freeze optimization\n encoder_jit = torch.jit.freeze(encoder_jit)\n encoder_path = os.path.join(path, \"encoder.pt\")\n torch.jit.save(encoder_jit, encoder_path)\n\n try:\n # torchscript decoder using scripting\n decoder_jit = torch.jit.script(decoder)\n # optionally apply jit.freeze optimization\n decoder_jit = torch.jit.freeze(decoder_jit)\n decoder_path = os.path.join(path, \"decoder.pt\")\n torch.jit.save(decoder_jit, decoder_path)\n except Exception as e:\n print(f'torch.jit.script(model.decoder) failed\\n{e}')\n\ndef load_model(path=EXPORT_PATH, mode=None):\n if mode == 'weights':\n model = get_model()\n weights_path = os.path.join(path,\"weights.pth\")\n state_dict = torch.load(weights_path, map_location=\"cpu\")\n model.load_state_dict(state_dict)\n return model.encoder, DecoderWrapper(model.decoder)\n elif mode == 'torchscript':\n encoder_path = os.path.join(path, \"encoder.pt\")\n decoder_path = os.path.join(path, \"decoder.pt\")\n encoder = torch.jit.load(encoder_path)\n decoder = torch.jit.load(decoder_path)\n # optionally apply target-device optimization \n encoder = torch.jit.optimize_for_inference(encoder)\n decoder = torch.jit.optimize_for_inference(decoder)\n return encoder, decoder\n else:\n model = get_model()\n return model.encoder, DecoderWrapper(model.decoder)\n```\n\nUnfortunately, our capture utility fails when trying to script the decoder model. A common problem with TorchScript is that it often fails on complex models. While many issues can be bypassed by using [torch.jit.trace](https://docs.pytorch.org/docs/stable/generated/torch.jit.trace.html), it cannot be applied to models with dynamic shapes or conditional logic, such as our decoder. It is often possible to rewrite the model implementation to be script-compliant, but it can require a lot of painstaking (and ugly) work. In the case of our decoder model, it would require a lot of intrusive patchwork to the transformers library source code.\nFor more on the topic of TorchScript, please see the official [documentation](https://docs.pytorch.org/docs/stable/jit.html).\n## Model Capturing with torch.export\nThe new way of capturing a model for deployment is [torch.export](https://docs.pytorch.org/docs/main/export.html). Similar to [torch.jit.trace](https://docs.pytorch.org/docs/stable/generated/torch.jit.trace.html), export works by [tracing](https://docs.pytorch.org/docs/stable/export.programming_model.html#basics-of-tracing) the model’s execution on input samples. However, unlike [torch.jit.trace](https://docs.pytorch.org/docs/stable/generated/torch.jit.trace.html), export includes support for [dynamism](https://docs.pytorch.org/docs/main/export.html#expressing-dynamism) and [conditional control flow](https://docs.pytorch.org/docs/main/cond.html). The output of the export utility is an intermediate graph representation called [Export IR](https://docs.pytorch.org/docs/stable/export.ir_spec.html), which can be loaded and executed in a clean inference environment. One of the advantages of our exported model is that, contrary to TorchScript models, they can be optimized for the inference environment using [torch.compile](https://docs.pytorch.org/tutorials/intermediate/torch_compile_tutorial.html). On the other hand, optimizations that require source code changes (e.g., configuring the [attn_implementation](https://huggingface.co/docs/transformers/en/main_classes/model#transformers.PreTrainedModel.from_pretrained.attn_implementation)) cannot be applied.\n### Overcoming Graph Breaks\nA _graph break_ occurs when the export function encounters an “untraceable” portion of Python code (e.g., see [here](https://docs.pytorch.org/docs/stable/generated/exportdb/index.html#not-supported-yet) for unsupported operations). We encountered the concept of _graph breaks_ in our [previous post](https://towardsdatascience.com/maximizing-ai-ml-model-performance-with-pytorch-compilation/) on [graph compilation](https://docs.pytorch.org/tutorials/intermediate/torch_compile_tutorial.html). However, contrary to model compilation where PyTorch will simply fall back to eager mode, torch.export forbids the presence of _graph breaks_. If export fails on your model due to a graph break, you will have to rewrite your code to bypass it.\nThere are several resources at your disposal for overcoming graph breaks, including the [Draft Export](https://docs.pytorch.org/docs/2.8/draft_export.html) utility, which generates a detailed report of export issues, [ExportDB](https://docs.pytorch.org/docs/stable/generated/exportdb/index.html), which maintains a list of supported and unsupported export cases, and a [tutorial](https://docs.pytorch.org/tutorials/recipes/torch_export_challenges_solutions.html) on overcoming common export issues.\n### Debugging an Exported Graph\nOn some occasions, you may succeed in exporting a graph only to find that running it on the target device either fails with an error or returns incorrect output. A common cause of such errors is that some of the variables from the export environment are treated as constants and baked into the exported graph.\nUnfortunately, (as of the time of this writing) the tools for debugging [Export IR](https://docs.pytorch.org/docs/stable/export.ir_spec.html) graphs are somewhat limited. Although the exported model is a [torch.nn.Module](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html) with a [forward](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module.forward) function, you cannot use a debugger to step into it to find the source of the errors.\nWe can, however, inspect the contents of the generated graph using [GraphModule.print_readable](https://docs.pytorch.org/docs/stable/fx.html#torch.fx.GraphModule.print_readable). This will print out all of the graph operations along with comments that point to the source code from which they were generated. Sometimes, this, combined with the output error information, is enough to find the source of the errors and tweak the source code accordingly. See below for an example.\n### Exporting a HuggingFace Model\nTo apply [torch.export](https://docs.pytorch.org/docs/stable/export.html) to our toy model, we first make sure to update the PyTorch library to the latest version (2.8.0 at the time of this writing). The export utility is under rapid development and we want to make sure to get the most up-do-date feature support.\nIn the code block below we revise our capture and load utility functions to support [torch.export](https://docs.pytorch.org/docs/stable/export.html). Note our use of [torch.export.Dim](https://docs.pytorch.org/docs/stable/export.html#expressing-dynamism) to specify dynamic dimensions:\npython\n\n```\ndef capture_model(model, path=EXPORT_PATH):\n encoder = model.encoder\n decoder = DecoderWrapper(model.decoder)\n\n # define dynamic dimensions\n batch = torch.export.Dim(\"batch\")\n seq_len = torch.export.Dim(\"seq_len\", min=2, max=MAX_SEQ_LEN)\n\n # export encoder\n # sample input\n example = torch.randn(4, 3, 224, 224)\n encoder_export = torch.export.export(\n encoder,\n (example,),\n dynamic_shapes=((batch,\n torch.export.Dim.STATIC,\n torch.export.Dim.STATIC,\n torch.export.Dim.STATIC),\n )\n )\n torch.export.save(encoder_export, os.path.join(path, \"encoder.pt2\"))\n\n\n # export decoder\n # get sample input for decoder\n encoder_hidden_states = encoder_export.module()(example)[0]\n decoder_input_ids = torch.ones((4, MAX_SEQ_LEN),\n dtype=torch.long)*START_ID\n\n\n decoder_export = torch.export.export(\n decoder,\n (decoder_input_ids, encoder_hidden_states),\n dynamic_shapes={\n 'input_ids': (batch,seq_len),\n 'encoder_hidden_states': (batch,\n torch.export.Dim.STATIC,\n torch.export.Dim.STATIC)\n }\n )\n torch.export.save(decoder_export, os.path.join(path, \"decoder.pt2\"))\n\n\ndef load_model(path=EXPORT_PATH, mode=None):\n if mode == 'weights':\n model = get_model()\n weights_path = os.path.join(path,\"weights.pth\")\n state_dict = torch.load(weights_path, map_location=\"cpu\")\n model.load_state_dict(state_dict)\n return model.encoder, DecoderWrapper(model.decoder)\n elif mode == 'export':\n encoder_path = os.path.join(path, \"encoder.pt2\")\n decoder_path = os.path.join(path, \"decoder.pt2\")\n encoder = load(encoder_path).module()\n decoder = load(decoder_path).module()\n return encoder, decoder\n else:\n model = get_model()\n return model.encoder, DecoderWrapper(model.decoder)\n```\n\nContrary to TorchScript, [torch.export](https://docs.pytorch.org/docs/stable/export.html) had no issues capturing our decoder and encoder models. In particular, no graph breaks were encountered during tracing.\n## Deploying an Exported Model\nTo complete our demonstration, we’ll test our exported model in a clean inference environment. For this experiment, we’ll use an [Amazon EC2 g5.xlarge](https://aws.amazon.com/ec2/instance-types/g5/) instance (containing an NVIDIA A10G GPU and 4 vCPUs) running a [PyTorch (2.7) Deep Learning AMI (DLAMI)](https://docs.aws.amazon.com/dlami/latest/devguide/aws-deep-learning-ami-gpu-pytorch-2.7-ubuntu-22-04.html). We’ll update to the latest PyTorch version but intentionally won’t install the transformers package\nUnfortunately, our excitement at the success of exporting our model was premature, as running the exported decoder on the GPU results in a runtime error, a portion of which we’ve pasted below:\nshell-session\n\n```\nFile \".24\", line 306, in forwardFile \"/opt/pytorch/lib/python3.12/site-packages/torch/_ops.py\", line 829, in __call__ return self._op(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^RuntimeError: Expected all tensors to be on the same device, but got index is on cpu, different from other tensors on cuda:0 (when checking argument in method wrapper_CUDA__index_select)\n```\n\nThe error indicates during the execution the exported graph, some of the tensors reside on the CPU when they are expected to be on the GPU. Since we explicitly copy the input tensors onto the GPU, we can conclude that these refer to tensors that the graph is creating internally. Since the graph was exported on a CPU device, the use of _device_ =_”cpu”_ was baked into the graph creation, resulting in a runtime error when running on a GPU.\nAlthough the error message points to the faulty line of code (_File “ .24”, line 306, in forward_), there is no actual file that we can add a breakpoint to and debug. We can, however, inspect the contents of the decoder graph and search for places where the use of the CPU device has been inadvertently baked in:\npython\n\n```\ndecoder_export.module().print_readable()\n```\n\nAnalyzing the output (searching for “cpu” references) and cross-referencing the source code using the embedded comments, we discover four locations where the _transformers_ library ([_modeling_gpt2.py_](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py) file) creates tensors on the CPU:\n 1. If not specified, the [GPT2 model](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py#L685) [creates of a ](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py#L854)[ _cache_position_](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py#L854) tensor on the baked-in CPU device using [torch.arange](https://docs.pytorch.org/docs/stable/generated/torch.arange.html). This can be fixed by passing in a user-defined value for _cache_position_.\n 2. On line [861](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py#L861), a [torch.Tensor.to](https://docs.pytorch.org/docs/stable/generated/torch.Tensor.to.html) operation is performed to ensure proper placement of the _position_embeds_ tensor. While this may be required in a case of [model parallelism](https://huggingface.co/docs/transformers/v4.13.0/en/parallelism), we do not require it.\n 3. If not specified, the model [creates a causal mask](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py#L867) using tensors that it creates on the baked-in CPU device. We could bypass this by passing in a user-defined causal mask, but we are perfectly happy keeping this _None_ and relying on the use of the _is_causal_ flag of the [sdpa attention function](https://docs.pytorch.org/docs/stable/generated/torch.nn.functional.scaled_dot_product_attention.html).\n 4. If not specified, the model [creates an ](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py#L878)[ _ecnoder_attention_mask_](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py#L878) tensor on the baked-in CPU. Once again, we could specify a value for this tensor, but since the mask would be all _True_ , setting it _None_ achieves the same purpose.\n\n\nThe following patch summarizes the changes we performed on the [ _modeling_gpt2.py_](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py) file. Importantly, these changes were specific to our toy model and will not generalize to all use cases. This kind of _monkey-patching_ is ill-advised and requires extensive testing before being used in a production setting.\ndiff\n\n```\n@@ -861 +861 @@\n- hidden_states = inputs_embeds + position_embeds.to(inputs_embeds.device)\n+ hidden_states = inputs_embeds + position_embeds\n@@ -867,3 +867 @@\n- causal_mask = self._update_causal_mask(\n- attention_mask, inputs_embeds, cache_position, past_key_values, output_attentions\n- )\n+ causal_mask = None\n@@ -877 +875 @@\n- if encoder_attention_mask is None:\n+ if not _use_sdpa and encoder_attention_mask is None:\n@@ -880,3 +878 @@\n- encoder_attention_mask = _prepare_4d_attention_mask_for_sdpa(\n- mask=encoder_attention_mask, dtype=inputs_embeds.dtype, tgt_len=input_shape[-1]\n- )\n+ pass\n```\n\nIn the following code block, we extend the _decoder_ definitions and export implementation with an explicit value for _cache_position_.\npython\n\n```\ndef generate_token(decoder, encoder_hidden_states, sequence):\n outputs = decoder(\n sequence,\n encoder_hidden_states,\n torch.arange(sequence.shape[1], device=sequence.device)\n )\n logits = outputs[0][:, -1, :]\n return torch.argmax(logits, dim=-1, keepdim=True)\n\nclass DecoderWrapper(torch.nn.Module):\n def __init__(self, decoder_model):\n super().__init__()\n self.decoder = decoder_model\n\n def forward(self, input_ids, encoder_hidden_states, cache_position):\n return self.decoder(\n input_ids=input_ids,\n cache_position=cache_position,\n encoder_hidden_states=encoder_hidden_states,\n use_cache=False,\n output_attentions=False,\n output_hidden_states=False,\n return_dict=False\n )\n\ndef capture_model(model, path=EXPORT_PATH):\n encoder = model.encoder\n decoder = DecoderWrapper(model.decoder)\n\n # define dynamic dimensions\n batch = torch.export.Dim(\"batch\")\n seq_len = torch.export.Dim(\"seq_len\", min=2, max=MAX_SEQ_LEN)\n\n # export encoder\n # sample tensor\n example = torch.randn(4, 3, 224, 224)\n encoder_export = torch.export.export(\n encoder,\n (example,),\n dynamic_shapes=((batch,\n torch.export.Dim.STATIC,\n torch.export.Dim.STATIC,\n torch.export.Dim.STATIC),\n )\n )\n torch.export.save(encoder_export, os.path.join(path, \"encoder.pt2\"))\n\n # export decoder\n # get sample input for decoder\n encoder_hidden_states = encoder_export.module()(example)[0]\n decoder_input_ids = torch.ones((4, MAX_SEQ_LEN), \n dtype=torch.long)*START_ID\n\n decoder_args = (\n decoder_input_ids,\n encoder_hidden_states,\n torch.arange(MAX_SEQ_LEN)\n )\n\n dynamic_shapes = {\n 'input_ids': (batch,seq_len),\n 'encoder_hidden_states': (batch,\n torch.export.Dim.STATIC,\n torch.export.Dim.STATIC),\n 'cache_position': (seq_len,),\n }\n\n decoder_export = torch.export.export(\n decoder,\n decoder_args,\n dynamic_shapes=dynamic_shapes\n )\n torch.export.save(decoder_export, os.path.join(path, \"decoder.pt2\"))\n```\n\nFollowing these change, the exported decoder succeeds in generating sequences on the GPU device. We hope (and expect) that as the torch.export feature evolves, these kinds of issues will be handled automatically by the internal tracing mechanism.\nWe further test the potential for machine-specific optimizations by applying graph compilation. For details on our choice of compilation parameters, see our [previous post](https://towardsdatascience.com/maximizing-ai-ml-model-performance-with-pytorch-compilation/).\npython\n\n```\nencoder = torch.compile(encoder, mode=\"reduce-overhead\")\ndecoder = torch.compile(decoder, dynamic=True)\n```\n\nThe table below captures the execution time of our model on a batch of random images, with and without torch.compile. (Note, that running the original model requires installation of the transformers library, version 4.54.0).\n![](../images/001.webp)\nRuntime Results (by Author)\nWe can see that exporting the model to graph representation results in a speed-up of 10.7%. The model compilation, however, had the opposite effect, significantly increasing the execution time. It is likely that this could be fixed through appropriate tuning.\n## Summary\nIn this post we explored the new torch.export utility and demonstrated its use in capturing and deploying a toy HuggingFace model. We found that is has a number of powerful and compelling properties, including:\n * **Support for complex models** : torch.export succeeded in capturing models that failed with TorchScript.\n * **Portability** : Exported models can be loaded and executed as standalone programs without specific package dependencies.\n * **Machine-specific optimizations** : exported models are compatible with graph compilation, enabling the application of machine-specific optimizations.\n\n\nWe also encountered some of torch.export's limitations:\n * **Unintended consequences of graph creation:** If we are not careful about how we design our model, values from the export environment can be inadvertently baked into the resulting graph, breaking its compatibility with the inference environment.\n * **Limited debugging tools:** As of this writing, the tools for debugging the execution of exported graphs are limited.\n\n\nAlthough it still requires some refinement, torch.export is already a huge improvement over previous capturing solutions like TorchScript. We look forward to seeing it continue to evolve and improve.\nFor more details on capturing AI/ML models using PyTorch Export and to keep track of API changes, please see the [torch.export](https://docs.pytorch.org/docs/stable/export.html) documentation. If you’re running inference on an edge device, also see the [ExecuTorch](https://docs.pytorch.org/executorch/stable/index.html) solution for model deployment, which is based on [torch.export](https://docs.pytorch.org/docs/stable/export.html).\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](../images/body-97f5483d57a9b8741146.jpeg)\nWritten By\n## [Chaim Rand](https://towardsdatascience.com/author/chaimrand/)\n[See all from Chaim Rand](https://towardsdatascience.com/author/chaimrand/)\n[Deep Dives](https://towardsdatascience.com/tag/deep-dives/)[Deep Learning](https://towardsdatascience.com/tag/deep-learning/)[Model Deployment](https://towardsdatascience.com/tag/model-deployment/)[Pytorch](https://towardsdatascience.com/tag/pytorch/)[Torchscript](https://towardsdatascience.com/tag/torchscript/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fcapturing-and-deploying-pytorch-models-with-torch-export%2F&title=Capturing%20and%20Deploying%20PyTorch%20Models%20with%20torch.export)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fcapturing-and-deploying-pytorch-models-with-torch-export%2F&title=Capturing%20and%20Deploying%20PyTorch%20Models%20with%20torch.export)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fcapturing-and-deploying-pytorch-models-with-torch-export%2F&text=Capturing%20and%20Deploying%20PyTorch%20Models%20with%20torch.export)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [![](../images/002.png)](https://towardsdatascience.com/experiments-peeking-and-optimal-stopping-954506cec665/)\n## [Experiments, Peeking, and Optimal Stopping](https://towardsdatascience.com/experiments-peeking-and-optimal-stopping-954506cec665/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nHow to run valid experiments with smaller sample sizes with the Sequential Probability Ratio Test\n[Matteo Courthoud](https://towardsdatascience.com/author/matteo.courthoud/)\nMay 2, 202214 min read\n * [![](../images/003.jpg)](https://towardsdatascience.com/what-it-means-for-something-to-be-learnable-pac-learnability-c84de9c061ad/)\n## [Defining Learnability with the PAC Framework](https://towardsdatascience.com/what-it-means-for-something-to-be-learnable-pac-learnability-c84de9c061ad/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nIn machine learning we often say something is \"learnable\". What does that really mean? Read on to find out.\n[Wilson Wang](https://towardsdatascience.com/author/wilson-wang/)\nAugust 12, 20214 min read\n * [![](../images/004.png)](https://towardsdatascience.com/all-deep-learning-is-statistical-model-building-fc310328f07/)\n## [All deep learning is statistical model building](https://towardsdatascience.com/all-deep-learning-is-statistical-model-building-fc310328f07/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nDeep learning is often used to make predictions for data driven analysis. But what are the meanings of these predictions? This post...\n[Tom Charnock](https://towardsdatascience.com/author/tomcharnock/)\nAugust 3, 202017 min read\n * [![](../images/005.jpeg)](https://towardsdatascience.com/xgboost-vs-lightgbm-on-a-high-dimensional-dataset-bbd5b5174274/)\n## [XGBoost vs LightGBM on a High Dimensional Dataset](https://towardsdatascience.com/xgboost-vs-lightgbm-on-a-high-dimensional-dataset-bbd5b5174274/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nComparison of both speed and performance\n[Soner Yıldırım](https://towardsdatascience.com/author/sonery/)\nSeptember 29, 20205 min read\n * [![](../images/006.jpg)](https://towardsdatascience.com/speak-to-me-how-many-words-a-model-is-reading-331e3af86d27/)\n## [Speak to me: How many words a model is reading](https://towardsdatascience.com/speak-to-me-how-many-words-a-model-is-reading-331e3af86d27/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nWhy and how to overcome the inner limit of a Large Language Model\n[Salvatore Raieli](https://towardsdatascience.com/author/salvatore-raieli/)\nJuly 14, 202319 min read\n * [![](../images/007.jpeg)](https://towardsdatascience.com/multilingual-nlp-get-started-with-the-tydiqa-goldp-dataset-in-10-minutes-or-less-c56c01ad47d2/)\n## [Multilingual NLP: Get Started with the TyDiQA-GoldP Dataset in 10 Minutes or Less](https://towardsdatascience.com/multilingual-nlp-get-started-with-the-tydiqa-goldp-dataset-in-10-minutes-or-less-c56c01ad47d2/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nA hands-on tutorial for retrieving, processing and using the dataset\n[Yousef Nami](https://towardsdatascience.com/author/namiyousef96/)\nOctober 25, 202210 min read\n * [![](../images/008.png)](https://towardsdatascience.com/learn-python-data-analytics-by-example-chicago-traffic-crashes-4840f6f15ed/)\n## [Learn Python Data Analytics by Example - Chicago Traffic Crashes](https://towardsdatascience.com/learn-python-data-analytics-by-example-chicago-traffic-crashes-4840f6f15ed/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nA fun project and detailed walkthrough of data analytics steps to help you learn Python, pandas, matplotlib, seaborn and Folium\n[Nick Cox](https://towardsdatascience.com/author/nickdcox/)\nFebruary 9, 202112 min read\n * [![](../images/009.jpg)](https://towardsdatascience.com/critical-facts-that-every-data-scientist-should-know-part-2-c9c06cde6e21/)\n## [Critical Facts That Every Data Scientist Should Know - Part 2](https://towardsdatascience.com/critical-facts-that-every-data-scientist-should-know-part-2-c9c06cde6e21/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nIn the second part of this article, we will conclude the list of some of the critical facts you need to know to become a super data...\n[Mario Larcher](https://towardsdatascience.com/author/mnslarcher/)\nApril 19, 202213 min read\n * [![](../images/010.png)](https://towardsdatascience.com/dynamic-pricing-using-reinforcement-learning-and-neural-networks-cc3abe374bf5/)\n## [Dynamic Pricing using Reinforcement Learning and Neural Networks](https://towardsdatascience.com/dynamic-pricing-using-reinforcement-learning-and-neural-networks-cc3abe374bf5/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nAn intelligent system that can increase e-commerce sales and profits\n[Reslley Gabriel](https://towardsdatascience.com/author/reslleygabriel/)\nFebruary 16, 20215 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n#### Recommended Articles \nClose\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/4970/image?etag=1780565225463&width=500&height=500) Productivity Tips, Data Career Insights, and Other Recent Must-Reads ](https://towardsdatascience.com/productivity-tips-data-career-insights-and-other-recent-must-reads-5c47eb7a2fe0/ \"Productivity Tips, Data Career Insights, and Other Recent Must-Reads\")\n\n\n", "metadata": { "title": "Capturing and Deploying PyTorch Models with torch.export | Towards Data Science", "description": "A demonstration of PyTorch’s exciting new export feature on a HuggingFace model", "keywords": null, "author": "Chaim Rand", "og:locale": "en_US", "og:type": "article", "og:title": "Capturing and Deploying PyTorch Models with torch.export", "og:description": "A demonstration of PyTorch’s exciting new export feature on a HuggingFace model", "og:url": "https://towardsdatascience.com/capturing-and-deploying-pytorch-models-with-torch-export/", "og:site_name": "Towards Data Science", "og:image": "https://assets.insightmediagroup.io/media/wp-content/uploads/2025/08/1-1-scaled-1.webp", "og:image:width": "2560", "og:image:height": "1501", "og:image:type": "image/webp", "twitter:card": "summary_large_image", "twitter:site": "@TDataScience", "twitter:creator": "@TDataScience", "twitter:label1": "Written by", "twitter:data1": "Chaim Rand", "twitter:label2": "Est. reading time", "twitter:data2": "15 min read", "article:published_time": "2025-08-19T19:20:12.000Z", "article:modified_time": "2025-08-19T19:20:12.000Z", "article:author": "https://towardsdatascience.com/author/chaimrand/" }, "paywall": { "listed_paid": false, "detected": false, "marker": "", "public_only": false, "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙" }, "crawl": { "success": true, "status_code": 202, "error": "" }, "code_blocks": [ { "language": "txt", "file": "attachments/cac89aaeead0/code/block_000.txt", "chars": 918 }, { "language": "txt", "file": "attachments/cac89aaeead0/code/block_001.txt", "chars": 924 }, { "language": "txt", "file": "attachments/cac89aaeead0/code/block_002.txt", "chars": 1049 }, { "language": "txt", "file": "attachments/cac89aaeead0/code/block_003.txt", "chars": 635 }, { "language": "txt", "file": "attachments/cac89aaeead0/code/block_004.txt", "chars": 594 }, { "language": "txt", "file": "attachments/cac89aaeead0/code/block_005.txt", "chars": 68 }, { "language": "txt", "file": "attachments/cac89aaeead0/code/block_006.txt", "chars": 1840 }, { "language": "txt", "file": "attachments/cac89aaeead0/code/block_007.txt", "chars": 2106 }, { "language": "txt", "file": "attachments/cac89aaeead0/code/block_008.txt", "chars": 387 }, { "language": "txt", "file": "attachments/cac89aaeead0/code/block_009.txt", "chars": 41 }, { "language": "txt", "file": "attachments/cac89aaeead0/code/block_010.txt", "chars": 725 }, { "language": "txt", "file": "attachments/cac89aaeead0/code/block_011.txt", "chars": 2345 }, { "language": "txt", "file": "attachments/cac89aaeead0/code/block_012.txt", "chars": 104 } ], "images": [ { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T18:08:41Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T18:08:41Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T18:09:22Z" }, { "status": "http_404", "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/4970/image?etag=1780565225463&width=500&height=500", "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/4970/image?etag=1780565225463&width=500&height=500", "alt": "", "downloaded": false, "discovery": "saved_markdown", "retried_at": "2026-10-20T18:09:22Z" } ], "audio": [], "videos": [ { "url": "https://www.youtube.com/c/TowardsDataScience", "kind": "channel_reference", "status": "reference_only" } ], "links": [ { "type": "internal", "href": "https://towardsdatascience.com/" }, { "type": "internal", "href": "https://towardsdatascience.com/latest/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/editors-pick/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/the-variable/" }, { "type": "internal", "href": "https://towardsdatascience.com/shipai/" }, { "type": "internal", "href": "https://towardsdatascience.com/submissions/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/artificial-intelligence/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/chaimrand/" }, { "type": "internal", "href": "https://towardsdatascience.com/maximizing-ai-ml-model-performance-with-pytorch-compilation/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/deep-dives/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/deep-learning/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/model-deployment/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/pytorch/" }, { "type": "internal", "href": "https://towardsdatascience.com/tag/torchscript/" }, { "type": "internal", "href": "https://towardsdatascience.com/experiments-peeking-and-optimal-stopping-954506cec665/" }, { "type": "internal", "href": "https://towardsdatascience.com/category/data-science/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/matteo.courthoud/" }, { "type": "internal", "href": "https://towardsdatascience.com/what-it-means-for-something-to-be-learnable-pac-learnability-c84de9c061ad/" }, { "type": "internal", "href": "https://towardsdatascience.com/author/wilson-wang/" }, { "type": "internal", "href": "https://towardsdatascience.com/all-deep-learning-is-statistical-model-building-fc310328f07/" }, { "type": "external", "href": "https://write.insightmediagroup.io" }, { "type": "external", "href": "https://www.linkedin.com/company/towards-data-science/" }, { "type": "external", "href": "https://x.com/TDataScience" }, { "type": "external", "href": "https://unsplash.com/@rionjau" }, { "type": "external", "href": "https://unsplash.com/" }, { "type": "external", "href": "https://docs.pytorch.org/docs/stable/export.html" }, { "type": "external", "href": "https://huggingface.co/" }, { "type": "external", "href": "https://huggingface.co/docs/transformers/en/index" }, { "type": "external", "href": "https://aws.amazon.com/ec2/instance-types/g5/" }, { "type": "external", "href": "https://docs.aws.amazon.com/dlami/latest/devguide/aws-deep-learning-ami-gpu-pytorch-2.7-ubuntu-22-04.html" }, { "type": "external", "href": "https://docs.pytorch.org/docs/stable/torch.compiler_dynamo_overview.html" }, { "type": "external", "href": "https://docs.pytorch.org/tutorials/intermediate/torch_compile_tutorial.html" }, { "type": "external", "href": "https://roadmap.sh/ai-engineer" }, { "type": "external", "href": "https://docs.pytorch.org/tutorials/intermediate/torch_export_tutorial.html" }, { "type": "external", "href": "https://docs.pytorch.org/docs/2.8/export.programming_model.html" }, { "type": "external", "href": "https://docs.pytorch.org/tutorials/recipes/torch_export_challenges_solutions.html" }, { "type": "external", "href": "https://huggingface.co/blog/not-lain/kv-caching" }, { "type": "external", "href": "https://docs.pytorch.org/tutorials/beginner/saving_loading_models.html" }, { "type": "external", "href": "https://docs.pytorch.org/docs/stable/jit.html" }, { "type": "external", "href": "https://docs.pytorch.org/docs/stable/generated/torch.jit.script.html" } ], "repair_version": 2, "repaired_at": "2026-10-20T17:33:58Z", "source_url": "https://towardsdatascience.com/capturing-and-deploying-pytorch-models-with-torch-export/", "delivery_status": "unverified", "discovery_status": "http_403", "missing_reason": "源页面未重新核验:http_403；已按保留正文补采图片", "repair_error": "", "actual_video_files": [], "expected_saved_body_images": 4, "image_discovery": "saved_markdown", "source_body_markdown": "![Revisit consent button](https://cdn-cookieyes.com/assets/images/revisit.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](https://cdn-cookieyes.com/assets/images/close.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n * Cookie\nBCTempID\n * Duration\n10 minutes\n * Description\nNo description available.\n\n\n * Cookie\n__cf_bm\n * Duration\n1 hour\n * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n * Cookie\nAWSALBCORS\n * Duration\n7 days\n * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n * Cookie\n_cfuvid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n * Cookie\nli_gc\n * Duration\n6 months\n * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n * Cookie\n__hssrc\n * Duration\nsession\n * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n * Cookie\n__hssc\n * Duration\n1 hour\n * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n * Cookie\nwpEmojiSettingsSupports\n * Duration\nsession\n * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n * Cookie\nBCSessionID\n * Duration\n1 year 1 month 4 days\n * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n * Cookie\n_octo\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nlogged_in\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\n__Secure-YEC\n * Duration\npast\n * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n * Cookie\n__eoi\n * Duration\n6 months\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTGCORS\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\nlogin-status-p\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nAWSALBTG\n * Duration\n7 days\n * Description\nNo description available.\n\n\n * Cookie\ncsrf_token\n * Duration\nsession\n * Description\nNo description available.\n\n\n * Cookie\ntoken_v2\n * Duration\n1 day\n * Description\nDescription is currently not available.\n\n\n * Cookie\nD\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nPHPSESSID\n * Duration\nsession\n * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n * Cookie\nVISITOR_PRIVACY_METADATA\n * Duration\n6 months\n * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n * Cookie\ncookietest\n * Duration\nsession\n * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n * Cookie\n__Host-airtable-session\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\n__Host-airtable-session.sig\n * Duration\n1 year\n * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n * Cookie\nm\n * Duration\n1 year 1 month 4 days\n * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n * Cookie\nBIGipServer*\n * Duration\nsession\n * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n * Cookie\n__cfruid\n * Duration\nsession\n * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n * Cookie\n_GRECAPTCHA\n * Duration\n6 months\n * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n * Cookie\n__Secure-YNID\n * Duration\n6 months\n * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n * Cookie\ncookieyes-consent\n * Duration\n1 year\n * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n * Cookie\nlidc\n * Duration\n1 day\n * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n * Cookie\nbrw\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nbrwConsent\n * Duration\n5 minutes\n * Description\nDescription is currently not available.\n\n\n * Cookie\nWMF-Uniq\n * Duration\n1 year\n * Description\nDescription is currently not available.\n\n\n * Cookie\nloom_anon_comment\n * Duration\n1 year\n * Description\nNo description available.\n\n\n * Cookie\nloom_referral_video\n * Duration\nsession\n * Description\nDescription is currently not available.\n\n\n * Cookie\nVISITOR_INFO1_LIVE\n * Duration\n6 months\n * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n * Cookie\nyt-remote-connected-devices\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n * Duration\nNever Expires\n * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n * Cookie\nyt-remote-device-id\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-name\n * Duration\nsession\n * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\nyt-remote-fast-check-period\n * Duration\nsession\n * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n * Cookie\nyt-remote-session-app\n * Duration\nsession\n * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n * Cookie\nyt-remote-cast-available\n * Duration\nsession\n * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n * Cookie\nyt-remote-cast-installed\n * Duration\nsession\n * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n * Cookie\ncp_session\n * Duration\n3 months\n * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n * Cookie\nloid\n * Duration\n1 year 1 month 4 days\n * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n * Cookie\n__hstc\n * Duration\n6 months\n * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n * Cookie\nhubspotutk\n * Duration\n6 months\n * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n * Cookie\n_ga\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n * Cookie\n_ga_*\n * Duration\n1 year 1 month 4 days\n * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n * Cookie\n__Host-psifi.analyticsTrace\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n__Host-psifi.analyticsTraceV2\n * Duration\n6 hours\n * Description\nDescription is currently not available.\n\n\n * Cookie\n_gh_sess\n * Duration\nsession\n * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n * Cookie\nYSC\n * Duration\nsession\n * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n * Cookie\najs_anonymous_id\n * Duration\n1 year\n * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n * Cookie\nvuid\n * Duration\n1 year 1 month 4 days\n * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n * Cookie\nAWSALB\n * Duration\n7 days\n * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n * Cookie\nacq\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nacq.sig\n * Duration\npast\n * Description\nDescription is currently not available.\n\n\n * Cookie\nptc\n * Duration\n2 years\n * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n * Cookie\nmuc_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n * Cookie\nguest_id_marketing\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\nguest_id_ads\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n * Cookie\npersonalization_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n * Cookie\nguest_id\n * Duration\n1 year 1 month 4 days\n * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n * Cookie\nbcookie\n * Duration\n1 year\n * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n * Cookie\n__Secure-ROLLOUT_TOKEN\n * Duration\n6 months\n * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n * Cookie\nyt.innertube::nextId\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nyt.innertube::requests\n * Duration\nNever Expires\n * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n * Cookie\nsession_tracker\n * Duration\nsession\n * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n * Cookie\nedgebucket\n * Duration\nsession\n * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n * Cookie\ndid\n * Duration\n1 year\n * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\n# Capturing and Deploying PyTorch Models with torch.export\nA demonstration of PyTorch’s exciting new export feature on a HuggingFace model\n[Chaim Rand](https://towardsdatascience.com/author/chaimrand/)\nAugust 19, 202515 min read\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/08/1-1-scaled-1.webp)\nPhoto by [Alferio Njau](https://unsplash.com/@rionjau?utm_source=medium&utm_medium=referral) on [Unsplash](https://unsplash.com/)\nWhen we think of starting a new AI/ML model project, most of our attention tends to focus on the big problems, such as: creating and curating datasets, designing the best ML architecture, acquiring appropriately large GPU clusters for training, and building an inference solution that meets target quality-of-service (QOS) requirements. However, it’s often the small details that become our _Achilles’ heel_ , leading to unanticipated bugs and significant production delays.\nOne detail that’s often overlooked is the handoff of a trained model to the inference environment. While this handoff may seem trivial, it can easily become the source of a great deal of frustration. The training and inference environments are rarely identical, with differences varying from runtime libraries to hardware targets. To navigate these differences, the AI/ML model developer must ensure:\n 1. that the model definition, along with its trained weights, is loaded properly into the inference environment, and\n 2. that the model’s behavior does not change.\n\n\nThis post will focus on the first challenge — reliable restoration of the model definition and state in an inference environment. We will survey some of the legacy options and their shortcomings. We will then introduce the new [torch.export](https://docs.pytorch.org/docs/stable/export.html) API and demonstrate it in action on a toy model built with [HuggingFace’s](https://huggingface.co/) [transformers](https://huggingface.co/docs/transformers/en/index) library (version 4.54.0). For our experiments, we will use an [Amazon EC2 g5.xlarge](https://aws.amazon.com/ec2/instance-types/g5/) instance (containing an NVIDIA A10G GPU and 4 vCPUs) running a [PyTorch (2.7) Deep Learning AMI (DLAMI)](https://docs.aws.amazon.com/dlami/latest/devguide/aws-deep-learning-ami-gpu-pytorch-2.7-ubuntu-22-04.html).\nOne of the technologies underlying [torch.export](https://docs.pytorch.org/docs/stable/export.html) is [Torch Dynamo](https://docs.pytorch.org/docs/stable/torch.compiler_dynamo_overview.html), a key component of PyTorch’s graph compilation solution, [torch.compile](https://docs.pytorch.org/tutorials/intermediate/torch_compile_tutorial.html). In a [recent post](https://towardsdatascience.com/maximizing-ai-ml-model-performance-with-pytorch-compilation/) we demonstrated the power of [torch.compile](https://docs.pytorch.org/tutorials/intermediate/torch_compile_tutorial.html) and its importance in optimizing the runtime performance of AI/ML models. In some ways this post can be viewed as a sequel: we will revisit some of the key concepts — particularly graph-breaks, we will reuse the same toy model, and we will demonstrate the use of torch.compile for just-in-time (JIT) compilation in our inference environment. While we recommend reading the [previous post](https://towardsdatascience.com/maximizing-ai-ml-model-performance-with-pytorch-compilation/), it is not a prerequisite for this one.\nLearn this step by step with the interactive [AI Engineer](https://roadmap.sh/ai-engineer) roadmap.\n### Disclaimers:\nAs of the time of this writing (PyTorch 2.8.0), [torch.export](https://docs.pytorch.org/docs/stable/export.html) is a “prototype” feature. While the behaviors discussed in this post are likely to mostly remain, the API definitions could change. Be sure to align your use of [torch.export](https://docs.pytorch.org/docs/stable/export.html) to the latest API versions.\nThis post will cover only a subset of torch.export’s features and behaviors. It should not be viewed as an alternative to the official PyTorch [documentation](https://docs.pytorch.org/tutorials/intermediate/torch_export_tutorial.html). There are a number of pages covering the export feature, including an [introductory tutorial](https://docs.pytorch.org/tutorials/intermediate/torch_export_tutorial.html), an [overview of the programming model](https://docs.pytorch.org/docs/2.8/export.programming_model.html), and a [guide to solving export challenges](https://docs.pytorch.org/tutorials/recipes/torch_export_challenges_solutions.html).\nWhile we will demonstrate its functionality on a toy example, torch.export depends heavily on the model’s details and may exhibit very different behavior in your own project.\nThe code we will share is for demonstration purposes only and should not be relied on for correctness or optimality. Please do not interpret our choice of platform, framework, or any other tool or library as an endorsement for its use.\n## A Toy HuggingFace Model\nTo facilitate our discussion, we define a simple image-to-text generative model using the HuggingFace [transformers](https://huggingface.co/docs/transformers/en/index) library. For simplicity, we skip the training phase and assume that the default constructor returns a pretrained model.\npython\n\n```\nimport torch\n\nNUM_TOKENS = 1024\nMAX_SEQ_LEN = 256\nPAD_ID = 0\nSTART_ID = 1\nEND_ID = 2\n\n# Set up an image-to-text model.\ndef get_model():\n\n # import transformers utilities\n from transformers import (\n VisionEncoderDecoderModel,\n VisionEncoderDecoderConfig,\n AutoConfig\n )\n\n config = VisionEncoderDecoderConfig.from_encoder_decoder_configs(\n encoder_config=AutoConfig.for_model(\"vit\"), # vit encoder\n decoder_config=AutoConfig.for_model(\"gpt2\") # gpt2 decoder\n )\n config.decoder.vocab_size = NUM_TOKENS\n config.decoder.use_cache = False\n config.decoder_start_token_id = START_ID\n config.pad_token_id = PAD_ID\n config.eos_token_id = END_ID\n config.max_length = MAX_SEQ_LEN\n\n model = VisionEncoderDecoderModel(config=config)\n model.encoder.pooler = None # remove unused pooler\n model.eval() # prepare the model for evaluation\n return model\n```\n\nWe define an auto-regressive image-to-text generator that uses the encoder and decoder components of the model to produce a caption for the input image. For simplicity, we use a basic implementation, leaving out common optimization techniques, such as [ _KV caching_](https://huggingface.co/blog/not-lain/kv-caching).\npython\n\n```\n# generate the next token\ndef generate_token(decoder, encoder_hidden_states, sequence):\n outputs = decoder(\n sequence,\n encoder_hidden_states\n )\n logits = outputs[0][:, -1, :]\n return torch.argmax(logits, dim=-1, keepdim=True)\n\n# simple auto-regressive sequence generator\ndef image_to_text_generator(encoder, decoder, image):\n # run encoder\n encoder_hidden_states = encoder(image)[0]\n\n # initialize sequence\n generated_ids = torch.ones(\n (image.shape[0], 1),\n dtype=torch.long,\n device=image.device\n ) * START_ID\n\n for _ in range(MAX_SEQ_LEN):\n # generate next token\n next_token = generate_token(\n decoder,\n encoder_hidden_states,\n generated_ids\n )\n generated_ids = torch.cat([generated_ids, next_token], dim=-1)\n if (next_token == END_ID).all():\n break\n\n return generated_ids\n```\n\nThe following code block demonstrates the use of our generator on a batch of random input images.\npython\n\n```\nimport os, time, random, torch\n\ntorch.manual_seed(42)\nrandom.seed(42)\n\nBATCH_SIZE = 64\nEXPORT_PATH = '/tmp/export/'\n\ndef test_inference(model_path=EXPORT_PATH, mode=None, compile=False):\n device = 'cuda' if torch.cuda.is_available() else 'cpu'\n rnd_image = torch.randn(BATCH_SIZE, 3, 224, 224).to(device)\n encoder, decoder = load_model(model_path, mode)\n encoder = encoder.to(device)\n decoder = decoder.to(device)\n\n if compile:\n encoder = torch.compile(encoder, mode=\"reduce-overhead\")\n decoder = torch.compile(decoder, dynamic=True)\n # run a few warmup rounds\n for i in range(10):\n image_to_text_generator(encoder, decoder, random_image)\n\n t0 = time.perf_counter()\n \n # optionally enable mixed precision\n with torch.amp.autocast(device, dtype=torch.bfloat16, enabled=True):\n with torch.no_grad():\n caption = image_to_text_generator(encoder, decoder, rnd_image)\n\n total_time = time.perf_counter() - t0\n print(f'batched inference total time: {total_time}')\n```\n\nThe encoder and decoder models are loaded via a _load_model_ utility. We define an initial implementation and amend it later based on our choice of model capturing strategy.\nTo prepare our model for the export process, we define a pass-through wrapper class for the decoder. This wrapper ensures the model can be traced using positional (rather than keyword) arguments, which is a current requirement of torch.export.\npython\n\n```\nclass DecoderWrapper(torch.nn.Module):\n def __init__(self, decoder_model):\n super().__init__()\n self.decoder = decoder_model\n\n def forward(self, input_ids, encoder_hidden_states):\n return self.decoder(\n input_ids=input_ids,\n encoder_hidden_states=encoder_hidden_states,\n use_cache=False,\n output_attentions=False,\n output_hidden_states=False,\n return_dict=False\n )\n\ndef load_model(path=EXPORT_PATH, mode=None):\n model = get_model()\n encoder = model.encoder\n decoder = model.decoder\n return encoder, DecoderWrapper(decoder)\n```\n\nNow that we have defined our toy model, let’s explore different strategies for capturing and deploying it to an inference environment.\n## Model Capturing and Deployment Strategies\nIn this section, we’ll review two common methods for capturing and deploying an AI/ML model’s state: _weights-only capture_ and conversion to a serializable intermediate graph-representation using _TorchScript_.\n### Weights Only Capture\nThe first option is to use [torch.save](https://docs.pytorch.org/tutorials/beginner/saving_loading_models.html) to capture only the model weights, not the model definition. This requires you to explicitly redefine the PyTorch model in the inference environment. There are several ways to carryg over the code definition including: copy-pasting (which is very error-prone), pulling from a shared code repository, or using Python packages. If the model definition relies on specific Python package dependencies, you will need to make sure that those packages exist in the inference environment and that their versions match the training environment.\nThe following code block demonstrates capturing and loading the model weights for our toy model:\npython\n\n```\ndef capture_model(model, path=EXPORT_PATH):\n # weights only\n weights_path = os.path.join(EXPORT_PATH, \"weights.pth\")\n torch.save(model.state_dict(), weights_path)\n\ndef load_model(path=EXPORT_PATH, mode=None):\n if mode == 'weights':\n model = get_model()\n weights_path = os.path.join(path,\"weights.pth\")\n state_dict = torch.load(weights_path, map_location=\"cpu\")\n model.load_state_dict(state_dict)\n return model.encoder, DecoderWrapper(model.decoder)\n else:\n model = get_model()\n return model.encoder, DecoderWrapper(model.decoder)\n```\n\nOne advantage of this method is that it provides maximum flexibility for tuning the model’s configuration to the inference environment. For example, you can apply machine-specific optimizations that increase the throughput of the inference workload. This freedom is especially important for very large models that require advanced sharding techniques for model execution.\nHowever, this method assumes you can easily design and configure the inference environment as desired — which cannot be taken for granted. Some inference environments are extremely constrained — with limited control over the runtime libraries and package installations.\nThe separation between the model definition and the model weights also can be the source of all sorts of ugly bugs. Ensuring appropriate alignment between the source code and the model weights requires disciplined version management. A preferred approach would be to bundle the model definition and weights into a single archive.\n### TorchScript Variants\nFor many years, the primary method for capturing a PyTorch model along with its weights was [TorchScript](https://docs.pytorch.org/docs/stable/jit.html). And despite its recent deprecation, it remains widely popular. TorchScript encapsulates two different capturing solutions, [torch.jit.script](https://docs.pytorch.org/docs/stable/generated/torch.jit.script.html) and [torch.jit.trace](https://docs.pytorch.org/docs/stable/generated/torch.jit.trace.html), both of which convert PyTorch models into serializable graph representations. This graph can be loaded into another environment and run as a standalone PyTorch program, with minimal runtime dependencies.\nThe [scripting](https://docs.pytorch.org/docs/stable/generated/torch.jit.script.html) and [tracing](https://docs.pytorch.org/docs/stable/generated/torch.jit.trace.html) functionalities are complementary. [Scripting](https://docs.pytorch.org/docs/stable/generated/torch.jit.script.html) performs _ahead-of-time_ (AOT) static analysis of the source code, whereas [tracing](https://docs.pytorch.org/docs/stable/generated/torch.jit.trace.html) traces the actual execution of the model on a sample input. Scripting is able to capture more complexity in the graph, such as conditional control-flow and dynamic shapes. This is contrary to tracing that captures just the execution path and tensor shapes dictated by the input sample it runs on. On the other hand, torch.jit.trace supports more operations than torch.jit.script (e.g., see [here](https://docs.pytorch.org/docs/stable/jit_language_reference.html)). Often, scripting will succeed where tracing will fail and vice versa. Sometimes, some form of [combination of both](https://docs.pytorch.org/docs/stable/jit.html#mixing-tracing-and-scripting) methods is required.\nLet’s now attempt to convert our toy model to TorchScript. Since the sequence generator makes separate calls to the encoder (once) and the decoder (iteratively) separately, we’ll capture them as separate graphs. The encoder takes fixed-shape inputs and has no input-dependent conditional logic, so we can apply the more flexible tracing option. However, the sequence that we input to the decoder increases in size each time a token is generated, so we have no choice but to use the scripting option.\nHuggingFace supports TorchScript via a dedicated configuration. See the [HuggingFace documentation](https://huggingface.co/docs/transformers/en/torchscript) for more details.\npython\n\n```\nconfig.decoder.torchscript = True\nconfig.encoder.torchscript = True\n```\n\nIn the code block below, we extend our capture and loading utilities with TorchScript support. Note the inclusion of the [torch.jit.freeze](https://docs.pytorch.org/docs/stable/generated/torch.jit.freeze.html) optimization during capture and the use the [torch.jit.optimize_for_inference](https://docs.pytorch.org/docs/stable/generated/torch.jit.optimize_for_inference.html) optimization on the target device.\npython\n\n```\ndef capture_model(model, path=EXPORT_PATH):\n # weights only\n weights_path = os.path.join(EXPORT_PATH, \"weights.pth\")\n torch.save(model.state_dict(), weights_path)\n\n encoder = model.encoder\n decoder = DecoderWrapper(model.decoder)\n\n # torchscript encoder using trace\n example = torch.randn(1, 3, 224, 224)\n encoder_jit = torch.jit.trace(encoder, example)\n # optionally apply jit.freeze optimization\n encoder_jit = torch.jit.freeze(encoder_jit)\n encoder_path = os.path.join(path, \"encoder.pt\")\n torch.jit.save(encoder_jit, encoder_path)\n\n try:\n # torchscript decoder using scripting\n decoder_jit = torch.jit.script(decoder)\n # optionally apply jit.freeze optimization\n decoder_jit = torch.jit.freeze(decoder_jit)\n decoder_path = os.path.join(path, \"decoder.pt\")\n torch.jit.save(decoder_jit, decoder_path)\n except Exception as e:\n print(f'torch.jit.script(model.decoder) failed\\n{e}')\n\ndef load_model(path=EXPORT_PATH, mode=None):\n if mode == 'weights':\n model = get_model()\n weights_path = os.path.join(path,\"weights.pth\")\n state_dict = torch.load(weights_path, map_location=\"cpu\")\n model.load_state_dict(state_dict)\n return model.encoder, DecoderWrapper(model.decoder)\n elif mode == 'torchscript':\n encoder_path = os.path.join(path, \"encoder.pt\")\n decoder_path = os.path.join(path, \"decoder.pt\")\n encoder = torch.jit.load(encoder_path)\n decoder = torch.jit.load(decoder_path)\n # optionally apply target-device optimization \n encoder = torch.jit.optimize_for_inference(encoder)\n decoder = torch.jit.optimize_for_inference(decoder)\n return encoder, decoder\n else:\n model = get_model()\n return model.encoder, DecoderWrapper(model.decoder)\n```\n\nUnfortunately, our capture utility fails when trying to script the decoder model. A common problem with TorchScript is that it often fails on complex models. While many issues can be bypassed by using [torch.jit.trace](https://docs.pytorch.org/docs/stable/generated/torch.jit.trace.html), it cannot be applied to models with dynamic shapes or conditional logic, such as our decoder. It is often possible to rewrite the model implementation to be script-compliant, but it can require a lot of painstaking (and ugly) work. In the case of our decoder model, it would require a lot of intrusive patchwork to the transformers library source code.\nFor more on the topic of TorchScript, please see the official [documentation](https://docs.pytorch.org/docs/stable/jit.html).\n## Model Capturing with torch.export\nThe new way of capturing a model for deployment is [torch.export](https://docs.pytorch.org/docs/main/export.html). Similar to [torch.jit.trace](https://docs.pytorch.org/docs/stable/generated/torch.jit.trace.html), export works by [tracing](https://docs.pytorch.org/docs/stable/export.programming_model.html#basics-of-tracing) the model’s execution on input samples. However, unlike [torch.jit.trace](https://docs.pytorch.org/docs/stable/generated/torch.jit.trace.html), export includes support for [dynamism](https://docs.pytorch.org/docs/main/export.html#expressing-dynamism) and [conditional control flow](https://docs.pytorch.org/docs/main/cond.html). The output of the export utility is an intermediate graph representation called [Export IR](https://docs.pytorch.org/docs/stable/export.ir_spec.html), which can be loaded and executed in a clean inference environment. One of the advantages of our exported model is that, contrary to TorchScript models, they can be optimized for the inference environment using [torch.compile](https://docs.pytorch.org/tutorials/intermediate/torch_compile_tutorial.html). On the other hand, optimizations that require source code changes (e.g., configuring the [attn_implementation](https://huggingface.co/docs/transformers/en/main_classes/model#transformers.PreTrainedModel.from_pretrained.attn_implementation)) cannot be applied.\n### Overcoming Graph Breaks\nA _graph break_ occurs when the export function encounters an “untraceable” portion of Python code (e.g., see [here](https://docs.pytorch.org/docs/stable/generated/exportdb/index.html#not-supported-yet) for unsupported operations). We encountered the concept of _graph breaks_ in our [previous post](https://towardsdatascience.com/maximizing-ai-ml-model-performance-with-pytorch-compilation/) on [graph compilation](https://docs.pytorch.org/tutorials/intermediate/torch_compile_tutorial.html). However, contrary to model compilation where PyTorch will simply fall back to eager mode, torch.export forbids the presence of _graph breaks_. If export fails on your model due to a graph break, you will have to rewrite your code to bypass it.\nThere are several resources at your disposal for overcoming graph breaks, including the [Draft Export](https://docs.pytorch.org/docs/2.8/draft_export.html) utility, which generates a detailed report of export issues, [ExportDB](https://docs.pytorch.org/docs/stable/generated/exportdb/index.html), which maintains a list of supported and unsupported export cases, and a [tutorial](https://docs.pytorch.org/tutorials/recipes/torch_export_challenges_solutions.html) on overcoming common export issues.\n### Debugging an Exported Graph\nOn some occasions, you may succeed in exporting a graph only to find that running it on the target device either fails with an error or returns incorrect output. A common cause of such errors is that some of the variables from the export environment are treated as constants and baked into the exported graph.\nUnfortunately, (as of the time of this writing) the tools for debugging [Export IR](https://docs.pytorch.org/docs/stable/export.ir_spec.html) graphs are somewhat limited. Although the exported model is a [torch.nn.Module](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html) with a [forward](https://docs.pytorch.org/docs/stable/generated/torch.nn.Module.html#torch.nn.Module.forward) function, you cannot use a debugger to step into it to find the source of the errors.\nWe can, however, inspect the contents of the generated graph using [GraphModule.print_readable](https://docs.pytorch.org/docs/stable/fx.html#torch.fx.GraphModule.print_readable). This will print out all of the graph operations along with comments that point to the source code from which they were generated. Sometimes, this, combined with the output error information, is enough to find the source of the errors and tweak the source code accordingly. See below for an example.\n### Exporting a HuggingFace Model\nTo apply [torch.export](https://docs.pytorch.org/docs/stable/export.html) to our toy model, we first make sure to update the PyTorch library to the latest version (2.8.0 at the time of this writing). The export utility is under rapid development and we want to make sure to get the most up-do-date feature support.\nIn the code block below we revise our capture and load utility functions to support [torch.export](https://docs.pytorch.org/docs/stable/export.html). Note our use of [torch.export.Dim](https://docs.pytorch.org/docs/stable/export.html#expressing-dynamism) to specify dynamic dimensions:\npython\n\n```\ndef capture_model(model, path=EXPORT_PATH):\n encoder = model.encoder\n decoder = DecoderWrapper(model.decoder)\n\n # define dynamic dimensions\n batch = torch.export.Dim(\"batch\")\n seq_len = torch.export.Dim(\"seq_len\", min=2, max=MAX_SEQ_LEN)\n\n # export encoder\n # sample input\n example = torch.randn(4, 3, 224, 224)\n encoder_export = torch.export.export(\n encoder,\n (example,),\n dynamic_shapes=((batch,\n torch.export.Dim.STATIC,\n torch.export.Dim.STATIC,\n torch.export.Dim.STATIC),\n )\n )\n torch.export.save(encoder_export, os.path.join(path, \"encoder.pt2\"))\n\n\n # export decoder\n # get sample input for decoder\n encoder_hidden_states = encoder_export.module()(example)[0]\n decoder_input_ids = torch.ones((4, MAX_SEQ_LEN),\n dtype=torch.long)*START_ID\n\n\n decoder_export = torch.export.export(\n decoder,\n (decoder_input_ids, encoder_hidden_states),\n dynamic_shapes={\n 'input_ids': (batch,seq_len),\n 'encoder_hidden_states': (batch,\n torch.export.Dim.STATIC,\n torch.export.Dim.STATIC)\n }\n )\n torch.export.save(decoder_export, os.path.join(path, \"decoder.pt2\"))\n\n\ndef load_model(path=EXPORT_PATH, mode=None):\n if mode == 'weights':\n model = get_model()\n weights_path = os.path.join(path,\"weights.pth\")\n state_dict = torch.load(weights_path, map_location=\"cpu\")\n model.load_state_dict(state_dict)\n return model.encoder, DecoderWrapper(model.decoder)\n elif mode == 'export':\n encoder_path = os.path.join(path, \"encoder.pt2\")\n decoder_path = os.path.join(path, \"decoder.pt2\")\n encoder = load(encoder_path).module()\n decoder = load(decoder_path).module()\n return encoder, decoder\n else:\n model = get_model()\n return model.encoder, DecoderWrapper(model.decoder)\n```\n\nContrary to TorchScript, [torch.export](https://docs.pytorch.org/docs/stable/export.html) had no issues capturing our decoder and encoder models. In particular, no graph breaks were encountered during tracing.\n## Deploying an Exported Model\nTo complete our demonstration, we’ll test our exported model in a clean inference environment. For this experiment, we’ll use an [Amazon EC2 g5.xlarge](https://aws.amazon.com/ec2/instance-types/g5/) instance (containing an NVIDIA A10G GPU and 4 vCPUs) running a [PyTorch (2.7) Deep Learning AMI (DLAMI)](https://docs.aws.amazon.com/dlami/latest/devguide/aws-deep-learning-ami-gpu-pytorch-2.7-ubuntu-22-04.html). We’ll update to the latest PyTorch version but intentionally won’t install the transformers package\nUnfortunately, our excitement at the success of exporting our model was premature, as running the exported decoder on the GPU results in a runtime error, a portion of which we’ve pasted below:\nshell-session\n\n```\nFile \".24\", line 306, in forwardFile \"/opt/pytorch/lib/python3.12/site-packages/torch/_ops.py\", line 829, in __call__ return self._op(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^RuntimeError: Expected all tensors to be on the same device, but got index is on cpu, different from other tensors on cuda:0 (when checking argument in method wrapper_CUDA__index_select)\n```\n\nThe error indicates during the execution the exported graph, some of the tensors reside on the CPU when they are expected to be on the GPU. Since we explicitly copy the input tensors onto the GPU, we can conclude that these refer to tensors that the graph is creating internally. Since the graph was exported on a CPU device, the use of _device_ =_”cpu”_ was baked into the graph creation, resulting in a runtime error when running on a GPU.\nAlthough the error message points to the faulty line of code (_File “ .24”, line 306, in forward_), there is no actual file that we can add a breakpoint to and debug. We can, however, inspect the contents of the decoder graph and search for places where the use of the CPU device has been inadvertently baked in:\npython\n\n```\ndecoder_export.module().print_readable()\n```\n\nAnalyzing the output (searching for “cpu” references) and cross-referencing the source code using the embedded comments, we discover four locations where the _transformers_ library ([_modeling_gpt2.py_](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py) file) creates tensors on the CPU:\n 1. If not specified, the [GPT2 model](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py#L685) [creates of a ](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py#L854)[ _cache_position_](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py#L854) tensor on the baked-in CPU device using [torch.arange](https://docs.pytorch.org/docs/stable/generated/torch.arange.html). This can be fixed by passing in a user-defined value for _cache_position_.\n 2. On line [861](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py#L861), a [torch.Tensor.to](https://docs.pytorch.org/docs/stable/generated/torch.Tensor.to.html) operation is performed to ensure proper placement of the _position_embeds_ tensor. While this may be required in a case of [model parallelism](https://huggingface.co/docs/transformers/v4.13.0/en/parallelism), we do not require it.\n 3. If not specified, the model [creates a causal mask](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py#L867) using tensors that it creates on the baked-in CPU device. We could bypass this by passing in a user-defined causal mask, but we are perfectly happy keeping this _None_ and relying on the use of the _is_causal_ flag of the [sdpa attention function](https://docs.pytorch.org/docs/stable/generated/torch.nn.functional.scaled_dot_product_attention.html).\n 4. If not specified, the model [creates an ](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py#L878)[ _ecnoder_attention_mask_](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py#L878) tensor on the baked-in CPU. Once again, we could specify a value for this tensor, but since the mask would be all _True_ , setting it _None_ achieves the same purpose.\n\n\nThe following patch summarizes the changes we performed on the [ _modeling_gpt2.py_](https://github.com/huggingface/transformers/blob/v4.54.0/src/transformers/models/gpt2/modeling_gpt2.py) file. Importantly, these changes were specific to our toy model and will not generalize to all use cases. This kind of _monkey-patching_ is ill-advised and requires extensive testing before being used in a production setting.\ndiff\n\n```\n@@ -861 +861 @@\n- hidden_states = inputs_embeds + position_embeds.to(inputs_embeds.device)\n+ hidden_states = inputs_embeds + position_embeds\n@@ -867,3 +867 @@\n- causal_mask = self._update_causal_mask(\n- attention_mask, inputs_embeds, cache_position, past_key_values, output_attentions\n- )\n+ causal_mask = None\n@@ -877 +875 @@\n- if encoder_attention_mask is None:\n+ if not _use_sdpa and encoder_attention_mask is None:\n@@ -880,3 +878 @@\n- encoder_attention_mask = _prepare_4d_attention_mask_for_sdpa(\n- mask=encoder_attention_mask, dtype=inputs_embeds.dtype, tgt_len=input_shape[-1]\n- )\n+ pass\n```\n\nIn the following code block, we extend the _decoder_ definitions and export implementation with an explicit value for _cache_position_.\npython\n\n```\ndef generate_token(decoder, encoder_hidden_states, sequence):\n outputs = decoder(\n sequence,\n encoder_hidden_states,\n torch.arange(sequence.shape[1], device=sequence.device)\n )\n logits = outputs[0][:, -1, :]\n return torch.argmax(logits, dim=-1, keepdim=True)\n\nclass DecoderWrapper(torch.nn.Module):\n def __init__(self, decoder_model):\n super().__init__()\n self.decoder = decoder_model\n\n def forward(self, input_ids, encoder_hidden_states, cache_position):\n return self.decoder(\n input_ids=input_ids,\n cache_position=cache_position,\n encoder_hidden_states=encoder_hidden_states,\n use_cache=False,\n output_attentions=False,\n output_hidden_states=False,\n return_dict=False\n )\n\ndef capture_model(model, path=EXPORT_PATH):\n encoder = model.encoder\n decoder = DecoderWrapper(model.decoder)\n\n # define dynamic dimensions\n batch = torch.export.Dim(\"batch\")\n seq_len = torch.export.Dim(\"seq_len\", min=2, max=MAX_SEQ_LEN)\n\n # export encoder\n # sample tensor\n example = torch.randn(4, 3, 224, 224)\n encoder_export = torch.export.export(\n encoder,\n (example,),\n dynamic_shapes=((batch,\n torch.export.Dim.STATIC,\n torch.export.Dim.STATIC,\n torch.export.Dim.STATIC),\n )\n )\n torch.export.save(encoder_export, os.path.join(path, \"encoder.pt2\"))\n\n # export decoder\n # get sample input for decoder\n encoder_hidden_states = encoder_export.module()(example)[0]\n decoder_input_ids = torch.ones((4, MAX_SEQ_LEN), \n dtype=torch.long)*START_ID\n\n decoder_args = (\n decoder_input_ids,\n encoder_hidden_states,\n torch.arange(MAX_SEQ_LEN)\n )\n\n dynamic_shapes = {\n 'input_ids': (batch,seq_len),\n 'encoder_hidden_states': (batch,\n torch.export.Dim.STATIC,\n torch.export.Dim.STATIC),\n 'cache_position': (seq_len,),\n }\n\n decoder_export = torch.export.export(\n decoder,\n decoder_args,\n dynamic_shapes=dynamic_shapes\n )\n torch.export.save(decoder_export, os.path.join(path, \"decoder.pt2\"))\n```\n\nFollowing these change, the exported decoder succeeds in generating sequences on the GPU device. We hope (and expect) that as the torch.export feature evolves, these kinds of issues will be handled automatically by the internal tracing mechanism.\nWe further test the potential for machine-specific optimizations by applying graph compilation. For details on our choice of compilation parameters, see our [previous post](https://towardsdatascience.com/maximizing-ai-ml-model-performance-with-pytorch-compilation/).\npython\n\n```\nencoder = torch.compile(encoder, mode=\"reduce-overhead\")\ndecoder = torch.compile(decoder, dynamic=True)\n```\n\nThe table below captures the execution time of our model on a batch of random images, with and without torch.compile. (Note, that running the original model requires installation of the transformers library, version 4.54.0).\n![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/08/2-1.webp)\nRuntime Results (by Author)\nWe can see that exporting the model to graph representation results in a speed-up of 10.7%. The model compilation, however, had the opposite effect, significantly increasing the execution time. It is likely that this could be fixed through appropriate tuning.\n## Summary\nIn this post we explored the new torch.export utility and demonstrated its use in capturing and deploying a toy HuggingFace model. We found that is has a number of powerful and compelling properties, including:\n * **Support for complex models** : torch.export succeeded in capturing models that failed with TorchScript.\n * **Portability** : Exported models can be loaded and executed as standalone programs without specific package dependencies.\n * **Machine-specific optimizations** : exported models are compatible with graph compilation, enabling the application of machine-specific optimizations.\n\n\nWe also encountered some of torch.export's limitations:\n * **Unintended consequences of graph creation:** If we are not careful about how we design our model, values from the export environment can be inadvertently baked into the resulting graph, breaking its compatibility with the inference environment.\n * **Limited debugging tools:** As of this writing, the tools for debugging the execution of exported graphs are limited.\n\n\nAlthough it still requires some refinement, torch.export is already a huge improvement over previous capturing solutions like TorchScript. We look forward to seeing it continue to evolve and improve.\nFor more details on capturing AI/ML models using PyTorch Export and to keep track of API changes, please see the [torch.export](https://docs.pytorch.org/docs/stable/export.html) documentation. If you’re running inference on an edge device, also see the [ExecuTorch](https://docs.pytorch.org/executorch/stable/index.html) solution for model deployment, which is based on [torch.export](https://docs.pytorch.org/docs/stable/export.html).\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](https://assets.insightmediagroup.io/media/1787657094227_7ymyo0.jpeg)\nWritten By\n## [Chaim Rand](https://towardsdatascience.com/author/chaimrand/)\n[See all from Chaim Rand](https://towardsdatascience.com/author/chaimrand/)\n[Deep Dives](https://towardsdatascience.com/tag/deep-dives/)[Deep Learning](https://towardsdatascience.com/tag/deep-learning/)[Model Deployment](https://towardsdatascience.com/tag/model-deployment/)[Pytorch](https://towardsdatascience.com/tag/pytorch/)[Torchscript](https://towardsdatascience.com/tag/torchscript/)\nShare This Article\n * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fcapturing-and-deploying-pytorch-models-with-torch-export%2F&title=Capturing%20and%20Deploying%20PyTorch%20Models%20with%20torch.export)\n * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fcapturing-and-deploying-pytorch-models-with-torch-export%2F&title=Capturing%20and%20Deploying%20PyTorch%20Models%20with%20torch.export)\n * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fcapturing-and-deploying-pytorch-models-with-torch-export%2F&text=Capturing%20and%20Deploying%20PyTorch%20Models%20with%20torch.export)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/05/1WpFA73Y1tEb7NotyTanhdQ.png)](https://towardsdatascience.com/experiments-peeking-and-optimal-stopping-954506cec665/)\n## [Experiments, Peeking, and Optimal Stopping](https://towardsdatascience.com/experiments-peeking-and-optimal-stopping-954506cec665/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nHow to run valid experiments with smaller sample sizes with the Sequential Probability Ratio Test\n[Matteo Courthoud](https://towardsdatascience.com/author/matteo.courthoud/)\nMay 2, 202214 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/08/0hO3ZEc2_TsOexXc3-scaled.jpg)](https://towardsdatascience.com/what-it-means-for-something-to-be-learnable-pac-learnability-c84de9c061ad/)\n## [Defining Learnability with the PAC Framework](https://towardsdatascience.com/what-it-means-for-something-to-be-learnable-pac-learnability-c84de9c061ad/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nIn machine learning we often say something is \"learnable\". What does that really mean? Read on to find out.\n[Wilson Wang](https://towardsdatascience.com/author/wilson-wang/)\nAugust 12, 20214 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/08/162JNtKp1yLIKKksF-PuHzQ.png)](https://towardsdatascience.com/all-deep-learning-is-statistical-model-building-fc310328f07/)\n## [All deep learning is statistical model building](https://towardsdatascience.com/all-deep-learning-is-statistical-model-building-fc310328f07/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nDeep learning is often used to make predictions for data driven analysis. But what are the meanings of these predictions? This post...\n[Tom Charnock](https://towardsdatascience.com/author/tomcharnock/)\nAugust 3, 202017 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/09/1yg7lLOo6SW-L6nrfmQynOQ-scaled.jpeg)](https://towardsdatascience.com/xgboost-vs-lightgbm-on-a-high-dimensional-dataset-bbd5b5174274/)\n## [XGBoost vs LightGBM on a High Dimensional Dataset](https://towardsdatascience.com/xgboost-vs-lightgbm-on-a-high-dimensional-dataset-bbd5b5174274/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nComparison of both speed and performance\n[Soner Yıldırım](https://towardsdatascience.com/author/sonery/)\nSeptember 29, 20205 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/07/0NyXFp_gHYM4JgRS0-scaled.jpg)](https://towardsdatascience.com/speak-to-me-how-many-words-a-model-is-reading-331e3af86d27/)\n## [Speak to me: How many words a model is reading](https://towardsdatascience.com/speak-to-me-how-many-words-a-model-is-reading-331e3af86d27/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nWhy and how to overcome the inner limit of a Large Language Model\n[Salvatore Raieli](https://towardsdatascience.com/author/salvatore-raieli/)\nJuly 14, 202319 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/10/0vmcl1E1fKSGYp2xU.jpeg)](https://towardsdatascience.com/multilingual-nlp-get-started-with-the-tydiqa-goldp-dataset-in-10-minutes-or-less-c56c01ad47d2/)\n## [Multilingual NLP: Get Started with the TyDiQA-GoldP Dataset in 10 Minutes or Less](https://towardsdatascience.com/multilingual-nlp-get-started-with-the-tydiqa-goldp-dataset-in-10-minutes-or-less-c56c01ad47d2/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nA hands-on tutorial for retrieving, processing and using the dataset\n[Yousef Nami](https://towardsdatascience.com/author/namiyousef96/)\nOctober 25, 202210 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1w4AZk1uQZoK-tWfepHw3sg.png)](https://towardsdatascience.com/learn-python-data-analytics-by-example-chicago-traffic-crashes-4840f6f15ed/)\n## [Learn Python Data Analytics by Example - Chicago Traffic Crashes](https://towardsdatascience.com/learn-python-data-analytics-by-example-chicago-traffic-crashes-4840f6f15ed/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nA fun project and detailed walkthrough of data analytics steps to help you learn Python, pandas, matplotlib, seaborn and Folium\n[Nick Cox](https://towardsdatascience.com/author/nickdcox/)\nFebruary 9, 202112 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/04/0waavr452mc1Ywufw-scaled.jpg)](https://towardsdatascience.com/critical-facts-that-every-data-scientist-should-know-part-2-c9c06cde6e21/)\n## [Critical Facts That Every Data Scientist Should Know - Part 2](https://towardsdatascience.com/critical-facts-that-every-data-scientist-should-know-part-2-c9c06cde6e21/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nIn the second part of this article, we will conclude the list of some of the critical facts you need to know to become a super data...\n[Mario Larcher](https://towardsdatascience.com/author/mnslarcher/)\nApril 19, 202213 min read\n * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/02/1GZtfpHMxf03765DlVaxohg.png)](https://towardsdatascience.com/dynamic-pricing-using-reinforcement-learning-and-neural-networks-cc3abe374bf5/)\n## [Dynamic Pricing using Reinforcement Learning and Neural Networks](https://towardsdatascience.com/dynamic-pricing-using-reinforcement-learning-and-neural-networks-cc3abe374bf5/)\n[Deep Learning](https://towardsdatascience.com/category/artificial-intelligence/deep-learning/)\nAn intelligent system that can increase e-commerce sales and profits\n[Reslley Gabriel](https://towardsdatascience.com/author/reslleygabriel/)\nFebruary 16, 20215 min read\n\n\n * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n * [X](https://x.com/TDataScience)\n * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n * [Threads](https://www.threads.net/@towardsdatascience)\n * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n#### Recommended Articles \nClose\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/4970/image?etag=1780565225463&width=500&height=500) Productivity Tips, Data Career Insights, and Other Recent Must-Reads ](https://towardsdatascience.com/productivity-tips-data-career-insights-and-other-recent-must-reads-5c47eb7a2fe0/ \"Productivity Tips, Data Career Insights, and Other Recent Must-Reads\")\n\n\n", "body_base_path": "attachments/cac89aaeead0/pages/", "body_markdown_path": "attachments/cac89aaeead0/pages/archived-body.md", "local_html": "attachments/cac89aaeead0/pages/archived-body.html", "body_source": "prior_capture" }