{
  "source_file": "0930",
  "source_platform": "Medium",
  "author": "Naser Tamimi",
  "title_listed": "A Comprehensive Introduction to Marketing Data Engineering",
  "keywords": "",
  "domain": "数据处理",
  "content_type": "深度技术解析",
  "is_paid_flag": "否",
  "url": "https://towardsdatascience.com/a-comprehensive-introduction-to-marketing-data-engineering-f292a4a6568b/",
  "publish_date": "2024-08-30",
  "corpus_stage": "",
  "notes": "",
  "id": "c16dddde4b1c",
  "collected_at": "2026-10-02T05:09:12",
  "title": "A Comprehensive Introduction to Marketing Data Engineering | Towards Data Science",
  "body_markdown": "![Revisit consent button](../images/body-df29beacd7496220cc94.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n  * Cookie\nBCTempID\n  * Duration\n10 minutes\n  * Description\nNo description available.\n\n\n  * Cookie\n__cf_bm\n  * Duration\n1 hour\n  * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n  * Cookie\nAWSALBCORS\n  * Duration\n7 days\n  * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n  * Cookie\n_cfuvid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n  * Cookie\nli_gc\n  * Duration\n6 months\n  * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n  * Cookie\n__hssrc\n  * Duration\nsession\n  * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n  * Cookie\n__hssc\n  * Duration\n1 hour\n  * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n  * Cookie\nwpEmojiSettingsSupports\n  * Duration\nsession\n  * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n  * Cookie\nBCSessionID\n  * Duration\n1 year 1 month 4 days\n  * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n  * Cookie\n_octo\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nlogged_in\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\n__Secure-YEC\n  * Duration\npast\n  * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n  * Cookie\n__eoi\n  * Duration\n6 months\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTGCORS\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\nlogin-status-p\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTG\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\ncsrf_token\n  * Duration\nsession\n  * Description\nNo description available.\n\n\n  * Cookie\ntoken_v2\n  * Duration\n1 day\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nD\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nPHPSESSID\n  * Duration\nsession\n  * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n  * Cookie\nVISITOR_PRIVACY_METADATA\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n  * Cookie\ncookietest\n  * Duration\nsession\n  * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n  * Cookie\n__Host-airtable-session\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\n__Host-airtable-session.sig\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\nm\n  * Duration\n1 year 1 month 4 days\n  * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n  * Cookie\nBIGipServer*\n  * Duration\nsession\n  * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n  * Cookie\n__cfruid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n  * Cookie\n_GRECAPTCHA\n  * Duration\n6 months\n  * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n  * Cookie\n__Secure-YNID\n  * Duration\n6 months\n  * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n  * Cookie\ncookieyes-consent\n  * Duration\n1 year\n  * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n  * Cookie\nlidc\n  * Duration\n1 day\n  * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n  * Cookie\nbrw\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nbrwConsent\n  * Duration\n5 minutes\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nWMF-Uniq\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nloom_anon_comment\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nloom_referral_video\n  * Duration\nsession\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nVISITOR_INFO1_LIVE\n  * Duration\n6 months\n  * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n  * Cookie\nyt-remote-connected-devices\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n  * Duration\nNever Expires\n  * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n  * Cookie\nyt-remote-device-id\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-name\n  * Duration\nsession\n  * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\nyt-remote-fast-check-period\n  * Duration\nsession\n  * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-app\n  * Duration\nsession\n  * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-available\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-installed\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\ncp_session\n  * Duration\n3 months\n  * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n  * Cookie\nloid\n  * Duration\n1 year 1 month 4 days\n  * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n  * Cookie\n__hstc\n  * Duration\n6 months\n  * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n  * Cookie\nhubspotutk\n  * Duration\n6 months\n  * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n  * Cookie\n_ga\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n  * Cookie\n_ga_*\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n  * Cookie\n__Host-psifi.analyticsTrace\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n__Host-psifi.analyticsTraceV2\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n_gh_sess\n  * Duration\nsession\n  * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n  * Cookie\nYSC\n  * Duration\nsession\n  * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n  * Cookie\najs_anonymous_id\n  * Duration\n1 year\n  * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n  * Cookie\nvuid\n  * Duration\n1 year 1 month 4 days\n  * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n  * Cookie\nAWSALB\n  * Duration\n7 days\n  * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n  * Cookie\nacq\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nacq.sig\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nptc\n  * Duration\n2 years\n  * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n  * Cookie\nmuc_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n  * Cookie\nguest_id_marketing\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\nguest_id_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\npersonalization_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n  * Cookie\nguest_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n  * Cookie\nbcookie\n  * Duration\n1 year\n  * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n  * Cookie\n__Secure-ROLLOUT_TOKEN\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n  * Cookie\nyt.innertube::nextId\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nyt.innertube::requests\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nsession_tracker\n  * Duration\nsession\n  * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n  * Cookie\nedgebucket\n  * Duration\nsession\n  * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n  * Cookie\ndid\n  * Duration\n1 year\n  * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Data Science](https://towardsdatascience.com/category/data-science/)\n# A Comprehensive Introduction to Marketing Data Engineering\nFundamentals, responsibilities, and challenges\n[Naser Tamimi](https://towardsdatascience.com/author/tamimi-naser/)\nAugust 30, 202418 min read\n### DATA ENGINEERING\n![Image generated by the author.](../images/000.jpeg)\nImage generated by the author.\nIn today's digital landscape, marketing is increasingly driven by data. Unlike traditional marketing (examples: television, radio, billboards, signs, and print), where measuring the impact of marketing channels, strategies, and ads was often unclear, digital marketing allows us to measure these impacts with precision. In fact, digital marketing has become a more effective approach not just because of the large audiences online, but because it enables us to evaluate our marketing efforts with greater accuracy and adjust its targeting more effectively.\nTherefore, marketing teams need robust and scalable data infrastructure that allows them to ingest, process, and analyze data from multiple sources or channels. Of course, this is where Marketing Data Engineering becomes an important topic. Marketing Data Engineering is the backbone of modern digital marketing. The quality of data engineering products in this specific field, enables data-driven decisions that can significantly impact a company's goals.\nWhile Marketing Data Engineering is a vast field that can't be fully covered in a single article, I've aimed to provide a concise introduction packed with essential information. Each of the topics mentioned has much more depth, which I hope to explore in future articles. To stay updated and receive the latest insights, be sure to [subscribe to my Medium channel](https://tamimi-naser.medium.com/subscribe).\n## Introduction\nMarketing Data Engineering involves the design, development, and maintenance of data products (including tables, data pipelines, dashboards, etc.) that ingest and process data from various marketing channels and partners. It requires a deep understanding of both digital marketing concepts and data engineering principles.\nIn one sentence, if I want to express the goal of marketing data engineering, I should say:\n> The goal of Marketing Data Engineering is to ensure that marketing data are:\n>   1. Reliable: accurately and continuously validated\n>   2. Available: consistently available and meets its SLA\n>   3. Scalable: capable of accommodating more data, partners, or channels\n>   4. Actionable: ready for use by decision-makers\n> \n\nAt its core, Marketing Data Engineering is about transforming raw marketing and business data into actionable marketing insights. Later you will see a whole section in this article on Marketing Data Engineers responsibilities.\n## Digital Marketing Channels\nBefore diving into the roles of Marketing Data Engineers, it's essential to understand marketing channels. Marketing channels are the various platforms through which companies reach their target audience, including search engines, media outlets, social media platforms, and email, among others. Each marketing channel may encompass different partners and platforms. For instance, Google and Bing both fall under the search engine marketing channel, even though they are distinct platforms (often referred to as marketing partners). They share similar marketing components and concepts, so they're grouped within the same channel but recognized as different partners. However, the data generated from search marketing campaigns is vastly different from that of social media campaigns. The challenge for Marketing Data Engineers is to standardize this diverse data to enable consistent and meaningful analysis.\nHere, we will briefly introduce a few widely-used marketing channels.\n### Search Engine Channels\nSearch Engines are critical digital marketing platforms and can be divided into three main marketing channels: Organic (SEO), Paid (SEM), and Metasearch\n**SEO (Search Engine Optimization) or Organic Search Channel:** This involves optimizing your website and content to rank higher in search engine results pages (SERPs) without paying for placement. The collected data here must show how to optimize your website to rank higher in each search engine (impression) and attract more audience (click). Google, Bing, and Yahoo are some well-known search engines that many SEO optimization techniques are studied and developed for them.\n**SEM (Search Engine Marketing) or Paid Search Channel:** This involves paying for ads to appear in Search Engineer Results Pages (SERPs). Unlike SEO channel, SEM pricing model involves bidding processes for different keywords. Your goal is to collect data that identifies which keywords with which bidding prices are effective in your ads and which are not. Similar to SEO, Google, Bing, and Yahoo are big names in SEM world.\n**Metasearch Engine Channel:** These marketing channels are slowly become popular in recent years specifically for e-commerce, hotels, air travel advertisements. Examples are Google, Bing, TripAdvisor, Kayak, Trivago, and Skyscanner all offer metasearch.\nThe following figure shows all three mentioned marketing channels associated with search engines on a sample Google result page.\n![Paid \\(SEM\\), metasearch, and organic \\(SEO\\) search results on a sample Google page \\(\\(Image by the author\\).](../images/001.png)\nPaid (SEM), metasearch, and organic (SEO) search results on a sample Google page ((Image by the author).\nRegardless of which search engine channels we use, a marketing data engineer must be able to handle data from all kinds of search engine campaigns, ensuring that it can be analyzed side by side to measure overall search marketing effectiveness.\n### Media Marketing Channel\nMedia marketing is traditionally associated with channels like TV, radio, and newspaper advertisements. However, many of these traditional media outlets have recently transitioned to the digital realm or recognized the value of incorporating digital marketing strategies. As a result, they have adopted digital tools to create, monitor, and measure the effectiveness of ad campaigns on their platforms. While it may still be challenging to track every user interaction with ads through these channels, we can assess their effectiveness in a broader sense.\n### Social Media Marketing (SMM) Channels\nSocial media marketing is another key area that has both organic and paid components as well. Meta (with Facebook and Instagram platforms), TikTok, X (or Twitter), LinkedIn, Snapchat, YouTube, and Pinterest are a few examples of partners in this type of marketing. Although it seems that SMM is a single marketing channel, we have two distinct marketing channels under SMM which have different campaigns and ways of measurement.\n**Organic Social Media Marketing Channel:** This refers to the content you post on social media platforms without paying for distribution (for example through your company social media pages). Here, key metrics include likes, shares, comments, and follower growth. Also, we might be able to link conversion on our website to this channel if we have a good tracking system in place.\n**Paid Social Media Marketing Channel:** This involves investing in ads or boosted posts on social media platforms. Key metrics include ad spend, impressions, clicks, conversions, etc. The challenge lies in crafting effective campaigns and managing them regularly to optimize costs while maximizing conversions. Cost Per Click (pay each time a user clicks), Cost Per Thousand Impressions (pay for every thousand times the ad is shown), Cost Per Action (pay when a user takes a specific action), and Cost Per Engagement (pay based on user interactions with the ad, such as likes, shares, comments, or other forms of engagement) are few pricing model in this type of marketing channel.\n### Email Marketing Channel\nThough email marketing is considered a traditional digital marketing channel, it remains one of the most widely used. Recently, it has successfully integrated with advanced content marketing techniques like automation and personalization. A prominent service provider in this space is Mailchimp. Price modeling for this kind of marketing is usually based on the number of emails and contacts per month.\n### **App Marketing Channel**\nThis channel is particularly important for businesses that have apps and promote their apps. Currently, Apple App Store and Google Plays App are the two most important platforms for promoting apps. The goal for these kinds of marketing campaigns is to install the app for the targeted audience. Therefore the way we measure conversion and define the goals for them are different. [Apple Search Ads](https://searchads.apple.com/) and [Google Ads](https://ads.google.com/intl/en_us/home/campaigns/app-ads/) are two important marketing partners in this type of marketing. Cost Per Tap (pay when a user tap on your ad), and Cost Per Install (pay when a user installs your app) are two pricing models in this marketing channel.\n## Generic Ad Structure\nUnderstanding the structure of digital advertising is crucial for effective campaign management and data analysis. The typical hierarchy in most advertising platforms, such as Google Ads or Facebook Ads, follows a multi-level structure that includes the Ad Account, Ad Campaign, Ad Group (sometimes called Ad Set too), and Ads (see the following figure).\n![A typical ad structure includes ad account, campaign, ad group, ad levels \\(Image by the author\\).](../images/002.png)\nA typical ad structure includes ad account, campaign, ad group, ad levels (Image by the author).\nEach level serves a specific purpose and allows for detailed targeting, budgeting, and optimization.\n### Ad Account\nThe Ad Account is the highest level in the advertising hierarchy. It represents the organization or individual running the ads and is where billing and payment information is managed. An Ad Account typically contains multiple campaigns and is the central hub for tracking overall ad spend and performance across all campaigns.\nKey responsibilities at the Ad Account level include setting up payment methods, managing user permissions (e.g., who can create or view ads), and accessing reporting dashboards that provide insights into the performance of all campaigns under the account.\n### Campaign\nThe next level is the Ad Campaign, which is where you define the broader objectives of your advertising efforts. An Ad Campaign is designed around a specific goal, such as increasing brand awareness, driving traffic to a website, or generating leads.\nEach campaign has its budget and schedule, and you can choose different campaign types depending on your objectives, such as search campaigns, display campaigns, or video campaigns. The settings at the campaign level, such as targeting criteria and bid strategy, apply to all the ad groups and ads within that campaign.\nSince each campaign can have a unique name, it's more effective to use a standardized naming convention rather than descriptive names. By establishing clear naming patterns within the marketing team, data engineers can later parse essential information, such as campaign objectives or point of sale, directly from the campaign name.\n### Ad Group (Ad Set)\nWithin each campaign, you have Ad Groups (or Ad Sets, depending on the platform). Ad Groups allow for more granular control over your ads, enabling you to organize them based on specific targeting criteria, such as audience demographics, interests, or keywords.\nAd Groups are where you define more detailed settings, such as the bid for each click or impression and the specific targeting options, like location or device type. Each Ad Group can contain multiple ads, all of which share the same targeting and bidding settings defined at the group level.\nJust like campaign names, ad group names can follow a specific pattern to include key objectives, such as targeted device type or geo-location. It's crucial that everyone on the marketing team adheres to these naming conventions. This consistency allows data engineering teams to extract valuable information from the ad group names and provide more context for the measured data.\n### Ads\nAt the bottom of the hierarchy are the Ads themselves, which are the individual pieces of content that users will see. This is where you define the creative elements of your campaign, such as the headline, text, images, or video, as well as the call to action (CTA).\nEach Ad within an Ad Group is subject to the targeting and bidding settings of the group but can be tested against other ads in the same group to see which performs best. This process, known as A/B testing, allows advertisers to optimize their creative assets by comparing the performance of different ad variations.\n### Integrating the Structure\nThe structure of Ad Accounts, Campaigns, Ad Groups, and Ads is designed to give marketers flexibility and control over their advertising efforts. By organizing your ads in this hierarchical way, you can manage large-scale campaigns more effectively, ensuring that your marketing budget is allocated efficiently and that your ads are reaching the right audience with the right message.\nFor Marketing Data Engineers, understanding this structure is essential. It informs how data is collected and organized within the platform and impacts how performance metrics are reported and analyzed. Properly structuring your ad campaigns can lead to more accurate data insights, enabling better optimization and more effective marketing strategies.\n## Marketing Data\nAfter understanding different marketing channels and a generic ad structure, it is essential to get familiar with marketing data. Data engineers love to divide data into dimensions and facts (aligned with data warehousing). Here, I also divide the marketing data into these two groups to make it easier for you to understand them.\n### **Context and Dimensions**\nEverything that gives us a context about a marketing activity (such as ads) is considered as dimension (or context). Here are some common dimensions for marketing measurements.\n**Ads structure dimensions:** As mentioned in the previous section, each ad has a hierarchy of ad account, ad campaign, ad group (or ad set). Ids and names of different parts of this hierarchy (such as account id, account name, campaign id, campaign name, and so on). These data helps us to aggregate different level of reports easier.\n**Device and User Attributes:** Nowadays, you can get information far more details on user interactions with app. Information regarding the impression device (where the ad is shown) such as device OS (ios, Android), and device type (mobile, tablet, or desktop). Sometimes the same information is shown for the device that eventually the conversion happened (usually called conversion or action device). In addition, if the user breakdowns are selected properly, you might get user attributes such as age, gender, location, and so on in an aggregated way. In particular, location breakdown is important for many advertisers. Two common subdivisions in the location breakdown are Country and Metro.\n**Attribution Windows:** Conversion metrics have a close relationship with attribution windows. Attribution windows refer to the time period during which a conversion is attributed to a particular marketing effort. When we are talking about conversion, we are talking about many kinds of conversions which might be the purpose of an ad or campaign. Some examples are purchasing, checking out, booking, installing, registration, signing up, donating, etc. In addition, different platforms may have different default attribution windows, which can significantly impact your analysis. Some platforms can provide you data in different attribution windows based on your report request. For example, they let you choose one of these attribution windows: 1day-view, 7day-click, or 7day-view-30day-click. It is highly recommended to review different partners attribution windows and find common (or similar) attribution windows for your further analytics. For instance, a 7-day click attribution window means that if a user clicks on an ad and converts within 7 days, the conversion is attributed to that ad. Understanding and customizing attribution windows is crucial for accurately measuring the impact of your marketing activities.\n### Facts and Metrics\n**Basic Metrics:** These fundamental metrics are essential for any digital marketing campaign and often serve as a starting point for more detailed analysis and advanced KPIs.\n  1. **Spend:** This represents the total amount of money invested in a marketing campaign. It can be tracked at various levels, such as by day, platform, or ad unit.\n  2. **Impressions:** This metric counts how many times an ad is displayed to users. It measures the reach of your campaign at the top of the funnel.\n  3. **Clicks:** This measures the number of times users interact with your ad by clicking on it. It's crucial for evaluating the effectiveness of your ad copy and targeting.\n  4. **Engagement:** This includes various user interactions with ads, such as likes, comments, shares, and video plays. It reflects how engaging the ads are to those who see them.\n  5. **Reach:** Unlike impressions, which count the total number of ad displays, reach measures the number of unique users who saw the ad, regardless of how many times they viewed it.\n  6. **Frequency:** This metric indicates the average number of times a user has seen your ad.\n  7. **Video Metrics:** For ads with video content, metrics such as the number of video plays and milestones (e.g., 50% or 100% completion) provide insights into video effectiveness and viewer engagement.\n\n\n**Action and Conversion Metrics:** These metrics go beyond the basics to measure user engagement and conversion actions. Conversions can vary widely, including actions such as purchases, checkouts, bookings, installations, registrations, sign-ups, and donations. Conversion metrics are closely tied to the attribution window, which defines the time frame within which a conversion is attributed to an ad interaction. For example, you might want to track whether a user makes a purchase 1 day or 30 days after interacting with an ad. As the time between ad interaction and conversion increases, the direct causality between the ad and the conversion may diminish. Additionally, different platforms may report conversions based on their data and they are blind to all your marketing efforts. For instance, Google might attribute a specific conversion to its platform based on a 30-day click window, while Facebook (Meta) might attribute the same conversion to itseld based a 1-day click window conversion. As you see, both of these platforms are claiming this specific conversion to themselves, but ultimately, it's up to you to determine how to attribute it based on your internal data and analysis. This is where the value of marketing analytics becomes evident, as you'll explore in the following sections.\n## Marketing Data Engineering Responsibilities\nNow that you have a grasp of digital marketing fundamentals, the structure of generic ads, and key marketing data features, let's explore the specific roles and responsibilities of data engineers in supporting marketing goals.\n### Data Lake Design, Implementation, and Maintenance\nOne of the primary responsibilities in Marketing Data Engineering is designing, implementing, and maintaining marketing data lakes. Data engineers must ingest two main types of data from various sources into a centralized data lake:\n**1. Ads Data from Marketing Platforms:** This involves downloading data from platforms like Google, Meta, Bing, etc. Integrating these platforms through APIs requires a deep understanding of APIs, data formats, and data transformation techniques. Many platforms provide SDKs (e.g., [Meta Business SDK](https://developers.facebook.com/docs/business-sdk/)), while others rely on RESTful APIs (e.g., [[TikTok](https://www.catchr.io/metrics/tiktok-ads-metrics) API for Business](https://ads.tiktok.com/help/article/marketing-api?lang=en)). As a data engineer, you'll spend considerable time exploring API capabilities and limitations by reviewing official documentation and conducting API tests using tools like Postman. Often, official documentation may be insufficient, necessitating the use of third-party resources for more comprehensive insights. For example, here you can find third-party documentation for [Facebook ](https://javadoc.io/doc/com.facebook.business.sdk/facebook-java-business-sdk/latest/index.html)and TikTok. A proof-of-concept (POC) phase is highly recommended to thoroughly explore the APIs before building data pipelines. Understanding the granularity of the data available from APIs is crucial, as some may limit data to daily levels at the ad group level, while others allow more detailed breakdowns by factors like age, device, gender, and location. Additionally, assessing the available metrics, such as basic or action/conversion metrics with varying attribution windows, is vital for enhancing future analytics. Testing API thresholds for request limits and data volume per day is also critical. To manage these restrictions, you may need to divide requests into smaller, parallel ones using threading, which requires careful handling of its complexities.\n**2. Business Performance Data:** This includes data from business activities such as orders, invoicing, bookings, transactions, and clickstream data. Typically stored in transactional systems (OLTP), these data must first be ingested into the marketing data lake using batch- or stream-processing pipelines. It's essential to consider how to store this data in the marketing data lake, as some may be sensitive customer or financial information that cannot be stored in its raw form. The sheer volume of this data can also significantly increase costs if not managed carefully.\nRegardless of the data source or type, the primary goal is to ensure seamless data flow from source to destination. Given that this data serves as the foundation for many downstream processes, defining a reasonable SLA and developing a strategy to meet it is crucial. The processes at this stage often consume the most resources and incur the highest costs, making good design essential. For example, indiscriminately dumping large amounts of clickstream data without a clear business use case can lead to significant storage costs, while running API request processes without leveraging threading can result in longer processing times and higher costs. These examples underscore the importance of efficiency in this stage to meet SLAs and minimize expenses.\n### Data Warehousing and Dimensional Modeling\nAfter data is ingested into marketing data lakes, it often arrives in an unorganized, unstructured, and unclean state. Before business or analytics teams can utilize this data effectively, it needs to be cleaned, transformed, and validated - tasks where data engineers focus much of their efforts.\nApplying data warehousing principles is critical in this process, with dimensional modeling remaining a popular approach in marketing data warehousing. By organizing marketing data into fact and dimension tables, stakeholders, data analysts, and scientists can more easily conduct further analysis, driving insights that inform business decisions.\nOne of the common challenges of applying dimensional modeling to ad data is how to deal with slowly changing dimensions. Many marketing teams reuse an ad account, or campaign for different purposes over time. Deciding on how to keep track of these changes must be considered in dimensional modeling of marketing data.\nSince marketing data are coming from different advertising platforms, it is a challenge on how to define a more or less similar grain and schema. Each platform has its own limitations and naming convention, but for our stakeholders, it is much easier to find a similar grain and naming convention among all channels and partners.\nAnother challenge associated with marketing data warehousing is to choose a right approach for complex conversion metrics. Sometimes conversion metrics are reported in STRUCT or MAP formats due to their complex nature. It is the data engineer design choice on how to report these metrics in the business tables. Remember, your stakeholders are not data engineers and one of your responsibilities is to simplify the data as much as possible to make it available to most of your stakeholder. Keeping complex data structures, make your data products less interesting for them to use in their decision making process.\n### Data Sharing with Marketing Partners\nIn certain situations, it becomes necessary to share marketing and business data with your marketing partners. But why is this sharing important? While marketing and business data are valuable assets, sharing specific data with partners can optimize the marketing process and improve targeting efforts.\nConsider this: if your marketing partner drives traffic to your site but doesn't know whether that traffic leads to conversions, they're essentially working blind. You could rely on their algorithm and hope for the best, but a more strategic approach involves sharing conversion data with them. By providing insights into which redirected traffic results in conversions - such as purchases or app installs - you empower their algorithm to focus on the audience most likely to convert. This collaboration enhances the effectiveness of the partnership, benefiting both parties.\nOne of the key responsibilities of a data engineer is to establish secure and reliable connections between your business data resources and those of your partners. This allows for the seamless sharing of marketing and conversion data. Typically, this involves creating customized data pipelines and collaborating closely with the engineers on the marketing platform.\n### Mapping Website Events to Metrics\nA crucial task in Marketing Data Engineering is mapping website events to marketing metrics. Website events are user actions on your site, such as page views, purchases, bookings, or form submissions. These events need to be tracked, labeled, and mapped to relevant marketing metrics to provide a complete picture of user behavior.\nFor example, a button click on a product page might be mapped to a \"product interest\" metric, while a form submission could be mapped to a \"lead generation\" metric. This mapping process is essential for accurate attribution and performance measurement, enabling businesses to understand and optimize user interactions effectively.\n## Analytics\nOnce data is collected and processed, it needs to be analyzed to derive insights. Analytics involves applying statistical and machine learning techniques to marketing data to uncover trends, predict outcomes, and optimize campaigns, distribute budget among channels and partners.\nCommon analytics tasks include A/B testing, cohort analysis, and customer segmentation. The insights gained from these analyses can then be used to inform marketing strategy and drive better decision-making.\n## Conclusion\nIn conclusion, Marketing Data Engineering is a complex but essential field that bridges the gap between marketing strategy and data-driven insights. By understanding the fundamentals of digital marketing, integrating data from various channels, and applying advanced analytics, Marketing Data Engineers empower marketing teams to make informed decisions that drive business success.\nIn future, we will talk about some of the topic in this article in more details. Please ensure to follow me and [subscribe to my channel](https://tamimi-naser.medium.com/subscribe) for upcoming articles.\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](../images/body-909ce5c2500185af5a85.jpeg)\nWritten By\n## [Naser Tamimi](https://towardsdatascience.com/author/tamimi-naser/)\n[See all from Naser Tamimi](https://towardsdatascience.com/author/tamimi-naser/)\n[Data Engineering](https://towardsdatascience.com/tag/data-engineering/)[Data Science](https://towardsdatascience.com/tag/data-science/)[Digital Marketing](https://towardsdatascience.com/tag/digital-marketing/)[Getting Started](https://towardsdatascience.com/tag/getting-started/)[Marketing](https://towardsdatascience.com/tag/marketing/)\nShare This Article\n  * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fa-comprehensive-introduction-to-marketing-data-engineering-f292a4a6568b%2F&title=A%20Comprehensive%20Introduction%20to%20Marketing%20Data%20Engineering)\n  * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fa-comprehensive-introduction-to-marketing-data-engineering-f292a4a6568b%2F&title=A%20Comprehensive%20Introduction%20to%20Marketing%20Data%20Engineering)\n  * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fa-comprehensive-introduction-to-marketing-data-engineering-f292a4a6568b%2F&text=A%20Comprehensive%20Introduction%20to%20Marketing%20Data%20Engineering)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n  * [![](../images/003.jpeg)](https://towardsdatascience.com/supermarket-data-analysis-with-pandas-e4991f3e0a9/)\n## [Supermarket Data Analysis with Pandas](https://towardsdatascience.com/supermarket-data-analysis-with-pandas-e4991f3e0a9/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nA comprehensive practical guide for pandas\n[Soner Yıldırım](https://towardsdatascience.com/author/sonery/)\nJanuary 27, 20215 min read\n  * [![](../images/004.jpg)](https://towardsdatascience.com/what-is-data-literacy-in-2025-its-not-what-you-think/)\n## [What Is Data Literacy in 2025? It’s Not What You Think](https://towardsdatascience.com/what-is-data-literacy-in-2025-its-not-what-you-think/)\n[Data Visualization](https://towardsdatascience.com/category/data-science/data-visualization/)\nIn a world of shrinking attention spans and rising AI-generated noise, true data literacy is no longer about charts and stats — it's about context, clarity, and connecting with humans who are more distracted than ever.\n[Michal Szudejko](https://towardsdatascience.com/author/michalszudejko/)\nJuly 30, 202516 min read\n  * [![](../images/005.jpg)](https://towardsdatascience.com/5-data-science-projects-in-healthcare-that-will-get-you-hired-81003cadf2f3/)\n## [5 Data Science Projects in Healthcare that will get you hired](https://towardsdatascience.com/5-data-science-projects-in-healthcare-that-will-get-you-hired-81003cadf2f3/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nWant to apply AI in healthcare? Do these projects to hone your skills.\n[Huma Shah](https://towardsdatascience.com/author/shummashah/)\nOctober 25, 202110 min read\n  * [![](../images/006.jpg)](https://towardsdatascience.com/artificial-intelligence-without-labeled-data-54cdbfbdaad2/)\n## [Artificial Intelligence Without Labeled Data](https://towardsdatascience.com/artificial-intelligence-without-labeled-data-54cdbfbdaad2/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nLet's talk about Unsupervised Deep Learning AI, and Transfer Learning.\n[Daniel Shapiro, PhD](https://towardsdatascience.com/author/lemaysolutions/)\nSeptember 26, 20173 min read\n  * [![](../images/007.jpg)](https://towardsdatascience.com/save-time-writing-sql-with-udfs-24b002bf0192/)\n## [Save Time Writing SQL with UDFs](https://towardsdatascience.com/save-time-writing-sql-with-udfs-24b002bf0192/)\n[Coding](https://towardsdatascience.com/category/programming/coding/)\nAn introduction to SQL UDFs with examples\n[Jye Sawtell-Rickson](https://towardsdatascience.com/author/jyesr/)\nOctober 7, 20224 min read\n  * [![](../images/008.jpg)](https://towardsdatascience.com/data-pipeline-with-airflow-and-aws-tools-s3-lambda-glue-18585d269761/)\n## [Data Pipeline with Airflow and AWS Tools (S3, Lambda & Glue)](https://towardsdatascience.com/data-pipeline-with-airflow-and-aws-tools-s3-lambda-glue-18585d269761/)\n[Data Engineering](https://towardsdatascience.com/category/data-science/data-engineering/)\nLearning a little about these tools and how to integrate them\n[João Pedro](https://towardsdatascience.com/author/joaopedro214/)\nApril 6, 202317 min read\n  * [![](../images/009.jpg)](https://towardsdatascience.com/mastering-data-science-workflows-with-helper-classes-1134afbd0600/)\n## [Mastering Data Science Workflows with Helper Classes](https://towardsdatascience.com/mastering-data-science-workflows-with-helper-classes-1134afbd0600/)\n[Programming](https://towardsdatascience.com/category/programming/)\nPython Helper Classes for EDA, Feature Engineering and Machine Learning\n[Sadrach Pierre, Ph.D.](https://towardsdatascience.com/author/spierre91/)\nDecember 30, 202211 min read\n  * [![](../images/010.jpg)](https://towardsdatascience.com/so-you-want-to-bootstrap-with-pyspark-fd04d056e4aa/)\n## [So you want to bootstrap with PySpark](https://towardsdatascience.com/so-you-want-to-bootstrap-with-pyspark-fd04d056e4aa/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nIt's a little more complicated than you may think at first...\n[Celso M. Cantanhede Silva](https://towardsdatascience.com/author/celso-mattheus/)\nDecember 15, 20204 min read\n  * [![](../images/011.jpeg)](https://towardsdatascience.com/the-easiest-way-to-download-youtube-videos-using-python-2640958318ab/)\n## [The easiest way to download YouTube videos using Python](https://towardsdatascience.com/the-easiest-way-to-download-youtube-videos-using-python-2640958318ab/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nAnd how to use a custom class to extract frames as images\n[Eryk Lewinson](https://towardsdatascience.com/author/eryk-lewinson/)\nFebruary 16, 20205 min read\n\n\n  * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n  * [X](https://x.com/TDataScience)\n  * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n  * [Threads](https://www.threads.net/@towardsdatascience)\n  * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n####  Recommended Articles \nClose\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n",
  "metadata": {
    "title": "A Comprehensive Introduction to Marketing Data Engineering | Towards Data Science",
    "description": "Fundamentals, responsibilities, and challenges",
    "keywords": null,
    "author": "Naser Tamimi",
    "og:locale": "en_US",
    "og:type": "article",
    "og:title": "A Comprehensive Introduction to Marketing Data Engineering",
    "og:description": "Fundamentals, responsibilities, and challenges",
    "og:url": "https://towardsdatascience.com/a-comprehensive-introduction-to-marketing-data-engineering-f292a4a6568b/",
    "og:site_name": "Towards Data Science",
    "og:image": "https://assets.insightmediagroup.io/media/wp-content/uploads/2024/08/1UDPDCOtIlIbJuDnpBuhAJQ.jpeg",
    "og:image:width": "1280",
    "og:image:height": "1280",
    "og:image:type": "image/jpeg",
    "twitter:card": "summary_large_image",
    "twitter:site": "@TDataScience",
    "twitter:creator": "@TDataScience",
    "twitter:label1": "Written by",
    "twitter:data1": "Naser Tamimi",
    "twitter:label2": "Est. reading time",
    "twitter:data2": "18 min read",
    "article:published_time": "2024-08-30T03:12:34.000Z",
    "article:modified_time": "2024-08-30T03:12:34.000Z",
    "article:author": "https://towardsdatascience.com/author/tamimi-naser/"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [],
  "images": [
    {
      "status": "http_404",
      "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500",
      "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500",
      "alt": "",
      "downloaded": false,
      "discovery": "saved_markdown",
      "retried_at": "2026-10-20T18:04:54Z"
    },
    {
      "status": "http_404",
      "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500",
      "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500",
      "alt": "",
      "downloaded": false,
      "discovery": "saved_markdown",
      "retried_at": "2026-10-20T18:04:54Z"
    },
    {
      "status": "http_404",
      "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500",
      "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500",
      "alt": "",
      "downloaded": false,
      "discovery": "saved_markdown",
      "retried_at": "2026-10-20T18:04:54Z"
    },
    {
      "status": "http_404",
      "url": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500",
      "src": "https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500",
      "alt": "",
      "downloaded": false,
      "discovery": "saved_markdown",
      "retried_at": "2026-10-20T18:04:54Z"
    }
  ],
  "audio": [],
  "videos": [
    {
      "url": "https://www.youtube.com/c/TowardsDataScience",
      "kind": "channel_reference",
      "status": "reference_only"
    }
  ],
  "links": [
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/latest/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/editors-pick/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/the-variable/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/shipai/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/submissions/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/category/data-science/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/author/tamimi-naser/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/data-engineering/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/data-science/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/digital-marketing/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/getting-started/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/tag/marketing/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/supermarket-data-analysis-with-pandas-e4991f3e0a9/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/category/artificial-intelligence/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/author/sonery/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/what-is-data-literacy-in-2025-its-not-what-you-think/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/category/data-science/data-visualization/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/author/michalszudejko/"
    },
    {
      "type": "internal",
      "href": "https://towardsdatascience.com/5-data-science-projects-in-healthcare-that-will-get-you-hired-81003cadf2f3/"
    },
    {
      "type": "external",
      "href": "https://write.insightmediagroup.io"
    },
    {
      "type": "external",
      "href": "https://www.linkedin.com/company/towards-data-science/"
    },
    {
      "type": "external",
      "href": "https://x.com/TDataScience"
    },
    {
      "type": "external",
      "href": "https://tamimi-naser.medium.com/subscribe"
    },
    {
      "type": "external",
      "href": "https://searchads.apple.com/"
    },
    {
      "type": "external",
      "href": "https://ads.google.com/intl/en_us/home/campaigns/app-ads/"
    },
    {
      "type": "external",
      "href": "https://developers.facebook.com/docs/business-sdk/"
    },
    {
      "type": "external",
      "href": "https://ads.tiktok.com/help/article/marketing-api?lang=en"
    },
    {
      "type": "external",
      "href": "https://javadoc.io/doc/com.facebook.business.sdk/facebook-java-business-sdk/latest/index.html"
    },
    {
      "type": "external",
      "href": "https://www.facebook.com/sharer/sharer.php?title=A+Comprehensive+Introduction+to+Marketing+Data+Engineering&u=https%3A%2F%2Ftowardsdatascience.com%2Fa-comprehensive-introduction-to-marketing-data-engineering-f292a4a6568b%2F"
    },
    {
      "type": "external",
      "href": "https://www.linkedin.com/shareArticle?mini=true&title=A+Comprehensive+Introduction+to+Marketing+Data+Engineering&url=https%3A%2F%2Ftowardsdatascience.com%2Fa-comprehensive-introduction-to-marketing-data-engineering-f292a4a6568b%2F"
    },
    {
      "type": "external",
      "href": "https://x.com/share?text=A+Comprehensive+Introduction+to+Marketing+Data+Engineering&url=https%3A%2F%2Ftowardsdatascience.com%2Fa-comprehensive-introduction-to-marketing-data-engineering-f292a4a6568b%2F"
    },
    {
      "type": "external",
      "href": "https://www.youtube.com/c/TowardsDataScience"
    },
    {
      "type": "external",
      "href": "https://www.threads.net/@towardsdatascience"
    },
    {
      "type": "external",
      "href": "https://bsky.app/profile/towardsdatascience.com"
    },
    {
      "type": "external",
      "href": "https://thenewstack.io"
    },
    {
      "type": "external",
      "href": "https://roadmap.sh"
    },
    {
      "type": "external",
      "href": "https://insightmediagroup.io/creator-network"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-20T17:31:51Z",
  "source_url": "https://towardsdatascience.com/a-comprehensive-introduction-to-marketing-data-engineering-f292a4a6568b/",
  "delivery_status": "unverified",
  "discovery_status": "http_403",
  "missing_reason": "源页面未重新核验:http_403；已按保留正文补采图片",
  "repair_error": "",
  "actual_video_files": [],
  "expected_saved_body_images": 4,
  "image_discovery": "saved_markdown",
  "source_body_markdown": "![Revisit consent button](https://cdn-cookieyes.com/assets/images/revisit.svg)\nWe value your privacy\nWe use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking \"Accept All\", you consent to our use of cookies.\nCustomise Reject All Accept All\nCustomise Consent Preferences ![](https://cdn-cookieyes.com/assets/images/close.svg)\nWe use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below.\nThe cookies that are categorised as \"Necessary\" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more\nNecessaryAlways Active\nNecessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.\n  * Cookie\nBCTempID\n  * Duration\n10 minutes\n  * Description\nNo description available.\n\n\n  * Cookie\n__cf_bm\n  * Duration\n1 hour\n  * Description\nThis cookie, set by Cloudflare, is used to support Cloudflare Bot Management. \n\n\n  * Cookie\nAWSALBCORS\n  * Duration\n7 days\n  * Description\nAmazon Web Services set this cookie for load balancing.\n\n\n  * Cookie\n_cfuvid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services\n\n\n  * Cookie\nli_gc\n  * Duration\n6 months\n  * Description\nLinkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes.\n\n\n  * Cookie\n__hssrc\n  * Duration\nsession\n  * Description\nThis cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session.\n\n\n  * Cookie\n__hssc\n  * Duration\n1 hour\n  * Description\nHubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie.\n\n\n  * Cookie\nwpEmojiSettingsSupports\n  * Duration\nsession\n  * Description\nWordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly.\n\n\n  * Cookie\nBCSessionID\n  * Duration\n1 year 1 month 4 days\n  * Description\nBlueconic sets this cookie as a unique identifier for the BlueConic profile.\n\n\n  * Cookie\n_octo\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nlogged_in\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\n__Secure-YEC\n  * Duration\npast\n  * Description\nYouTube sets this cookie to stores the user's video player preferences using embedded YouTube video\n\n\n  * Cookie\n__eoi\n  * Duration\n6 months\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTGCORS\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\nlogin-status-p\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nAWSALBTG\n  * Duration\n7 days\n  * Description\nNo description available.\n\n\n  * Cookie\ncsrf_token\n  * Duration\nsession\n  * Description\nNo description available.\n\n\n  * Cookie\ntoken_v2\n  * Duration\n1 day\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nD\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nPHPSESSID\n  * Duration\nsession\n  * Description\nThis cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed.\n\n\n  * Cookie\nVISITOR_PRIVACY_METADATA\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to store the user's cookie consent state for the current domain. \n\n\n  * Cookie\ncookietest\n  * Duration\nsession\n  * Description\nThe cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience.\n\n\n  * Cookie\n__Host-airtable-session\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\n__Host-airtable-session.sig\n  * Duration\n1 year\n  * Description\nThis cookie is used to enable us to integrate the services of Airtable.\n\n\n  * Cookie\nm\n  * Duration\n1 year 1 month 4 days\n  * Description\nStripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly.\n\n\n  * Cookie\nBIGipServer*\n  * Duration\nsession\n  * Description\nMarketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. \n\n\n  * Cookie\n__cfruid\n  * Duration\nsession\n  * Description\nCloudflare sets this cookie to identify trusted web traffic.\n\n\n  * Cookie\n_GRECAPTCHA\n  * Duration\n6 months\n  * Description\nGoogle Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks.\n\n\n  * Cookie\n__Secure-YNID\n  * Duration\n6 months\n  * Description\nGoogle cookie used to protect user security and prevent fraud, especially during the login process.\n\n\n  * Cookie\ncookieyes-consent\n  * Duration\n1 year\n  * Description\nCookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors.\n\n\nFunctional\nFunctional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.\n  * Cookie\nlidc\n  * Duration\n1 day\n  * Description\nLinkedIn sets the lidc cookie to facilitate data center selection.\n\n\n  * Cookie\nbrw\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nbrwConsent\n  * Duration\n5 minutes\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nWMF-Uniq\n  * Duration\n1 year\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nloom_anon_comment\n  * Duration\n1 year\n  * Description\nNo description available.\n\n\n  * Cookie\nloom_referral_video\n  * Duration\nsession\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nVISITOR_INFO1_LIVE\n  * Duration\n6 months\n  * Description\nA cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface.\n\n\n  * Cookie\nyt-remote-connected-devices\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nytidb::LAST_RESULT_ENTRY_KEY\n  * Duration\nNever Expires\n  * Description\nThe cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future.\n\n\n  * Cookie\nyt-remote-device-id\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to store the user's video preferences using embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-name\n  * Duration\nsession\n  * Description\nThe yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\nyt-remote-fast-check-period\n  * Duration\nsession\n  * Description\nThe yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos.\n\n\n  * Cookie\nyt-remote-session-app\n  * Duration\nsession\n  * Description\nThe yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-available\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player.\n\n\n  * Cookie\nyt-remote-cast-installed\n  * Duration\nsession\n  * Description\nThe yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video.\n\n\n  * Cookie\ncp_session\n  * Duration\n3 months\n  * Description\nCodepen sets this cookie for Help systems found in the website.\n\n\n  * Cookie\nloid\n  * Duration\n1 year 1 month 4 days\n  * Description\nThis cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform.\n\n\nAnalytics\nAnalytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.\n  * Cookie\n__hstc\n  * Duration\n6 months\n  * Description\nHubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session).\n\n\n  * Cookie\nhubspotutk\n  * Duration\n6 months\n  * Description\nHubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts.\n\n\n  * Cookie\n_ga\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors.\n\n\n  * Cookie\n_ga_*\n  * Duration\n1 year 1 month 4 days\n  * Description\nGoogle Analytics sets this cookie to store and count page views.\n\n\n  * Cookie\n__Host-psifi.analyticsTrace\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n__Host-psifi.analyticsTraceV2\n  * Duration\n6 hours\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\n_gh_sess\n  * Duration\nsession\n  * Description\nGitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form.\n\n\n  * Cookie\nYSC\n  * Duration\nsession\n  * Description\nYSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages.\n\n\n  * Cookie\najs_anonymous_id\n  * Duration\n1 year\n  * Description\nThis cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before.\n\n\n  * Cookie\nvuid\n  * Duration\n1 year 1 month 4 days\n  * Description\nVimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. \n\n\nPerformance\nPerformance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.\n  * Cookie\nAWSALB\n  * Duration\n7 days\n  * Description\nAWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target.\n\n\n  * Cookie\nacq\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nacq.sig\n  * Duration\npast\n  * Description\nDescription is currently not available.\n\n\n  * Cookie\nptc\n  * Duration\n2 years\n  * Description\nNo description available.\n\n\nAdvertisement\nAdvertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.\n  * Cookie\nmuc_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to collect user behaviour and interaction data to optimize the website.\n\n\n  * Cookie\nguest_id_marketing\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\nguest_id_ads\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor.\n\n\n  * Cookie\npersonalization_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting.\n\n\n  * Cookie\nguest_id\n  * Duration\n1 year 1 month 4 days\n  * Description\nTwitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences.\n\n\n  * Cookie\nbcookie\n  * Duration\n1 year\n  * Description\nLinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs.\n\n\n  * Cookie\n__Secure-ROLLOUT_TOKEN\n  * Duration\n6 months\n  * Description\nYouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment.\n\n\n  * Cookie\nyt.innertube::nextId\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nyt.innertube::requests\n  * Duration\nNever Expires\n  * Description\nYouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen.\n\n\n  * Cookie\nsession_tracker\n  * Duration\nsession\n  * Description\nThis cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website.\n\n\n  * Cookie\nedgebucket\n  * Duration\nsession\n  * Description\nReddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content.\n\n\n  * Cookie\ndid\n  * Duration\n1 year\n  * Description\nArbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year.\n\n\nUncategorised\nOther uncategorised cookies are those that are being analysed and have not been classified into a category as yet.\nNo cookies to display.\nReject All Save My Preferences Accept All\nToggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation\n[](https://towardsdatascience.com/)\nPublish AI, ML & data-science insights to a global community of data professionals.\nSign in[Submit an Article](https://write.insightmediagroup.io)\n[Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/)\n* * *\n[Write For TDS](https://towardsdatascience.com/submissions/)\n[LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search\n[Data Science](https://towardsdatascience.com/category/data-science/)\n# A Comprehensive Introduction to Marketing Data Engineering\nFundamentals, responsibilities, and challenges\n[Naser Tamimi](https://towardsdatascience.com/author/tamimi-naser/)\nAugust 30, 202418 min read\n### DATA ENGINEERING\n![Image generated by the author.](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/08/1UDPDCOtIlIbJuDnpBuhAJQ.jpeg)\nImage generated by the author.\nIn today's digital landscape, marketing is increasingly driven by data. Unlike traditional marketing (examples: television, radio, billboards, signs, and print), where measuring the impact of marketing channels, strategies, and ads was often unclear, digital marketing allows us to measure these impacts with precision. In fact, digital marketing has become a more effective approach not just because of the large audiences online, but because it enables us to evaluate our marketing efforts with greater accuracy and adjust its targeting more effectively.\nTherefore, marketing teams need robust and scalable data infrastructure that allows them to ingest, process, and analyze data from multiple sources or channels. Of course, this is where Marketing Data Engineering becomes an important topic. Marketing Data Engineering is the backbone of modern digital marketing. The quality of data engineering products in this specific field, enables data-driven decisions that can significantly impact a company's goals.\nWhile Marketing Data Engineering is a vast field that can't be fully covered in a single article, I've aimed to provide a concise introduction packed with essential information. Each of the topics mentioned has much more depth, which I hope to explore in future articles. To stay updated and receive the latest insights, be sure to [subscribe to my Medium channel](https://tamimi-naser.medium.com/subscribe).\n## Introduction\nMarketing Data Engineering involves the design, development, and maintenance of data products (including tables, data pipelines, dashboards, etc.) that ingest and process data from various marketing channels and partners. It requires a deep understanding of both digital marketing concepts and data engineering principles.\nIn one sentence, if I want to express the goal of marketing data engineering, I should say:\n> The goal of Marketing Data Engineering is to ensure that marketing data are:\n>   1. Reliable: accurately and continuously validated\n>   2. Available: consistently available and meets its SLA\n>   3. Scalable: capable of accommodating more data, partners, or channels\n>   4. Actionable: ready for use by decision-makers\n> \n\nAt its core, Marketing Data Engineering is about transforming raw marketing and business data into actionable marketing insights. Later you will see a whole section in this article on Marketing Data Engineers responsibilities.\n## Digital Marketing Channels\nBefore diving into the roles of Marketing Data Engineers, it's essential to understand marketing channels. Marketing channels are the various platforms through which companies reach their target audience, including search engines, media outlets, social media platforms, and email, among others. Each marketing channel may encompass different partners and platforms. For instance, Google and Bing both fall under the search engine marketing channel, even though they are distinct platforms (often referred to as marketing partners). They share similar marketing components and concepts, so they're grouped within the same channel but recognized as different partners. However, the data generated from search marketing campaigns is vastly different from that of social media campaigns. The challenge for Marketing Data Engineers is to standardize this diverse data to enable consistent and meaningful analysis.\nHere, we will briefly introduce a few widely-used marketing channels.\n### Search Engine Channels\nSearch Engines are critical digital marketing platforms and can be divided into three main marketing channels: Organic (SEO), Paid (SEM), and Metasearch\n**SEO (Search Engine Optimization) or Organic Search Channel:** This involves optimizing your website and content to rank higher in search engine results pages (SERPs) without paying for placement. The collected data here must show how to optimize your website to rank higher in each search engine (impression) and attract more audience (click). Google, Bing, and Yahoo are some well-known search engines that many SEO optimization techniques are studied and developed for them.\n**SEM (Search Engine Marketing) or Paid Search Channel:** This involves paying for ads to appear in Search Engineer Results Pages (SERPs). Unlike SEO channel, SEM pricing model involves bidding processes for different keywords. Your goal is to collect data that identifies which keywords with which bidding prices are effective in your ads and which are not. Similar to SEO, Google, Bing, and Yahoo are big names in SEM world.\n**Metasearch Engine Channel:** These marketing channels are slowly become popular in recent years specifically for e-commerce, hotels, air travel advertisements. Examples are Google, Bing, TripAdvisor, Kayak, Trivago, and Skyscanner all offer metasearch.\nThe following figure shows all three mentioned marketing channels associated with search engines on a sample Google result page.\n![Paid \\(SEM\\), metasearch, and organic \\(SEO\\) search results on a sample Google page \\(\\(Image by the author\\).](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/08/10xAMfYjxc2-RfucKd5rMTA.png)\nPaid (SEM), metasearch, and organic (SEO) search results on a sample Google page ((Image by the author).\nRegardless of which search engine channels we use, a marketing data engineer must be able to handle data from all kinds of search engine campaigns, ensuring that it can be analyzed side by side to measure overall search marketing effectiveness.\n### Media Marketing Channel\nMedia marketing is traditionally associated with channels like TV, radio, and newspaper advertisements. However, many of these traditional media outlets have recently transitioned to the digital realm or recognized the value of incorporating digital marketing strategies. As a result, they have adopted digital tools to create, monitor, and measure the effectiveness of ad campaigns on their platforms. While it may still be challenging to track every user interaction with ads through these channels, we can assess their effectiveness in a broader sense.\n### Social Media Marketing (SMM) Channels\nSocial media marketing is another key area that has both organic and paid components as well. Meta (with Facebook and Instagram platforms), TikTok, X (or Twitter), LinkedIn, Snapchat, YouTube, and Pinterest are a few examples of partners in this type of marketing. Although it seems that SMM is a single marketing channel, we have two distinct marketing channels under SMM which have different campaigns and ways of measurement.\n**Organic Social Media Marketing Channel:** This refers to the content you post on social media platforms without paying for distribution (for example through your company social media pages). Here, key metrics include likes, shares, comments, and follower growth. Also, we might be able to link conversion on our website to this channel if we have a good tracking system in place.\n**Paid Social Media Marketing Channel:** This involves investing in ads or boosted posts on social media platforms. Key metrics include ad spend, impressions, clicks, conversions, etc. The challenge lies in crafting effective campaigns and managing them regularly to optimize costs while maximizing conversions. Cost Per Click (pay each time a user clicks), Cost Per Thousand Impressions (pay for every thousand times the ad is shown), Cost Per Action (pay when a user takes a specific action), and Cost Per Engagement (pay based on user interactions with the ad, such as likes, shares, comments, or other forms of engagement) are few pricing model in this type of marketing channel.\n### Email Marketing Channel\nThough email marketing is considered a traditional digital marketing channel, it remains one of the most widely used. Recently, it has successfully integrated with advanced content marketing techniques like automation and personalization. A prominent service provider in this space is Mailchimp. Price modeling for this kind of marketing is usually based on the number of emails and contacts per month.\n### **App Marketing Channel**\nThis channel is particularly important for businesses that have apps and promote their apps. Currently, Apple App Store and Google Plays App are the two most important platforms for promoting apps. The goal for these kinds of marketing campaigns is to install the app for the targeted audience. Therefore the way we measure conversion and define the goals for them are different. [Apple Search Ads](https://searchads.apple.com/) and [Google Ads](https://ads.google.com/intl/en_us/home/campaigns/app-ads/) are two important marketing partners in this type of marketing. Cost Per Tap (pay when a user tap on your ad), and Cost Per Install (pay when a user installs your app) are two pricing models in this marketing channel.\n## Generic Ad Structure\nUnderstanding the structure of digital advertising is crucial for effective campaign management and data analysis. The typical hierarchy in most advertising platforms, such as Google Ads or Facebook Ads, follows a multi-level structure that includes the Ad Account, Ad Campaign, Ad Group (sometimes called Ad Set too), and Ads (see the following figure).\n![A typical ad structure includes ad account, campaign, ad group, ad levels \\(Image by the author\\).](https://assets.insightmediagroup.io/media/wp-content/uploads/2024/08/1yRYf_SAvo2wuOxMOvRhluA.png)\nA typical ad structure includes ad account, campaign, ad group, ad levels (Image by the author).\nEach level serves a specific purpose and allows for detailed targeting, budgeting, and optimization.\n### Ad Account\nThe Ad Account is the highest level in the advertising hierarchy. It represents the organization or individual running the ads and is where billing and payment information is managed. An Ad Account typically contains multiple campaigns and is the central hub for tracking overall ad spend and performance across all campaigns.\nKey responsibilities at the Ad Account level include setting up payment methods, managing user permissions (e.g., who can create or view ads), and accessing reporting dashboards that provide insights into the performance of all campaigns under the account.\n### Campaign\nThe next level is the Ad Campaign, which is where you define the broader objectives of your advertising efforts. An Ad Campaign is designed around a specific goal, such as increasing brand awareness, driving traffic to a website, or generating leads.\nEach campaign has its budget and schedule, and you can choose different campaign types depending on your objectives, such as search campaigns, display campaigns, or video campaigns. The settings at the campaign level, such as targeting criteria and bid strategy, apply to all the ad groups and ads within that campaign.\nSince each campaign can have a unique name, it's more effective to use a standardized naming convention rather than descriptive names. By establishing clear naming patterns within the marketing team, data engineers can later parse essential information, such as campaign objectives or point of sale, directly from the campaign name.\n### Ad Group (Ad Set)\nWithin each campaign, you have Ad Groups (or Ad Sets, depending on the platform). Ad Groups allow for more granular control over your ads, enabling you to organize them based on specific targeting criteria, such as audience demographics, interests, or keywords.\nAd Groups are where you define more detailed settings, such as the bid for each click or impression and the specific targeting options, like location or device type. Each Ad Group can contain multiple ads, all of which share the same targeting and bidding settings defined at the group level.\nJust like campaign names, ad group names can follow a specific pattern to include key objectives, such as targeted device type or geo-location. It's crucial that everyone on the marketing team adheres to these naming conventions. This consistency allows data engineering teams to extract valuable information from the ad group names and provide more context for the measured data.\n### Ads\nAt the bottom of the hierarchy are the Ads themselves, which are the individual pieces of content that users will see. This is where you define the creative elements of your campaign, such as the headline, text, images, or video, as well as the call to action (CTA).\nEach Ad within an Ad Group is subject to the targeting and bidding settings of the group but can be tested against other ads in the same group to see which performs best. This process, known as A/B testing, allows advertisers to optimize their creative assets by comparing the performance of different ad variations.\n### Integrating the Structure\nThe structure of Ad Accounts, Campaigns, Ad Groups, and Ads is designed to give marketers flexibility and control over their advertising efforts. By organizing your ads in this hierarchical way, you can manage large-scale campaigns more effectively, ensuring that your marketing budget is allocated efficiently and that your ads are reaching the right audience with the right message.\nFor Marketing Data Engineers, understanding this structure is essential. It informs how data is collected and organized within the platform and impacts how performance metrics are reported and analyzed. Properly structuring your ad campaigns can lead to more accurate data insights, enabling better optimization and more effective marketing strategies.\n## Marketing Data\nAfter understanding different marketing channels and a generic ad structure, it is essential to get familiar with marketing data. Data engineers love to divide data into dimensions and facts (aligned with data warehousing). Here, I also divide the marketing data into these two groups to make it easier for you to understand them.\n### **Context and Dimensions**\nEverything that gives us a context about a marketing activity (such as ads) is considered as dimension (or context). Here are some common dimensions for marketing measurements.\n**Ads structure dimensions:** As mentioned in the previous section, each ad has a hierarchy of ad account, ad campaign, ad group (or ad set). Ids and names of different parts of this hierarchy (such as account id, account name, campaign id, campaign name, and so on). These data helps us to aggregate different level of reports easier.\n**Device and User Attributes:** Nowadays, you can get information far more details on user interactions with app. Information regarding the impression device (where the ad is shown) such as device OS (ios, Android), and device type (mobile, tablet, or desktop). Sometimes the same information is shown for the device that eventually the conversion happened (usually called conversion or action device). In addition, if the user breakdowns are selected properly, you might get user attributes such as age, gender, location, and so on in an aggregated way. In particular, location breakdown is important for many advertisers. Two common subdivisions in the location breakdown are Country and Metro.\n**Attribution Windows:** Conversion metrics have a close relationship with attribution windows. Attribution windows refer to the time period during which a conversion is attributed to a particular marketing effort. When we are talking about conversion, we are talking about many kinds of conversions which might be the purpose of an ad or campaign. Some examples are purchasing, checking out, booking, installing, registration, signing up, donating, etc. In addition, different platforms may have different default attribution windows, which can significantly impact your analysis. Some platforms can provide you data in different attribution windows based on your report request. For example, they let you choose one of these attribution windows: 1day-view, 7day-click, or 7day-view-30day-click. It is highly recommended to review different partners attribution windows and find common (or similar) attribution windows for your further analytics. For instance, a 7-day click attribution window means that if a user clicks on an ad and converts within 7 days, the conversion is attributed to that ad. Understanding and customizing attribution windows is crucial for accurately measuring the impact of your marketing activities.\n### Facts and Metrics\n**Basic Metrics:** These fundamental metrics are essential for any digital marketing campaign and often serve as a starting point for more detailed analysis and advanced KPIs.\n  1. **Spend:** This represents the total amount of money invested in a marketing campaign. It can be tracked at various levels, such as by day, platform, or ad unit.\n  2. **Impressions:** This metric counts how many times an ad is displayed to users. It measures the reach of your campaign at the top of the funnel.\n  3. **Clicks:** This measures the number of times users interact with your ad by clicking on it. It's crucial for evaluating the effectiveness of your ad copy and targeting.\n  4. **Engagement:** This includes various user interactions with ads, such as likes, comments, shares, and video plays. It reflects how engaging the ads are to those who see them.\n  5. **Reach:** Unlike impressions, which count the total number of ad displays, reach measures the number of unique users who saw the ad, regardless of how many times they viewed it.\n  6. **Frequency:** This metric indicates the average number of times a user has seen your ad.\n  7. **Video Metrics:** For ads with video content, metrics such as the number of video plays and milestones (e.g., 50% or 100% completion) provide insights into video effectiveness and viewer engagement.\n\n\n**Action and Conversion Metrics:** These metrics go beyond the basics to measure user engagement and conversion actions. Conversions can vary widely, including actions such as purchases, checkouts, bookings, installations, registrations, sign-ups, and donations. Conversion metrics are closely tied to the attribution window, which defines the time frame within which a conversion is attributed to an ad interaction. For example, you might want to track whether a user makes a purchase 1 day or 30 days after interacting with an ad. As the time between ad interaction and conversion increases, the direct causality between the ad and the conversion may diminish. Additionally, different platforms may report conversions based on their data and they are blind to all your marketing efforts. For instance, Google might attribute a specific conversion to its platform based on a 30-day click window, while Facebook (Meta) might attribute the same conversion to itseld based a 1-day click window conversion. As you see, both of these platforms are claiming this specific conversion to themselves, but ultimately, it's up to you to determine how to attribute it based on your internal data and analysis. This is where the value of marketing analytics becomes evident, as you'll explore in the following sections.\n## Marketing Data Engineering Responsibilities\nNow that you have a grasp of digital marketing fundamentals, the structure of generic ads, and key marketing data features, let's explore the specific roles and responsibilities of data engineers in supporting marketing goals.\n### Data Lake Design, Implementation, and Maintenance\nOne of the primary responsibilities in Marketing Data Engineering is designing, implementing, and maintaining marketing data lakes. Data engineers must ingest two main types of data from various sources into a centralized data lake:\n**1. Ads Data from Marketing Platforms:** This involves downloading data from platforms like Google, Meta, Bing, etc. Integrating these platforms through APIs requires a deep understanding of APIs, data formats, and data transformation techniques. Many platforms provide SDKs (e.g., [Meta Business SDK](https://developers.facebook.com/docs/business-sdk/)), while others rely on RESTful APIs (e.g., [[TikTok](https://www.catchr.io/metrics/tiktok-ads-metrics) API for Business](https://ads.tiktok.com/help/article/marketing-api?lang=en)). As a data engineer, you'll spend considerable time exploring API capabilities and limitations by reviewing official documentation and conducting API tests using tools like Postman. Often, official documentation may be insufficient, necessitating the use of third-party resources for more comprehensive insights. For example, here you can find third-party documentation for [Facebook ](https://javadoc.io/doc/com.facebook.business.sdk/facebook-java-business-sdk/latest/index.html)and TikTok. A proof-of-concept (POC) phase is highly recommended to thoroughly explore the APIs before building data pipelines. Understanding the granularity of the data available from APIs is crucial, as some may limit data to daily levels at the ad group level, while others allow more detailed breakdowns by factors like age, device, gender, and location. Additionally, assessing the available metrics, such as basic or action/conversion metrics with varying attribution windows, is vital for enhancing future analytics. Testing API thresholds for request limits and data volume per day is also critical. To manage these restrictions, you may need to divide requests into smaller, parallel ones using threading, which requires careful handling of its complexities.\n**2. Business Performance Data:** This includes data from business activities such as orders, invoicing, bookings, transactions, and clickstream data. Typically stored in transactional systems (OLTP), these data must first be ingested into the marketing data lake using batch- or stream-processing pipelines. It's essential to consider how to store this data in the marketing data lake, as some may be sensitive customer or financial information that cannot be stored in its raw form. The sheer volume of this data can also significantly increase costs if not managed carefully.\nRegardless of the data source or type, the primary goal is to ensure seamless data flow from source to destination. Given that this data serves as the foundation for many downstream processes, defining a reasonable SLA and developing a strategy to meet it is crucial. The processes at this stage often consume the most resources and incur the highest costs, making good design essential. For example, indiscriminately dumping large amounts of clickstream data without a clear business use case can lead to significant storage costs, while running API request processes without leveraging threading can result in longer processing times and higher costs. These examples underscore the importance of efficiency in this stage to meet SLAs and minimize expenses.\n### Data Warehousing and Dimensional Modeling\nAfter data is ingested into marketing data lakes, it often arrives in an unorganized, unstructured, and unclean state. Before business or analytics teams can utilize this data effectively, it needs to be cleaned, transformed, and validated - tasks where data engineers focus much of their efforts.\nApplying data warehousing principles is critical in this process, with dimensional modeling remaining a popular approach in marketing data warehousing. By organizing marketing data into fact and dimension tables, stakeholders, data analysts, and scientists can more easily conduct further analysis, driving insights that inform business decisions.\nOne of the common challenges of applying dimensional modeling to ad data is how to deal with slowly changing dimensions. Many marketing teams reuse an ad account, or campaign for different purposes over time. Deciding on how to keep track of these changes must be considered in dimensional modeling of marketing data.\nSince marketing data are coming from different advertising platforms, it is a challenge on how to define a more or less similar grain and schema. Each platform has its own limitations and naming convention, but for our stakeholders, it is much easier to find a similar grain and naming convention among all channels and partners.\nAnother challenge associated with marketing data warehousing is to choose a right approach for complex conversion metrics. Sometimes conversion metrics are reported in STRUCT or MAP formats due to their complex nature. It is the data engineer design choice on how to report these metrics in the business tables. Remember, your stakeholders are not data engineers and one of your responsibilities is to simplify the data as much as possible to make it available to most of your stakeholder. Keeping complex data structures, make your data products less interesting for them to use in their decision making process.\n### Data Sharing with Marketing Partners\nIn certain situations, it becomes necessary to share marketing and business data with your marketing partners. But why is this sharing important? While marketing and business data are valuable assets, sharing specific data with partners can optimize the marketing process and improve targeting efforts.\nConsider this: if your marketing partner drives traffic to your site but doesn't know whether that traffic leads to conversions, they're essentially working blind. You could rely on their algorithm and hope for the best, but a more strategic approach involves sharing conversion data with them. By providing insights into which redirected traffic results in conversions - such as purchases or app installs - you empower their algorithm to focus on the audience most likely to convert. This collaboration enhances the effectiveness of the partnership, benefiting both parties.\nOne of the key responsibilities of a data engineer is to establish secure and reliable connections between your business data resources and those of your partners. This allows for the seamless sharing of marketing and conversion data. Typically, this involves creating customized data pipelines and collaborating closely with the engineers on the marketing platform.\n### Mapping Website Events to Metrics\nA crucial task in Marketing Data Engineering is mapping website events to marketing metrics. Website events are user actions on your site, such as page views, purchases, bookings, or form submissions. These events need to be tracked, labeled, and mapped to relevant marketing metrics to provide a complete picture of user behavior.\nFor example, a button click on a product page might be mapped to a \"product interest\" metric, while a form submission could be mapped to a \"lead generation\" metric. This mapping process is essential for accurate attribution and performance measurement, enabling businesses to understand and optimize user interactions effectively.\n## Analytics\nOnce data is collected and processed, it needs to be analyzed to derive insights. Analytics involves applying statistical and machine learning techniques to marketing data to uncover trends, predict outcomes, and optimize campaigns, distribute budget among channels and partners.\nCommon analytics tasks include A/B testing, cohort analysis, and customer segmentation. The insights gained from these analyses can then be used to inform marketing strategy and drive better decision-making.\n## Conclusion\nIn conclusion, Marketing Data Engineering is a complex but essential field that bridges the gap between marketing strategy and data-driven insights. By understanding the fundamentals of digital marketing, integrating data from various channels, and applying advanced analytics, Marketing Data Engineers empower marketing teams to make informed decisions that drive business success.\nIn future, we will talk about some of the topic in this article in more details. Please ensure to follow me and [subscribe to my channel](https://tamimi-naser.medium.com/subscribe) for upcoming articles.\n## Register Now to Continue Reading\n### Unlock Towards Data Science with a Free Account\nAll articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more.\nRegister for freeSign in to your account\n* * *\n![](https://assets.insightmediagroup.io/media/1787657191379_0jh99m.jpeg)\nWritten By\n## [Naser Tamimi](https://towardsdatascience.com/author/tamimi-naser/)\n[See all from Naser Tamimi](https://towardsdatascience.com/author/tamimi-naser/)\n[Data Engineering](https://towardsdatascience.com/tag/data-engineering/)[Data Science](https://towardsdatascience.com/tag/data-science/)[Digital Marketing](https://towardsdatascience.com/tag/digital-marketing/)[Getting Started](https://towardsdatascience.com/tag/getting-started/)[Marketing](https://towardsdatascience.com/tag/marketing/)\nShare This Article\n  * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Fa-comprehensive-introduction-to-marketing-data-engineering-f292a4a6568b%2F&title=A%20Comprehensive%20Introduction%20to%20Marketing%20Data%20Engineering)\n  * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Fa-comprehensive-introduction-to-marketing-data-engineering-f292a4a6568b%2F&title=A%20Comprehensive%20Introduction%20to%20Marketing%20Data%20Engineering)\n  * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Fa-comprehensive-introduction-to-marketing-data-engineering-f292a4a6568b%2F&text=A%20Comprehensive%20Introduction%20to%20Marketing%20Data%20Engineering)\n\n\nTowards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program.\n[Write for TDS](https://towardsdatascience.com/submissions/)\n## Related Articles\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/01/1r90q4P8IKMlOUd1h8Z8lXQ-scaled.jpeg)](https://towardsdatascience.com/supermarket-data-analysis-with-pandas-e4991f3e0a9/)\n## [Supermarket Data Analysis with Pandas](https://towardsdatascience.com/supermarket-data-analysis-with-pandas-e4991f3e0a9/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nA comprehensive practical guide for pandas\n[Soner Yıldırım](https://towardsdatascience.com/author/sonery/)\nJanuary 27, 20215 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2025/07/nick-mollenbeck-dJEPPwXAk7w-unsplash-scaled-1.jpg)](https://towardsdatascience.com/what-is-data-literacy-in-2025-its-not-what-you-think/)\n## [What Is Data Literacy in 2025? It’s Not What You Think](https://towardsdatascience.com/what-is-data-literacy-in-2025-its-not-what-you-think/)\n[Data Visualization](https://towardsdatascience.com/category/data-science/data-visualization/)\nIn a world of shrinking attention spans and rising AI-generated noise, true data literacy is no longer about charts and stats — it's about context, clarity, and connecting with humans who are more distracted than ever.\n[Michal Szudejko](https://towardsdatascience.com/author/michalszudejko/)\nJuly 30, 202516 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2021/10/0rx1J7zmXNYhDZbcr-scaled.jpg)](https://towardsdatascience.com/5-data-science-projects-in-healthcare-that-will-get-you-hired-81003cadf2f3/)\n## [5 Data Science Projects in Healthcare that will get you hired](https://towardsdatascience.com/5-data-science-projects-in-healthcare-that-will-get-you-hired-81003cadf2f3/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nWant to apply AI in healthcare? Do these projects to hone your skills.\n[Huma Shah](https://towardsdatascience.com/author/shummashah/)\nOctober 25, 202110 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2017/09/0wQnGm8jRwwCn9Fz7.jpg)](https://towardsdatascience.com/artificial-intelligence-without-labeled-data-54cdbfbdaad2/)\n## [Artificial Intelligence Without Labeled Data](https://towardsdatascience.com/artificial-intelligence-without-labeled-data-54cdbfbdaad2/)\n[Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/)\nLet's talk about Unsupervised Deep Learning AI, and Transfer Learning.\n[Daniel Shapiro, PhD](https://towardsdatascience.com/author/lemaysolutions/)\nSeptember 26, 20173 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/10/0EYwcevoACAGnO5hI.jpg)](https://towardsdatascience.com/save-time-writing-sql-with-udfs-24b002bf0192/)\n## [Save Time Writing SQL with UDFs](https://towardsdatascience.com/save-time-writing-sql-with-udfs-24b002bf0192/)\n[Coding](https://towardsdatascience.com/category/programming/coding/)\nAn introduction to SQL UDFs with examples\n[Jye Sawtell-Rickson](https://towardsdatascience.com/author/jyesr/)\nOctober 7, 20224 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2023/04/0y_AgxJqpYtFk0VQC-scaled.jpg)](https://towardsdatascience.com/data-pipeline-with-airflow-and-aws-tools-s3-lambda-glue-18585d269761/)\n## [Data Pipeline with Airflow and AWS Tools (S3, Lambda & Glue)](https://towardsdatascience.com/data-pipeline-with-airflow-and-aws-tools-s3-lambda-glue-18585d269761/)\n[Data Engineering](https://towardsdatascience.com/category/data-science/data-engineering/)\nLearning a little about these tools and how to integrate them\n[João Pedro](https://towardsdatascience.com/author/joaopedro214/)\nApril 6, 202317 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2022/12/1HvrGeRUpwJCPP77UbhE-yA.jpg)](https://towardsdatascience.com/mastering-data-science-workflows-with-helper-classes-1134afbd0600/)\n## [Mastering Data Science Workflows with Helper Classes](https://towardsdatascience.com/mastering-data-science-workflows-with-helper-classes-1134afbd0600/)\n[Programming](https://towardsdatascience.com/category/programming/)\nPython Helper Classes for EDA, Feature Engineering and Machine Learning\n[Sadrach Pierre, Ph.D.](https://towardsdatascience.com/author/spierre91/)\nDecember 30, 202211 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/12/0d1Y2Bd0E-v2xLN-8-scaled.jpg)](https://towardsdatascience.com/so-you-want-to-bootstrap-with-pyspark-fd04d056e4aa/)\n## [So you want to bootstrap with PySpark](https://towardsdatascience.com/so-you-want-to-bootstrap-with-pyspark-fd04d056e4aa/)\n[Data Science](https://towardsdatascience.com/category/data-science/)\nIt's a little more complicated than you may think at first...\n[Celso M. Cantanhede Silva](https://towardsdatascience.com/author/celso-mattheus/)\nDecember 15, 20204 min read\n  * [![](https://assets.insightmediagroup.io/media/wp-content/uploads/2020/02/1CFBRJJaHQ7LBoJvuUTikig.jpeg)](https://towardsdatascience.com/the-easiest-way-to-download-youtube-videos-using-python-2640958318ab/)\n## [The easiest way to download YouTube videos using Python](https://towardsdatascience.com/the-easiest-way-to-download-youtube-videos-using-python-2640958318ab/)\n[Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/)\nAnd how to use a custom class to extract frames as images\n[Eryk Lewinson](https://towardsdatascience.com/author/eryk-lewinson/)\nFebruary 16, 20205 min read\n\n\n  * [YouTube](https://www.youtube.com/c/TowardsDataScience)\n  * [X](https://x.com/TDataScience)\n  * [LinkedIn](https://www.linkedin.com/company/towards-data-science/)\n  * [Threads](https://www.threads.net/@towardsdatascience)\n  * [Bluesky](https://bsky.app/profile/towardsdatascience.com)\n\n\n[](https://towardsdatascience.com/)\nYour home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals.\nSubscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/)\n© Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network)\n####  Recommended Articles \nClose\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607686/image?etag=1782161464448&width=500&height=500) How Relevance Models Foreshadowed Transformers for NLP ](https://towardsdatascience.com/how-relevance-models-foreshadowed-transformers-for-nlp/ \"How Relevance Models Foreshadowed Transformers for NLP\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ \"Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ \"The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs\")\n  * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ \"Does Your Company Have a Data Strategy?\")\n\n\n",
  "body_base_path": "attachments/c16dddde4b1c/pages/",
  "body_markdown_path": "attachments/c16dddde4b1c/pages/archived-body.md",
  "local_html": "attachments/c16dddde4b1c/pages/archived-body.html",
  "body_source": "prior_capture"
}