![Revisit consent button](../images/body-df29beacd7496220cc94.svg) We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences ![](../images/body-2159bb67f3de8ad870eb.svg) We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data. * Cookie BCTempID * Duration 10 minutes * Description No description available. * Cookie __cf_bm * Duration 1 hour * Description This cookie, set by Cloudflare, is used to support Cloudflare Bot Management. * Cookie AWSALBCORS * Duration 7 days * Description Amazon Web Services set this cookie for load balancing. * Cookie _cfuvid * Duration session * Description Cloudflare sets this cookie to track users across sessions to optimize user experience by maintaining session consistency and providing personalized services * Cookie li_gc * Duration 6 months * Description Linkedin set this cookie for storing visitor's consent regarding using cookies for non-essential purposes. * Cookie __hssrc * Duration session * Description This cookie is set by Hubspot whenever it changes the session cookie. The __hssrc cookie set to 1 indicates that the user has restarted the browser, and if the cookie does not exist, it is assumed to be a new session. * Cookie __hssc * Duration 1 hour * Description HubSpot sets this cookie to keep track of sessions and to determine if HubSpot should increment the session number and timestamps in the __hstc cookie. * Cookie wpEmojiSettingsSupports * Duration session * Description WordPress sets this cookie when a user interacts with emojis on a WordPress site. It helps determine if the user's browser can display emojis properly. * Cookie BCSessionID * Duration 1 year 1 month 4 days * Description Blueconic sets this cookie as a unique identifier for the BlueConic profile. * Cookie _octo * Duration 1 year * Description No description available. * Cookie logged_in * Duration 1 year * Description No description available. * Cookie __Secure-YEC * Duration past * Description YouTube sets this cookie to stores the user's video player preferences using embedded YouTube video * Cookie __eoi * Duration 6 months * Description Description is currently not available. * Cookie AWSALBTGCORS * Duration 7 days * Description No description available. * Cookie login-status-p * Duration past * Description Description is currently not available. * Cookie AWSALBTG * Duration 7 days * Description No description available. * Cookie csrf_token * Duration session * Description No description available. * Cookie token_v2 * Duration 1 day * Description Description is currently not available. * Cookie D * Duration 1 year * Description Description is currently not available. * Cookie PHPSESSID * Duration session * Description This cookie is native to PHP applications. The cookie stores and identifies a user's unique session ID to manage user sessions on the website. The cookie is a session cookie and will be deleted when all the browser windows are closed. * Cookie VISITOR_PRIVACY_METADATA * Duration 6 months * Description YouTube sets this cookie to store the user's cookie consent state for the current domain. * Cookie cookietest * Duration session * Description The cookietest cookie is typically used to determine whether the user's browser accepts cookies, essential for website functionality and user experience. * Cookie __Host-airtable-session * Duration 1 year * Description This cookie is used to enable us to integrate the services of Airtable. * Cookie __Host-airtable-session.sig * Duration 1 year * Description This cookie is used to enable us to integrate the services of Airtable. * Cookie m * Duration 1 year 1 month 4 days * Description Stripe sets this cookie for fraud prevention purposes. It identifies the device used to access the website, allowing the website to be formatted accordingly. * Cookie BIGipServer* * Duration session * Description Marketo sets this cookie to collect information about the user's online activity and build a profile about their interests to provide advertisements relevant to the user. * Cookie __cfruid * Duration session * Description Cloudflare sets this cookie to identify trusted web traffic. * Cookie _GRECAPTCHA * Duration 6 months * Description Google Recaptcha service sets this cookie to identify bots to protect the website against malicious spam attacks. * Cookie __Secure-YNID * Duration 6 months * Description Google cookie used to protect user security and prevent fraud, especially during the login process. * Cookie cookieyes-consent * Duration 1 year * Description CookieYes sets this cookie to remember users' consent preferences so that their preferences are respected on subsequent visits to this site. It does not collect or store any personal information about the site visitors. Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features. * Cookie lidc * Duration 1 day * Description LinkedIn sets the lidc cookie to facilitate data center selection. * Cookie brw * Duration 1 year * Description No description available. * Cookie brwConsent * Duration 5 minutes * Description Description is currently not available. * Cookie WMF-Uniq * Duration 1 year * Description Description is currently not available. * Cookie loom_anon_comment * Duration 1 year * Description No description available. * Cookie loom_referral_video * Duration session * Description Description is currently not available. * Cookie VISITOR_INFO1_LIVE * Duration 6 months * Description A cookie set by YouTube to measure bandwidth that determines whether the user gets the new or old player interface. * Cookie yt-remote-connected-devices * Duration Never Expires * Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos. * Cookie ytidb::LAST_RESULT_ENTRY_KEY * Duration Never Expires * Description The cookie ytidb::LAST_RESULT_ENTRY_KEY is used by YouTube to store the last search result entry that was clicked by the user. This information is used to improve the user experience by providing more relevant search results in the future. * Cookie yt-remote-device-id * Duration Never Expires * Description YouTube sets this cookie to store the user's video preferences using embedded YouTube videos. * Cookie yt-remote-session-name * Duration session * Description The yt-remote-session-name cookie is used by YouTube to store the user's video player preferences using embedded YouTube video. * Cookie yt-remote-fast-check-period * Duration session * Description The yt-remote-fast-check-period cookie is used by YouTube to store the user's video player preferences for embedded YouTube videos. * Cookie yt-remote-session-app * Duration session * Description The yt-remote-session-app cookie is used by YouTube to store user preferences and information about the interface of the embedded YouTube video player. * Cookie yt-remote-cast-available * Duration session * Description The yt-remote-cast-available cookie is used to store the user's preferences regarding whether casting is available on their YouTube video player. * Cookie yt-remote-cast-installed * Duration session * Description The yt-remote-cast-installed cookie is used to store the user's video player preferences using embedded YouTube video. * Cookie cp_session * Duration 3 months * Description Codepen sets this cookie for Help systems found in the website. * Cookie loid * Duration 1 year 1 month 4 days * Description This cookie is set by the Reddit. The cookie enables the sharing of content from the website onto the social media platform. Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc. * Cookie __hstc * Duration 6 months * Description Hubspot set this main cookie for tracking visitors. It contains the domain, initial timestamp (first visit), last timestamp (last visit), current timestamp (this visit), and session number (increments for each subsequent session). * Cookie hubspotutk * Duration 6 months * Description HubSpot sets this cookie to keep track of the visitors to the website. This cookie is passed to HubSpot on form submission and used when deduplicating contacts. * Cookie _ga * Duration 1 year 1 month 4 days * Description Google Analytics sets this cookie to calculate visitor, session and campaign data and track site usage for the site's analytics report. The cookie stores information anonymously and assigns a randomly generated number to recognise unique visitors. * Cookie _ga_* * Duration 1 year 1 month 4 days * Description Google Analytics sets this cookie to store and count page views. * Cookie __Host-psifi.analyticsTrace * Duration 6 hours * Description Description is currently not available. * Cookie __Host-psifi.analyticsTraceV2 * Duration 6 hours * Description Description is currently not available. * Cookie _gh_sess * Duration session * Description GitHub sets this cookie for temporary application and framework state between pages like what step the user is on in a multiple step form. * Cookie YSC * Duration session * Description YSC cookie is set by Youtube and is used to track the views of embedded videos on Youtube pages. * Cookie ajs_anonymous_id * Duration 1 year * Description This cookie is set by Segment to count the number of people who visit a certain site by tracking if they have visited before. * Cookie vuid * Duration 1 year 1 month 4 days * Description Vimeo installs this cookie to collect tracking information by setting a unique ID to embed videos on the website. Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors. * Cookie AWSALB * Duration 7 days * Description AWSALB is an application load balancer cookie set by Amazon Web Services to map the session to the target. * Cookie acq * Duration past * Description Description is currently not available. * Cookie acq.sig * Duration past * Description Description is currently not available. * Cookie ptc * Duration 2 years * Description No description available. Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns. * Cookie muc_ads * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to collect user behaviour and interaction data to optimize the website. * Cookie guest_id_marketing * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. * Cookie guest_id_ads * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. * Cookie personalization_id * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to integrate and share features for social media and also store information about how the user uses the website, for tracking and targeting. * Cookie guest_id * Duration 1 year 1 month 4 days * Description Twitter sets this cookie to identify and track the website visitor. It registers if a user is signed in to the Twitter platform and collects information about ad preferences. * Cookie bcookie * Duration 1 year * Description LinkedIn sets this cookie from LinkedIn share buttons and ad tags to recognize browser IDs. * Cookie __Secure-ROLLOUT_TOKEN * Duration 6 months * Description YouTube sets this cookie to manage feature rollout and experimentation. It helps Google control which new features or interface changes are shown to users as part of testing and staged rollouts, ensuring consistent experience for a given user during an experiment. * Cookie yt.innertube::nextId * Duration Never Expires * Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen. * Cookie yt.innertube::requests * Duration Never Expires * Description YouTube sets this cookie to register a unique ID to store data on what videos from YouTube the user has seen. * Cookie session_tracker * Duration session * Description This cookie is set by the Reddit. This cookie is used to identify trusted web traffic. It also helps in adverstising on the website. * Cookie edgebucket * Duration session * Description Reddit sets this cookie to save the information about a log-on Reddit user, for the purpose of advertisement recommendations and updating the content. * Cookie did * Duration 1 year * Description Arbor sets this cookie to show targeted ads to site visitors.This cookie expires after 2 months or 1 year. Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle Search[](https://towardsdatascience.com/)Toggle Mobile Navigation [](https://towardsdatascience.com/) Publish AI, ML & data-science insights to a global community of data professionals. Sign in[Submit an Article](https://write.insightmediagroup.io) [Latest](https://towardsdatascience.com/latest/)[Editor's Picks](https://towardsdatascience.com/tag/editors-pick/)[Newsletter](https://towardsdatascience.com/tag/the-variable/)[ShipAI](https://towardsdatascience.com/shipai/) * * * [Write For TDS](https://towardsdatascience.com/submissions/) [LinkedIn](https://www.linkedin.com/company/towards-data-science/)[X](https://x.com/TDataScience)Toggle Search [Data Science](https://towardsdatascience.com/category/data-science/) # Transcribing Poetry And Speeches With Wav2Vec2 Efficient transcription of audio files has been one of the major "missing links" in modern NLP - till now. Enter Hugging Face's... [Chua Chin Hon](https://towardsdatascience.com/author/chuachinhon/) February 26, 202115 min read ### Efficient transcription of audio files has been one of the major "missing links" in modern NLP - till now. Enter Hugging Face's implementation of Facebook's Wav2Vec2 model, which produces impressive out-of-the-box results. ![Poet Amanda Gorman delivering the inauguration poem on Jan 20, 2021. Screen-capture via PBS NewsHour's YouTube clip.](../images/000.png) Poet Amanda Gorman delivering the inauguration poem on Jan 20, 2021. Screen-capture via [PBS NewsHour's YouTube clip](https://www.youtube.com/watch?v=LZ055ilIiN4). AI can't write great poetry on its own (yet?). But it can now transcribe a poetry recital really well, if results from the [Wav2Vec2 transformer model](https://huggingface.co/transformers/model_doc/wav2vec2.html) is anything to go by. My trials using audio clips ranging in length from 62s to 12.5 minutes, including the evocative [Inaugural Poem by youth poet Amanda Gorman](https://www.youtube.com/watch?v=LZ055ilIiN4), turned up pretty impressive results. Efficient audio-to-text transcription has been one of the "missing links" in the modern Natural Language Processing (NLP) toolkit. Not anymore it seems, thanks to [Hugging Face's implementation of the Wav2Vec2](https://huggingface.co/facebook/wav2vec2-base-960h) model by Facebook. Learn this step by step with the interactive [AI and Data Scientist](https://roadmap.sh/ai-data-scientist) and [Machine Learning](https://roadmap.sh/machine-learning) roadmaps. What's exciting about this is that it opens up new possibilities of "chain linking" NLP tasks from audio files to, say, textual sentiment analysis or even translation in one go. The caveat here is that long audio clips are really memory intensive when processed this way, and require additional work arounds for folks (like me) who don't have access to a [super-duper computer](https://www.apple.com/shop/buy-mac/mac-pro/rack). This post outlines results from my recent trials, and includes links to a [repo](https://github.com/chuachinhon/wav2vec2_transformers) with notebooks and sample files to run your own experiment. You can check out the output transcripts of the two longish clips [here](https://www.dropbox.com/sh/evmoflm2w3a1zo1/AAAbKTt2fL45Sf8lIOS7RuJwa?dl=0). * * * ## REPO, REQUIREMENTS, AND REFERENCES My [repo](https://github.com/chuachinhon/wav2vec2_transformers) contains 3 notebooks (2 Jupyter, 1 Colab) and 3 sets of audio files. To run them, you'll need: * [Transformers ≥ 4.3](https://pypi.org/project/transformers/) * [Librosa](https://pypi.org/project/librosa/) (to manage the audio files) The [audio files in the repo](https://github.com/chuachinhon/wav2vec2_transformers/tree/main/audio) have been split-up and downsampled to 16kHz. If you wish to use your own audio clips, make sure they are sampled at 16kHz as that's the frequency that the Wav2Vec2 base model was pre-trained and fine tuned on. How long should each audio clip be? That would depend on the amount of compute at your disposal. The Wav2Vec2 transcription process is very memory intensive. Attempts to transcribe audio files longer than 90s have all crashed on me, even on Colab Pro. As such, I've kept the audio clips to about 60s max. To process longer audio clips, you'll have to split them up into equal parts of a minute (or whatever your machine can deal with) each. We'll get into this later. I used [Audacity](https://www.audacityteam.org/) to split up the audio files in my trials. It's free and more than adequate for the task at hand. I won't get into the technical details in this post. But the documentation and paper on Wav2Vec2 are worth a read: * Documentation on [Hugging Face](https://huggingface.co/transformers/model_doc/wav2vec2.html) * Paper on [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://arxiv.org/abs/2006.11477) * Hosted inference [API on Hugging Face](https://huggingface.co/facebook/wav2vec2-base-960h) There are several versions of the Wav2Vec2 model on Hugging Face's model hub. For this exercise, I'm sticking with the [wav2vec2-base-960h](https://huggingface.co/facebook/wav2vec2-base-960h) base model. * * * ## TRIAL #1: "ASK WHAT YOU CAN DO FOR YOUR COUNTRY " ![Former US President John F Kennedy delivering his inaugural address on 20 January 1961. Screen-grab via video on the website of the JFK Presidential Library and Museum.](../images/001.png) Former US President John F Kennedy delivering his inaugural address on 20 January 1961. Screen-grab via [video on the website of the JFK Presidential Library and Museum](https://www.jfklibrary.org/learn/about-jfk/historic-speeches/inaugural-address). The simplest way to try out Wav2Vec2 is with a small audio clip that would be relatively fast for the model to transcribe in one go. I picked a [62s clip](https://github.com/chuachinhon/wav2vec2_transformers/blob/main/audio/jfk.flac) from late US President John F Kennedy's famous inaugural address in 1961 (13th-minute mark onwards), and this is the transcript from Wav2Vec2: > IN THE LONG HISTORY OF THE WORLD ONLY A FEW GENERATION HAVE BEEN GRANTED THE ROLE OF DEFENDING FREEDOM IN ITS **OUR O** MAXIMUM DANGER I DO NOT SHRINK FROM THIS RESPONSIBILITY I WELCOME IT I DO NOT BELIEVE THAT ANY OF US WOULD EXCHANGE PLACES WITH ANY OTHER PEOPLE OR ANY OTHER GENERATION THE ENERGY THE FAITH THE DEVOTION WHICH **WANGE BRANG TO THIS END OF UP WILL NIT** OUR COUNTRY AND ALL WHO SERVE IT AND THE GLOW FROM THAT FIRE **AND** TRULY **LIKE** THE WORLD AND SO MY FELLOW **AMERICA** ASK NOT WHAT YOUR COUNTRY CAN DO FOR YOU ASK WHAT YOU CAN DO FOR YOUR COUNTRY The parts in bold in the Wav2Vec2 transcript are the problematic parts. Overall, the results are pretty impressive, in my view. But how would the model perform on a longer audio clip, with a change in accent and cadence of speaking? ## TRIAL #2: "THE HILL WE CLIMB" ![Poet Amanda Gorman delivering the inauguration poem on Jan 20, 2021. Screen-capture via PBS NewsHour's YouTube clip.](../images/002.png) Poet Amanda Gorman delivering the inauguration poem on Jan 20, 2021. Screen-capture via [PBS NewsHour's YouTube clip](https://www.youtube.com/watch?v=LZ055ilIiN4). For a second trial that would feature distinct contrast with the first, I jumped 40 years ahead to another US Presidential Inauguration and picked a 5 minutes 34s clip of Amanda Gorman delivering a beautiful and evocative poem from the steps of the US Capitol building. However, attempts to transcribe the 334s clip in one go proved futile, with out-of-memory issues crashing my [notebook](https://github.com/chuachinhon/wav2vec2_transformers/blob/main/notebooks/2.0_wav2vec2_poetry.ipynb) repeatedly. I eventually settled on a simple if clumsy workaround by splitting the original clip into smaller 35s clips and transcribing them one at a time. See this notebook. This is the [transcript](https://www.dropbox.com/s/zx4bfct1zhl18az/amanda_gorman.txt?dl=0) from Wav2Vec2: text ``` MISTER PRESIDENT DOCTER BYDEN MADAM VICE PRESIDENT MISTER MHOFF AMERICANS AND THE WORLD WHEN DAY COMES WE ASK OURSELVES WHERE CAN WE FIND LIGHT IN THIS NEVER ENDING SHADE THE LOSS WE CARRY A SEA WE MUST WADE WE BRAVE THE BELLY OF THE BEAST WE'VE LEARNED THAT QUIET ISN'T ALWAYS PEACE IN THE NORMS IN NOTIONS OF WHAT JUST IS ISN'T ALWAYS JUST IS AND YET THE DAWN IS HOURS BEFORE WE KNEW IT SOMEHOW WE DO IT SOMEHOW WE'VE WEATHERED AND WITNESSED A NATION THAT ISN'T BROKEN BUT SIMPLY UNFINISHED WE THE SUCCESSORS OF A COUNTRY AND A TIME WERE A SKINNY BLACK GIRL DESCENDED FROM SLAVES AND RAISED BY A SINGLE MOTHER CAN DREAM OF BECOMING PRESIDENT ONLY TO FIND HERSELF RECITING FOR ONE AND YES WE ARE FAR FROM POLISHED FAR FROM PESTIM BUT THAT DOESN'T MEAN WE ARE STRIVING TO FORM A UNION THAT IS PERFECT WE OR STRIVING TO FORGE OR UNION WITH PURPOSE TO COMPOSE A COUNTRY COMMITTED TO ALL CULTURES COLORS CHARACTERS AND CONDITIONS OF MAN AND SO WE LIFT OUR GAZES NOT TO WHAT STANDS BETWEEN US BUT WHAT STANDS BEFORE US WE CLOSE THE DIVIDE BECAUSE WE KNOT A PUT OUR FUTSURE FIRST WE MUST FIRST PUT OUR DIFFERENCES ASIDE WE LAY DOWN OUR ARMS TIL WE CAN REACH OUT OUR ARMS TO ONE ANOTHER WE SEEK HARM TO NONE AND HARMONY FOR ALL LET THE GLOBE IF NOTHING ELSE SAY THIS IS TRUE THAT EVEN AS WE GRIEVED WE GREWE THAT EVEN AS WE HURT WE HOPED THAT EVEN AS WE TIRED WE TRIE THAT WILL FOREVER BE TIED TOGETHER VICTORIOUS NOT BECAUSE WE WILL NEVER AGAIN KNO DEFEAT BUT BECAUSE WE WILL N EVER AGAIN SO DIVISION SKIPSER TELLS US TO INVISION THAT EVERYONE SHALL SIT UNDER THE OWN VINE AND FIG TREE AND NO ONE SHALL MAKE THEM AFRAID IF WE'RE TO LIVE UP TO OUR OWN TIME THAN VICTORY WON'T LIE IN THE BLADE BUT IN ALL THE BRIDGES WE'VE MADE THAT IS THE PROMISE TO GLADE THE HILL WE CLIME IF ONLY WE DARE IT BECAUSE BEING AMERICAN IS MORE THAN A PRIDE WE INHERIT IT'S THE PAST WE STEP INTO AND HOW WE RE HAR IT WE'VE SEEN A FOREST THAT WOULD SHATTER OR NATION RATHER THAN SHARE IT WELD DESTROY OUR COUNTRY IF IT MEANT DELAYING DEMOCRACY AND THIS EFFORT VERY NEARLY SUCCEEDED BUT WHILE DEMOCRACY CAN BE PERIODICALLY DELAYED IT CAN NEVER BE PERMANENTLY DEFEATED IN THIS TRUTH IN THIS FAITH WE TRUST FOR WHILE WE HAVE OUR EYES ON THE FUTURE OR HISTORY HAS ITS EYES ON US THIS IS THE ERA OF JUST REDEMPTION WE FEARED A CEPTION WE DID NOT FEEL PREPARED TO BE THE EIRS OF SUCH A TERRIFYING HOUR BUT WITHIN IT WE FOUND THE POWER TO AUTHOR A NEW CHAPTER TO OFFER HOPE AND LAUGHTER TO OURSELVES SO WHILE ONCE WE ASKED HOW COULD WE POSSIBLY PREVAIL OVER CATASTROPHE NOW WE ASSERT HOW COULD CATASTROPHE POSSIBLY PREVAIL OVER US WE WILL NOT MARCH BACK TO WHAT WAS BUT MOVE TO WHAT SHALL BE A COUNTRY THAT IS BRUISED BUT WHOLE BE VOLENCE BUT BOLD FIERCE AND FREE WE WILL NOT BE TURNED AROUND OR INTERRUPTED BY INTIMIDATION BECAUSE WE KNOW OUR INACTION AND INERTIA WILL BE THE INHERITANCE OF THE NEXT GENERATION OUR BLENDERS BECOME THEIR BURDENS BUT ONE THING IS CERTAIN IF WE MERGE MERCY WITH MIGHT AND MIGHT WITH MIGHT THEN LOVE BECOMES OUR LEGACY IN CHANGE OUR CHILDREN'S BIRTHRIGHT SO LET US LEAVE BEHIND A COUNTRY BETTER THAN ONE WE WERE LEFT WITH EVERY BREATH FROM OUR BRONZE POUNDED CHEST WE WILL RAISE THIS WOUNDED WORLD INTO A WONDROUS ONE WE WILL RISE FROM THE GOLDLIMD HILLS OF THE WEST WE WILL RISE FROM THE WIND SWEPT NORTHEAST WHERE OUR FORFATHER'S FIRST REALISE REVOLUTION WE WILL RISE FROM THE LAKE RIMD CITIES OF THE MIDWESTERN STATES WE WILL RISE FROM THE SUNBAKED SOUTH WE WILL REBUILD RECONCILE AND BECOVER AND EVERY KNOWN NOOK OF OUR NATION IN EVERY CORNER CALLED OUR COUNTRY OUR PEOPLE DIVERSE AND BEAUTIFUL WILL EMMERGE BATTERED AND BEAUTIFUL WHEN DAY COMES WE STEP OUT OF THE SHADE AFLAME AND UNAFRAID THE NEW DAWN BALLOONS AS WE FREE IT FOR THERE WAS ALWAYS LIGHT IF ONLY WERE BRAVE ENOUGH TO SEE IT IF ONLY WERE BRAVE ENOUGH TO BE IT ``` Again, the parts in bold are the problematic parts that the model failed to get right. But they are pretty minor issues I would say, and can be easily cleaned up. As the Wav2Vec2 models get more sophisticated, I'm pretty sure the results will be better. ## TRIAL #3: SINGAPORE AND THE WEF ![Screen-capture via Singapore Prime Minister's Office's YouTube Channel.](../images/003.png) Screen-capture via [Singapore Prime Minister's Office's YouTube Channel.](https://www.youtube.com/watch?v=izrdoAm4_Gw) For the third trial, I wanted to assess the model on a clip longer than 10 minutes featuring a speaker with an Asian accent. I settled on a January 29 2021 speech by Singapore Prime Minister Lee Hsien Loong at the [World Economic Forum Davos Agenda Week](https://www.youtube.com/watch?v=izrdoAm4_Gw). This time round, I split the 12 minute 30s speech into 13 parts (12x60s clips and 1x30s clips) and ran it on [Colab Pro](https://github.com/chuachinhon/wav2vec2_transformers/blob/main/notebooks/2.1_wav2vec2_colab.ipynb). The [transcript](https://www.dropbox.com/s/gu3e6ns4x4tty61/lhl_wef.txt?dl=0) took 2 minutes 4s to produce: text ``` IAM VERY HONOR TO SPEAK AT DISCLOSING ADDRESS AND I LIKE TO CONGRATULATE PROFESSOR SCHOEB YOURSELF AND THE WHOLE BLW F TEAM FOR PUTTING TOGETHER A SUCCESSFUL PROGRAMM IT 'S BEEN A YEAR SINCE WE WERE ALL PHYSICALLY GATHERED IN DAVORCE FOR THE FIFTIETH ANNUAL MEETING OFER THE DBU F AT THAT TIME WE WERE JUST STARTING TO HEAR ABOUT THIS NEW VIRUS AND TRYING TO UNDERSTAND WHAT WAS HAPPENING NONE OF US ANTICIPATED HOW QUICKLY A FULL SCALE PANDAMIC WOULD BLOW UP AND DRAMATICALLY CHANGE OUR WORLD THE DESRUPTION TO LIVES AND LIVELIHOODS HAS BEEN MASSIVE AND UNPRECEDENTED THE VIRAS IS STILL RAGING IN MANY COUNTRIES IN THE DEVELOPED WORLD IN THE US AN EUROPE AND ALSO IN THE DEVELOPING WORLD IN AFRICA SOUTH AMERICA AND SOUTH ASIA THANKFULLY WITH BAXINES BECOMING AVAILABLE THERE IS SOME LIGHT AT THE END OF THE TUNNEL IT IS NOW CRITICAL THAT VAXINES ARE RULED OUT QUICKLY ACROSS THE WORLD BUT EVEN WITH VAXINES THE PANDAMIC IS FAR FROM BEING QUELLED THE NEW VARIANCE DISCOVERED AND THE U K IN SOUTH AFRICA AND BRASIL ARE WARRYING AND FURTHER MUTATIONS WILL SURELY EMERGE UNTIL A LARGE PART OF THE WORLD'S POPULATION IS VACCINATED WE STILL NEEDS STRONG PUBLIC HEALTH MEASURES EVERYWHERE TO SUPPRESS THE SPREAD OF THE VIOLUS AND KEEP POPULATION SAFE WHAT WILL THE POSTCOVE NINETEEN WOLL LOOK LIKE WILL COUNTRIES EMERGE MORE RESOLVE TO BUILD A MORE RESILIANT BUT STILL GLOBALIZED WELL OR ARE WE HEADED TOWARDS A LESS INTEGRATED GLOBAL ECONOMY A LESS STABLE INTERNATIONAL ORDER THE ANSWER DEPENDS ON THE DECISIONS THAT COUNTRIES TAKE NOW EVEN BEFORE COVER NINETEEN GLOBILIZATION WAS ALREADY UNDER PRESSURE CONFIDENCE AND MALTILACERAL INSTITUTIONS AND RULES AND NORMES WAS ERODING POPULOUS POLITICS NATIVISM NATIONALISM PROTECTIONISM WERE ON THE RISE COUNTRYS INITIAL REACTIONS TO THE PENDEMIC SEEMED TO HERL GRUBILIZATIONS DEMIES BORDERS WERE CLOSE SUPPLY CHANGE WERE BADLY DISRUPTED EACH COUNTRY SCRAMBLE TO SECURE ITS OWN SUPPLIES OF ESSENTIAL GOODS ESPECIALLY IN MEDICINES FACE MASKS AND VENTILATORS IT WAS EACH MAN FOR HIMSELF BUT AS THEIR SITUATION UNFOLDED WE WERE FORCEFULLY REMINDED THAT OUR FATES WERE INTERTWINED AND THAT WE HAD TO WORK TOGETHER AND SO WE DID IN MANY AREAS WE RESTORED SUPPLY CHANGE WE REPATRATED EACH OTHER'S CITIZENS STUCK OVER SEAS WE SHARED TESTS AND MEDICAL SUPPLIES WE SUPPORTED VAXYNG MALTILATERALISM INITIATIVES LIKE THE KOVACS SO THAT ALL COUNTRIES AV SPECIALLY THE LEAST DEVELOP ONES WOULD HAVE ACCESS TO VACSCENES AND AS WE GRADUALLY REBUILT CONFIDENCE IN ONE ANOTHER WE OPENED UP CONTROL CORRIDORS FOR TRAVEL AND TRADE BETWEEN COUNTRIES CRUCIALLY INTERNATIONAL SCIENTIFIC CO OPERATION IN THE FIGHT AGAINST COVERNINETEEN CONTINUE DOCTORS AND SCIENTISTS SHARED INFORMATION ABOUT THE DISEASE AND THE VIRUS STUDYING THEM DEVELOPING TREATMENTS AND TESTING VAKSCENES THIS ENABLE US TO IMPROVE PATIENT CARE AND TO PRODUCE EFFECTIVE ACTENES IN RECORD TIME SOME USING NEW ECKNOWLEDGIES SUCH INTERNATIONAL CO OPERATION AND MALTILATERAL EFFORTS REMAIN ESSENTIAL TO TACKLE THE GLOBAL PANDEMIC COHERENTLY WITH BORDER CLOSURES AND LOCK DOWNS ECONOMIES HAVE ALL TAKEN A DEEP PLUNGE THE LIBLIHOODS OF MILLIONS CAME UNDER IN NORMOUS STRESS ONLY UNPRECEDENTED LEVELS OF EMERGENCY SPENDING AND MAGETARY STIMULUS HAVE KEPT US AFLOAD PROVIDING A LIFE LINE TO COMPANIES WORKERS AND FAMILIES CENTRAL BANKS HAVE PLAYED THEIR PART TO PREVENT FINANCIAL SYSTEMS AND GLOBAL CAPITAL MARKETS FROM SEIZING UP UNLIKEN PREVIOUS CRISES THESE EXTRAORDINARY MEASURES CANNOT BE SUSTAINED INDEFINITELY IN FACT SPENDING PACKAGES ARE ALREADY TAPERING OFF BUT HOPEFULLY AS FASCINATION BECOMES MORE WIDESPREAD AND WE MAKE HEADWAYS SUPPRESSING THE VIOLUS COVET NINETEEN RESTRICTIONS CAN BE PROGRESSIVELY EASE AND ECONOMIES WILL REBOUN THE WORLD BANK AND I MY FORECAST GLOBAL GROWTH TO RECOVER THIS YEAR IT WILL NOT RESTORE OUTPUT TO PRICOVENINETEEN LEVELS BUT TI SOMETHING STILL TO BE THANKFUL FOR NOW WE ARE ENTERING A NEW PHASE THE PANDEMIC HAS EXPOSED BUSINESSES AND JOBS WHICH ARE NOT GOING TO REMAIN VIABLE THEY HAVE TO BE LET GO TO ALLOW NEW GROWTH AND BETTER JOBS TO BE CREATED IN THEIR PLACE HARD DECISIONS HAVE TO BE MADE AND THIS WILL EXASCEBATE EXISTING STRESSES GOVERNMENTS WILL COME UNDER MORE PRESSURE TO ADOPT PROTECTIONISTS AND NATIVEST POSITIONS TO RESUME GROWTH WE MUST LOOK BEYOND RETURNING TO THE STATEST QUA ANTI WE MUST LOOK AHEAD WILEN COUNTRIES GOVERNMENTS AND BUSINESSES MUST COLLABORATE TO TACT NEW MARKETS AND DEVELOP NOVELT ACKNOWLEDGES EXTERNALLY COUNTRIES NEED TO STRENGTHEN THE FRAMEWORK FOR INTERNATIONAL CO OPERATION AS AN IMMEDIATE TASK COUNTRIES SHOULD COLLABORATE TO DEVELOP A STANDARDIZE ROBUST SYSTEM TO VERIFY THE AUTHENTICITY OF TESTS AND VACCINATIONS THIS IS ESSENTIAL TO REOPEN B ORDERS AND RESUME INTERNATIONAL TRAVEL IN THE LONGER TERM COUNTRIES SHOULD WORK TOGETHER TO UPDATE AND STRENGTHENINTS TERNATIONAL INSTITUTIONS LIKE THE W TEOL AND CREATE NEW RULES TO GOVERN AND FOSTER NOVEL FORMS OF ECONOMIC ACTIVITY FOR EXAMPLE TO SUSTAIN THE GROWTH OF THE DIGITAL ECONOMY AND FACILITATE SAFE SECURE AND EFFICIENT CROSS BORDER E PAYMENTS AND DATA FLOWS WE HAVE TO DEVELOP NEW E TRADE REGULATIONS SINGAPOLE HAS CONCLUDED DIGITAL ECONOMY AGREEMENTS WITH LIKE MINDED COUNTRIES LIKE AUSTRALIA CHILLE AND NEW ZEALAN WE HOPE THAT THIS IS ONLY THE BEGINNING WE ENCOURAGE ALL COUNTRIES TO COME TOGETHER TO SHAPE AND GROW THE DIGITAL ECONOMY GLOBLY THUSSIGNING OF THE REGINAL COMPREHENSIVE ECONOMIC PARTNERSHIP OR THE ARLC E P LAST TIA BY FIFTEEN COUNTRIES IN ASIA WAS ALSO A MAJOR COLLECTIVE COMMITMENT TO TRADE AN ECONOMIC INTEGRATION AMIDST THE PANDEMIC THE ARC P WILL BROADEN TRAN OPEN AT MARKETS IN EAST AND SOUTH EASTATIA AND AUSTRALASIA AND HOPEFULLY PREVENT THE PUSH FOR RASILIANTS AND SELF RELIANCE FROM GOING TOO FAR WHILE DEALING WITH THE AFTERMATH OF COVET NINETEEN WE MUST NOT LOSE SIGHT OF OTHER LONG TERM CHALLENGES THAT AFFECT ALL OF US ONE MAJOR PROBLEM IS CLIMATE CHANGE TWENTY TWENTY WAS THE WORLD'S HOTTESEER ON RECORD EXTREME WEATHER EVENTS HAVE BECOME MUCH MORE FREQUENT LAST YIAR CARBON EMISSIONS WENT DOWN BUT ONLY BECAUSE OF COVET NINETEEN OTHERWISE THE TREN HAS BEEN INEXORABLY UPWARDS CLIMATE CHANGE IS CLEARLY EX CELERATING DANGEROUSLY AND IT IS LATE IN THE DAY BUT IF COUNTRIES ACT NOW AND IN CONCERT HUMAN KIND CAN STILL HOPE TO AVERT A CATASTROPHE WE ALL KNOW WHAT WE NEED TO DO WITHIN INDIVIDUAL COUNTRIES TO MUSTER SUPPORT FOR POLICIES AND MEASURES THAT WILL SLOW THE CHANGES AND LIMIT GLOBLE WARNING COLLECTIVELY TO SET HIGHER COMMON STANDARDS AND HOLD ONE ANOTHER TO OUR MUTUAL COMMITMENTS WHETHER IT'S TIGHTENING EMISSION RULES FASING OUT FOSSILFIELD SUBSIDIES OR PROMOTING RENEWABLE ENERGY WE CAN TAKE SOME COMFORT THAT COUNTRIES ARE NOW TAKING CLIMATE CHANGE MORE SERIOUSLY THE US HAS REJOINED THE PARIS AGREEMENT CHINA HAS ANNOUNCED A ZERO EMISSIONS TARGET BY TWENTY SIXLY BUT MUCH MORE STILL NEEDS TO BE DONE GOING BEYOND OUR PARIS COMMITMENTS OTHERWISE WE RISK GRAVE CONSEQUENCES IN THE NOT TOO DISTANT FUTURE EVEN WITHIN OUR OWN LIFETIMES TO TACKLE THESE CHALLENGES COVET NINETEEN ECONOMIC RECOVERY AND CLIMATE CHANGE GLOBA CO OPERATION IS ESSENTIAL BUT GETTING COUNTRIES TO WORK TOGETHER IS NOT SIMPLY A MATTER OF NURTURING AND SHOWING GOODWILL THE INTERNATIONAL ORDER MUST BE UNDERPINNED BY STABLE GREAT POWER RELATIONS BIG COUNTRIES NATURALLY JOSTLE AND COMPETE WITH ONE ANOTHER FOR INFLUENCE AND POWER BUT THEY ALSO NEED TO WORK WITH ONE ANOTHER THROUGH ESTABLISH AN ACCEPTED RULES AND NORMS ON ISSUES WHICH AFFECT US ALL BE IT PANDAMICS ECONOMIC CO OPERATION OR CLIMATE CHANGE RECENT YEARS HAVE WITNESSED GROWING FRICTION AND DISTRUST RATHER THAN CO OPERATION AND CONFIDENCE BUILDING AMONG MAJOR POWERS THE MOST WORRYING TREND I S THE W S CHINA RELATIONS THIS REMAINS THE MOST IMPORTANT BYLATURAL RELATIONSHIP FOR THE WORLD IN THE YEARS AHEAD OVER THE LAST FOUR YEARS TENSIONS BETWEEN THE U S AND CHINA HAVE INTENSIFIED SHARPLY BOTH POWERS HAVE ADOPTED MORE SERTIVE AND UNCOMPROMISING POSTURES THE U S NOW SEES CHINAS AS STRATEGIC RIVAL AND CHALLENGER TO ITS PRE EMINENT POSITION AND CHINA IS VIGOROUSLY ASSERTING WHAT IT CONSIDERS ITS RIGHTFUL PLACE IN THE WORLD ON BOTH SIDES DOMESTIC PRESSURES TO HARDEN THEIR EXTERNAL POSITIONS ARE CONSIDERABLE AND MODERATE VOICES HAVE BEEN MARGINALIZE GIVEN THE ENORMOUS STAKES DIFFICULT AS IT WILL BE IT CANNOT POSSIBLY BE TOO LATE FOR THE U S AND CHINA TO RESET THE TONE OF THEIR INTERACTIONS AND AVERT A CLASH BETWEEN THEM WHICH WILL BECOME A GENERATIONAL TWILIGHT STRUGGLE THE NEW US ADMINISTRATION IS AN OPPORTUNITY TO STEER THE RELATIONSHIP TOWARDS SAFER WATERS AMID PRESIDENT BIDONS MANY URGENT PREOCCUPATIONS THE U S CHINA RELATIONSHIP SHOULD BECOME A KEY STRATEGIC PRORITY TO BILD A STABLE INTERNATIONAL ORDER REGULAR CONSTRUCTIVE DIALOGUE IS CRITICAL IAM THUS HAPPY TO SEE MANY DISTINGUISHED PARTICIPANTS TAKING PART IN THE DAVOSA GENDER WEEK THE WORLD ECONOMIC FORUM PLAYS AN IMPORTANT RULE PROMOTING DIALOGUE BRINGING TOGETHER LEADERS IN GOVERNMENT INDUSTRY AND CIVIL SOCIETY IT'S A FORUM WHERE LEADERS FROM COUNTRIES LARGE AND SMALLER LIKE CAN SPEAK AND BE HEARD AND THIS IS WHY WHEN PROFESSOR SHROB ASK ME WHETHER SINGO POET HOSTESS SPECIAL ANNUAL MEETING OF THE W E F I AGREED IT WAS NOT A DECISION LIGHTLY TAKEN BUT WE ARE HAPPY TO MAKE A MODEST CONTRIBUTION TO THE GLOBAL DISCUSSION AS THE HOST COUNTRY WE WILL WORK WITH A W E F TO ENSURE THE HEALTH AND SAFETY OF ALL I WELCOME ALL OF YOU TO SING A POR IN MAY SO THAT WE CAN TAKE THESE DISCUSSIONS FORWARD AND FORGE A NEW PARTH AHEAD TOGETHER THANK YOU ``` As per the first two trials, the parts in bold indicate the problematic areas. The speech had a number of technical terms which clearly tripped up the model, as did the PM's pronunciation of some words. A lot more cleaning up is required this time. But this took all of 2 minute 4s. Just think about the amount of time one could save from manually transcribing the speech. * * * The Wav2Vec2 transcripts aren't perfect by any means. The lack of punctuation could be an issue for some use cases, and the management of audio clips would require additional resources. But all things considered, I would say the Wav2Vec2 model opens up exciting new possibilities in NLP, beyond just fast and accurate transcripts. I'm particularly excited by the prospects of using the transcripts as inputs for translation, sentiment analysis, or summaries via other transformer models. This could get really interesting. As always, if you spot mistakes in this or any of my earlier posts, ping me at: * Twitter: [Chua Chin Hon](https://medium.com/u/b3d8090c0aee?source=post_page-----aad7f2e1d0a0----------------------) * LinkedIn: [www.linkedin.com/in/chuachinhon](http://www.linkedin.com/in/chuachinhon) The repo for this post, containing the data and notebooks for the charts, can be found [here](https://github.com/chuachinhon/wav2vec2_transformers). ## Register Now to Continue Reading ### Unlock Towards Data Science with a Free Account All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account * * * ![](../images/body-d56b810b4697a0fbf983.png) Written By ## [Chua Chin Hon](https://towardsdatascience.com/author/chuachinhon/) [See all from Chua Chin Hon](https://towardsdatascience.com/author/chuachinhon/) [AI](https://towardsdatascience.com/tag/ai/)[Data Science](https://towardsdatascience.com/tag/data-science/)[Hugging Face](https://towardsdatascience.com/tag/hugging-face/)[Machine Learning](https://towardsdatascience.com/tag/machine-learning/)[NLP](https://towardsdatascience.com/tag/nlp/) Share This Article * [](https://www.facebook.com/sharer/sharer.php?u=https%3A%2F%2Ftowardsdatascience.com%2Ftranscribing-poetry-and-speeches-with-wav2vec2-2658c6510f06%2F&title=Transcribing%20Poetry%20And%20Speeches%20With%20Wav2Vec2) * [](https://www.linkedin.com/shareArticle?mini=true&url=https%3A%2F%2Ftowardsdatascience.com%2Ftranscribing-poetry-and-speeches-with-wav2vec2-2658c6510f06%2F&title=Transcribing%20Poetry%20And%20Speeches%20With%20Wav2Vec2) * [](https://x.com/share?url=https%3A%2F%2Ftowardsdatascience.com%2Ftranscribing-poetry-and-speeches-with-wav2vec2-2658c6510f06%2F&text=Transcribing%20Poetry%20And%20Speeches%20With%20Wav2Vec2) Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. [Write for TDS](https://towardsdatascience.com/submissions/) ## Related Articles * [![](../images/004.gif)](https://towardsdatascience.com/the-arti-canon-neural-text-generation-2a8f032c2a68/) ## [The Arti Canon: Neural Text Generation](https://towardsdatascience.com/the-arti-canon-neural-text-generation-2a8f032c2a68/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) Building a computer generated text based on classic Buddhist writings. [Jake Grigsby](https://towardsdatascience.com/author/__jakegrigsby__/) October 22, 201812 min read * [![](../images/005.jpeg)](https://towardsdatascience.com/the-seven-stages-of-mlops-maturity-ccb029530f2/) ## [The Seven Stages of MLOps Maturity](https://towardsdatascience.com/the-seven-stages-of-mlops-maturity-ccb029530f2/) [Artificial Intelligence](https://towardsdatascience.com/category/artificial-intelligence/) A practical guide to building critical MLOps capabilities that maximize data science ROI [Josh Poduska](https://towardsdatascience.com/author/josh_domino/) August 25, 20228 min read * [![](../images/body-63beb50f664a055ed739.jpg)](https://towardsdatascience.com/managing-data-as-a-data-engineer-part-3-key-principles-lessons-learnt-920d9ebead1d/) ## [Managing Data as a Data Engineer - Part 3: Key Principles & Lessons Learnt](https://towardsdatascience.com/managing-data-as-a-data-engineer-part-3-key-principles-lessons-learnt-920d9ebead1d/) [Data Science](https://towardsdatascience.com/category/data-science/) In the last two articles, we have looked at how users view data and how data changes over time. In this part of the article, I am going to... [Loh Meng Xin](https://towardsdatascience.com/author/lohmengxin/) February 11, 20218 min read * [![](../images/007.png)](https://towardsdatascience.com/application-of-algorithms-for-natural-language-processing-in-it-monitoring-with-python-libraries-57eddc45c4ac/) ## [Application of algorithms for natural language processing in IT-monitoring with Python libraries](https://towardsdatascience.com/application-of-algorithms-for-natural-language-processing-in-it-monitoring-with-python-libraries-57eddc45c4ac/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) As applied to systems for monitoring of IT infrastructure and business processes, NLP algorithms can be used to solve problems of text... [Nick Gan](https://towardsdatascience.com/author/nikolay-gan/) August 24, 202113 min read * [![](../images/008.jpg)](https://towardsdatascience.com/how-to-get-a-technical-internship-a-detailed-guide-ecd4e1b68154/) ## [How to Get a Technical Internship - A Detailed Guide](https://towardsdatascience.com/how-to-get-a-technical-internship-a-detailed-guide-ecd4e1b68154/) [Programming](https://towardsdatascience.com/category/programming/) Familiarize yourself with the three-step process for navigating the tech landscape. [Murtaza Ali](https://towardsdatascience.com/author/murtaza5152-ali/) April 8, 202212 min read * [![](../images/009.png)](https://towardsdatascience.com/3-ways-to-deal-with-heteroskedasticity-in-time-series-831f6499e688/) ## [3 Ways to Deal with Heteroskedasticity in Time Series](https://towardsdatascience.com/3-ways-to-deal-with-heteroskedasticity-in-time-series-831f6499e688/) [Machine Learning](https://towardsdatascience.com/category/artificial-intelligence/machine-learning/) How to stabilize the variance of time series and improve forecasting performance [Vitor Cerqueira](https://towardsdatascience.com/author/vcerq/) December 30, 20226 min read * [![](../images/010.png)](https://towardsdatascience.com/model-predictive-control-basics/) ## [Model Predictive Control Basics](https://towardsdatascience.com/model-predictive-control-basics/) [Data Science](https://towardsdatascience.com/category/data-science/) A hands-on tutorial with Python and CasADi [Willem Esterhuizen](https://towardsdatascience.com/author/willem.esterhuizen/) August 12, 20258 min read * [![](../images/011.png)](https://towardsdatascience.com/omop-datashield-a-perfect-match-to-elevate-privacy-enhancing-healthcare-analytics-041b425d532a/) ## [OMOP & DataSHIELD: A perfect match to elevate privacy-enhancing healthcare analytics?](https://towardsdatascience.com/omop-datashield-a-perfect-match-to-elevate-privacy-enhancing-healthcare-analytics-041b425d532a/) [Data Science](https://towardsdatascience.com/category/data-science/) Exploring synergies between DataSHIELD and OHDSI/OMOP for collaborative healthcare analytics [Eric Boernert](https://towardsdatascience.com/author/eric.boernert/) July 3, 20247 min read * [![](../images/012.png)](https://towardsdatascience.com/getting-started-with-task-groups-in-airflow-2-0-d85b7517767e/) ## [Getting started with Task Groups in Airflow 2.0](https://towardsdatascience.com/getting-started-with-task-groups-in-airflow-2-0-d85b7517767e/) [Data Science](https://towardsdatascience.com/category/data-science/) A simple pipeline with two groups of tasks, using the @taskgroup decorator of the TaskFlow API from Airflow 2. [Pedro Madruga](https://towardsdatascience.com/author/pmadruga/) August 24, 20213 min read * [YouTube](https://www.youtube.com/c/TowardsDataScience) * [X](https://x.com/TDataScience) * [LinkedIn](https://www.linkedin.com/company/towards-data-science/) * [Threads](https://www.threads.net/@towardsdatascience) * [Bluesky](https://bsky.app/profile/towardsdatascience.com) [](https://towardsdatascience.com/) Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our Newsletter[Write For TDS](https://towardsdatascience.com/submissions/)•[About](https://towardsdatascience.com/about-towards-data-science-d691af11cc2f/)•[Advertise](https://contact.towardsdatascience.com/advertise-with-towards-data-science)•[Privacy Policy](https://towardsdatascience.com/privacy-policy/)•[Terms of Use](https://towardsdatascience.com/website-terms-of-use/) © Insight Media Group, LLC 2026•[The New Stack](https://thenewstack.io)•[roadmap.sh](https://roadmap.sh)•[Creator Network](https://insightmediagroup.io/creator-network) #### Recommended Articles Close * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607639/image?etag=1781250125848&width=500&height=500) Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1 ](https://towardsdatascience.com/javascript-fatigue-you-dont-need-js-to-build-chatgpt/ "Javascript Fatigue: HTMX Is All You Need to Build ChatGPT — Part 1") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/607930/image?etag=1782124566054&width=500&height=500) The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs ](https://towardsdatascience.com/subset-sum-problem-solved-in-linear-time-for-dense-enough-inputs/ "The Subset Sum Problem Solved in Linear Time for Dense Enough Inputs") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/3660/image?etag=1780176246070&width=500&height=500) Does Your Company Have a Data Strategy? ](https://towardsdatascience.com/does-your-company-have-a-data-strategy-e8944dbec4ae/ "Does Your Company Have a Data Strategy?") * [ ![](https://h030.towardsdatascience.com/rest/v2/contentStores/d7d3a4ab-7caf-4383-bbd2-fbf7fb4f1787/items/4970/image?etag=1780565225463&width=500&height=500) Productivity Tips, Data Career Insights, and Other Recent Must-Reads ](https://towardsdatascience.com/productivity-tips-data-career-insights-and-other-recent-must-reads-5c47eb7a2fe0/ "Productivity Tips, Data Career Insights, and Other Recent Must-Reads")