保留正文与补采附件。源页面当前未重新核验；具体限制见交付清单。

Revisit consent button We value your privacy We use cookies to enhance your browsing experience, serve personalised ads or content, and analyse our traffic. By clicking "Accept All", you consent to our use of cookies. Customise Reject All Accept All Customise Consent Preferences We use cookies to help you navigate efficiently and perform certain functions. You will find detailed information about all cookies under each consent category below. The cookies that are categorised as "Necessary" are stored on your browser as they are essential for enabling the basic functionalities of the site. ... Show more NecessaryAlways Active Necessary cookies are required to enable the basic features of this site, such as providing secure log-in or adjusting your consent preferences. These cookies do not store any personally identifiable data.

Functional Functional cookies help perform certain functionalities like sharing the content of the website on social media platforms, collecting feedback, and other third-party features.

Analytics Analytical cookies are used to understand how visitors interact with the website. These cookies help provide information on metrics such as the number of visitors, bounce rate, traffic source, etc.

Performance Performance cookies are used to understand and analyse the key performance indexes of the website which helps in delivering a better user experience for the visitors.

Advertisement Advertisement cookies are used to provide visitors with customised advertisements based on the pages you visited previously and to analyse the effectiveness of the ad campaigns.

Uncategorised Other uncategorised cookies are those that are being analysed and have not been classified into a category as yet. No cookies to display. Reject All Save My Preferences Accept All Toggle SearchToggle Mobile Navigation Publish AI, ML & data-science insights to a global community of data professionals. Sign inSubmit an Article LatestEditor's PicksNewsletterShipAI

Write For TDS LinkedInXToggle Search Data Science

Beautiful Boxplots With Statistical Significance Annotation

Super short tutorial for boxplots with significance annotation in Python Serafeim Loukas, PhD July 9, 20206 min read

Introduction & Motivation

I always remember myself reading some nice scientific publications where the authors would have created some nice boxplots with statistical annotations. In most of these cases, a statistical test had been used to determine whether there was a statistically significant difference in the mean value of a specific feature between different groups. I have now managed to create some custom python code to do exactly this: produce beautiful boxplots with statistical annotations integrated. In this short article, I just show how to create such beautiful boxplots in Python.

The dataset

We will use the Iris Dataset as we have done in all my previous posts. The dataset contains four features (length and width of sepals and petals) of 50 samples of three species of Iris (Iris setosa , Iris virginica and Iris versicolor). The dataset is often used in data mining, classification and clustering examples and to test algorithms. For reference, here are pictures of the three flowers species : Learn this step by step with the interactive AI and Data Scientist, Computer Science and Machine Learning roadmaps. Figure made by the author. Figure made by the author. For this short tutorial, we will be only using 2 out of the 3 classes i.e. the setosa and versicolor classes. This is done only for the sake of simplicity.

Working example in Python

Step 1 : Let's load the data and sub-select the desired 2 flower classes : text

from sklearn.datasets import load_iris
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
import numpy as np

text

# Load the Iris dataset
X = load_iris().data
y = load_iris().target
feature_names = load_iris().feature_names
classes_names = load_iris().target_names

text

# Use only 2 classes for this example
mask = y!=2
X,y  = X[mask,:], y[mask]

text

# Get the remained class names
classes_names[[0,1]]
# array(['setosa', 'versicolor'], dtype='<U10')

Step 2 :We have now selected all the samples for the 2 classes: setosa & versicolor flower classes**.** We will put the data into a panda dataframe to make our lives easier: text

df = pd.DataFrame(X,columns=feature_names)
df['Group'] = [i for i in y]
df_long = pd.melt(df, 'Group', var_name='Feature', value_name='Value') # this is needed for the boxplots later on

text

df.head()

Step 3: Let's inspect the dataframe: As we can see, we have 4 features and the last column denote the group membership of the corresponding sample.

The statistical tests

Step 4 : Now it's time to do the statistical tests. We will use a two-sample t-test (since our group are independent) to test if the mean value of any of these 4 features (i.e. sepal length, sepal width, petal length, petal width) is statistically different between the 2 groups of flowers (setosa and versicolor). text

#* Statistical tests for differences in the features across groups
from scipy import stats
all_t = list()
all_p = list()
for case in range(len(feature_names)):
    sub_df = df_long[df_long.Feature == feature_names[case]]
    g1 = sub_df[sub_df['Group'] == 0]['Value'].values
    g2 = sub_df[sub_df['Group'] == 1]['Value'].values
    t, p = stats.ttest_ind(g1, g2)
    all_t.append(t)
    all_p.append(p)

To do the statistical test we just used: text

t, p = stats.ttest_ind(g1, g2)

Here we compare the mean of g1 (group 1: setosa) to the mean of g2 (group 2: versicolor) and we do that for all 4 features (using the for loop).

But how can we know if the mean of g1 (group 1: setosa) was significantly greater or smaller than the mean of g2 (group 2: versicolor)? For this, we need to look at the t-values. text

print(all_t)
[-10.52098626754911, 9.454975848128596, -39.492719391538095, -34.08034154357719]

text

print(feature_names)
['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)']

Interpretation:

Reminder : _feature names = ['sepal length (cm)', 'sepal width (cm)', 'petal length (cm)', 'petal width (cm)'].

Step 5 : Check the t-test results text

print(np.count_nonzero(np.array(feature_names)[np.array(all_p) < 0.05]))
# 4

Interpretation: We can see that there is a statistically significant difference in all 4 features between setosa and versicolor classes.

Step 6 : Here is the magic. Let's create some beautiful boxplots and annotate them with the estimated statistical significance. text

# renaming so that class 0 will appear as setosa and class 1 as versicolor
df_long.loc[df_long.Group==0, 'Group'] = classes_names[0]
df_long.loc[df_long.Group==1, 'Group'] = classes_names[1]

text

# Boxplots
fig, axes = plt.subplots(2,2, figsize=(14,10), dpi=100)
axes = axes.flatten()

text

for idx, feature in enumerate(feature_names):
    ax = sns.boxplot(x="Feature", hue="Group", y="Value", data = df_long[df_long.Feature == feature], linewidth=2, showmeans=True, meanprops={"marker":"*","markerfacecolor":"white", "markeredgecolor":"black"}, ax=axes[idx])
    #* tick params
    axes[idx].set_xticklabels([str(feature)], rotation=0)
    axes[idx].set(xlabel=None)
    axes[idx].set(ylabel=None)
    axes[idx].grid(alpha=0.5)
    axes[idx].legend(loc="lower right", prop={'size': 11})

    #*set edge color = black
    for b in range(len(ax.artists)):
        ax.artists[b].set_edgecolor('black')
        ax.artists[b].set_alpha(0.8)

    #* statistical tests
    x1, x2 = -0.20, 0.20
    y, h, col = df_long[df_long.Feature == feature]["Value"].max()+1, 2, 'k'
    axes[idx].plot([x1, x1, x2, x2], [y, y+h, y+h, y], lw=1.5, c=col)
    axes[idx].text((x1+x2)*.5, y+h, "statistically significant", ha='center', va='bottom', color=col)

text

fig.suptitle("Significant feature differences between setosa and versicolor classes/groups", size=14, y=0.93)
plt.show()

Figure produced by the author. Figure produced by the author.

Conclusions

As we can see from the statistical tests, we can conclude that only the mean value of sepal widthof group 1 (setosa) was statistically greater than the mean value of sepal widthof group 2 (versicolor). On the other hand, the mean value of sepal _length, petal length and petal width of the Setosa group_ was statistically smaller than the mean value of the Versicolor group. These observations can be also verified by looking at boxplots. That's all folks! Hope you liked this article!

Stay tuned & support this effort

If you liked and found this article useful, follow me to be able to see all my new posts. Questions? Post them as a comment and I will reply as soon as possible.

NEW : We offer our expertise through a platform, the "Data Science Hub" on Patreon (https://www.patreon.com/TheDataScienceHub). This hub is our way of providing you with bespoke consulting services and comprehensive responses to all your inquiries , ranging from Machine Learning to strategic data analytics planning.

Latest posts

Time-Series Forecasting: Predicting Stock Prices Using Facebook's Prophet Model ROC Curve Explained using a COVID-19 hypothetical example: Binary & Multi-Class Classification... Support Vector Machines (SVM) clearly explained: A python tutorial for classification problems... PCA clearly explained - How, when, why to use it and feature importance: A guide in Python Everything you need to know about Min-Max normalization in Python How Scikit-Learn's StandardScaler works

Get in touch with me

Register Now to Continue Reading

Unlock Towards Data Science with a Free Account

All articles on Towards Data Science are available to registered users. Create a free account today to access this article and thousands more. Register for freeSign in to your account

Written By

Serafeim Loukas, PhD

See all from Serafeim Loukas, PhD Box PlotData ScienceMachine LearningProgrammingVisualization Share This Article

Towards Data Science is a community publication. Submit your insights to reach our global audience and earn through the TDS Author Payment Program. Write for TDS

Related Articles

Type-Hinting DataFrames for Static Analysis and Runtime Validation

Programming How StaticFrame Enables Comprehensive DataFrame Type Hints Christopher Ariza November 16, 202310 min read

How Does It Feel to Make a Career Change to Become a Data Scientist?

Artificial Intelligence And why I'm glad I made it Data science has gained tremendous popularity in recent... Soner Yıldırım May 1, 20215 min read

4 Advice for Every Data Enthusiast- My Professional Experience

Data Science How different the classroom and industrial employment experience Cornellius Wijaya June 25, 20218 min read

Implementing a Decision Tree From Scratch

Artificial Intelligence Recursively refine your understanding by implementing a decision tree with just Python and NumPy Marvin Lanhenke February 2, 20227 min read

A Machine Learning Approach to Predicting MGMT Methylation Status in Glioblastoma Patients

Machine Learning Radiomics in Oncology Jarrett Evans July 26, 20235 min read

Modeling Urban Walking Risk Using Spatial-Temporal Machine Learning

Machine Learning Estimating neighborhood-level pedestrian risk from real-world incident data Aneesh Patil January 28, 202610 min read

Now is the best time to be a Data Scientist or a Data Steward in Europe

Data Science Why does the European Union plan to train over half a million data scientists and data stewards Ioana Spanache, PhD October 9, 20209 min read

CIFAR 100: Transfer Learning using EfficientNet

Artificial Intelligence Transfer learning using state-of-the-art EfficientNet-B0 Chetna Khanna March 30, 202110 min read

Attempting to Predict the Result of an NBA Game before you could Finish Watching the First Half

Machine Learning Using the Support Vector Machines algorithm Christophe Brown January 3, 202215 min read

Your home for data science and AI. The world's leading publication for data science, data analytics, data engineering, machine learning, and artificial intelligence professionals. Subscribe to Our NewsletterWrite For TDS•About•Advertise•Privacy Policy•Terms of Use © Insight Media Group, LLC 2026•The New Stack•roadmap.sh•Creator Network

Recommended Articles

Close