{
    "cells": [
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "# Lab 5: Visualization\n",
       "\n",
       "In this lab we will introduce two popular visualization tools commonly used in python. We will use data from the US Federal Election Commission. The lab will contain live QA and coding that will give you a few minutes to complete.\n",
       "\n",
       "One of the most popular plotting libraries today is [Matplotlib](https://matplotlib.org/). According to the documentation, \"Matplotlib tries to make easy things easy and hard things possible.\" For the most basic plots, line graphs, single bar plot's etc., matplotlib has functions for just these things.\n",
       "\n",
       "As plots get more complex, many data scientists use [Seaborn](https://seaborn.pydata.org/), a wrapper around matplotlib that improves on its theme, and has convenience functions for more complex plots that tend to be common. You can see a ton of examples in Seaborn's [example gallery](https://seaborn.pydata.org/examples/index.html).\n",
       "\n",
       "With Matplotlib and Seaborn you often have to manually move labels, adjust axes, add data labels, etc. An alternative is declarative visualization. Akin to the difference between imperative approaches like Pandas vs declarative approaches like SQL for manipulating tabular data, declarative visualizations allow users to specify what visualization they want over their data by describing the connection between data and visual encodings.\n",
       "\n",
       "In this lab we will use a python declarative visualization library, [Altair](https://altair-viz.github.io/), itself built atop the popular declarative visualization tool [Vega](https://vega.github.io/vega/), to demonstrate how to get some effective visualizations in relatively few lines of code. \n",
       "\n",
       "While it is difficult to be comprehensive in a short lecture, the idea behind this excercize is to give an introduction to the flavor of some of these tools so that you can try out things on your own later. As usual, useful resource include the documentation and StackOverflow."
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "## Data Overview\n",
       "Let's briefly describe the data before starting our visualizations."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "import pandas as pd\n",
       "cands = pd.read_csv(\"data/cand_summary.txt\", delimiter=\"|\")\n",
       "cands[\"CAND_OFFICE\"] = cands.CAND_ID.str[:1]\n",
       "pacs = pd.read_csv(\"data/pac_summary.txt\", delimiter=\"|\")\n",
       "dist_pop = pd.read_csv(\"data/dist_pop.txt\", delimiter=\"|\")"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "The ``cands`` table contains information about candidates in each election year. We are primarily interested in visualization election funding information, so let us remove unneeded information."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "# Removing US territories\n",
       "cands = cands[~cands.CAND_OFFICE_ST.isin([\"AS\", \"GU\", \"MP\", \"US\", \"DC\", \"MH\", \"PR\", \"VI\"])]\n",
       "\n",
       "# Add a column for CAND_OFFICE ('P', 'H' or 'S', for President, House, and Senate respectively)\n",
       "cands[\"CAND_OFFICE\"] = cands.CAND_ID.str[:1]\n",
       "\n",
       "# We keep candidate id, name, state, office, year, party, total funding, funding from individual contributions.\n",
       "cands = cands[['CAND_ID', 'CAND_NAME', 'CAND_OFFICE_ST', 'CAND_OFFICE', 'ELECTION_CYCLE_YR', 'PTY_AFFILIATION', 'TTL_RECEIPTS', 'TTL_INDIV_CONTRIB']]\n",
       "cands"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "The `pacs` table contains information about Political Action Committee which are defined as:\n",
       "> (in the US) an organization that raises money privately to influence elections or legislation, especially at the federal level.\n",
       "\n",
       "Let's again remove unneeded information."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "# We keep PAC id, name, type, election year, and total funding.\n",
       "pacs = pacs[[\"CMTE_ID\", \"CMTE_NM\", \"CMTE_TP\", \"ELECTION_CYCLE_YR\", \"TTL_RECEIPTS\"]]"
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "pacs.info()"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "Finally the `dist_pop` table contains the population US districts."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "dist_pop"
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "dist_pop.info()"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "## Part 1: Matplotlib + Seaborn"
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "import seaborn as sns\n",
       "import matplotlib.pyplot as plt"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "### Viz: Funding of House Canditates in 2018.\n",
       "In this first visualization task, we want to compare individual contributions and total funding for house canditates in 2018. We will walk you through this task, but the later ones will be more open-ended.\n",
       "We can get the 2018 house candidates as follows, sorted by toal fundraising receipts."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "house_2018 = cands[(cands.ELECTION_CYCLE_YR == 2018) &\n",
       "                   (cands.CAND_OFFICE == \"H\") &\n",
       "                   (cands.PTY_AFFILIATION.isin([\"REP\", \"DEM\"]))] \\\n",
       "            .sort_values(\"TTL_RECEIPTS\", ascending=False)\n",
       "house_2018"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "Now let's start making some plots. We'll start by looking at the ratio of individual contributions to total receipts for all 2018 House candidates using a scatterplot. Here is first attempt:"
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "plt.scatter(house_2018.TTL_RECEIPTS, house_2018.TTL_INDIV_CONTRIB)\n",
       "\n",
       "# Increase the size of the figure for visibility\n",
       "plt.gcf().set_size_inches(10,6) # gcf() means get current figure."
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "It's worth thinking about what is going on here. Matplotlib has a default **figure** that starts with one **axes** object. When using the ``plt.`` functions we edit the default axis or the default figure (depending on what property we are manipulating). Alternatively we can fetch the axis or figure object and operate on that directly. This is useful if you want to create multiple plots in the same figure. An example of multiple plots on the same figure is [here](https://matplotlib.org/devdocs/gallery/subplots_axes_and_figures/subplots_demo.html).\n",
       "\n",
       "There are a couple of issues with the figure above: It has no title, no axis labels, and our area of interest is pretty hard to see. Let's fix that."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "# Get the current axes object\n",
       "ax = plt.gca()\n",
       "\n",
       "# Create Scatterplot\n",
       "ax.scatter(house_2018.TTL_RECEIPTS, house_2018.TTL_INDIV_CONTRIB)\n",
       "\n",
       "# Set the title\n",
       "ax.set_title(\"2018 House Individual Contributions vs Total Fundraising\")\n",
       "\n",
       "# Set the axis labels\n",
       "ax.set_xlabel(\"TTL_RECEIPTS ($)\")\n",
       "ax.set_ylabel(\"TTL_INDIV_CONTRIB ($)\")\n",
       "\n",
       "# Adjust the axis limits \n",
       "ax.set_xlim((0, 20000000))\n",
       "ax.set_ylim((0, 20000000))\n",
       "\n",
       "# Increase figure size on the current figure\n",
       "plt.gcf().set_size_inches(10,8)"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "This is looking better, but who are these outliers who receive tons of money but almost none by individuals? \n",
       "\n",
       "The first thing we can do is add multiple series on the same axis to see which party these house candidates belong to. We'll split our data into two sets, one for Republicans and one for Democrats. We will then add both scatterplots to the same axes object and plot them over each other.\n",
       "\n",
       "We will also add a line y=x to better see how much the individual contributions differ from the total funding."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "# Get the current figure and axes\n",
       "ax = plt.gca()\n",
       "\n",
       "# Split rep and dem.\n",
       "rep_2018 = house_2018[house_2018.PTY_AFFILIATION ==\"REP\"]\n",
       "dem_2018 = house_2018[house_2018.PTY_AFFILIATION ==\"DEM\"]\n",
       "\n",
       "# TODO: Make scatter plot colored red for republicans.\n",
       "# TODO: Make scatter plot colored blue for democrats.\n",
       "# TODO: Add Add line y=x.\n",
       "\n",
       "\n",
       "# Set the title\n",
       "ax.set_title(\"2018 House Individual Contributions vs Total Fundraising\")\n",
       "\n",
       "# Set the axis labels\n",
       "ax.set_xlabel(\"TTL_RECEIPTS ($)\")\n",
       "ax.set_ylabel(\"TTL_INDIV_CONTRIB ($)\")\n",
       "\n",
       "#Add a legend\n",
       "plt.gcf().legend()\n",
       "\n",
       "#Adjust the axis limits \n",
       "ax.set_xlim((0, 20000000))\n",
       "ax.set_ylim((0, 20000000))\n",
       "\n",
       "#Increase figure size\n",
       "plt.gcf().set_size_inches(10,8)"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "Now we can see that there are a number of politicians raising a ton of money with almost none of it coming from indivudal contributers. But who exactly are those outliers? One thing we could do is to add data labels for some of these points (See the example [here](https://matplotlib.org/3.1.1/tutorials/text/annotations.html)). We'll come back to this later.\n",
       "\n",
       "Even with these simple plots this is starting to get pretty verbose. Matplotlib tends to be a manual process in this way. However, One advantage is the ability to do nearly anything you want. \n",
       "\n",
       "Seaborn is a wrapper around matplotlib that tends to be a bit easier to use and more easily produces cleaner plots without much manipulation. In the next few steps we'll look at how to show how the total amount of money raised changes each election cycle. Fortunately this data goes back to 1980!\n",
       "\n",
       "But first we'll revisit the same plot, to see how we would do the same thing in Seaborn. Using [sns.relplot](https://seaborn.pydata.org/generated/seaborn.relplot.html) documentation, make a scatter plot in which republicans and democrats are colored differently."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "# Set the Seaborn default theme on Matmplotlib\n",
       "sns.set()\n",
       "\n",
       "# TODO: Use sns.relplot to automatically splits the data into separate series and colors them accordingly.\n",
       "\n",
       "\n",
       "# Adjust the axis limits using Matplotlib\n",
       "plt.xlim((0, 20000000))\n",
       "plt.ylim((0, 20000000))\n",
       "\n",
       "# Increase figure size\n",
       "plt.gcf().set_size_inches(10,8)"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "Instead of plotting Democrats and Republicans atop one another, we can also use a \"Facet Grid\" to split out different categories into different plots. Take a look at [sns.FacetGrid](https://seaborn.pydata.org/generated/seaborn.FacetGrid.html) and split the two plots."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "# TODO: Create facet grid object. You'll only need the 'col' and 'hue' option.\n",
       "\n",
       "# TODO: Use map to draw each figure.\n",
       "\n",
       "#Adjust the axis limits \n",
       "plt.xlim((0, 20000000))\n",
       "plt.ylim((0, 20000000))\n",
       "\n",
       "#Increase figure size\n",
       "plt.gcf().set_size_inches(10,8)"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "### Viz: Spending Over Time."
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "In this task, we will look at how campaign contributions change over time using Seaborn."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "# Get house candidates before 2020 (years with complete data).\n",
       "house_cands = cands[(cands.CAND_OFFICE == 'H') & (cands.ELECTION_CYCLE_YR < 2020) & (cands.PTY_AFFILIATION.isin([\"REP\", \"DEM\"]))]"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "Take a look at the documentation for [sns.barplot](https://seaborn.pydata.org/generated/seaborn.barplot.html). Use it to show the evolution of campaign spending over time for both parties combined. \n",
       "\n",
       "Readability guidelines:\n",
       "* Set the figure size to (10, 8).\n",
       "* Rotate the x labels by 45 degree using [plt.xticks](https://matplotlib.org/3.5.0/api/_as_gen/matplotlib.pyplot.xticks.html) "
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "contrib_sum = house_cands.groupby(\"ELECTION_CYCLE_YR\").sum().reset_index()\n",
       "contrib_sum\n",
       "# TODO: evolution of election spending."
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "We can see the rapid increase in spending on elections over the past 40 years. Although the US population has increased by 50% and the inflation rate is about 3x since 1980, the spending on elections far outpaces this.\n",
       "\n",
       "Next, create a grouped bar plot using Seaborn that compares Democrat and Republican spending on **House** Campaigns since 1980. Use [seaborn's grouped barplot example](https://seaborn.pydata.org/examples/grouped_barplot.html) for inspiration. Use the same readability guidelines as above."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "contrib_sum = house_cands.groupby([\"ELECTION_CYCLE_YR\", \"PTY_AFFILIATION\"]).sum().reset_index()\n",
       "# TODO: Grouped bar plot"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "**Q: Since we are tracking data over time, what might be a different plot that could be a useful way to view this data?**\n",
       "\n",
       "Now change the above plot to instead be of the type we just decided. Feel free to use Google or Duckduckgo to find documentation."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "# TODO: Create your plot here"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "### Viz: Sources of Funding"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "Election funding can come from the candidate's own campaign or from PACs. For each year since 2000, we would like to visualize the proportion of funding from campaigns (senate, house, presidential) and from PACs according to their types.\n",
       "\n",
       "Let's take a look at how we would accomplish this. The first thing we have to do is manipulate our data so that it contains what we want. Remember that we want to separate out spending on house, presidential, and Senate elections, as well as different spending on political action committees (PACs) per year. \n",
       "\n",
       "You've already done enough pandas manipulation so we'll just give you the code to gather the data in one dataframe. You don't need to read the details of the code, just read the output. In the next section of this tutorial, we'll see how to avoid most of this manipulation."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "#Sum up and get only the fields we are interested in\n",
       "election_spending = cands[(cands.ELECTION_CYCLE_YR >= 2000) & (cands.ELECTION_CYCLE_YR < 2020)]. \\\n",
       "                    groupby([\"ELECTION_CYCLE_YR\", \"CAND_OFFICE\"])\\\n",
       "                    .sum().reset_index()\n",
       "\n",
       "# We will change the column name to match with the pacs table.\n",
       "election_spending = election_spending[[\"ELECTION_CYCLE_YR\", \"CAND_OFFICE\", \"TTL_RECEIPTS\"]].rename({\"CAND_OFFICE\":\"SPENDING_TYPE\"}, axis=\"columns\")\n",
       "\n",
       "\n",
       "pac_spending = pacs[pacs.ELECTION_CYCLE_YR < 2020].groupby([\"ELECTION_CYCLE_YR\", \"CMTE_TP\"]).sum().reset_index()\n",
       "\n",
       "#Get the top 3 Pacs\n",
       "top_pacs = pac_spending.groupby(\"CMTE_TP\").sum().sort_values(\"TTL_RECEIPTS\", ascending=False).index[:3]\n",
       "pac_spending = pac_spending[pac_spending.CMTE_TP.isin(top_pacs)]\n",
       "pac_spending = pac_spending[[\"ELECTION_CYCLE_YR\", \"CMTE_TP\", \"TTL_RECEIPTS\"]].rename({\"CMTE_TP\":\"SPENDING_TYPE\"}, axis=\"columns\")\n",
       "\n",
       "total_spending = pd.concat([election_spending, pac_spending])\n",
       "\n",
       "\n",
       "#Rename to meaningful spending types\n",
       "spending_type_map = {\"H\":\"House\", \"S\":\"Senate\", \"P\":\"President\", \"Q\":\"Standard PAC\", \"O\":\"Super PAC\", \"Y\":\"Party PAC\"}\n",
       "total_spending.SPENDING_TYPE = total_spending.SPENDING_TYPE.apply(lambda x: spending_type_map[x])\n",
       "\n",
       "# This will create rows for election cycle years, and columns for each distinct spending type.\n",
       "total_spending = total_spending.pivot(index=\"ELECTION_CYCLE_YR\", columns=\"SPENDING_TYPE\", values=\"TTL_RECEIPTS\")\n",
       "\n",
       "total_spending"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "Once your data is in the above format (x-axis represented by index and each stack element as a column), the builtin pandas plot function allows to make stacked bar plots as follows:"
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "total_spending.plot.bar(stacked=True)\n",
       "plt.gcf().set_size_inches(15,10)\n",
       "plt.legend()"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "One interesting thing to see here is that Super PAC did not contribute to elections before 2010. [According to Wikipedia](https://en.wikipedia.org/wiki/Political_action_committee#Super_PACs), they came into existence in 2010 after a court decision.\n",
       "\n",
       "Seaborn allows you to do many more things as shown in the [gallery](https://seaborn.pydata.org/examples/index.html) but, for complex visualizations, using Altair and Vega-lite are likely more convenient."
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "## Part 2: Altair & Vega-Lite\n",
       "\n",
       "With matplotlib alone, even simple plots require a lot of hand-written code to get a presentable plot. Seaborn significantly simplifies matplotlib, but (1) does not easily support advanced techniques like interactivity, (2) gets increasingly tedious for complex visualizations like , (3) often requires some matplotlib style manual adjustments for best readability.\n",
       "\n",
       "Altair further simplifies visualizations. It is to seaborn what SQL is to Pandas. Despite having a higher learning curve, it aims to allow for more sophisticated and better looking visualizations with less code. From the documentation:\n",
       "\n",
       "> The key idea is that you are declaring links between data columns and visual encoding channels, such as the x-axis, y-axis, color, etc. The rest of the plot details are handled automatically. Building on this declarative plotting idea, a surprising range of simple to sophisticated plots and visualizations can be created using a relatively concise grammar.\n",
       "\n",
       "Altair is a python wrapper around Vega-lite, a declarative visualization grammar. "
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "### Basics"
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "import altair as alt\n",
       "from vega_datasets import data\n",
       "from altair import datum\n",
       "alt.renderers.enable('mimetype')\n",
       "alt.data_transformers.disable_max_rows()\n",
       "\n",
       "house_2018 = cands[(cands.ELECTION_CYCLE_YR == 2018) &\n",
       "                   (cands.CAND_OFFICE == \"H\") & (cands.TTL_RECEIPTS < 20000000)]"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "We'll start by revisiting the initial scatterplot. We'll start with the full code then walk through it."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "alt.Chart(house_2018).mark_point().encode(\n",
       "    x=\"TTL_RECEIPTS:Q\",\n",
       "    y=\"TTL_INDIV_CONTRIB:Q\",\n",
       "    color=\"PTY_AFFILIATION:N\",\n",
       ").transform_filter(\n",
       "    (datum.PTY_AFFILIATION == \"REP\") | (datum.PTY_AFFILIATION == \"DEM\")\n",
       ").properties(\n",
       "    title={\n",
       "        \"text\": \"Individual Contrib vs Total Receipts\"\n",
       "    }\n",
       ")"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "There is a bunch going on here so let's break it down a step at a time.\n",
       "\n",
       "The code starts with `alt.Chart(data)` which specifies that you want to draw a figure using the provided data. Underneath, the `Chart` object compiles to `JSON` then get used by the underlying library.\n",
       "\n",
       "Then you specify a `mark` to indicate what the data should look like. Because we are doing a scatter plot, we want individual points, so we use `mark_point()`. We'll see other marks later in the tutorial."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "chart = alt.Chart(house_2018).mark_point()\n",
       "chart"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "This is not very useful as it stacks all points in one. We need to specify more visual elements using the `encode` function. Possible encodings include x-axis, y-axis, color, interactive tooltip, labels on points, etc.\n",
       "\n",
       "Let's start by specifying an x-axis."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "# Q indicates that the variable is quantitative.\n",
       "chart = chart.encode(x=\"TTL_RECEIPTS:Q\")\n",
       "chart"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "The data is now spread through the x-axis. Let's add a y-axis."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "chart = chart.encode(y=\"TTL_INDIV_CONTRIB:Q\")\n",
       "chart"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "We now want to color points by party affiliation. We once again use the `encode()` function."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "# N means nominal: unordered categorical variable.\n",
       "chart = chart.encode(color=\"PTY_AFFILIATION:N\")\n",
       "chart"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "That's a lot of parties; we only want republicans and democrats. Instead of using pandas to filter the dataframe, we can filter individual points using `transform_filter`. When your data is in pandas format already, you should likely just use pandas since you are more familiar with it. But transforms are more general as they operate across data formats. So we show an example here."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "chart = chart.transform_filter((datum.PTY_AFFILIATION == \"REP\") | (datum.PTY_AFFILIATION == \"DEM\"))\n",
       "chart"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "For best practice, let's add a title. We do so by using the `properties` function."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "chart = chart.properties(\n",
       "    title={\n",
       "        \"text\": \"Individual Contrib vs Total Receipts\"\n",
       "    }\n",
       ")\n",
       "chart"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "Note that the readability of the plot is taken care of by the library's default configurations, which are fairly decent in most cases.\n",
       "\n",
       "Let's add a few more features to our plot.\n",
       "\n",
       "Support we wanted to add an y=x line to the plot above. Let's first draw it in isolation. Use [this example from the documentation](https://altair-viz.github.io/gallery/simple_line_chart.html) to draw a single y=x line."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "df = pd.DataFrame({\n",
       "    'X': [0, 20000000],\n",
       "    'Y': [0, 20000000],\n",
       "})\n",
       "\n",
       "line = None\n",
       "# TODO: Draw line. Store the resulting chart in the 'line' variable. You'll need a Chart, a mark, and a few encodings."
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "Altair allow to place multiple plots in the same figure using the `+` operation."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "full_chart = chart + line\n",
       "full_chart"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "Likewise, you may want to split party the figure by party affiliation. You do that by using facets."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "multi_chart = chart.facet(\"PTY_AFFILIATION\")\n",
       "multi_chart"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "Finally, suppose we wanted to interactively identify outliers by hovering over the points with our mouse. Adding interactivity is trivial by adding a `tooltip` to the encoding."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "chart = chart.encode(\n",
       "    tooltip=[\"CAND_NAME\", \"CAND_OFFICE_ST\"] # Specify the columns you want to see.\n",
       ")\n",
       "\n",
       "chart"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "We've now explored a small subset of the things you can do with Altair. So here's a chance for you to play around.\n",
       "\n",
       "### Practice 1: Spending over Time by Party\n",
       "Try recreating the histogram showing spending over time by each party. Make it interactive by showing the exact value for a bar when you hover over it. It doesn't have to be exactly the same as the histogram above, but it should be close. For inspiration, start from [this example in the documentation](https://altair-viz.github.io/gallery/grouped_bar_chart.html). Also feel free to use Google and SO."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "# Use pandas for simple filtering.\n",
       "house_cands = cands[(cands.CAND_OFFICE == 'H') & (cands.ELECTION_CYCLE_YR < 2020) & (cands.PTY_AFFILIATION.isin([\"REP\", \"DEM\"]))]\n",
       "\n",
       "# TODO: Recreate the plot showing spending by party over time without using pandas aggregation."
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "### Practice 2: Share of Receipts by Campaign Type\n",
       "Create a stacked bar chart showing the proportion of the funding that comes from presidential, senate or house campaigns. You do not need to show PAC data here. Start from [this example in the documentation](https://altair-viz.github.io/gallery/stacked_bar_chart.html)."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "all_cands = cands\n",
       "\n",
       "# TODO: Create stacked bar plot showing the funding of all three campaign types."
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "### Map Visualization\n",
       "There are many more things we can do with Altair. Take a look at [Altair's gallery](https://altair-viz.github.io/gallery/index.html) for more examples.\n",
       "\n",
       "Let's say we want to see which states in the last election cycles had Senate races that raised a disproportionate amount of money per capita. Senate elections are every 6 years so looking at data from 2012 to 2018 data ensures that all Senate seats have had an election.\n",
       "\n",
       "We'll create Choropleth map. That is, a map that is shaded relative to some statistic.\n",
       "\n",
       "Let's first compute the per capita receipts in each state. You only need to look at the output; don't worry about the manipulation."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "# Use dist_pop to get state population.\n",
       "state_pop = dist_pop[[\"state\", \"population\"]].groupby(\"state\").sum().reset_index()\n",
       "# Get total receipts per candidate.\n",
       "senate_cands = cands[(cands.ELECTION_CYCLE_YR >= 2012) &\n",
       "                                 (cands.ELECTION_CYCLE_YR < 2018) &\n",
       "                                 (cands.CAND_OFFICE==\"S\")]\n",
       "senate_receipts = senate_cands.groupby(\"CAND_OFFICE_ST\").agg({\"TTL_RECEIPTS\": \"sum\"}).reset_index()\n",
       "\n",
       "# Merge\n",
       "senate_pop = pd.merge(left=senate_funds, right=state_pop, left_on=\"CAND_OFFICE_ST\", right_on=\"state\")\n",
       "\n",
       "# Compute per capita receipts\n",
       "senate_pop[\"PER_CAPITA_RECEIPTS\"] = funds_pop.TTL_RECEIPTS / funds_pop.population\n",
       "\n",
       "# Add state id: needed for map\n",
       "# Mapping of postal code to the state id used in the geographic data\n",
       "states = {'AK': 'Alaska','AL': 'Alabama','AR': 'Arkansas','AS': 'American Samoa','AZ': 'Arizona','CA': 'California','CO': 'Colorado','CT': 'Connecticut','DC': 'District of Columbia','DE': 'Delaware','FL': 'Florida','GA': 'Georgia','GU': 'Guam','HI': 'Hawaii','IA': 'Iowa','ID': 'Idaho','IL': 'Illinois','IN': 'Indiana','KS': 'Kansas','KY': 'Kentucky','LA': 'Louisiana','MA': 'Massachusetts','MD': 'Maryland','ME': 'Maine','MI': 'Michigan','MN': 'Minnesota','MO': 'Missouri','MP': 'Northern Mariana Islands','MS': 'Mississippi','MT': 'Montana','NA': 'National','NC': 'North Carolina','ND': 'North Dakota','NE': 'Nebraska','NH': 'New Hampshire','NJ': 'New Jersey','NM': 'New Mexico','NV': 'Nevada','NY': 'New York','OH': 'Ohio','OK': 'Oklahoma','OR': 'Oregon','PA': 'Pennsylvania','PR': 'Puerto Rico','RI': 'Rhode Island','SC': 'South Carolina','SD': 'South Dakota','TN': 'Tennessee','TX': 'Texas','UT': 'Utah','VA': 'Virginia','VI': 'Virgin Islands','VT': 'Vermont','WA': 'Washington','WI': 'Wisconsin','WV': 'West Virginia','WY': 'Wyoming'}\n",
       "state_to_id = {\"Alabama\":\"1\",\"Alaska\":\"2\",\"Arizona\":\"4\",\"Arkansas\":\"5\",\"California\":\"6\",\"Colorado\":\"8\",\"Connecticut\":\"9\",\"Delaware\":\"10\",\"District of Columbia\":\"11\",\"Florida\":\"12\",\"Georgia\":\"13\",\"Hawaii\":\"15\",\"Idaho\":\"16\",\"Illinois\":\"17\",\"Indiana\":\"18\",\"Iowa\":\"19\",\"Kansas\":\"20\",\"Kentucky\":\"21\",\"Louisiana\":\"22\",\"Maine\":\"23\",\"Maryland\":\"24\",\"Massachusetts\":\"25\",\"Michigan\":\"26\",\"Minnesota\":\"27\",\"Mississippi\":\"28\",\"Missouri\":\"29\",\"Montana\":\"30\",\"Nebraska\":\"31\",\"Nevada\":\"32\",\"New Hampshire\":\"33\",\"New Jersey\":\"34\",\"New Mexico\":\"35\",\"New York\":\"36\",\"North Carolina\":\"37\",\"North Dakota\":\"38\",\"Ohio\":\"39\",\"Oklahoma\":\"40\",\"Oregon\":\"41\",\"Pennsylvania\":\"42\",\"Rhode Island\":\"44\",\"South Carolina\":\"45\",\"South Dakota\":\"46\",\"Tennessee\":\"47\",\"Texas\":\"48\",\"Utah\":\"49\",\"Vermont\":\"50\",\"Virginia\":\"51\",\"Washington\":\"53\",\"West Virginia\":\"54\",\"Wisconsin\":\"55\",\"Wyoming\":\"56\",\"Puerto Rico\":\"72\"}\n",
       "\n",
       "# Set the state name in our data instead of the postal code\n",
       "senate_pop[\"STATE_ID\"] = senate_pop.CAND_OFFICE_ST.apply(lambda x: state_to_id[states[x]])\n",
       "\n",
       "senate_pop"
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "Starting from [the example in Altair's documentation](https://altair-viz.github.io/gallery/choropleth.html), draw a map shaded by per capita receipts. Make the map interactive by showing a state's name, population and total receipts when you hover over it."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "# Fetch the state geograpy data\n",
       "state_geo = alt.topo_feature(data.us_10m.url, 'states')\n",
       "\n",
       "# TODO: Make choropleth map.\n",
       "# Tip: Use senate_pop and STATE_ID in the transform lookup.\n",
       "# Tip: Use state_geo above instead of 'counties' in the example."
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "### Optional: Other Visualizations.\n",
       "Think of something interesting you'd like to visualize. It doesn't have to be complex. You can reuse the election dataset, [the datasets in Altair's docs](https://altair-viz.github.io/gallery/index.html), or any other dataset."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": [
       "# ..."
      ]
     },
     {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
       "## Take Home\n",
       "We will release the rest of Lab 5 after the due date of Lab 4. Stay tuned on Piazza for updates."
      ]
     },
     {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": []
     }
    ],
    "metadata": {
     "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
     },
     "language_info": {
      "codemirror_mode": {
       "name": "ipython",
       "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.9.12"
     }
    },
    "nbformat": 4,
    "nbformat_minor": 4
   }