{
  "source_file": "0930",
  "source_platform": "个人博客",
  "author": "Jay Alammar",
  "title_listed": "A Visual Intro to NumPy and Data Representation",
  "keywords": "NumPy,数据表示,可视化",
  "domain": "数据处理",
  "content_type": "图解教程",
  "is_paid_flag": "否",
  "url": "https://jalammar.github.io/visual-numpy/",
  "publish_date": "2019-06",
  "corpus_stage": "数据处理",
  "notes": "NumPy与数据表示可视化入门",
  "id": "6e810fcbddb0",
  "collected_at": "2026-10-02T14:21:32",
  "title": "A Visual Intro to NumPy and Data Representation – Jay Alammar – Visualizing machine learning one concept at a time.",
  "body_markdown": "A Visual Intro to NumPy and Data Representation\nDiscussions:\nHacker News (366 points, 21 comments)\n,\nReddit r/MachineLearning (256 points, 18 comments)\nTranslations:\nChinese 1\n,\nChinese 2\n,\nJapanese\n,\nKorean\nThe\nNumPy\npackage is the workhorse of data analysis, machine learning, and scientific computing in the python ecosystem. It vastly simplifies manipulating and crunching vectors and matrices. Some of python’s leading package rely on NumPy as a fundamental piece of their infrastructure (examples include scikit-learn, SciPy, pandas, and tensorflow). Beyond the ability to slice and dice numeric data, mastering numpy will give you an edge when dealing and debugging with advanced usecases in these libraries.\nIn this post, we’ll look at some of the main ways to use NumPy and how it can represent different types of data (tables, images, text…etc) before we can serve them to machine learning models.\nimport\nnumpy\nas\nnp\nCreating Arrays\nWe can create a NumPy array (a.k.a. the mighty\nndarray\n) by passing a python list to it and using ` np.array()`. In this case, python creates the array we can see on the right here:\nThere are often cases when we want NumPy to initialize the values of the array for us. NumPy provides methods like ones(), zeros(), and random.random() for these cases. We just pass them the number of elements we want it to generate:\nOnce we’ve created our arrays, we can start to manipulate them in interesting ways.\nArray Arithmetic\nLet’s create two NumPy arrays to showcase their usefulness. We’ll call them\ndata\nand\nones\n:\nAdding them up position-wise (i.e. adding the values of each row) is as simple as typing\ndata + ones\n:\nWhen I started learning such tools, I found it refreshing that an abstraction like this makes me not have to program such a calculation in loops. It’s a wonderful abstraction that allows you to think about problems at a higher level.\nAnd it’s not only addition that we can do this way:\nThere are often cases when we want to carry out an operation between an array and a single number (we can also call this an operation between a vector and a scalar). Say, for example, our array represents distance in miles, and we want to convert it to kilometers. We simply say\ndata * 1.6\n:\nSee how NumPy understood that operation to mean that the multiplication should happen with each cell? That concept is called\nbroadcasting\n, and it’s very useful.\nIndexing\nWe can index and slice NumPy arrays in all the ways we can slice python lists:\nAggregation\nAdditional benefits NumPy gives us are aggregation functions:\nIn addition to\nmin\n,\nmax\n, and\nsum\n, you get all the greats like\nmean\nto get the average,\nprod\nto get the result of multiplying all the elements together,\nstd\nto get standard deviation, and\nplenty of others\n.\nIn more dimensions\nAll the examples we’ve looked at deal with vectors in one dimension. A key part of the beauty of NumPy is its ability to apply everything we’ve looked at so far to any number of dimensions.\nCreating Matrices\nWe can pass python lists of lists in the following shape to have NumPy create a matrix to represent them:\nnp\n.\narray\n([[\n1\n,\n2\n],[\n3\n,\n4\n]])\nWe can also use the same methods we mentioned above (\nones()\n,\nzeros()\n, and\nrandom.random()\n) as long as we give them a tuple describing the dimensions of the matrix we are creating:\nMatrix Arithmetic\nWe can add and multiply matrices using arithmetic operators (\n+-*/\n) if the two matrices are the same size. NumPy handles those as position-wise operations:\nWe can get away with doing these arithmetic operations on matrices of different size only if the different dimension is one (e.g. the matrix has only one column or one row), in which case NumPy uses its broadcast rules for that operation:\nDot Product\nA key distinction to make with arithmetic is the case of\nmatrix multiplication\nusing the dot product. NumPy gives every matrix a\ndot()\nmethod we can use to carry-out dot product operations with other matrices:\nI’ve added matrix dimensions at the bottom of this figure to stress that the two matrices have to have the same dimension on the side they face each other with. You can visualize this operation as looking like this:\nMatrix Indexing\nIndexing and slicing operations become even more useful when we’re manipulating matrices:\nMatrix Aggregation\nWe can aggregate matrices the same way we aggregated vectors:\nNot only can we aggregate all the values in a matrix, but we can also aggregate across the rows or columns by using the\naxis\nparameter:\nTransposing and Reshaping\nA common need when dealing with matrices is the need to rotate them. This is often the case when we need to take the dot product of two matrices and need to align the dimension they share. NumPy arrays have a convenient property called\nT\nto get the transpose of a matrix:\nIn more advanced use case, you may find yourself needing to switch the dimensions of a certain matrix. This is often the case in machine learning applications where a certain model expects a certain shape for the inputs that is different from your dataset. NumPy’s\nreshape()\nmethod is useful in these cases. You just pass it the new dimensions you want for the matrix. You can pass -1 for a dimension and NumPy can infer the correct dimension based on your matrix:\nYet More Dimensions\nNumPy can do everything we’ve mentioned in any number of dimensions. Its central data structure is called ndarray (N-Dimensional Array) for a reason.\nIn a lot of ways, dealing with a new dimension is just adding a comma to the parameters of a NumPy function:\nNote: Keep in mind that when you print a 3-dimensional NumPy array, the text output visualizes the array differently than shown here.  NumPy’s order for printing n-dimensional arrays is that the last axis is looped over the fastest, while the first is the slowest. Which means that\nnp.ones((4,3,2))\nwould be printed as:\narray\n([[[\n1.\n,\n1.\n],\n[\n1.\n,\n1.\n],\n[\n1.\n,\n1.\n]],\n[[\n1.\n,\n1.\n],\n[\n1.\n,\n1.\n],\n[\n1.\n,\n1.\n]],\n[[\n1.\n,\n1.\n],\n[\n1.\n,\n1.\n],\n[\n1.\n,\n1.\n]],\n[[\n1.\n,\n1.\n],\n[\n1.\n,\n1.\n],\n[\n1.\n,\n1.\n]]])\nPractical Usage\nAnd now for the payoff. Here are some examples of the useful things NumPy will help you through.\nFormulas\nImplementing mathematical formulas that work on matrices and vectors is a key use case to consider NumPy for. It’s why NumPy is the darling of the scientific python community. For example, consider the mean square error formula that is central to supervised machine learning models tackling regression problems:\nImplementing this is a breeze in NumPy:\nThe beauty of this is that numpy does not care if\npredictions\nand\nlabels\ncontain one or a thousand values (as long as they’re both the same size). We can walk through an example stepping sequentially through the four operations in that line of code:\nBoth the predictions and labels vectors contain three values. Which means n has a value of three. After we carry out the subtraction, we end up with the values looking like this:\nThen we can square the values in the vector:\nNow we sum these values:\nWhich results in the error value for that prediction and a score for the quality of the model.\nData Representation\nThink of all the data types you’ll need to crunch and build models around (spreadsheets, images, audio…etc). So many of them are perfectly suited for representation in an n-dimensional array:\nTables and Spreadsheets\nA spreadsheet or a table of values is a two dimensional matrix. Each sheet in a spreadsheet can be its own variable. The most popular abstraction in python for those is the\npandas dataframe\n, which actually uses NumPy and builds on top of it.\nAudio and Timeseries\nAn audio file is a one-dimensional array of samples. Each sample is a number representing a tiny chunk of the audio signal. CD-quality audio may have 44,100 samples per second and each sample is an integer between -32767 and 32768. Meaning if you have a ten-seconds WAVE file of CD-quality, you can load it in a NumPy array with length 10 * 44,100 = 441,000 samples. Want to extract the first second of audio? simply load the file into a NumPy array that we’ll call\naudio\n, and get\naudio[:44100]\n.\nHere’s a look at a slice of an audio file:\nThe same goes for time-series data (for example, the price of a stock over time).\nImages\nAn image is a matrix of pixels of size (height x width).\nIf the image is black and white (a.k.a. grayscale), each pixel can be represented by a single number (commonly between 0 (black) and 255 (white)). Want to crop the top left 10 x 10 pixel part of the image? Just tell NumPy to get you\nimage[:10,:10]\n.\nHere’s a look at a slice of an image file:\nIf the image is colored, then each pixel is represented by three numbers - a value for each of red, green, and blue. In that case we need a 3rd dimension (because each cell can only contain one number). So a colored image is represented by an ndarray of dimensions: (height x width x 3).\nLanguage\nIf we’re dealing with text, the story is a little different. The numeric representation of text requires a step of building a vocabulary (an inventory of all the unique words the model knows) and an\nembedding step\n. Let us see the steps of numerically representing this (translated) quote by an ancient spirit:\n“Have the bards who preceded me left any theme unsung?”\nA model needs to look at a large amount of text before it can numerically represent the anxious words of this warrior poet. We can proceed to have it process a\nsmall dataset\nand use it to build a vocabulary (of 71,290 words):\nThe sentence can then be broken into an array of tokens (words or parts of words based on common rules):\nWe then replace each word by its id in the vocabulary table:\nThese ids still don’t provide much information value to a model. So before feeding a sequence of words to a model, the tokens/words need to be replaced with their embeddings (50 dimension\nword2vec embedding\nin this case):\nYou can see that this NumPy array has the dimensions [embedding_dimension x sequence_length]. In practice these would be the other way around, but I’m presenting it this way for visual consistency. For performance reasons, deep learning models tend to preserve the first dimension for batch size (because the model can be trained faster if multiple examples are trained in parallel). This is a clear case where\nreshape()\nbecomes super useful. A model like\nBERT\n, for example, would expect its inputs in the shape: [batch_size, sequence_length, embedding_size].\nThis is now a numeric volume that a model can crunch and do useful things with. I left the other rows empty, but they’d be filled with other examples for the model to train on (or predict).\n(It turned out the\npoet’s words\nin our example were immortalized more so than those of the other poets which trigger his anxieties. Born a slave owned by his father,\nAntarah’s\nvalor and command of language gained him his freedom and the mythical status of having his poem as one of\nseven poems suspended in the kaaba\nin pre-Islamic Arabia).\nWritten on June 26, 2019",
  "metadata": {
    "title": "A Visual Intro to NumPy and Data Representation – Jay Alammar – Visualizing machine learning one concept at a time.",
    "description": "Discussions:\nHacker News (366 points, 21 comments), Reddit r/MachineLearning (256 points, 18 comments)\n\n\nTranslations: Chinese 1, Chinese 2, Japanese, Korean\n\n\n  \n  \n\n\nThe NumPy package is the workhorse of data analysis, machine learning, and scientific computing in the python ecosystem. It vastly simplifies manipulating and crunching vectors and matrices. Some of python’s leading package rely on NumPy as a fundamental piece of their infrastructure (examples include scikit-learn, SciPy, pandas, and tensorflow). Beyond the ability to slice and dice numeric data, mastering numpy will give you an edge when dealing and debugging with advanced usecases in these libraries.\n\nIn this post, we’ll look at some of the main ways to use NumPy and how it can represent different types of data (tables, images, text…etc) before we can serve them to machine learning models.",
    "keywords": null,
    "author": "Jay Alammar",
    "og:description": "Discussions:\nHacker News (366 points, 21 comments), Reddit r/MachineLearning (256 points, 18 comments)\n\n\nTranslations: Chinese 1, Chinese 2, Japanese, Korean\n\n\n  \n  \n\n\nThe NumPy package is the workhorse of data analysis, machine learning, and scientific computing in the python ecosystem. It vastly simplifies manipulating and crunching vectors and matrices. Some of python’s leading package rely on NumPy as a fundamental piece of their infrastructure (examples include scikit-learn, SciPy, pandas, and tensorflow). Beyond the ability to slice and dice numeric data, mastering numpy will give you an edge when dealing and debugging with advanced usecases in these libraries.\n\nIn this post, we’ll look at some of the main ways to use NumPy and how it can represent different types of data (tables, images, text…etc) before we can serve them to machine learning models.",
    "og:title": "A Visual Intro to NumPy and Data Representation"
  },
  "paywall": {
    "listed_paid": false,
    "detected": false,
    "marker": "",
    "public_only": false,
    "policy": "付费/会员内容只保留页面公开可见部分，不绕过付费墙"
  },
  "crawl": {
    "success": true,
    "status_code": 200,
    "error": ""
  },
  "code_blocks": [
    {
      "file": "attachments/6e810fcbddb0/code/body-000.txt",
      "chars": 19,
      "language": "text"
    },
    {
      "file": "attachments/6e810fcbddb0/code/body-001.txt",
      "chars": 24,
      "language": "text"
    },
    {
      "file": "attachments/6e810fcbddb0/code/body-002.txt",
      "chars": 224,
      "language": "text"
    }
  ],
  "images": [],
  "audio": [],
  "videos": [],
  "links": [
    {
      "href": "https://news.ycombinator.com/item?id=20282985",
      "label": "Hacker News (366 points, 21 comments)",
      "type": "a"
    },
    {
      "href": "https://www.reddit.com/r/MachineLearning/comments/c5nc89/p_a_visual_intro_to_numpy_and_data_representation/",
      "label": "Reddit r/MachineLearning (256 points, 18 comments)",
      "type": "a"
    },
    {
      "href": "http://www.junphy.com/wordpress/index.php/2019/10/24/visual-numpy/",
      "label": "Chinese 1",
      "type": "a"
    },
    {
      "href": "https://github.com/kevingo/blog/blob/master/ML/visual-numpy.md",
      "label": "Chinese 2",
      "type": "a"
    },
    {
      "href": "https://note.mu/sayajewels/n/n95edaedb0fc5",
      "label": "Japanese",
      "type": "a"
    },
    {
      "href": "https://chloamme.github.io/2021/12/20/visual-numpy-korean.html",
      "label": "Korean",
      "type": "a"
    },
    {
      "href": "https://www.numpy.org/",
      "label": "NumPy",
      "type": "a"
    },
    {
      "href": "https://docs.scipy.org/doc/numpy/reference/arrays.ndarray.html",
      "label": "ndarray",
      "type": "a"
    },
    {
      "href": "https://jakevdp.github.io/PythonDataScienceHandbook/02.04-computation-on-arrays-aggregates.html",
      "label": "plenty of others",
      "type": "a"
    },
    {
      "href": "https://www.mathsisfun.com/algebra/matrix-multiplying.html",
      "label": "matrix multiplication",
      "type": "a"
    },
    {
      "href": "https://jalammar.github.io/gentle-visual-intro-to-data-analysis-python-pandas/",
      "label": "pandas dataframe",
      "type": "a"
    },
    {
      "href": "https://jalammar.github.io/illustrated-word2vec/",
      "label": "embedding step",
      "type": "a"
    },
    {
      "href": "http://mattmahoney.net/dc/textdata.html",
      "label": "small dataset",
      "type": "a"
    },
    {
      "href": "https://jalammar.github.io/illustrated-bert/",
      "label": "BERT",
      "type": "a"
    },
    {
      "href": "https://en.wikisource.org/wiki/The_Poem_of_Antara",
      "label": "poet’s words",
      "type": "a"
    },
    {
      "href": "https://en.wikipedia.org/wiki/Antarah_ibn_Shaddad",
      "label": "Antarah’s",
      "type": "a"
    },
    {
      "href": "https://en.wikipedia.org/wiki/Mu%27allaqat",
      "label": "seven poems suspended in the kaaba",
      "type": "a"
    }
  ],
  "repair_version": 2,
  "repaired_at": "2026-10-14T10:52:31Z",
  "source_url": "https://jalammar.github.io/visual-numpy/",
  "papers": [],
  "actual_video_files": [],
  "body_chars": 10970,
  "expected_body_images": 0,
  "discovery_status": "ok",
  "delivery_status": "ok",
  "missing_reason": "",
  "raw_html": "attachments/6e810fcbddb0/pages/source.html",
  "local_html": "attachments/6e810fcbddb0/pages/article.html",
  "body_base_path": "attachments/6e810fcbddb0/pages/",
  "body_markdown_path": "attachments/6e810fcbddb0/pages/article.md"
}