Series and DataFrame — the two things in pandas

The problem we are solving

In the last chapter one line was written, and it worked:

print(sales["quantity"].sum())

19

The line is easy to read, and there is a question hiding inside it. What did sales["quantity"] hand back? A list? Lists have no .sum() — writing [12, 5, 2].sum() in Python raises an error.

The answer is worth knowing, because in pandas you are always holding one of two things, and nearly every confusion starts with the same question: which one am I holding right now? A column, or a whole table?

This chapter is about those two things: their names, their shapes, and what separates them.

By the end of this chapter you can

• Say in one sentence what separates a Series from a DataFrame
• Build a Series and inspect its index, values and type separately
• Explain why every value in a column has to share one type
• Use df["col"] and df[["col"]] knowing which is which
• Read a KeyError and tell exactly what went wrong with a column name

Prerequisites: installing pandas and your first table.

Series — one column

Start with the smaller unit. A Series is a run of values, each paired with a name:

import pandas as pd

quantity = pd.Series([12, 5, 2])

print(quantity)

0    12
1     5
2     2
dtype: int64

Three values went in, and three pairs came out. The 0, 1, 2 down the left are the index — a name for each value. You did not supply them; pandas counted from zero.

And along the bottom, dtype: int64 — the type of the whole series. One type, not one per value. That is what makes pandas fast, and it is also its main constraint.

The three parts can be looked at separately:

import pandas as pd

quantity = pd.Series([12, 5, 2])

print(quantity.index)
print(quantity.values)
print(quantity.dtype)

RangeIndex(start=0, stop=3, step=1)
[12  5  2]
int64

RangeIndex means the index is nothing but a count — starting at zero, going up by one. values is the values without their names. And dtype is the type of the whole column.

An index does not have to be numbers:

import pandas as pd

quantity = pd.Series([12, 5, 2], index=["pen", "notebook", "bag"])

print(quantity)
print(quantity["notebook"])

pen         12
notebook     5
bag          2
dtype: int64
5

Now the rows are named after the products, and a value can be fetched by that name. This sounds like a Python dictionary, and the resemblance is real — with one large difference:

import pandas as pd

quantity = pd.Series([12, 5, 2], index=["pen", "notebook", "bag"])

print(quantity * 2)

pen         24
notebook    10
bag          4
dtype: int64

The whole column was multiplied by two at once, with no loop. Doing that to a dictionary means walking over every value yourself. Run an operation on a Series and pandas applies it to every value — and every remaining chapter stands on that behaviour.

DataFrame — several columns, one index

A DataFrame is several Series side by side that share one index.

import pandas as pd

sales = pd.DataFrame({
    "product": ["pen", "notebook", "bag"],
    "quantity": [12, 5, 2],
    "price": [15.0, 60.0, 850.0],
})

print(sales)
print(sales.shape)
print(sales.columns)

product  quantity  price
0       pen        12   15.0
1  notebook         5   60.0
2       bag         2  850.0
(3, 3)
Index(['product', 'quantity', 'price'], dtype='object')

That phrase "share one index" is the substance of it. The product, quantity and price at index 0 are all talking about the same thing, one pen. The index is what ties three columns into rows.

One table, several types — but one column, one type

import pandas as pd

sales = pd.DataFrame({
    "product": ["pen", "notebook", "bag"],
    "quantity": [12, 5, 2],
    "price": [15.0, 60.0, 850.0],
})

print(sales.dtypes)

product      object
quantity      int64
price       float64
dtype: object

Three columns, three types. int64 is whole numbers, float64 decimals, and object — which usually means text.

Notice there is no str. Pandas types come from NumPy, where every value occupies the same amount of space — possible for numbers, not for text of varying length. So to hold text, pandas keeps pointers to Python objects instead, and that arrangement is called object.

Now the important rule: every value in a column shares one type. So what happens when numbers and text are mixed into one?

import pandas as pd

mixed = pd.Series([1, "two", 3.0])

print(mixed)
print(mixed.dtype)

0      1
1    two
2    3.0
dtype: object

Pandas does not raise an error — it makes the whole column object, falling back to the loosest type available. The consequence is not dramatic, it is silent: the column is no longer numeric, so .mean() will fail on it and .sum() will start joining text. One stray N/A in one row of a CSV does exactly this, and that is the subject of chapter six.

Which do I have — a Series or a DataFrame?

import pandas as pd

sales = pd.DataFrame({
    "product": ["pen", "notebook", "bag"],
    "quantity": [12, 5, 2],
    "price": [15.0, 60.0, 850.0],
})

column = sales["quantity"]

print(type(column))
print(type(sales))
print(column.mean())

<class 'pandas.core.series.Series'>
<class 'pandas.core.frame.DataFrame'>
6.333333333333333

And there is the answer to the question this chapter opened with. sales["quantity"] is not a list, it is a Series — and a Series has .sum(), .mean(), .max() and the rest.

There is also a pair here that stops everyone the first time:

import pandas as pd

sales = pd.DataFrame({
    "product": ["pen", "notebook", "bag"],
    "quantity": [12, 5, 2],
    "price": [15.0, 60.0, 850.0],
})

print(sales["quantity"])
print("---")
print(sales[["quantity"]])

0    12
1     5
2     2
Name: quantity, dtype: int64
---
   quantity
0        12
1         5
2         2

One extra bracket, and the result is a different kind of thing. The first is a Series — one column, with its name and type printed underneath. The second is a DataFrame — a table that happens to have one column, so the column heading sits on top.

The rule is that the inner square brackets mean a list of columns, and asking for a list gets you a table. So sales[["product", "price"]] gives a two-column table.

Why keep this distinction in mind? Because it is where the error messages diverge. A Series has .mean() but no column names; a DataFrame selects columns but does not hand you a single number. When a message says an object has no such attribute, the first question is: which of the two am I actually holding?

A complete example

inventory.py:

import pandas as pd

# Product names become the index, so a row can be reached by name.
stock = pd.DataFrame(
    {
        "quantity": [12, 5, 2, 30],
        "price": [15.0, 60.0, 850.0, 8.0],
    },
    index=["pen", "notebook", "bag", "eraser"],
)

print(stock)
print()
print("Shape      :", stock.shape)
print("Columns    :", list(stock.columns))
print("Index      :", list(stock.index))
print()

# A column is a Series, so an operation applies to every value at once.
value = stock["quantity"] * stock["price"]
print("Value per product:")
print(value)
print()
print("Total value:", value.sum())

quantity  price
pen             12   15.0
notebook         5   60.0
bag              2  850.0
eraser          30    8.0

Shape      : (4, 2)
Columns    : ['quantity', 'price']
Index      : ['pen', 'notebook', 'bag', 'eraser']

Value per product:
pen          180.0
notebook     300.0
bag         1700.0
eraser       240.0
dtype: float64

Total value: 2420.0

Two things are worth noticing.

First, stock["quantity"] * stock["price"] multiplied two Series and produced another Series. Pandas matched them up by index — the pen's quantity with the pen's price, the bag's with the bag's. By name, not by row order. That behaviour is the foundation of joining two tables later on.

Second, the resulting column's type is float64, though quantity was int64. Multiplying a whole number by a decimal gives a decimal — the same rule as in plain Python; pandas is not doing anything of its own here.

When it breaks

KeyError: 'Quantity' There is no such column. Column names are case-sensitive, and names that came from a CSV file often carry a leading or trailing space. Run print(list(df.columns)) to see exactly what is there — printing them as a list puts quotes around each one, so any space shows up.

AttributeError: 'DataFrame' object has no attribute 'mean_price' df.column_name also fetches a column, but only for simple names — not when the name has a space in it, starts with a digit, or collides with a pandas method such as count. df["column_name"] always works, so make that the one you write.

TypeError: unsupported operand type(s) while calculating on a numeric column The column is probably not numeric. Check df.dtypes; an object there means text is mixed in somewhere.

.sum() is joining text instead of adding numbers Same cause — the column is object. One N/A or one empty cell in a single row is enough to change the type of the whole column.

ValueError: All arrays must be of the same length The lists given to pd.DataFrame({...}) are not all the same length. Every column needs the same number of values, because the rows have to line up on one shared index.

Your turn

Write a file called library.py holding a DataFrame of at least five books — the titles as the index, and columns for the number of copies (whole numbers), the price (decimals) and the author's name (text).

Then print:

1. The whole table, its shape, and its dtypes
2. The price column on its own — and show with type() that it is a Series
3. The price column with double brackets — and show that this one is a DataFrame
4. The total value of each book (copies × price), and the total across all of them

Then run one experiment: replace one of the copy counts with "unknown" and run the file again. What does dtypes say now, and what happened to the total?

Those are the most valuable five minutes in this chapter. One word in one cell changes the type of a whole column and breaks the arithmetic without raising anything — and in real data it happens constantly.