Adding and Removing Rows

Pandas: Adding and Removing Rows

Just like columns, the rows in a DataFrame are dynamic. In a real-world scenario, your dataset changes constantly: new transactions arrive (adding rows), or invalid data needs to be cleaned (removing rows).

In this comprehensive tutorial, we will master the different techniques to manipulate rows, including the modern pd.concat method and conditional deletion.

---

🔹 1. Adding Rows

Adding rows can be done in two main ways: manually adding a single entry or merging two datasets together.

Method A: The .loc Method (Manual Entry)

If you know the specific index where you want to place the data, you can assign it directly using .loc.

🧪 Code Example

import pandas as pd

# Setup sample data
data = {
    "Name": ["Atul", "Aman"],
    "Class": [11, 39],
    "Status": ["pass", "fail"]
}
table = pd.DataFrame(data)

# Add a new row explicitly at index 2
table.loc[2] = ["Harsh", 47, "pass"]

# Add a new row automatically at the end (using len)
# len(table) gives 3, so this adds data at index 3
table.loc[len(table)] = ["Rishi", 56, "pass"]

print(table)

📤 Output

    Name  Class Status
0   Atul     11   pass
1   Aman     39   fail
2  Harsh     47   pass
3  Rishi     56   pass

⚠ Warning: Overwriting Data
If you use an index that already exists (e.g., table.loc[0] = [...]), Pandas will overwrite the existing row without asking. Always ensure your index is unique when adding new data.

---

Method B: The concat Method (Combining Tables)

The most professional way to add data is to create a new DataFrame and combine it with the old one.
Note: The old table.append() command was removed in Pandas 2.0. You must use concat now.

🧪 Code Example

# Create a new mini-table with new students
new_students = pd.DataFrame({
    "Name": ["Devansh", "Karan"],
    "Class": [76, 12],
    "Status": ["fail", "pass"]
})

# Combine the original table and the new students
# ignore_index=True resets the row numbers (0, 1, 2, 3...)
table = pd.concat([table, new_students], ignore_index=True)

print(table)

📤 Output

      Name  Class Status
0     Atul     11   pass
1     Aman     39   fail
2    Harsh     47   pass
3    Rishi     56   pass
4  Devansh     76   fail
5    Karan     12   pass

---

🔹 2. Removing Rows (The .drop Method)

We use the .drop() method to delete rows. Unlike columns, we do not need to specify axis=1 because the default axis (0) refers to rows.

A. Removing by Index Label

This is useful if you know the exact ID (index) of the row you want to delete.

🧪 Code Example

# Remove the row at index 1 (Aman) temporarily
# We pass a list [1, 5] to remove multiple rows at once
table_cleaned = table.drop([1, 5])

print("--- Data after dropping rows 1 and 5 ---")
print(table_cleaned)

📤 Output

--- Data after dropping rows 1 and 5 ---
      Name  Class Status
0     Atul     11   pass
2    Harsh     47   pass
3    Rishi     56   pass
4  Devansh     76   fail

Notice the "Index Gap": The row numbers now jump from 0 to 2. The index 1 is gone forever.

---

B. Conditional Deletion (The Professional Way)

In real projects, you don't delete "Row 4". You delete "Rows where Status is Fail". We do this by Filtering. We keep only the rows that we want.

🧪 Code Example

# "Keep only rows where Status is pass"
# Effectively deletes all "fail" rows
passed_students_only = table[table["Status"] == "pass"]

print("--- Only Passed Students ---")
print(passed_students_only)

📤 Output

--- Only Passed Students ---
      Name  Class Status
0     Atul     11   pass
2    Harsh     47   pass
3    Rishi     56   pass
5    Karan     12   pass

---

🔹 3. Fixing the Index (Reset Index)

After adding or removing rows, your index numbers (0, 1, 3, 5...) often become messy and non-sequential. We use reset_index() to fix this.

🧪 Code Example

# Step 1: Messy Index
print("--- Messy Index ---")
print(passed_students_only)

# Step 2: Fix Index
# drop=True prevents the old index from becoming a new column
passed_students_only.reset_index(drop=True, inplace=True)

print("\n--- Clean Index (0, 1, 2, 3...) ---")
print(passed_students_only)

📤 Output

--- Messy Index ---
0     Atul ...
2    Harsh ...
3    Rishi ...
5    Karan ...

--- Clean Index (0, 1, 2, 3...) ---
      Name  Class Status
0     Atul     11   pass
1    Harsh     47   pass
2    Rishi     56   pass
3    Karan     12   pass

---

📌 Summary

Action	Syntax Code	Pro Tip
Add Single Row	df.loc[len(df)] = [...]	Uses table length to find next index.
Merge Tables	pd.concat([df1, df2])	Best for combining large datasets.
Remove by ID	df.drop([0, 2])	Deletes specific index numbers.
Remove by Logic	df[df["Col"] == "Value"]	"Filter to keep" is safer than delete.
Fix Index	df.reset_index(drop=True)	Always do this after deleting rows.