Dataframes
## Learning Objectives
- Create and manipulate dataframes
- Subset rows and columns
- Understand factors and dates
- Master dataframe operations
- Handle missing data
## Creating Dataframes
### Basic Creation
```r
# From vectors
df <- data.frame(
name = c("Alice", "Bob", "Charlie"),
age = c(25, 30, 35),
score = c(95.5, 87.3, 92.1)
)
# Print dataframe
print(df)
df
```
### with stringsAsFactors
```r
# Default: character converted to factor
df <- data.frame(
name = c("Alice", "Bob"),
age = c(25, 30)
)
str(df)
# $ name: Factor w/ 2 "Alice" "Bob"
# $ age : num 25 30
# Prevent conversion
df <- data.frame(
name = c("Alice", "Bob"),
age = c(25, 30),
stringsAsFactors = FALSE
)
str(df)
# $ name: chr "Alice" "Bob"
```
### from Matrix
```r
mat <- matrix(1:9, nrow = 3)
df <- as.data.frame(mat)
colnames(df) <- c("A", "B", "C")
```
## Viewing Dataframes
### Inspection Functions
```r
df <- data.frame(
name = c("Alice", "Bob", "Charlie"),
age = c(25, 30, 35)
)
# Structure
str(df)
# 'data.frame': 3 obs. of 2 variables:
# $ name: chr "Alice" "Bob" "Charlie"
# $ age : num 25 30 35
# Dimensions
nrow(df) # 3
ncol(df) # 2
dim(df) # 3 2
# Column names
colnames(df) # "name" "age"
names(df) # Same
# First/last rows
head(df) # First 6 rows
head(df, 3) # First 3 rows
tail(df) # Last 6 rows
```
## Subsetting
### Row Subsetting
```r
df <- data.frame(
name = c("Alice", "Bob", "Charlie", "Diana"),
age = c(25, 30, 35, 28),
score = c(95.5, 87.3, 92.1, 88.7)
)
# By row number
df[1, ] # First row
df[1:2, ] # First two rows
# By condition
df[df$age > 28, ]
# With subset()
subset(df, age > 28)
```
### Column Subsetting
```r
df <- data.frame(
name = c("Alice", "Bob"),
age = c(25, 30),
score = c(95.5, 87.3)
)
# By name
df$name # Returns vector
df[, "name"] # Returns vector
df[, "name", drop = FALSE] # Returns dataframe
# By position
df[, 1] # First column as vector
df[, 1:2] # First two columns
# With select
df[, c("name", "age")]
```
### Row and Column
```r
df <- data.frame(
name = c("Alice", "Bob", "Charlie"),
age = c(25, 30, 35)
)
# Single cell
df[1, 2] # By position: 25
df$age[1] # Via column: 25
df[1, "age"] # By name: 25
```
### Using dplyr
```r
library(dplyr)
df <- data.frame(
name = c("Alice", "Bob", "Charlie"),
age = c(25, 30, 35)
)
# Select columns
select(df, name)
select(df, name, age)
# Filter rows
filter(df, age > 28)
# Chain operations
df %>%
filter(age > 25) %>%
select(name)
```
## Adding/Removing Columns
### Add Column
```r
df <- data.frame(
name = c("Alice", "Bob"),
age = c(25, 30)
)
# Direct assignment
df$score <- c(95.5, 87.3)
# Using transform
df <- transform(df, score = c(95.5, 87.3))
# Using cbind
df <- cbind(df, score = c(95.5, 87.3))
```
### Remove Column
```r
df <- data.frame(
name = c("Alice", "Bob"),
age = c(25, 30),
score = c(95.5, 87.3)
)
# Set to NULL
df$score <- NULL
# Using subset
df <- subset(df, select = -score)
# Using dplyr
df <- select(df, -score)
```
## Adding/Removing Rows
### Add Row
```r
df <- data.frame(
name = c("Alice", "Bob"),
age = c(25, 30)
)
new_row <- data.frame(
name = "Charlie",
age = 35
)
df <- rbind(df, new_row)
```
### Remove Row
```r
df <- data.frame(
name = c("Alice", "Bob", "Charlie"),
age = c(25, 30, 35)
)
# Remove second row
df <- df[-2, ]
# Remove by condition
df <- df[df$name != "Bob", ]
```
## Sorting
### Order
```r
df <- data.frame(
name = c("Charlie", "Alice", "Bob"),
age = c(35, 25, 30)
)
# Order by column
df[order(df$age), ]
# Descending order
df[order(df$age, decreasing = TRUE), ]
# Multiple columns
df[order(df$age, df$name), ]
```
### Sorting with dplyr
```r
library(dplyr)
df <- data.frame(
name = c("Charlie", "Alice", "Bob"),
age = c(35, 25, 30)
)
df %>% arrange(age)
df %>% arrange(desc(age))
df %>% arrange(age, name)
```
## Merging
### rbind and cbind
```r
# Combine rows (same columns)
df1 <- data.frame(name = "Alice", age = 25)
df2 <- data.frame(name = "Bob", age = 30)
rbind(df1, df2)
# Combine columns (same rows)
df1 <- data.frame(name = c("Alice", "Bob"))
df2 <- data.frame(age = c(25, 30))
cbind(df1, df2)
```
### merge (Join)
```r
df1 <- data.frame(
id = c(1, 2, 3),
name = c("Alice", "Bob", "Charlie")
)
df2 <- data.frame(
id = c(2, 3, 4),
score = c(87.3, 92.1, 88.7)
)
# Inner join (matching IDs)
merge(df1, df2, by = "id")
# Left join (all from df1)
merge(df1, df2, by = "id", all.x = TRUE)
# Outer join (all)
merge(df1, df2, by = "id", all = TRUE)
```
## Missing Data
### Checking for NA
```r
df <- data.frame(
name = c("Alice", "Bob", NA),
age = c(25, NA, 35)
)
# Find NA cells
is.na(df)
# name age
# [1,] FALSE FALSE
# [2,] FALSE TRUE
# [3,] TRUE FALSE
# Count NA per column
colSums(is.na(df))
# Any NA?
anyNA(df) # TRUE
```
### Handling NA
```r
df <- data.frame(
x = c(1, 2, NA, 4),
y = c(NA, 2, 3, 4)
)
# Remove rows with any NA
na.omit(df)
# Remove rows with NA in specific column
df[!is.na(df$x), ]
# Fill NA with value
df$x[is.na(df$x)] <- 0
# Fill NA with mean
df$x[is.na(df$x)] <- mean(df$x, na.rm = TRUE)
```
### Complete Cases
```r
# Get complete cases only
df <- data.frame(
x = c(1, 2, NA),
y = c(1, NA, 3)
)
complete.cases(df) # TRUE FALSE FALSE
df[complete.cases(df), ] # First row only
```
## Summarizing
### Basic Summary
```r
df <- data.frame(
name = c("Alice", "Bob", "Charlie"),
age = c(25, 30, 35),
score = c(95.5, 87.3, 92.1)
)
# Summary statistics
summary(df)
# Specific functions
mean(df$age)
sd(df$score)
min(df$age)
max(df$age)
range(df$age)
```
### Summarizing with dplyr
```r
library(dplyr)
df <- data.frame(
group = c("A", "A", "B", "B"),
value = c(10, 20, 30, 40)
)
# Group by and summarize
df %>%
group_by(group) %>%
summarize(
mean = mean(value),
sum = sum(value),
n = n()
)
```
## Factors
### Creating Factors
```r
# Create factor
gender <- factor(c("male", "female", "male", "female"))
levels(gender) # "female" "male"
as.numeric(gender) # 2 1 2 1
# With levels specified
size <- factor(c("small", "large", "medium"),
levels = c("small", "medium", "large"))
```
### Ordered Factors
```r
education <- ordered(c("high school", "phd", "bachelor", "high school"),
levels = c("high school", "bachelor", "master", "phd"))
education[2] < education[4] # TRUE
```
## Dates
### Date Class
```r
# Current date
today <- Sys.Date()
class(today) # "Date"
# Create date
as.Date("2024-01-15")
as.Date("2024/01/15")
# From components
as.Date(paste(2024, 1, 15, sep = "-"))
```
### Date Operations
```r
# Date arithmetic
as.Date("2024-01-15") + 7 # 2024-01-22
as.Date("2024-01-15") - as.Date("2024-01-10") # 5 days
# Extract parts
date <- as.Date("2024-01-15")
format(date, "%Y") # "2024"
format(date, "%m") # "01"
format(date, "%d") # "15"
format(date, "%B") # "January"
format(date, "%a") # "Mon"
```
### POSIXlt/POSIXct
```r
# Current time
now <- Sys.time()
class(now) # "POSIXct" "POSIXt"
# From string
as.POSIXct("2024-01-15 10:30:00")
# Extract components
now_ct <- as.POSIXlt(now)
now_ct$hour
now_ct$min
now_ct$sec
```
## Summary
- Dataframes are tabular data structures (rows x columns)
- Use `data.frame()` to create, `str()` to inspect
- Subset with `[row, col]`, `$column`, or `subset()`
- Add columns with `$` or `cbind`, rows with `rbind`
- Use `merge()` to join dataframes by key columns
- Handle missing values with `is.na()`, `na.omit()`, `complete.cases()`
- Factors for categorical data, `levels()` to see categories
- Dates stored as `Date` or `POSIXct` class
- `dplyr` package provides modern dataframe manipulation
Comments
Comments powered by Giscus
To enable comments, add your Giscus embed code here.
Learn more about Giscus →