How to import multiple .csv files simultaneously in R and create a data frame

Have you ever struggled to import hundred of small datasets files? Can be very time consuming or maybe impossible. I was in this situation some time ago when I had a folder with approximately three thousand CSV files, and I was interested in creating a single dataset.

At the time I was thinking to create a for loop for importing each file separately and then to merge all small datasets.

# file1 = read_csv("file1.csv")
# file2 = read_csv("file2.csv")
# file3 = read_csv("file3.csv")

I didn't know how that would work, or even it would be possible to merge 3000 datasets easily. Anyway, I started searching for similar questions, and I don't remember that I found something helpful until I discovered the plyr package. I am happy to share it with you. There are no many codes.

Load the package

library(plyr)
library(readr)

For this post, I created 3 CSV files and put them in a folder (i.e., cvsfolder) in my desktop. You can do the same if you want to replicate this post. I set the directory in R and used the function list.files to list all files in folder with extension CSV.

setwd("~/Desktop")
mydir = "csvfolder"
myfiles = list.files(path=mydir, pattern="*.csv", full.names=TRUE)
myfiles
## [1] "csvfolder/file1.csv" "csvfolder/file2.csv" "csvfolder/file3.csv"

In the R Studio environment, I have only the location of CSV files; no file is uploaded yet. To upload all files and create a dataset will use ldply and applied the read_csv function.

dat_csv = ldply(myfiles, read_csv)
dat_csv
##    a  b  c
## 1  1 34 98
## 2 23 55 10
## 3 43 67  3
## 4 32 21 56
## 5 34 23 57
## 6 31 24 58
## 7 43 65 77
## 8 45 63 78
## 9 57 61 79

Done!

You can apply the same function for importing .txt files as well. The function read.table shall be used for .txt files. See code below:

# dat_txt = ldply(myfiles, read.table, sep = "\t", fill=TRUE, header = TRUE)

Extra

Below I will import each file separately to show that the dataset and variable names correspondent with the dat_csv above.

read_csv(myfiles[1])
## # A tibble: 3 x 3
##       a     b     c
##   <int> <int> <int>
## 1     1    34    98
## 2    23    55    10
## 3    43    67     3

read_csv(myfiles[2])
## # A tibble: 3 x 3
##       a     b     c
##   <int> <int> <int>
## 1    32    21    56
## 2    34    23    57
## 3    31    24    58

read_csv(myfiles[3])
## # A tibble: 3 x 3
##       a     b     c
##   <int> <int> <int>
## 1    43    65    77
## 2    45    63    78
## 3    57    61    79

I hope you learned something new today and share it with your peers. Who knows it may be helpful for someone else.