Quantcast
Channel: Active questions tagged r - Stack Overflow
Viewing all articles
Browse latest Browse all 205278

Importing and merging files in loop

$
0
0

I have to combine datasets. They are .sav files, and I have 6-7 datasets per month, per year - a total of 13 years. That's a lot of datasets to import and combine, and I want to automate this using loops.

As I'm a beginner, I wrote the first loop to simply combine the datasets for one year (so only looping over the months). This was my code, and it did what I wanted perfectly. It's not the fastest and for sure not the prettiest or most efficient, but it worked. Note: I shortened the "C..." path in the code posted for brevity: in my real code it's the full path.

 for (m in months) {
setwd(paste("C:.... survey\\DANE 2005\\",m,sep=""))
files_2005 <- list.files(path=(paste("C:\....survey\\DANE 2005\\",m,sep="")),  pattern=("Area.*.sav"))

#for (i in (paste("files_",m,sep=""))){
   df_2005 <- lapply(files_2005, read_sav)
  assign(paste("DANE2005_",m,sep=""), df_2005 %>% reduce(rbind.fill))

#}

df_2005 <- mget(ls(pattern="DANE2005_"))
dane_2005 <- df_2005 %>% reduce(rbind.fill)

}

And here is my current code, looping over years & months (thanks to @Onyambu for the comments). However, it still does not work; if I dont use setwd R says that the "current file does not exist in the directory" (and refers back to my main directory, not the path specified). If I do use setwd, I get the "cannot change working directory" error.

for (y in years) {
  for (m in months) {

    #Go to a folder per year/month
    path <- paste("C:.... survey\\DANE ",y,"\\",m,sep="")
    #Create a list of all the files in that folder by month, based on a pattern
    list_data<-list.files(path=path,  pattern=("Area.*.sav"))

    if(!is_empty(list_data)){
    #Read in all the files in the folder by month, based on the list
    df_2005 <- lapply(list_data, read_sav)
    #bind the files for one month together based on the list
    assign(paste("DANE2005_",m,sep=""), df_2005 %>% reduce(rbind.fill))
    }
  }
  #Bind together all the files for one year
  df_2005 <- mget(ls(pattern="DANE2005_"))
  dane_2005 <- df_2005 %>% reduce(left_join)
}

Any help is much appreciated.

EDIT: cleaned up the code and re-posed the question for clarity after initial comments.


Viewing all articles
Browse latest Browse all 205278

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>