Quantcast
Channel: Active questions tagged r - Stack Overflow
Viewing all articles
Browse latest Browse all 201839

Loop to download Google search data for multiple keywords using gtrends in r

$
0
0

I want to download daily Google search data for multiple keywords using the gtrends package in r. I need search data for 30 keywords between 2004-18. Since Google allows to extract daily data only for 9 months at a time, I have to download the data 6 months at a time for each keyword . I also do some additional calculations for the 6-month data (see code below). After downloading the data for 6 months at a time, I want to combine the data to a one time series. After that, I want to omit NAs, regress on weekday dummies and keep the residual and finally scale the time series by its own standard deviation. In the end I would want to save the adjusted data as a vector with the name of the search term (see code below).

How do I create a loop which does the search and calculations for each search term separately and saves the adjusted data as a vector? I've tried to use different kinds of loops and apply functions, but do not understand how to use them with the gtrends package.

#define the keywords
keywords=c("Charity")

#set the geographic area: GB = Great Britain
country=c('GB')

#timeframe
time=("2004-01-01 2004-06-30")
#set channels 
channel='web'
trends = gtrends(keywords, gprop =channel,geo=country, time = time )
#select only interest over time 
time_trend=trends$interest_over_time
time_trend$hits[time_trend$hits=="0"]<-1
time_trend$change <- c(NA,diff(log(time_trend$hits)))
set1=time_trend[which(weekdays(as.Date(time_trend$date, format = "%m/%d/%Y"))
                 %in% c('Monday','Tuesday', 'Wednesday', 'Thursday', 'Friday')), ]

This goes on until set30, after which:

### Combine each 6 month data set ####

set <- rbind(set1,..,set30)

#omit NAs from the set
set <- na.omit(set)

# Regress on weekday and month dummies and keep the residual
set$weekday <- weekdays(set$date) #dummy for weekdays
weekday <- set$weekday

setti$month <- months(setti$date) #dummy for months
month <- set$month
mod <- lm(set$change~month+weekday)

#keep the residuals after the regression
set$residuals <- residuals(mod)

# Scale each by the time-series standard deviation #
sd <- sd(set$residuals)
set$adj_residuals=((set$residuals)/(sd))
adj_svi <- set$adj_residuals

# Save the deseasonalized and standardized ln daily change in keyword search volume as a vector

charity <- adj_svi

Viewing all articles
Browse latest Browse all 201839

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>