R Change color of a ggplot background with regard to semesters

I have a stock-market dataframe and I want to create a plot with a colored background. But the color should be based on the semesters (first or second half of the year).

Here you can get the data:

library(tidyverse)
library(tidyquant)
library(lubridate)

GDAXI<-tq_get("^GDAXI", get = "stock.prices")

GDAXI%>%
  ggplot(aes(x=date, y=close))+geom_line()

And here is my plot:

Now I would like to have a colored background. The first half year(jan-june) should be blue an the second half (july-dec) should be red.

With

GDAXI<-GDAXI%>%mutate(semester=lubridate::semester(date))

you can add these semesters. (jan-june ==1 and july-dec ==2)

And now I want to add a colored background with geom_rect()

GDAXI%>%
  ggplot(aes(x=date, y=close))+geom_line()+ 
  geom_rect(aes(xmin = date, xmax = date, ymin = -Inf, ymax = Inf, fill = semesters)

But this is not working. Can someone help me to solve this problem?