How to insert grouped median segments in violin plot in ggplot2

I'd like to insert median lines for factor levels into a violin plot in ggplot2. Here's some reproducible data:

set.seed(12)
FactorVar <- sample(LETTERS[1:5], 500, replace = T)
NumericVar <- abs(rnorm(500))
df <- data.frame(FactorVar, NumericVar)

To get the grouped medians I use tapply:

medians <- tapply(df$NumericVar, df$FactorVar, FUN = median)

And this is the code for the plot. As can be seen, I'm inserting each median line individually. That's cumbersome and uneconomical:

library(ggplot2)    
g <- 
  ggplot(data = df, 
         aes(x = FactorVar, y = NumericVar, fill = FactorVar)) +
  geom_violin(scale = "count", trim = F, adjust = 0.75) +
  geom_point(aes(y = NumericVar), 
             position = position_jitter(width = .15), size = 0.9, alpha = 0.8) +
  geom_hline(yintercept = mean(NumericVar), color = "blue", size = 0.8, linetype = 4) +
  geom_segment(x = 0.5, xend = 1.5, y= medians[1],  yend = medians[1], color = "red", linetype = 2) +
  geom_segment(x = 1.5, xend = 2.5, y = medians[2], yend = medians[2], color = "red", linetype = 2) +
  geom_segment(x = 2.5, xend = 3.5, y = medians[3], yend = medians[3], color = "red", linetype = 2) +
  geom_segment(x = 3.5, xend = 4.5, y = medians[4], yend = medians[4], color = "red", linetype = 2) +
  geom_segment(x = 4.5, xend = 5.5, y = medians[5], yend = medians[5], color = "red", linetype = 2) +
  guides(fill = FALSE) +
  guides(color = FALSE) +
  coord_flip() + 
  theme_gray(); g

How can the median segments be inserted in a single command? Also, observe how the median line for factor A is thinner than the others? Why's that?