ggplot of Time Series Forecast of Daily Data

I have some daily data that look like below:

        Date Weight
1 2019-09-18   69.4
2 2019-09-19   69.3
3 2019-09-20   69.8

using the package forecast and ggfortify

I was able to get a graph that looks like the following:

The problem is that for Date axis, all the labels are numeric instead of Date

I tried alot of things, and what I thought would most likely work was to convert it back using a trans

tran <- trans_new("date-tran",
                  transform = as.Date,
                  inverse = as.numeric)
p <- autoplot(fit)
p <- p + labs(x = "Date",
              y = "Weight (kg)",
              title = title)
p <- p + scale_x_continuous(trans = tran)

But that doesn't work -

Error in as.Date.numeric(value) : 'origin' must be supplied

Is there a way for me to either replace the x-axis or make forecast work with Date instead of numeric?