How I can take mean for different subsets of a specific column in a data.table?

Given a sample data frame:

dt <- data.table(value=1:10,start=c(1,4,5,8,6,3,2,1,9,4),finish=c(3,7,8,9,10,10,4,10,10,8))

I want to add a new column which may be named as mean_column. i'th row of this colum should have the value

mean( value[ seq( from = start[i], to=finish[i] ) ] )

The real data I'm working on has 20 million row, so I need to find a fast way to do this calculation.

Edit: the value column in the data.table doesn't need to be an ordered sequence as in the example. Each value in this column may take any positive number.