Rstudio Keras network structure causing trouble

I am new to Keras and was trying to use bidirectional-lstm to classify each string in a text. Could someone please help out in getting the network structure right. On a higher level, I am trying to use the hidden state of each cell as an input to a classification problem.

*Shape of data tensor: 2090 500

Shape of label tensor: 2090 500 5*

embedding_dim      <- 100

model <- keras_model_sequential() %>%
  layer_embedding(input_dim = max_words, output_dim = 32) %>%
  bidirectional(layer_lstm(units = 32)) %>%
  layer_dense(units = 5, activation = "softmax")



model %>% compile(
  optimizer = "adam",
  loss = "categorical_crossentropy",
  metrics = c("acc")
)

history <- model %>% fit(
  data_seq, labels_seq,
  epochs = 10,
  batch_size = 128,
  validation_split = 0.2
)