filter function from tidyverse and as.numeric()

I had to force my data frame into character and pivot to get the final outpu I have. I am trying to do the following 3 tasks.

1. all the location of the cells that can't be turned into numeric using as.numeric()
2. filter rows where all the specific columns that end with "_chart" are NA.
3. filter rows where Ref value doesn't match with any of _chart columns

SAMPLE DATA

compared <- structure(list(ID = c("51-07518", "51-07518", "51-07518", "51-07518", 
"51-07518", "51-07518"), name = c("SC5pos", "SC7pos", "SC1", 
"SC4neg", "SC6pos", "Not found"), Ref = c("1", "1", "1", "1", 
"1", "1"), A_chart = c("R2", NA_character_, 
NA_character_, NA_character_, NA_character_, NA_character_), 
    A2_chart = c(NA_character_, NA_character_, NA_character_, 
    "1", "0", NA_character_), B_chart = c(NA_character_, 
    "0", NA_character_, NA_character_, NA_character_, 
    NA_character_), D_chart = c(NA_character_, NA_character_, 
    "1", NA_character_, NA_character_, NA_character_
    )), row.names = c(NA, -6L), class = c("tbl_df", "tbl", "data.frame"
))

# A tibble: 6 x 7
  ID       name      Ref   A_chart A2_chart B_chart D_chart
  <chr>    <chr>     <chr> <chr>    <chr>             <chr>   <chr>  
1 51-07518 SC5pos    1     R2       NA                NA      NA     
2 51-07518 SC7pos    1     NA       NA                0       NA     
3 51-07518 SC1       1     NA       NA                NA      1      
4 51-07518 SC4neg    1     NA       1                 NA      NA     
5 51-07518 SC6pos    1     NA       0                 NA      NA     
6 51-07518 Not found 1     NA       NA                NA      NA

I read the forums and commands from dplyr and thought filter_at? Something like this:

library(tidyverse)
filter_at(compared, vars(ends_with("chart")), all_vars(!is.na(.)))