Filter out both copies of duplicated files in a vector with dplyr

I'm processing files in a directory to convert a .raw filte to .wav. The output goes in the same folder and I need a way to generate a list of raw files that don't have a corresponding wav.

e.g convert this

files  <- c('2019-09-30_16:29.raw', '2019-09-30_16:29.raw.wav', '2019-10-06_09:13.raw', '2019-10-06_09:13.raw.wav', 
'2019-10-06_09:15.raw', '2019-10-06_09:15.raw.wav', '2019-10-11_06:18.raw', '2019-10-11_06:18.raw.wav', 
'2019-10-11_08:55.raw', '2019-10-11_17:42.raw', '2019-10-12_05:53.raw', '2019-10-12_07:34.raw',
'2019-10-12_08:43.raw')

to this

unpaired_files <- c('2019-10-11_08:55.raw', '2019-10-11_17:42.raw', '2019-10-12_05:53.raw', '2019-10-12_07:34.raw', '2019-10-12_08:43.raw')

If I strip the file extensions, duplicated or unique will get rid of the duplicates, but keep one copy of each pair. I want to get rid of both copies to just leave the unpaired files.