Match strings using regular expressions but ignore (not exclude) certain phrases containing the matched string

I'm trying to find a way to match certain strings within a dataset, but ignore (not exclude) some expressions that contain a match.

clin_pres <- c("Patient A received yellow fever vaccine, and had a fever", "Patient B received the yellow fever vaccine but had no fever", "Patient C returned from Bali yesterday and now has a fever", "Patient D had no fever last week but now has a fever")

So in this example, I would like to find all matches with the word "fever", but ignore matches with the strings "yellow fever vaccine" or "no fever"

I know I can do

grepl("fever",clin_pres, ignore.case = TRUE) & !grepl("yellow fever vaccine",clin_pres, ignore.case = TRUE) & !grepl("no fever",clin_pres, ignore.case = TRUE)

Which outputs: [1] FALSE FALSE TRUE FALSE

But I just want to ignore "yellow fever vaccine" and "no fever" as matches, not exclude them when matched, to obtain output: [1] TRUE FALSE TRUE TRUE

Any help or suggestions please?