I wouldn't necessarily say there are any "downsides" to holding CDRs, but there are a lot of "it depends" situations with them.
They are hedged investments, meaning it will protect you from currency fluctuations. However, you will pay for that hedging at the cost of around 0.6% a year.
My general thought process around CDRs is this:
If you are in retirement, or close to retirement and do not want large scale currency fluctuations impacting your investment portfolio, they are very strong options. In addition to this, if you are a retiree that doesn't travel much and utilizes the CAD the most, it can be very convenient to have these CDRs as you don't have to be constantly converting your USD dividends into CAD to use them in your home country. You simply buy the CAD listed CDR, and you get CAD dividends.
If your time horizon is long, I tend to lean towards paying the currency conversion and owning the USD stock. This is because currency fluctuations tend to even out over the long-term, and in the end you'll just end up paying a 0.6% annual fee to end up at the same place, minus all your fees.
Ultimately, it is all up to the individual though.