2004 5.4L exhaust leak or cam phasers??
 

I know what everyone's going to say:

"It's your cam phasers!" ... "timing chain issue!"

I did think that this was the problem at first but I'm starting to think that it may just be as simple as a really bad exhaust leak.
When it starts up cold(i mean like ice cold, engine hasn't ran for a day or two), this sound isn't here or at least extremely un-noticeable.
As the engine heats up, metal expands, so my thought is the leak opens up as the engine gets hotter. because the noise gets worse.

After the engine settles down to idle at around 800 rpm, its got that dreaded diesel sound. There really isnt any "ticking" as described with a bad/loose timing chain, so my thought is either an exhaust leak, cam phasers, or a combo of both.

I know this doesn't cause damage, but its just really annoying the crap out of me.

I'd appreciate some feedback from mechanics or anyone who's done work on this motor on how to fix it or what it might be.

Thanks!

(truck is an 04 XLT with the 5.4L triton)