An old (really old) trick for testing belts used to be to hold a bar of soap under the belt with light pressure; the soap would "lube" the belt for a few seconds and a squealing belt would become quiet for a minute, telling you the belt was glased.
I've never done this old trick on the new belts, as they are ribbed, so I'm not sure if it would work or not. If your belt has over 50K on it, I'd change it anyhow, and use the "gatorback" belt sold at NAPA and other parts places.
It could be a bearing in the alternator, or idler pulley, but I'd suspect the belt first. You should be able to pin point the noise with the hood open and engine running. I've also used a stick, placed on the alternator and the other end at my ear; if the bearing in the alternator is bad, the noise transfers through the stick and you'll hear it when you lightly press your ear to the stick. Another old trick if you don't have a stethiscope.
Give an answer, they learn for the day; suggest a search, they learn for a lifetime.
2005 F150 STX Regular Cab (4.2 liter V6)