Sounds like your pinion bearing is going out. If you get that swapped out now, you can probably avoid trashing the rest of the diff. Still probably looking at 4 hours labor to swap it assuming the shavings didn't get too far into everything else. Axle shafts and carrier have to come out, to pull the pinion to swap the pinion bearing.
FWIW - I drove a good 15,000+ miles after mine god noisy. I don't know if the worn bearing contributed to the pinion shaft shearing off (there was no scoring or other heat damage), but when the pinion gear got sucked into the diff, then push out the bottom of the housing, it wasn't pretty. I did not have much driveline play, so that's why I put it off. Since you have visible driveline slop, yours might be in worse shape.
2000 E450 dually V10 wagon