In neutral with the clutch out, they all rumble as if they're about to fall apart.
If this has become worse, it's likely that your clutch is getting a little old and as you bought it used, there's no way of knowing how it was treated before you got it.
The cam chain is always under tension whether the bike is in neutral or not, clutch in or out. Not likely to be the cam chain IMO.
If you're running a magnetic sump plug, what does that look like when you're changing the oil? Metallic fragments?
Linz