try some closed cell foam. you can get a scrap piece from an upholstery shop, usually for nothing. it is usually black, so it would be hard to see.
i have several scrap pieces of different types of foam, just in case i get a rattle i want to take care of.