Blackartemis: It's not too hard to take off the interior 1/4 panel. First, you need to unbolt the seatbelt where it mounts at the top with a T50 torx. Then the panel around the window is clipped in, so it just pulls out. Then for the bottom panel, there's a fastener holding it in place on the top. After that, there's a clip holding it in place in the front, right behind and about 8" or so up from the sill plate. Be careful when you take this one out, because the piece that holds the clip is fastened to the panel, it's not actually part of the panel, so it can rip out if you're not careful. Then there's just one or two more clips holding it in.
To help the rattles from the back, some 1/8-1/4" foam will help if you attach it underneath the rear shelf, where the rear seat and the panel attach to the front of the trunk, and where the panel mounts at the top. My door panels rattled a little bit, so after I deadened them, I used some 1/2" foam on top of that to get rid of the plastic rattling against the door. Silicone, like 302stang said, will also help to eliminate some rattles, it'll just smell for a day or two.
Also, to add to Joker's list of deadeners, you could also look into Second Skin.