ok i don't know if this idea will work, but i'll try to explain it the best i can, would it help if you put dynamat or any kind of sound deadner on the sub box, like how dynamat has the speaker kit where you put it on the door so it doesn't vibrate as much, so instead of the door what would happen if you put it on the outside of the sub box, i think its called the baffel, put dynamat all the way around it, wouldn't that help decrease the box vibrating.
I am getting a custom made sub box from my car place here that has something like that inside of the box... I can't remember what its called but they showed me a test car that had that box and damn it sounded better than just a striaght up box!