You could test for a current drain, rather than just guess at it.
Attach a DMM set to the DC amps range between the positive terminal of the battery and the positive battery cable. Close all the doors, but you can leave the trunk open. After a few minutes, the trunk light will go off, and the current will go down by almost 1 amp. Wait about thirty minutes (you must leave the meter connected and on the whole time). The current should now read 50 mA or less (that is 0.05 A or less). If it doesn't, you have an excess drain somewhere. You can start pulling fuses in the trunk to look for it. If one of the fuses you pull stops the drain, then check and see if that fuse goes to another fuse panel, if so, put the fuse back in (and open the door or hood and wait 30 minutes again). Now you can check the fuses at the sub-panels.
If you find that you don't have a high drain, then it might be an internal battery fault, or it could still be a load that just doesn't always show up. The best way to tell is to leave the battery disconnected. Wait the number of days it normally takes for the battery to go dead, and reconnect it. If it's dead, then the battery has an internal short and is draining itself. If it's still good, then the problem is in the car.