I work out, and my good drives go about 300 (I average about 260). I am not swinging for the fences either...

Hint: a golf swing involves your entire body, including your lower body! A full body workout routine will increase your distance dramatically. It also increases your consistency, since stronger muscles have better muscle memory.