I've finally moved away from working on the rendering side of things and have managed to implement a point collision detection system.
To do this I have a slightly decompressed version of the voxels in main memory (for use on the CPU not the GPU). I decided it was best to do the collision detection on the CPU since the GPU is already struggling with the insane number of triangles.
(Side Note: Have achieved slightly better performance by fixing a bug which caused half of the voxels to not be rendered...oops =P)
The CPU version of the data is stored as start and end indices of the voxel segments. Since these are in order I am able to binary search the data. Giving me O(Log2(N)) access to voxels.
This allows me to check whether a single point is filled or empty, therefore providing point collision detection.
I am trying to think of a way to check for intersections with shapes without checking per voxel but I'm not sure how this could work yet.