/etc/usbmount/usbmount.conf

There are a few changes you can make here, but the one we are interested in is one of the first ones available. Changing ENABLED to 0 will disable this add-on (at least not auto mount your drives).

# Change to zero to disable usbmount
ENABLED=0