This is sort of my version of Tiigers advice above...
When using my target rifle I have an adjustable rest at the front and bag at the rear. Because the rifle is supporter by the rest and bag all I really have on the rifle is my thumb on the back of the grip to support my hand and a finger on the trigger.
Once I'm nestled into the rifle I just do subtle movements with my shoulder to aim from the butt rather than pushing the grip around. I exhale before a shot and hold it so my body goes still, then all I do is pull one finger back on the trigger. There isn't much of me touching the rifle to shake it around.