If you always shoot the 55 gr well, the 100 gr is a big jump. The twist rate on your barrel may be set up for the lighter bullets (a higher numerically = slower) twist rate. The havier stuff needs a faster twist rate (lower number).
Check and see if you can find what twist rate you have. If it's a slow rate, it's not stabilizing those heavy bullets properly, and you won't get good accuracy.
EDIT... just re-read and saw that you have shot the 100 gr before... there goes all my typing for nothing.
If your gun likes the 100 gr, shoot that. Sorry, don't know what to say now.