EOM: equations of motion mode and allows you to fire off-boresight (e.g. you can fire while flanking the SAM site), and the range indicated is calculated depending on your aircraft's attitude, and obviously even less than in range-unknown mode if you're e.g. flanking the SAM site, because the missile will have to turn towards the site first, which costs energy, and as a result, reduces range. The most exact mode. Primarily used when we have a steerpoint on the exact location of the SAM. HARM missile lofts for greater range. EOM can fire highly off-azimuth. EOM mode is very useful when you are trying to get inactive radar to activate and start pinging you, you can pass the site close, for example on your left side, ready to bank and run, and fire the missile at the same time, you don't have to point your nose at the radar. Allows you to fire while not pointed at the target.
EOM has a precise footprint for when you know exactly where the emitter is and you don't want it to home in outside this area. For absolute standoff range I don't expect that EOM is dramatically worse than PB.
PB: prebriefed mode and has the longest range, because it assumes that the coordinates that were entered are very close to where the SAM's radar actually is. You will have to follow the cue on the HUD for the missile to be able to reach the target if you're firing at maximum range. A little less exact than EOM. Primarily used when we have a steerpoint in the vicinity of the SAM. HARM missile lofts for greater range. Lofting cues are also shown on HUD in this mode to increase range by aircraft pitch up. PB has greater range (range depends on altitude and speed of the aircraft that fires the missile) but nose needs to be pointed towards the radar, it is great to use against radars that are already emitting and you don't want to get close and risk getting shot. PB only permits a very few degrees off. Its mostly for long shots. Inhibit firing from more than 5 degrees off-target laterally.
Historically PB was a mode of launch in the F-4 (and other planes) involving a loft. The PB mode in the F-16 is the only one which inhibits launch outside of a certain azimuth of bearing directly to the target point. It has a relatively early and wide search program which is good for making sure it hits something. The downside is you can hit the wrong thing is a busy environment and off-axis attacks aren't supported.
The advantage of lofting in PB mode is that when the missile is fired at the indicated ideal loft angle, you will be able to engage the target from further away, meaning, it optimizes the missile's range.
RUK: range unknown mode, works pretty much like prebriefed, but due to the larger search area, the indicated range is smaller than with PB. Used when we only know the general direction of the SAM and not the range. HARM does not loft, and goes active shortly after launch to search for targets with a 120 degree FOV.
RUK is a degraded mode with the widest footprint which is terrible for being selective and worse energy management than PB.