It's based on how long the boss could survive if wailed on nonstop by the PCs - that is, if they didn't have to bother healing, reviving, curing status, etc. There are a lot of disagreements between people about how much HP a hit or round survived is worth, how much PCs even do at any given point of the game (this is a key issue with Grahf in particular), whether or not to give the boss a boost because he has someone else to divert damage from him in game (I find this practice retarded myself but some people swear by it), etc. But that's the basic idea. (There are also voters who as you said do not give a shit about any of the above considerations and vote based on story power instead, but they're rare and none of them are making anything about Grahf's fight this week)
"Highest HP of any boss in the game" is always kind of a misleading statement because bosses can take abuse for a long time in some games and a short time in others. That statement means a lot in, say, the original breath of fire where it takes 32 turns from the big uber final transformation to kill the final boss and... christ, it has to be over a hundred from a normal PC. But in final fantasy 6, where an endboss would be lucky to survive say 10 hits, it means much less. These are more extreme ends of the spectrum, XG I see as somewhere in the middle, a bit on the lower end but still high enough that its toughest boss can take 2 hits from Georg.
