[Edit 2/1/2021: Adding some more mechanics obscura here which I had back in 2014 but forgot about until now. As before, these are courtesy of slartifer and/or 1whoistornapart from GameFAQs]
[Edit 8/13/2021: Adding a few more bits of extreme obscura re: the Chance spell, credit to airkix72 from the DW3 speedrunning community on Twitch for digging this data up and to MisterHomes for the run that prompted even looking into it]
I've included everything from the old post as well, just so that everything is in one place.
Damage
DQ3 NES's standard damage formula is: DMG = (ATK-(DEF/2)) * [102..153]/256
This makes ATK/2-DEF/4 a reasonable approximation, with variance of roughly +/- 20%.
If the formula would return less than 1 damage, you have a 50% chance to miss and a 50% chance to hit for 1.
The weapons that do bonus damage to specific enemy types add a flat bonus of 16-31 damage. These are Dragon Killer (against any enemy flagged as a "dragon") and Zombie Slasher (checks resistance to Expel).
Bikill status doubles damage on non-critical hits. Critical hits are not affected by Bikill.
An attacker using the Multi-Edge Sword will receive backfire damage equal to (Damage/4)+1.
A defender wearing the Swordedge Armor will cause attackers to receive backfire damage equal to (Damage/2)+1 if the attacker fails an Expel resistance check.
Poison Needle bypasses the standard attack checks, including enemy evasion (read: it's autohit). It has a 1/8 chance to kill a non-boss target outright (regardless of Beat resistance). Otherwise, it will do exactly 1 damage. It cannot instant-kill bosses, and will always hit them for 1 damage.
Critical hits
Critical hits use their own damage formula: DMG = ATK * [54..64]/64
In other words, a critical hit does between 84.375% and 100% of the attacker's ATK in damage (average ~92%) and ignores the target's defense.
Most PCs have a 1/64 chance to critically hit on each attack. A PC wielding the Demon Axe has a 1/8 chance to critically hit which *replaces* the standard critical hit rate. A Fighter has a (Level/256) chance to critically hit *in addition to* the standard critical hit rate.
Evasion
PCs have a base 1/64 chance to evade any physical attack. Enemies can have different rates, but 1/64 is the most common evasion rate for enemies as well. After the standard evasion check, the following additional checks are made as well:
A PC wielding the Demon Axe has a 1/8 chance to miss.
A PC afflicted by Surround status has a 5/8 chance to miss.
A PC wearing a Cloak of Evasion has a 13/64 chance to evade any physical attack. (Note that this is a separate roll from the standard 1/64.)
Magic damage
Every spell in DQ3 has a set damage range that it deals, and these ranges differ for PCs and enemies.
The ranges are as follows for PCs:
Blaze 8-13, Blazemore 70-89, Blazemost 160-199
Firebal 16-23, Firebane 30-41, Firevolt 88-111
Bang 16-23, Boom 52-67, Explodet 120-159
IceBolt 25-34, Snowblast 42-57, Snowstorm 88-111, IceSpears 60-79
Infernos 8-23, Infermore 25-54, Infermost 60-119
Zap 70-89, Lightning 175-224
The ranges are as follows for enemies:
Blaze 7-12, Blazemore 52-62, Blazemost 92-128
Firebal 10-18, Firebane 22-34
Explodet 60-80
IceBolt 16-24, Snowblast 32-42, Snowstorm 55-67
Infernos 6-18, Infermore 14-34, Infermost 30-62
Additionally, these are the damage ranges on enemy breath attacks:
"Weak" fire breath (Flamapede, Heat Cloud, Lava Basher, Wyvern, Voodoo Warlock): 6-10
"Medium" fire breath (Evil Mage, Sky Dragon, Lava Basher, Wyvern, King Tortragon, Orochi, Hydra, Green Dragon, King Hydra): 30-40
"Strong" fire breath (Salamander, Baramos, Baramos Bomus): 80-100
"Weak" ice breath (Demonite, Glacier Basher, Snow Dragon, Frost Cloud, Minidemon, Scalgon, Archmage, Putregon): 9-21
"Medium" ice breath (Snow Dragon, Scalgon, Putregon, Zoma): 40-60
"Strong" ice breath (Zoma): 100-140
RobMagic normally steals 5-10 MP from its target if it hits; due to a bug, when used against a target under Bounce, the caster usually gains a very large random amount of MP due to the game reading garbage data.
The "Chance" spell
In random battles, Chance generates a random number between 0 and 16 and chooses an effect based on that number as follows:
0 - All surviving enemies are forced to flee the battle, ending it and awarding EXP/gold for any enemies killed previously
1 - Restores 100% HP to all living party members
2 - Revives all dead party members with 100% HP and restores 100% HP to all living party members
3 - Restores [41..48] HP to all living party members
4 - All party members and enemies become Confused [ignores status protection/resistance]
5 - All enemies are instantly killed [ignores status protection/resistance]
6 - All enemies are instantly killed [ignores status protection/resistance]
7 - All spells are nullified for the remainder of the battle
8 - All party members' physical attacks are guaranteed to critically hit for the remainder of the battle
9 - All party members and enemies are put to sleep [ignores status protection/resistance]
10 - All remaining actions in the current turn are skipped; for the next 3 turns, only the caster of Chance may act
11 - The marching order of the party is randomized
12 - The current battle immediately ends with no EXP or gold awarded, even for enemies previously killed
13 - All remaining enemy actions in the current turn are skipped; for the next turn, only the party acts
14 - Drains all MP from all enemies and gives it to the caster
15 - Cures Numb (paralysis) status for all party members
16 - Does nothing
In any scripted battle, Chance generates a random number between 0 and 15. (In technical terms, I believe it generates a number between 0 and 255 and then simply ignores the four high bits, but this shouldn't actually matter.) If the generated number is 12 or greater, then it restores 100% HP to all living party members; otherwise, it does nothing.
Initiative, speed, etc.
At the start of any non-boss battle, enemies have a 1/32 chance to ambush the party.
If they do not, the party then has a 1/32 chance to ambush the enemies.
Turn order is loosely determined by Agility, but the variance is extreme: the game picks a random number between 25% and 100% of each combatant's Agility stat, and then each combatant acts in order from highest result to lowest result. In case of ties, priority goes through all PCs left to right, then all enemies left to right.
Enemy HP and regeneration
Enemies are generated with HP randomly between 75% and 100% of their maximum.
Some enemies regenerate HP each round (mostly bosses) as follows:
Weak regeneration (16-23 per turn): Scalgon
Medium regeneration (44-55 per turn): Stone Hulk, Troll King, Baramos Bomus, Baramos Gonus, Kandar 2
Strong regeneration (90-109 per turn): Orochi, Boss Troll, King Hydra, Baramos, Zoma
Enemy regeneration and/or other healing is capped at maximum HP, regardless of the HP value the enemy actually generated with.
Encounter rates and the Golden Claw
DQ3 uses a simple probability check for random encounters happening, made every step, and varying by terrain. For every step, there is an X/256 chance for a battle to start, where X is equal to:
On ocean: 4 in the daytime, 5 at night.
On plains: 10 in the daytime, 13 at night.
On bushes or desert: 15 in the daytime, 19 at night.
On forest or swamp: 18 in the daytime, 22 at night.
On hills: 25 in the daytime, 31 at night.
On any tile in a dungeon: 10 regardless of time.
When stepping through a screen transition in a dungeon such as a doorway: 84 regardless of time.
If you're carrying the Golden Claw, X is always 100 regardless of terrain and time. Therefore, it multiplies the encounter rate by roughly 6.
Goof-Off Antics
Goof-offs learn new "antics" in the same way spellcasters learn spells. All goof-off antics use the 50/50 method of spell learning, meaning that once the minimum eligible level is reached, the antic has a 50% chance to be learned every level.
A Goof-Off has a 65/256 chance to perform an antic each turn instead of the command you selected. Here is the list of antics with the minimum level each can be learned; if no effect is stated in parentheses, the antic simply wastes the Goof-Off's turn.
L2: Assess the situation
L3: Sing a song
L5: Take a break
L7: Smile
L10: Get lost in thought
L11: Kind glance
L12: Cheerlead
L14: Attempt to flee (Prints the normal message for attempting to run, but always fails)
L15: Trip over self
L16: Spruce up
L18: Parry (actually executes the PARRY command)
L19: Beckon the heavens
L20: Hone body
L21: Smug remark
L24: Recoil
L26: Point finger
L27: Pinch
L28: Jazzercise
L29: Curious dance (Prints the same message as the enemy action, but always fails)
L30: Call for reinforcements (Prints the same message as the enemy action, but always fails)
L31: Fake Explodet
L32: Pokes dead monster
L33: Fake arms merchant
L34: Chew gum
L35: Numb self (Inflicts Numb status on the Goof-Off!)
L37: Feign death
L39: Badmouth
L40: Satori
L45: Bluster
L50: Soft soap
L60: Expose tattoo (Only female goof-offs will use this antic; male goof-offs will still "learn" it, but if it's selected in the process described below, they'll perform the command you selected instead of doing it)
L70: Blow kiss
L85: Awful pun
L95: Sock whiff
The antics are organized into magic "pages" as if they were actually magic, and the game selects which antic the Goof-Off uses the same way it would select the spell used by a confused character: it chooses a page of the magic list at random (with equal probability), then chooses a learned spell from that page (again with equal probability). These are the "pages" for Goof-Off antics:
Page 1 - beckon, recoil, take a break, lost in thought, spruce up, satori, hone body, jazzercise
Page 2 - sock whiff, assess situation, poke dead monster, trip over self, sing, dance, chew gum, smile
Page 3 - smug remark, point finger, badmouth, feign death, call reinforcements, attempt to flee, bluster, fake Explodet
Page 4 - pinch, cheerlead, expose tattoo, fake arms merchant, awful pun, blow kiss, soft soap, kind glance
Page 5 - numb self, parry
As mentioned above, if a male Goof-Off attempts to use the "expose tattoo" antic, it will be ignored, and the Goof-Off will instead use the intended command.
Running from battle
Under normal circumstances, the chance to successfully flee from battle in DQ3 is 50% on the first and second attempts, 75% on the third attempt, and 100% on the fourth attempt.
If the Hero's level exceeds the level of all enemies in the battle by 10 or more, fleeing will always succeed (unless the battle is a boss battle). The highest level of any normal enemy in the game is 45, so a level 55 Hero's party will always succeed at running away from almost any battle. (Exception: Red Slimes, for some reason, are flagged as level 63, which is the level assigned to boss enemies.)
Fleeing will always fail if the battle is not a random encounter.
Resistance values
Enemies in DQ3 have a resistance value between 0 and 3 for each of the following categories of attacks:
Blaze/Blazemore/Blazemost/Firebal/Firebane/Firevolt/Bang/Boom/Explodet
IceBolt/Snowblast/Snowstorm/IceSpears
Infernos/Infermore/Infermost
Zap/Lightning
Beat/Defeat
Sacrifice
Sleep
StopSpell
Sap/Defence
Surround
RobMagic
Chaos
Limbo/Slow
Expel
Resistance values against a magic category cause spells in that category to fail at the following rates:
0 resistance = 0%
1 resistance = 77/256 (approximately 30%)
2 resistance = 179/256 (approximately 70%)
3 resistance = 100%
Enemy targeting
Enemies' basic physical attacks will obey one of four targeting flags set on a monster-by-monster basis. The four possible targeting flags are as follows:
Normal: Target is chosen weighted by positioning at the start of the round. (The weighting will be explained below.)
Unweighted: Target is chosen randomly with an equal chance of hitting any party member at the start of the round.
Smart: Target is chosen weighted by positioning at the monster's turn (meaning it'll adjust for party members dying before its action).
Concentrated: Target is chosen randomly with an equal chance of hitting any party member at the start of the battle, and for the duration of the battle, all concentrated-targeting enemies in the battle will hit *only* this party member.
The targeting weighting uses four sequential rolls, presuming four party members:
First, the attack has a 113/256 chance to hit the lead party member.
If it does not, then it has a 180/256 chance to hit the second party member.
If it does not, then it has a 235/256 chance to hit the third party member.
If it does not, then it will hit the fourth party member.
This overall results in targeting chances by slot of approximately 44.14%/39.28%/15.22%/1.36%.
All attacks *other* than basic physical attacks always use unweighted targeting.
Other enemy AI bits
Some DQ3 enemies are flagged as "MP aware". Enemies with this flag will not attempt to cast spells they lack sufficient MP to cast and, if they are afflicted with StopSpell, will not waste turns trying to cast spells.