I can't speak for everyone obviously, but TWEWY's system annoyed me because I'm one of those people that hates grinding, and fighting randoms that are wholly optional like that feels like grinding. At least with randoms on the field they're actively trying to get you into a fight usually and as such encounters feel like part of the main sequence. When the encounters appear to be totally optional but really aren't since you need to do them regardless, it gets on my nerves. I'm not sure if I'm explaining this well or if anyone else feels this way, but figured I'd get this out there.
On a more mechanical level, if you do go with entirely optional randoms, how do you plan to balance any required fights? Assume no fighting? Assume fighting everything? Something in the middle? If you balance for no fights or make it so the rewards don't affect the balance much, then some people aren't going to fight anything. If you balance for a lot of fighting, why make the encounters optional in the first place, since you're just going to screw someone over.
One possible option is something like the random encounter system from Parasite Eve, changed a little bit. In PE, there were no random encounters, only areas where a fight had a chance to start when you walked into them. The first time you hit them, you always got a fight, and the chances went down from there for other times you wander into that screen. If you added onto that some on-screen indication that there were monsters at those spots, it may work for you.
I guess I really have to ask what it is people hate about the invisible randoms, so I know what kind of suggestions to make. I don't have a problem with them, so another thing I would have suggested is something like the Ar Tonelico 2 system, which has invisible randoms but also has a gauge that empties as you fight. As it goes down it seemed like the encounter rate got lower(not sure on that, that's just memories of vague impressions), and when the gauge emptied there were no more random encounters. For people that want to power up a bit more, if you exited and entered an area the gauge refilled. For people that just want to explore and look for stuff while not being bothered, eventually there were items you could use to empty the gauge. Worked pretty well for me but I, again, don't have anything against invisible randoms.