I feel like the gameplay really appeals to people who like that type of gameplay. The games are really enjoyable if you're into that rougelike/dungeon crawler-esque kind of gameplay, but I can see how more casual audiences might not be into that. Maybe that's why these games never really got great reviews or sold that well even though they were great (except for Gates to Infinity).
Actually I think that's the reason why Gates to Infinity wasn't very good - they tried to water down the gameplay to appeal to a more casual audience, but not only did they fail to capture that audience, they also let down their already-established audience by making the gameplay objectively worse than it had been in previous entries. I also feel like this is where Super succeeded - they improved on the gameplay and made it more accessible without dumbing it down or casualizing it.