First off I applaud RIOT for starting to experiment with new gametypes. The community obviously has been asking for variety, however, the current Hexakill game is a failure for the following reasons:

Adding another person on the map for each team does not add any new element to the game, other than simply...another ally to rage, feed, afk, etc. In an environment seeped in meta, it's hard enough trying to get a 5 player team on the same page, adding more players just adds more chaos.

With the short lobby time, there is not sufficient time to plan any real attempt at lanes other than blind picking champions, and deciding where the 6th man goes. Where he ends up, the experience curve is skewed and the players do not understand how to adjust. You get lanes severely behind others and the potential for enemy snowballing it greatly enhanced.

If 6 players were the norm, then players could learn to adapt playstyles and have fun experimenting. However, with the brevity of these custom modes, experimenting only leads to headaches and then the mode is removed.

Those who have played the customs have also experienced the increased lack of interest in playing games out, as in more likely to AFK. Since there isn't really anything new to engage players in this mode, there is no reason to stick around. Basically taking it less serious than even a nonranked normal match.

The reason why the last custom mode (all same champs) was successful was because players felt something different and exciting about the mode. It was hilarious, even if it was completely unbalanced. People enjoyed the novelty of that element. However, hexakill does not bring anything resembling novelty to the table.

Players want customization, they want new exciting elements, they want boundaries pushed (100% cooldown reduction, ridiculous movespeed, crazy physics, new objectives, etc). Take for example the successful custom modes in games like Halo or COD...rockets only, knives only, zero gravity, infinite ammo, etc. These modes offer novel adjustments to game elements they are familiar with but extend them beyond the norm, making the gameplay laughably enjoyable.