I have tried to run ipv6 several times, but every time I do, it always slow down the network even following guides to the dot.

I am running opnsense on bare metal (high power I9 9th gen and 32GB ram so no resource issues).

So this is the regular setup I am doing.

1. Firstly the ISP give native IPV6 so enabling IPV6 with mask 56 as the ISP has instructed
2. Setting the network interfaces and vlan to track iterface and set IPv6 Prefix ID to a different for each VLAN
3. Enable IPV6 in the firewall (add accept all in testing) on all vlans
4. Enable IPV6 in the settings (allow ipv6)
5. Enabling the IPV6 DHCP server
6. Turning on the assisted mode in Router Advertisement Daemon
7. Making sure all switches have IPV6 enabled
8. Adding both IPV4 and IPV6 dns in Unbound DNS over TLS
9. Doing a IPV6 check on https://test-ipv6.com/ and get all green

For some reason everytime I have done this I get some strange issues (random disconnects, routing issues and strange random slowdowns. I am not sure what to try anymore to get ipv6 to work properly. As from what I understand now with matter (smart home protocol) is needs ipv6, and also as a homelabber its important to start embracing ipv6, at least for dynamic setup. I just cant get the grasp over where I am stepping wrong.

Any suggestions?