Hey all, I'm facilitating a project related to improving the advanced search mechanics of a complex product. As search functionality is a very open ended and can be complex it must be one of the most difficult things to prototype and test for. I think that one reason for this is that any user might string together a multi-variable search in a variety of different ways, with different keywords and a different understanding of the syntax and mechanics. Accounting for all of these different possibilities is impossible and time-consuming to attempt.

At the moment, I'm planning on doing guided tests where I give instructions to the subject about what to search for and will be focused on qualitative feedback and comparing it against the same on the existing product. It doesn't feel like the insights gained from the user testing will be very strong as the guidance is very unlike a realistic environment.

Wondering others view on how to approach testing for search mechanics?