Is my mathematical thinking right?

First of all thanks for all of you, I am trying to build a research study where I work but anyone here seems to know anything about investigation so I am mostly alone.

Well some context I am reviewing a bank of questions it has like 8k of laws question from a platform that prepares you for your national exam.

The university where I work wants to pair the questions in the plataform with our curricular design so this way students can have the questions that they need to answer paired to our study plan.

I am not an expert, but experts in our school are busy to review 8k so I am reviewing whole questions and classifying them, and I need to prove that my work is as good as an expert

So I thought about conducting a kapp cohens where I ask 3 experts if they would include or not include the question, use the majority to make the final decision, then compare their mean results with mines so this way I can prove my selection has the same consistency as this 3 experts

For this obviously I need to calculate the size of my sample which I think I can calculate it by sample size test with a finite population like predefining a confidence level of 95, a p of 50%, error of 0.05 and my population size.

Can I do this? I ve searched for more literature doing this but most of it’s applied to other areas and I am afraid that for some mathematician thing I am not using the correct formulas and my whole thinking is wrong but I ve no one to discuss.

What do you think?