Hi. I've been thinking about this for way too long and have reached R's equivalent of writer's block. I have a dataset of continuous, ordinal and categorical variables with an ordinal outcome that has four levels, and the outcome of interest is rare (5%). I want to subset the data by outcome level, then take samples of one predictor from each subset and compare them. For example, lets say I sample 100 observations from a continuous variable at outcome levels 1 and 2, so I'l have two vectors from the same variable with different outcomes. I now want to compare those vectors with each other to see if their distributions vary by outcome. Am I causing myself trouble by doing this with a t test? Does bootstrapping this process and using the mean p-value of all boots impact my type 1 error rate?

I want to do the same thing with the categorical variables as well, comparing counts of each factor's levels between subsets. Will Fisher's exact test work on both the nominal and ordinal data? If not, what test should I read up on? Would bootstrapping here cause issues?

I think the samples could be considered independent because they come from people, and each person appears only once in the dataset. My thought with the bootstrapping is to create a new dataset with equal counts of outcome levels for easier model training, then using the orginal data as my test set. Kind of like over and undersampling at the same time I guess. I'm curious to hear what you think. Thank you!