So a bit of context, I'm new to driving having recently gotten my learners permit and I'm still quite nervous behind the wheel. This happened the third time I was driving. I live in a rural area, with a small town nearby. I had practiced by driving to and from a stores parking lot and that was where we were headed that day. My brother was in the passenger seat and we ended up reaching the local school zone at around 4 pm. I slowed down to 20mph and everything went smooth the first time through. On our way back about 10 minutes later, I slowed down again when we reached the school zone. As I drove through, the guy behind me began to honk which made me look into my review mirror for a second. He was flipping me off and throwing his hands up in the air. Eventually we got through and I got back home in one piece. I'm not sure if I did something wrong, but I do know the situation has made me alot less confident. Since then I haven't wanted to practice driving, especially in town. Am I the asshole in this situation?

Edit: The posted speed limit in effect during school hours is 7 am to 5 pm at 20mph in my state

Edit 2: I'm sure it seems like it should be obvious but I really don't know, I'm knew and nervous and just want to drive safe and hear others opinions on the situation so I can learn to be a better driver, especially in situations involving not so happy operates of big fast death machines