Set up three bowls for the battering process. In bowl one combine flour, pepper, salt & cayenne. In bowl two, whisk together egg and buttermilk. In bowl three add 1 cup of panko breadcrumbs to begin (add more as needed).
Line shallow rimmed baking sheet with parchment paper.
Working in small batches, dip shrimp into flour, then egg, then panko. Place on baking sheet. Chill coated shrimp for 30 minutes before frying.