When was his prime? Hard to say actually he went 49-20-1 with 31 ko's. He was ko'ed by Tiger Jack Fox when he was 23 and lost again the following year to him. He was knocked out by Abe Simon in 6 when he was 26. From 1946 to 49 he went 3-5 with losses to Charles, Louis twice , Maxim and Elmer Ray. From 1950-53 he went 2-4 with losses to Rocky twice, Charles and Rex Layne. He did win rematches with Maxim and Ray but Joe never really put together a solid string of wins during his career that stand out. You'd think a man was prime between 24 to 33 but with Walcott? Could be he really didn't have a stretch considered prime performances, he was simply consistently inconsistent. .
Later on, Walcott improved as an actor. His performance in "The Harder They Fall" (1956) starring Humphrey Bogart was more convincing than in "The Quicker They Fall" (1953) co-starring Rocky Marciano.