I took a look and its not exactly strong on specifics. Zimbabwe ruins are older than 1600's ... on the other hand I seem to recall Dias' expedition only interacted with Khoisan along the SA coast, and that was in 1488. I'd really have to study the history better to give specific answers though.
I checked my Jared Diamond book. There was a section of South Africa that the Bantu left to the Khoisan due to poor potential for agriculture, which would explain what you were referencing.