According to Uncommon Objects,
One of the oldest recorded uses of the swastika in Africa is found in artwork by the Akan people of Ghana. It is also found on servants’ dresses in the Ashanti Empire…

Click the picture to enlarge, then look at the cloth pattern of the woman in the middle.