Did you read the article? If you're white you don't get to say what's racist. Black people are experts on their own experience.
The Tea Party popularized the Gadsen flag as a symbol of opposition to our first Black president. Prior to that, it was just a historical relic. But its recent use characterizes it as racist.
It's the same way with the swastika, which originally represented the Sun as the source of energy and life on Earth. Traditionally, it is a good luck symbol in many cultures all over the world. But then the Nazis came along and reversed it and mis-appropriated it and now it's generally recognized as a symbol of hate.
Symbols mean whatever they are understood to mean, and their meanings can change over time.