Can data be used to harm other people? The reality is that in some parts of the world, it is already being used to trial increased surveillance and fuel oppressive social systems. This is a clear indication that technology will only ever be a reflection of the human hands responsible for its creation and maintenance.
In this episode, Alexander McCaig and Jason Rigby discuss the social implications of China’s authoritarian hold on its citizens—particularly its military-industrial complex’s creation of a three-billion US dollar supercomputer satellite center in the country’s Wenchang Spacecraft Launch Site.
Despite the official statement claiming that this enormous tech site will be used as part of a “massive constellation of commercial satellites” that can “offer services from high-speed internet for aircraft to tracking coal shipments,” we think it pays to be a little critical of the difference between what’s being reported—and what’s being done.
China has been the subject of widespread criticism after it announced the development of a social credit system in 2011. This system builds on a mass surveillance structure of more than 770 million cameras installed across the country as of 2019, with expectations that it will eventually hit the one billion mark by the end of 2021.
The social credit system is used to score individuals and companies based on a collection of data from different sources. Individuals are rewarded for appropriate social behavior, such as proper conduct on mass transportation systems and adherence to waste sorting rules in their city.
Conversely, they will be punished for “negative behavior” which can include elderly residents suing family members for not visiting regularly (Shanghai); cheating in online video games (Suzhou), failing to show for reservations at hotels or restaurants (Suzhou); and failing to pick up any take-out food that was ordered (Suzhou).
Individuals with poor credit score will face restrictions on loans, transportation, and even education. As part of the system’s effort to encourage good behavior, some local governments offer incentives to those with a higher credit score. These people will be prioritized in health care provision, and can even waive deposits to rent public housing.
Businesses are required to submit data on their operations and on their partners and suppliers. Their credit score can be influenced by their behavior and ratings from their suppliers.
Finally, individuals and businesses that are deemed “untrustworthy” will be publicly named and shamed.
A society where human behavior is closely controlled and dictated by the state, through the latest technological capabilities, sounds like the plot of a dystopian sci-fi novel like George Orwell’s 1984—but this is already the reality for more than a billion individuals and 28 million companies in China.
What are the implications of the social credit system? Critics are quick to point out that the government is using incomplete or inaccurate data to determine the provision of social privileges, and sometimes rights, for their own citizens.
The implementation of a stable credit system is also dependent on the strength of basic services, such as regulation in the credit industry and data protection. Those with lose credit scores may find it difficult to continue to progress in society, particularly if there is no concrete policy that can support their rehabilitation or reintegration.
Lastly, in a world where we have yet to fully account for all the factors that contribute to how and why we make decisions, this could easily turn into a system that disproportionately punishes people and businesses who are already struggling.
Beyond 1984, the hit television series Black Mirror also showed a glimpse into a society where people are controlled by their data. One of their episodes, entitled Nosedive, draw viewers into a world where everyone’s social status is controlled by the quality of their ratings on social media. A series of unfortunate events that are outside of the protagonist’s control have a massive impact on her credit score, which in turn cripples her socioeconomic status.
These important pieces of media and literature highlight how the agnosticism of data, alongside the impact of human intervention, can drastically change the impact of technology. It is our responsibility to ensure that AI is developed with a conscientious hand, and that it is capable of empowering minorities instead of widening the inequality gap.
Humanity’s thirst for innovating new and exciting ways of harnessing technology compels us to participate in a shared initiative: one that will help preserve our free will, personal autonomy, and human rights.
The TARTLE platform is our life’s work toward ensuring that your personal data remains personal. Everything you share is given with your full consent, and we help you connect to other like-minded individuals and organizations who can represent your interests.
