While AI technologies provide convenience across many areas of life, concerns about how user data is processed are growing significantly.
Recent allegations that widely used AI chat applications unlimitedly label and profile users’ sensitive information — including political views and religious beliefs — have raised serious questions.
The belief that AI always tells the absolute truth is crumbling. Investigations have been launched into the outputs of certain AI applications, and newer model releases have put the “political poisoning” debate in AI back on the agenda.
Companies developing large language models state in their official communications that they take technical and administrative measures to protect users’ personal data. They also frequently emphasize that users have the right to decide whether their data will be used for model training. However, some experts argue that data processing workflows are not sufficiently transparent.
Senior Data Science Engineer Emre Durgut, in statements to the press, drew attention to the fact that AI systems carry not only technical but also legal and social responsibility.
Durgut stated: “The biggest risk is the creation of a profile for each user through labeling of personal information obtained from users. These profiles can include sensitive personal data such as political views, religious beliefs, and health status. If this information is processed without the user’s explicit consent or in violation of the law, serious privacy violations occur.”
Noting that data processing workflows are often hidden from users and that legal frameworks are being violated, Durgut said: “If the allegations regarding the labeling and profiling of user data — especially sensitive personal data — are correct, this violates numerous legal frameworks, primarily the European Union’s General Data Protection Regulation (GDPR) and Turkey’s Personal Data Protection Law (KVKK).”
Pointing out that users generally cannot fully know how the system works and therefore cannot realize how their provided information is analyzed, Durgut added: “These laws are based on the principles of explicit consent, data minimization, purpose limitation, and transparency for the processing of personal data. For sensitive data (political views, religious beliefs, etc.), they impose much stricter conditions. If platforms process, label, and profile such data without a legal basis, this clearly constitutes an unlawful data processing activity.”
Noting that the responses given by the system become increasingly personalized over time, Durgut warned that this could lead to discrimination in areas such as advertising, recruitment, or financial decisions.
Durgut warned: “Imagine an AI system incorrectly labeling users — based on analysis of their text inputs — as holding a particular political view, ethnic background, or religious belief. This incorrect labeling can lead to concrete discrimination: from targeted ad delivery, to rejection of loan applications, to bias in recruitment processes.”
Emphasizing that the cascading effect of a data leak would be far more serious, Durgut noted that if information falls into the hands of malicious actors, security vulnerabilities such as social engineering attacks, blackmail, and fraud will emerge, and data belonging to individuals in critical positions could turn into a national security threat.
Durgut also warned users: “First and foremost, avoid giving sensitive personal information to such systems as much as possible. Try to reduce risk by distributing different information across different platforms rather than relying on a single one. Increase your awareness in using such systems and approach them with a skeptical mindset.”
Pointing out that technology firms producing such AI applications also have a major responsibility, Durgut stated: “These firms must transparently and openly explain their data processing workflows and usage policies in an understandable way. Companies must regularly audit their practices through independent audits and share results with the public. They must give users more control and management authority over their data (e.g., rights to delete, access, or correct data). They must establish strong internal rules through ethics boards and adopt enforceable policies.”
