Transforming a child’s face into a cute, playful AI image may appear creative and harmless, but uploading it to AI-powered online platforms, such as ChatGPT and Gemini, could pose risks that are often overlooked, according to authorities.
One AI-generated image that has become popular is the “AI baby sticker,” which displays a baby’s photo enhanced in multiple sticker-like versions to show different facial expressions.
Many netizens, especially parents, have followed the trend after finding the adorable AI-created photos of their babies. Some, however, are worried about the risks of uploading child photos to AI-powered platforms, aside from the environmental cost of AI infrastructure that uses huge volumes of water for their cooling system.
Salmo Deo T. Joaquin, a lawyer at the Legislative Division of the Cybercrime Investigation and Coordinating Center (CICC), said the uploaded photos could be used to create misleading content, fake social media accounts, or even deepfake videos. The CICC is an attached agency of the Department of Information and Communications Technology (DICT).
Once a photo is uploaded online, it may contain metadata and other information that can be accessed, he said.
“When children’s images are targeted for misuse, it can expose them to cyberbullying, impersonation, online grooming and other forms of child exploitation,” Joaquin told CoverStory.ph.
He added: “AI is a useful innovation, but it can also be abused. A simple photograph can be transformed into a fake image or video using the AI.”
Parents’ complaints
The CICC, through its Cybersecurity Complaint Center, has documented cases based on complaints of parents after photos of their children were uploaded to AI platforms. It said the photos could potentially be used to generate sexually explicit content involving minors and infants.
“We have taken down some platforms temporarily because there was no restriction on the use of the images of children being placed into a different body,” said Joaquin.
Some entrepreneurs, however, are turning to AI-powered platforms to produce their preferred design through prompts. Among them is Catherine Ardiente, 33, a Cebu-based businesswoman who runs a printing business.
She said she had used it for her flower craft business.
“I’d use it to change the background. But when AI-generated sticker labels became trending, that’s when I became interested in using AI,” Ardiente said in an interview with CoverStory.
Even after telling her clients about the risks associated with AI, she said many still choose to avail themselves of her services. She said she obtains the parents’ consent before generating AI images of their children.
Joaquin said that sometimes, a picture of a child’s face, school uniform and frequently visited places could be geolocated.
“So once a photo is posted online, it can be saved and it can be shared with multiple people without the consent of the parents,” he pointed out, emphasizing that the same risks apply when users upload a photo to AI-powered platforms.
According to the Council for the Welfare of Children (CWC), online sexual abuse or exploitation of children (OSAEC) and child sexual abuse or exploitation materials (CSAEM) remain one of the most serious risks associated with uploading and sharing children’s photos online.
Manipulated images
In particular, CWC, an agency attached to the Department of Social Welfare and Development, cited the increasing misuse of AI technologies to generate highly realistic images of children that are publicly accessible or shared on digital platforms.
“These manipulated images may be used by perpetrators to generate AI-produced CSAEM, further facilitating the sexual exploitation, abuse, and revictimization of children,” CWC said in response to questions sent by CoverStory.
“The creation, possession, distribution, and use of AI-generated CSAEM not only violate children’s rights to privacy, dignity, and protection but also inflict lasting psychological and emotional harm,” it said.
Last Jan. 16, the DICT banned access to Grok AI, X’s chatbot, following criticisms over its use in generating nonconsensual, sexually explicit, and manipulated images. “These concerns raised serious implications for digital safety, privacy, and the protection of vulnerable sectors, especially women and children,” it said in a statement.
The agency lifted the ban five days later, after xAI, the company behind Grok AI, implemented more safeguards and tightened content moderation to prevent the generation of explicit and nonconsensual content.
“These corrective actions include strengthened restrictions on prompts involving real individuals and improved enforcement of acceptable-use policies,” the DICT said.
The CICC said it had also received reports of some children whose profiles were being used by other parents to commit scams.
Last year, the CWC’s Makabata Helpline 1383 recorded and tracked 79 cases related to OSAEC-CSAEM. The hotline serves as the national reporting mechanism for concerns involving children in need of special protection and as an accessible platform for reporting and facilitating appropriate referrals and interventions.
Stored data
Images, personal data, and other content uploaded to AI-powered platforms do not simply disappear; they are stored on the platforms’ servers long after they are posted.
In a March 2025 article by the National Cybersecurity Alliance (NCA), a US-based nonprofit, public AI platforms often collect and retain the data users input into their systems.
For instance, ChatGPT, one of the most widely used generative AI platforms, stated in its privacy policy that the user’s personal data, including the prompts, files, images, audio and video, Sora characters, and data from connected services are collected and stored.
When asked where the uploaded data go, CICC’s Joaquin said data sovereignty depends on where the platform is based or where the data is stored.
“Let’s say there is a data center in the Philippines and that platform or AI platform is located in the Philippines, then it gets uploaded here,” he said.
For ChatGPT, the data uploaded to the platform is stored on OpenAI and its facilities and servers are in the United States.
Why do AI platforms collect users’ data? NCA said they “retain input data for training purposes, meaning that anything you share could be used to refine future responses—or worse, inadvertently exposed to other users.”
The same applies to ChatGPT. As stated in its policy, the content that the user provides is used to improve their services and to train the models that power ChatGPT.
“When you allow your content to be used to train our models, it helps our models become more accurate and better at solving your specific problems and it also helps improve their general capabilities and safety,” the company said.
Disabling the setting
If users do not want their data to be used by the platform, they can opt out by disabling the setting.
ChatGPT allows users to delete personal data stored in their account, conversations, or their entire account. Once a user chooses to do so, the platform will delete it within 30 days unless it is required to retain certain information for legal, security or other legitimate reasons.
However, Joaquin was doubtful that AI platforms actually delete user data, even if they state this in their terms and conditions. “We do not have a guarantee that that information is being deleted from their platform, especially if it is not within the jurisdiction of the Philippines,” he said.
Jonathan Rudolph Y. Ragsag, an information technology officer of the Data Security and Technology Standard Division at the National Privacy Commission (NPC), said the personal information controllers (PICs) must consider transparency and accountability about how they process personal data.
Ragsag said that as part of accountability, PICs must have “demonstrable” data security measures to protect personal data being processed by their systems.
As stated in the NPC Advisory No. 2024-04: “PICs shall inform their data subjects of the nature, purpose, and extent of the processing of personal data when such processing is involved in the development or deployment of AI systems, including its training and testing.”
The PICs should explain the purpose of processing the data, the factors and inputs considered by such AI systems, as well as the risks associated with the processing, the expected output and their impact on the data subject.
Ragsag said they must also disclose where the data is stored, how long it will be retained, the servers and service providers, and the countries where those servers are located.
Legal frameworks
Under Republic Act No. 10173, or the Data Privacy Act of 2012, data subjects have the right to and control over their personal information. “Under our law, we have the right as data subjects to decide whether we will provide our personal data for processing or not,” he said.
While the law broadly protects all individuals, it does not contain a section specifically dedicated to protecting children.
“In the United States, there’s a federal law governing children’s personal data, the Children’s Online Privacy Protection Act,” Ragsag said. “But we don’t have that. There is no specific provision that is mentioned with regard to children, so it needs to be amended.”
“There are now many new technologies, like generative AI. We also have AI agents, age assurance technologies, blockchain, etc. That’s why the law needs to be updated,” he added.
Republic Act No. 11930, or the Anti-Online Sexual Abuse or Exploitation of Children and Anti-Child Sexual Abuse or Exploitation Materials Act, criminalizes and penalizes online child sexual abuse.
Enacted in 2022, the law provides legal protection for children who become victims of online sexual exploitation and other forms of online abuse of minors.
The CWC, however, believes that there is a need to strengthen the legal framework in response to the rapid advancement of AI. It stresses that images of children uploaded to AI-powered platforms or publicly accessible online spaces carry risks, including manipulation and creation of deepfakes without the knowledge or consent of the child or parents.
“Given these emerging risks, legislation should establish stronger safeguards for the collection, use, processing, and sharing of children’s images, while reinforcing the accountability of AI developers, digital platforms, and online service providers,” CWC said.
“Rather than imposing a blanket prohibition on uploading children’s photos, the law should adopt a child rights-based, privacy-by-design, and safety-by-design approach,” it said.
This includes requiring AI-powered platforms to implement robust child protection measures, prevent the misuse of children’s images, strengthen transparency and reporting mechanisms, and ensure that children’s data and privacy are protected by default.
Joaquin encouraged victims to reach out to the CICC or call the 1326 hotline so the agency can provide assistance and protection.
Safe digital practices
While these efforts help strengthen child protection, experts strongly advised parents, guardians, and the public to be cautious when uploading photos of children.
“We therefore encourage parents and caregivers to make informed decisions, prioritize their children’s privacy and best interests, and practice safe and protective digital parenting at all times,” CWC said.
“Protecting children online is a shared responsibility that requires collaboration among families, government, schools, technology companies, and communities,” it said. “We encourage everyone to participate in awareness campaigns and capacity-building initiatives on AI, online safety, and the prevention of OSAEC and CSAEM.”
Joaquin said his office views the use of AI as merely a tool. “Technology should be used to protect children and we should not put them at risk,” he said. The CICC does not prohibit posting children’s pictures, but wants parents to be aware of the risks when they do so, he added.
Asked how she plans to move forward after learning about the risks, Ardiente, the businesswoman, said she had considered no longer generating AI images of children but does not see herself giving up AI anytime soon.
“AI has become a partner in my business. Why would I go back to doing things the harder and slower way when there’s an easier way?” she said. CS
Fionna Andrada, a third-year journalism student at the University of the Philippines’ College of Media and Communication in Diliman, is an intern at CoverStory.ph.
