Can "Recommended for You" Be Trusted? WKU Professor Waqar Ali's Team Tackles Attack-Resilient AI Recommendation Systems

Open any shopping app and "Recommended for You" seems to know exactly what you want; scroll through a video platform and the algorithm appears to have already read your interests. As recommendation systems grow increasingly attuned to us, how can we ensure they protect our privacy — and resist malicious manipulation?

Recently, Dr. Waqar Ali, Associate Professor in the College of Science, Mathematics and Technology at Wenzhou-Kean University (WKU), was awarded funding under the Research Fund for International Young Scientists (RFIS-I) of the National Natural Science Foundation of China (NSFC) for his project, "Responsible Recommendation Services with Attack-Resilient Federated Learning."

Focused on the security and governance of artificial intelligence, the project aims to build a line of defense for recommendation systems, making personalized recommendations safer, fairer, and more trustworthy.

　　Waqar Ali教授曾主持四川省科技计划项目，研究基于联邦学习的隐私保护推荐算法。此次国家自然科学基金项目则进一步聚焦攻击与防御，推动研究从“保护用户数据”走向“保障整个推荐系统安全”。

　　多年来，Waqar Ali教授的研究成果发表于 ACM Computing Surveys、IEEE Transactions on Knowledge and Data Engineering、ACM Transactions on Recommender Systems、Information Sciences、IEEE Internet of Things Journal 等国际学术期刊，并拥有3项专利。

Understanding you — and protecting you

To learn what users like, recommendation systems typically analyze data such as browsing history, clicks, and purchases. How can we put this data to use while safeguarding personal privacy? Federated learning offers one answer.

It allows user data to remain on local devices such as phones and computers. Each device trains the model locally, and the results are aggregated to jointly improve the recommendation system. In short: "data never leaves the device, yet AI can still be trained together."

But Dr. Ali has drawn attention to an easily overlooked issue: the fact that data is not shared directly does not mean the resulting model is inherently secure.

If a "malicious participant" sneaks into the training process, it can still submit manipulated information that quietly skews the AI's judgment.

This is like a group of people jointly "teaching" an AI which products are worth recommending. Most participants provide normal information — but if someone continuously feeds in carefully crafted false content, certain products could gain abnormal exposure, or the entire recommendation system could be undermined.

"When people use online shopping, entertainment, or travel platforms, they expect high-quality recommendations that genuinely match their needs — not misleading or manipulated results," said Dr. Ali.

The impact of this risk goes far beyond "recommending the wrong product." As recommendation technology moves into finance, healthcare, education, and e-commerce, a compromised or severely biased system could interfere with decisions that matter.

The project therefore seeks to answer a key question: when multiple participants jointly train a recommendation model, how can we detect malicious behavior in a timely manner and prevent it from affecting the final outcome?

Mapping attack types to lay the groundwork for defense

To counter different kinds of attacks, the first step is understanding how the "adversary" might act.

The project plans to systematically review the definitions, characteristics, implementation methods, and potential impacts of various attacks on federated recommendation systems, building a knowledge base for attack identification and defense. This will serve as a reference for researchers to spot potential threats, detect malicious behavior, and develop defense strategies.

Building on this, the team will study mechanisms for detecting malicious clients — analyzing data distributions, model updates, and training behavior across clients to identify anomalies and flag participants who may be manipulating the system.

The project will also explore more secure federated learning frameworks to strengthen the stability of recommendation systems in complex adversarial environments. In parallel, the team will examine new attack vectors enabled by generative AI, as well as the impact of attacks on recommendation fairness.
Popular content, for instance, tends to be recommended more because it attracts more attention. If attackers exploit this mechanism to manipulate outcomes, content that is already less visible could lose even more exposure. The implications extend beyond whether recommendations are accurate, to whether the system can serve different users and content providers fairly.

From mapping attack types and identifying malicious participants, to designing defense mechanisms and reducing algorithmic bias, the project forms a fairly complete research pathway — ultimately advancing safer, fairer, and more responsible recommendation services.

From privacy protection to trustworthy AI

The project continues a research direction Dr. Ali has pursued for years.About a decade ago, an academic discussion on optimizing search engine performance sparked his interest in recommendation systems. His research has since evolved from context-aware recommendation to privacy protection, federated learning, and responsible recommender systems.

He previously led a project under the Sichuan Provincial Science and Technology Program on privacy-preserving recommendation algorithms based on federated learning. The new RFIS-I project sharpens the focus on attacks and defense, moving the research from "protecting user data" to "safeguarding the entire recommendation system."

Over the years, Dr. Ali's findings have been published in international journals including ACM Computing Surveys, IEEE Transactions on Knowledge and Data Engineering, ACM Transactions on Recommender Systems, Information Sciences, and IEEE Internet of Things Journal, and he holds three patents.

"This means a great deal to me," he said of the grant approval. For him, it is both recognition of his research and a new responsibility. He hopes to build on it to help translate AI research into technology that is trustworthy, socially meaningful, and practically impactful.

Once the project is underway, WKU students will have the opportunity to participate in the research and engage with cutting-edge AI questions in practice. Dr. Ali also plans to strengthen research collaboration with Kean University in the United States and expand his network of partners with universities and research institutions at home and abroad.

As AI grows ever better at understanding us, how can we grow more confident in trusting AI? This inquiry into trustworthy artificial intelligence will continue to find new answers at Wenzhou-Kean University.