Over the past few decades, with the rise of the Internet and now, the metaverse, navigating the seemingly endless deluge of options online feels almost like a full-time occupation.

From e-commerce to music and movie suggestions, recommender systems are responsible for powering our on- and off-line activities: from Amazon’s product recommendations, restaurant suggestions and Facebook’s newsfeed personalisation, to movies to watch — handpicked by Netflix’s Recommendation Engine.

“As we grow increasingly dependent on digital services for most of our activities, I'm excited to learn about how technology can customise our online experiences,” says SMU Associate Professor of Computer Science Hady Lauw.

“The key lies in uncovering user preferences from data. For that, we rely on artificial intelligence and machine learning.”

Recommender systems, algorithms that suggest items for users based on their past behaviour, have become ubiquitous in modern life. As such, the dissertation Modelling Sentiments and Preferences from Multimodal Data by SMU School of Computing and Information Systems PhD candidate Truong Quoc Tuan aims to model the subjectivity of user review data to further fuel existing systems.  Prof Lauw is the advisor to his dissertation.

As Tuan expresses: “The need for recommendations is to guide users in browsing the myriad of options offered to them.”

Based on Tuan’s research and that of Prof Lauw, here are five trends that will supercharge the next generation of such systems.

#1 Exposure never-before-encountered items

Computer scientists have developed sophisticated programmes to analyse enormous volumes of data. They aid users' decisions on online platforms and ensure the picks meet their needs, says Tuan, whose research interests include user profiling and personalisation. 

Such recommender systems tap on various forms of data to customise recommendations. For example, a user’s interactions with a product, such as clicking on a listing, reading about the item, or purchasing it in the past, play a crucial role in machine-assisted suggestions. Other kinds of data, such as user reviews or the descriptive content of products, also contribute to a match.

According to Prof Lauw, it is important to pay attention not just to the user’s interaction with the item, but also auxiliary information or interactions.

During his research, Prof Lauw investigated various steps in the buying process — from the actual purchase to supporting actions that led to the cart check-out such as clicks. Insights from an e-commerce site’s organic feedback and that of a target advertising site could be used to predict ad performance on the target site. Moreover, auxiliary data from one domain could come in useful to facilitate cross-domain recommendations.

#2 Tapping on a wider range of interactions for better recommendations

“Since winning the Singapore National Research Foundation Fellowship, I am increasingly focused on mining preferences from multi-modal data,” shares Prof Lauw.

“For one, the kind of modality we get depends on a specific data source. On social media, we may be able to find images. But on e-commerce sites, we are more likely to find text reviews. For another, the kind of modality that is important depends on product category. For things that are visual in nature such as fashion items, image modality would be useful. For book recommendations, the text content is the primary information.”

For example, he examines social networks, features of items and also associations among items, for recommending the next item that a user may put into their basket.

The key to designing efficient algorithms is not only to rely on primary interactions between users, items and sites, but also to look at other forms of interactions, adds Tuan. This includes modalities, such as text or images, along with social connections between users, to provide the most accurate recommendation.

His dissertation centres on how technology can customise online experiences.

Through the paper, he hopes to leverage on artificial intelligence and machine learning to uncover user preferences from data, to develop algorithms and intelligent systems to extract information about users’ sentiments and preferences on various products.

#3 Moving towards hyperpersonalisation

Hyperpersonalisation is the use of technology to deliver highly customised content and experiences to users. It takes into account an individual's unique preferences, interests and behaviours in order to deliver personalised content that is relevant to them.

“The more often users go through personalised interactions, the more they might expect customised recommendations in the future, which will drive us down this lane of hyper-personalisation,” says  Prof Lauw.

Formerly a scientist with A*STAR Institute for Infocomm Research, one of  Prof Lauw’s major research goals is the study of algorithms for learning the preferences of consumers from large-scale data of diverse formats and types, to enable more accurate personalisation.

“Because we want more products that are a very good fit to our needs, hyperpersonalisation improves our daily life, in a sense,” explains Tuan.

“Given that we are exposed to a lot of options nowadays, and a lot of information that we need to filter, this form of recommendation is a must.”

#4 The power of images

Augmented convolutional neural networks (ACNNs) are a type of artificial intelligence that is often used in recommendation systems. ACNNs mines input from both sides of a user interface, making recommendations based on what the user has searched for in the past as well as what the user is currently viewing.

Besides text, the image representation of an image provides plenty of fodder for recommendation systems to understand a user's preferences. Leveraging his research,  Prof Lauw has proposed “ACNNs to detect the sentiments expressed by images, considering the user or item involved.”

According to Tuan, images can play a significant role in shopping for fashion items. Along with your past interactions such as what you might have bought previously, the recommender system can suggest outfits or accessories that are similar in style as well as items that can complete your look.

#5 A versatile algorithm across platforms

The beauty of tech-enabled recommender systems is that the algorithm can be applied across different products, industries, platforms and markets, says Tuan.

He believes that product recommendations go beyond just pitching the newest launch, and that businesses should not need to create different recommender systems for different products, be it fashion outfits or movie. Through using the interactive data between a user and items, he hopes to extract the similarities or patterns to recommend items that similar users might like, even if the user has never been exposed to the items before.

“The way we look at the problem is that we have a set of items that requires a general formulation,” he explains.

“We don't really need to look at a specific kind of items that have to be solved. We still can apply the same kind of algorithms and ensure it works on different platforms.”

More critically, the recommendation systems of tomorrow need to consider various modalities for greater accuracy. As such, Prof Lauw’s research team created Cornac, an open-source multimodal recommender library with more than 40 algorithms.

“The reason why this is important is because different algorithms work differently depending on specific recommendation datasets. We might not know beforehand what is best, and so a comparison capability is important. It even leads to surprising findings, such as an algorithm that was designed with image modality may work better with text or graph modality,” states Prof Lauw.

“The cool thing is how easy this software makes it not only to build recommendation models, but also to compare them side by side to see which is more effective under which dataset and which metric.”