PolyU-led research reveals that sensory and motor inputs help large language models represent complex concepts
HONG KONG SAR - Media OutReach Newswire - 9 June 2025 - Can one truly understand what 'flower' means without smelling a rose, touching a daisy or walking through a field of wildflowers? This question is at the core of a rich debate in philosophy and cognitive science. While embodied cognition theorists argue that physical, sensory experience is essential to concept formation, studies of the rapidly evolving large language models (LLMs) suggest that language alone can build deep, meaningful representations of the world.
A research team led by Prof. Li Ping, Sin Wai Kin Foundation Professor in Humanities and Technology, Dean of the PolyU Faculty of Humanities and Associate Director of the PolyU-Hangzhou Technology and Innovation Research Institute, explored the similarities between large language models and human representations, shedding new light on the extent to which language alone can shape the formation and learning of complex conceptual knowledge.
By exploring the similarities between LLMs and human representations, researchers at The Hong Kong Polytechnic University (PolyU) and their collaborators have shed new light on the extent to which language alone can shape the formation and learning of complex conceptual knowledge. Their findings also revealed how the use of sensory input for grounding or embodiment – connecting abstract with concrete concepts during learning – affects the ability of LLMs to understand complex concepts and form human-like representations. The study, in collaboration with scholars from Ohio State University, Princeton University and City University of New York, was recently published in Nature Human Behaviour.
Led by Prof. LI Ping, Sin Wai Kin Foundation Professor in Humanities and Technology, Dean of the PolyU Faculty of Humanities and Associate Director of the PolyU-Hangzhou Technology and Innovation Research Institute, the research team selected conceptual word ratings produced by state-of-the-art LLMs, namely ChatGPT (GPT-3.5, GPT-4) and Google LLMs (PaLM and Gemini). They compared them with human-generated word ratings of around 4,500 words across non-sensorimotor (e.g., valence, concreteness, imageability), sensory (e.g., visual, olfactory, auditory) and motor domains (e.g., foot/leg, mouth/throat) from the highly reliable and validated Glasgow Norms and Lancaster Norms datasets.
The research team first compared pairs of data from individual humans and individual LLM runs to discover the similarity between word ratings across each dimension in the three domains, using results from human-human pairs as the benchmark. This approach could, for instance, highlight to what extent humans and LLMs agree that certain concepts are more concrete than others. However, such analyses might overlook how multiple dimensions jointly contribute to the overall representation of a word. For example, the word pair 'pasta' and 'roses' might receive equally high olfactory ratings, but 'pasta' is in fact more similar to 'noodles' than to 'roses' when considering appearance and taste. The team therefore conducted representational similarity analysis of each word as a vector along multiple attributes of non-sensorimotor, sensory and motor dimensions for a more complete comparison between humans and LLMs.
The representational similarity analyses revealed that word representations produced by the LLMs were most similar to human representations in the non-sensorimotor domain, less similar for words in sensory domain and most dissimilar for words in motor domain. This highlights LLM limitations in fully capturing humans' conceptual understanding. Non-sensorimotor concepts are understood well but LLMs fall short when representing concepts involving sensory information like visual appearance and taste, and body movement. Motor concepts, which are less described in language and rely heavily on embodied experiences, are even more challenging to LLMs than sensory concepts like colour, which can be learned from textual data.
In light of the findings, the researchers examined whether grounding would improve the LLMs' performance. They compared the performance of more grounded LLMs trained on both language and visual input (GPT-4, Gemini) with that of LLMs trained on language alone (GPT-3.5, PaLM). They discovered that the more grounded models incorporating visual input exhibited a much higher similarity with human representations.
Prof. Li Ping said, 'The availability of both LLMs trained on language alone and those trained on language and visual input, such as images and videos, provides a unique setting for research on how sensory input affects human conceptualisation. Our study exemplifies the potential benefits of multimodal learning, a human ability to simultaneously integrate information from multiple dimensions in the learning and formation of concepts and knowledge in general. Incorporating multimodal information processing in LLMs can potentially lead to a more human-like representation and more efficient human-like performance in LLMs in the future.'
Interestingly, this finding is also consistent with those of previous human studies indicating the representational transfer. Humans acquire object-shape knowledge through both visual and tactile experiences, with seeing and touching objects activating the same regions in human brains. The researchers pointed out that – as in humans – multimodal LLMs may use multiple types of input to merge or transfer representations embedded in a continuous, high-dimensional space. Prof. Li added, 'The smooth, continuous structure of embedding space in LLMs may underlie our observation that knowledge derived from one modality could transfer to other related modalities. This could explain why congenitally blind and normally sighted people can have similar representations in some areas. Current limits in LLMs are clear in this respect'.
Ultimately, the researchers envision a future in which LLMs are equipped with grounded sensory input, for example, through humanoid robotics, allowing them to actively interpret the physical world and act accordingly. Prof. Li said, 'These advances may enable LLMs to fully capture embodied representations that mirror the complexity and richness of human cognition, and a rose in LLM's representation will then be indistinguishable from that of humans.'
Hashtag: #PolyU #HumanCognition #LargeLanguageModels #LLMs #GenerativeAI
The issuer is solely responsible for the content of this announcement.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Yahoo
41 minutes ago
- Yahoo
Blockchain Alliance International Launches Asia-Pacific Digital Economy Initiative in Hong Kong
Hong Kong SAR, 23–24 June 2025 HONG KONG, June 22, 2025 /PRNewswire/ -- Blockchain Alliance International (BAI), a New York-based non-profit organization in special consultative status with the United Nations Economic and Social Council (ECOSOC), is proud to announce the launch of its Asia-Pacific Digital Economy Initiative, taking place on 23–24 June 2025 in Hong Kong SAR, People's Republic of China. The two-day high-level forum comes in direct response to the 2024 UN Summit of the Future, which adopted the Pact for the Future, Global Digital Compact, and Declaration on Future Generations—three landmark frameworks that emphasize inclusive, rights-based, and sustainable digital transformation. As digital technologies like blockchain, AI, and Web3 platforms rapidly reshape global systems, the Initiative aims to ensure that developing countries - particularly those in the Asia-Pacific - are empowered to harness innovation in ways that address local needs and accelerate progress toward the Sustainable Development Goals (SDGs). The event will feature UN-style plenaries, thematic breakouts, case study showcases, and investment dialogues, culminating in the inauguration of BAI's new Hong Kong Regional Office. This office will serve as a regional hub for innovation, partnership-building, and digital capacity development. "Our mission is to bridge the digital divide through multi-stakeholder cooperation and ensure that digital progress leaves no one behind," said Ambassador Alexandru Cujba, Executive President of BAI. "With this new platform in Hong Kong, we are strategically positioned to foster cross-border dialogue and innovation that supports human-centered development." Participants include government officials, tech leaders, academics, entrepreneurs, and representatives of international organizations. Together, they will explore scalable solutions for digital inclusion, education, healthcare, financial access, and climate resilience - particularly benefiting youth, women, MSMEs, and underserved communities. The Asia-Pacific Digital Economy Initiative reflects BAI's commitment to "technology for good", and its alignment with global efforts to promote ethical, inclusive, and impactful digital governance. For additional information, please visit or contact wendy@ View original content: SOURCE Blockchain Alliance International Error in retrieving data Sign in to access your portfolio Error in retrieving data Error in retrieving data Error in retrieving data Error in retrieving data
Yahoo
an hour ago
- Yahoo
Xiaomi to unveil prices for new YU7 SUV at June 26 launch
SHANGHAI (Reuters) -Xiaomi's CEO Lei Jun said on Monday that the firm plans to hold an official launch event for its new YU7 electric sports utility vehicle on Thursday, where it is expected to announce pricing and start taking orders. Lei made the announcement on his Weibo account. Xiaomi earlier announced specifications for its second EV model in May. Error in retrieving data Sign in to access your portfolio Error in retrieving data Error in retrieving data Error in retrieving data Error in retrieving data
Yahoo
2 hours ago
- Yahoo
Scale AI's 30-year-old billionaire cofounder has a warning for anyone who craves work-life balance: ‘maybe you're not in the right work'
The billionaire cofounder of Scale AI, , has a message for anyone who craves work-life balance: Maybe you're in the wrong job. This millennial wakes up at 5:30 a.m. and doesn't clock off until midnight—and it's a philosophy that's catching on among founders now openly embracing China's 996 grind. Work-life balance has become the holy grail of modern employment. It's the non-negotiable perk that trumps salary and title—with Gen Z and millennial workers willing to walk away from jobs that don't deliver it in abundance. But what if instead of walking out on jobs that don't provide balance, they should leave the jobs that make them crave it instead? That's because, according to Lucy Guo, the 30-year-old billionaire cofounder of Scale AI, the need to clock off at 5 p.m. on the dot to unwind might signal that you're in the wrong job altogether. Guo, who dropped out of college and built her fortune in the tech industry, says her grueling daily schedule—waking up at 5:30 am and working until midnight—doesn't feel like work to her at all. 'I probably don't have work-life balance,' Guo tells Fortune. 'For me, work doesn't really feel like work. I love doing my job.' 'I would say that if you feel the need for work-life balance, maybe you're not in the right work.' That doesn't mean she's completely ignorant to life outside the office. The uber successful millennial, just dethroned Taylor Swift as the youngest self-made woman on the planet, according to Forbes' latest rankings. The 5% stake she held on to when she left her post at Scale AI is now worth an estimated $1.2 billion. Now, she's busy running another venture, the creator community platform Passes. Yet even when working '90-hour workweeks,' she says she still finds 'one to two hours' to squeeze in family and friends. 'You should always find time for that, regardless of how busy you are.' That, she suggests, is about making time for life—not running from your work. 5:30 a.m.: Wake upOn the morning of our interview in London, LA-based Guo says was up all night: 'I'm so jet lagged.' But she typically wakes up at around 5:30 and does two to three high-intensity workouts at Barry's every day. 9 a.m. onwards: In the office'Every day looks very different,' Guo says. 'Some days, I am doing more marketing pushes. I'm talking to our PR, I'm doing podcasts, etc. Other days I am more product-focused… Reviewing designs, giving user experience feedback.' She has her daily black coffee hit and lunch al desko. Midnight: BedtimeThe founder says she's typically working until 12 a.m.—when she finally will shut the laptop and go to sleep. The thing keeping her up so late? Keeping a beady eye on the customer support inbox. She gives her team just five minutes to respond to their customers before responding to them herself. 'Having that white glove customer service is what makes startups stand out from big tech,' Guo explains. 'While you have less customers, it's very possible for the CEO to answer everything which makes people more loyal. It's impossible for like the Uber CEO to do this nowadays. So that's the kind of mentality I have.' 'If you want to grow, your reputation is everything, and the best thing you do for your reputation is, offering the best, support to your customers. So I'm constantly doing that.' While Guo's routine may sound extreme to the regular worker, for founders, it's the new norm. Entrepreneurs have been taking to LinkedIn and claiming that the only way to succeed in the current climate is by copying China's 996 model. That is, working 9 am to 9 pm, six days a week. Harry Stebbings, founder of the 20VC fund, ignited the latest debate at the start of the month when he said Silicon Valley had 'turned up the intensity,' and European founders needed to take notice. '7 days a week is the required velocity to win right now. There is no room for slip up,' Stebbings wrote on LinkedIn. 'You aren't competing against random company in Germany etc but the best in the world.' 'Forget 9 to 5, 996 is the new startup standard,' Martin Mignot, partner at Index Ventures echoed on the networking platform. 'Back in 2018, Michael Moritz introduced the West to China's '996' work schedule… At the time, the piece was controversial. Now? That same schedule has quietly become the norm across tech,' Mignot added. 'And founders are no longer apologizing for it.' But it's not just startup chiefs that are having to put in overtime to get ahead, CEOs admitted to Fortune at our recent Most Powerful Women Summit in Riyadh that they work well beyond the 40-hour benchmark. 'I don't know that I finish work psychologically,' Leah Cotterill CEO of Cigna Healthcare Middle East and Africa revealed, adding that she fully immerses herself into work all day and night 'Monday through Thursday' but tries to 'ease that off' on Friday for the weekend. Others put a number on the hours they work, from up to 12 a day to 80 a week. But like Guo, many said they do it—not in reaction to the current market conditions, but because they're passionate about what they do. 'I'm always working 24/7 I'm a workaholic, so I don't stop working because I enjoy what I do,' Princess Noura bint Faisal Al Saud, Culture House's CEO added. And the next generation of workers probably needs to take note. Unfortunately for work-life balance-loving young people, experts have stressed that 40-hour workweeks aren't enough if they want to climb the corporate ladder. In a leaked memo to Google's AI workers, Sergey Brin suggested that 60 hours a week is the 'sweet spot'. This story was originally featured on
