PolyU-led research reveals that sensory and motor inputs help large language models represent complex concepts
A research team led by Prof. Li Ping, Sin Wai Kin Foundation Professor in Humanities and Technology, Dean of the PolyU Faculty of Humanities and Associate Director of the PolyU-Hangzhou Technology and Innovation Research Institute, explored the similarities between large language models and human representations, shedding new light on the extent to which language alone can shape the formation and learning of complex conceptual knowledge.
HONG KONG SAR - Media OutReach Newswire - 9 June 2025 - Can one truly understand what "flower" means without smelling a rose, touching a daisy or walking through a field of wildflowers? This question is at the core of a rich debate in philosophy and cognitive science. While embodied cognition theorists argue that physical, sensory experience is essential to concept formation, studies of the rapidly evolving large language models (LLMs) suggest that language alone can build deep, meaningful representations of the world.By exploring the similarities between LLMs and human representations, researchers at The Hong Kong Polytechnic University (PolyU) and their collaborators have shed new light on the extent to which language alone can shape the formation and learning of complex conceptual knowledge. Their findings also revealed how the use of sensory input for grounding or embodiment – connecting abstract with concrete concepts during learning – affects the ability of LLMs to understand complex concepts and form human-like representations. The study, in collaboration with scholars from Ohio State University, Princeton University and City University of New York, was recently published in Nature Human Behaviour Led by Prof. LI Ping, Sin Wai Kin Foundation Professor in Humanities and Technology, Dean of the PolyU Faculty of Humanities and Associate Director of the PolyU-Hangzhou Technology and Innovation Research Institute, the research team selected conceptual word ratings produced by state-of-the-art LLMs, namely ChatGPT (GPT-3.5, GPT-4) and Google LLMs (PaLM and Gemini). They compared them with human-generated word ratings of around 4,500 words across non-sensorimotor (e.g., valence, concreteness, imageability), sensory (e.g., visual, olfactory, auditory) and motor domains (e.g., foot/leg, mouth/throat) from the highly reliable and validated Glasgow Norms and Lancaster Norms datasets.The research team first compared pairs of data from individual humans and individual LLM runs to discover the similarity between word ratings across each dimension in the three domains, using results from human-human pairs as the benchmark. This approach could, for instance, highlight to what extent humans and LLMs agree that certain concepts are more concrete than others. However, such analyses might overlook how multiple dimensions jointly contribute to the overall representation of a word. For example, the word pair "pasta" and "roses" might receive equally high olfactory ratings, but "pasta" is in fact more similar to "noodles" than to "roses" when considering appearance and taste. The team therefore conducted representational similarity analysis of each word as a vector along multiple attributes of non-sensorimotor, sensory and motor dimensions for a more complete comparison between humans and LLMs.The representational similarity analyses revealed that word representations produced by the LLMs were most similar to human representations in the non-sensorimotor domain, less similar for words in sensory domain and most dissimilar for words in motor domain. This highlights LLM limitations in fully capturing humans' conceptual understanding. Non-sensorimotor concepts are understood well but LLMs fall short when representing concepts involving sensory information like visual appearance and taste, and body movement. Motor concepts, which are less described in language and rely heavily on embodied experiences, are even more challenging to LLMs than sensory concepts like colour, which can be learned from textual data.In light of the findings, the researchers examined whether grounding would improve the LLMs' performance. They compared the performance of more grounded LLMs trained on both language and visual input (GPT-4, Gemini) with that of LLMs trained on language alone (GPT-3.5, PaLM). They discovered that the more grounded models incorporating visual input exhibited a much higher similarity with human representations.Prof. Li Ping said, "The availability of both LLMs trained on language alone and those trained on language and visual input, such as images and videos, provides a unique setting for research on how sensory input affects human conceptualisation. Our study exemplifies the potential benefits of multimodal learning, a human ability to simultaneously integrate information from multiple dimensions in the learning and formation of concepts and knowledge in general. Incorporating multimodal information processing in LLMs can potentially lead to a more human-like representation and more efficient human-like performance in LLMs in the future."Interestingly, this finding is also consistent with those of previous human studies indicating the representational transfer. Humans acquire object-shape knowledge through both visual and tactile experiences, with seeing and touching objects activating the same regions in human brains. The researchers pointed out that – as in humans – multimodal LLMs may use multiple types of input to merge or transfer representations embedded in a continuous, high-dimensional space. Prof. Li added, "The smooth, continuous structure of embedding space in LLMs may underlie our observation that knowledge derived from one modality could transfer to other related modalities. This could explain why congenitally blind and normally sighted people can have similar representations in some areas. Current limits in LLMs are clear in this respect".Ultimately, the researchers envision a future in which LLMs are equipped with grounded sensory input, for example, through humanoid robotics, allowing them to actively interpret the physical world and act accordingly. Prof. Li said, "These advances may enable LLMs to fully capture embodied representations that mirror the complexity and richness of human cognition, and a rose in LLM's representation will then be indistinguishable from that of humans."Hashtag: #PolyU #HumanCognition #LargeLanguageModels #LLMs #GenerativeAI
The issuer is solely responsible for the content of this announcement.
Hashtags
Try Our AI Features
Explore what Daily8 AI can do for you:
Comments
No comments yet...
Related Articles
Malay Mail
13 hours ago
- Malay Mail
Tesla to build China's largest grid battery plant in RM2.3b Shanghai deal
NEW YORK, June 22 — Tesla announced Friday that it signed an agreement to build its first grid-scale energy storage power station project in mainland China. The project will help with the flexible adjustment of grid resources, and 'effectively solve pressures relating to urban power supply,' Tesla said in a post to the Chinese social media platform Weibo. 'After completion, this project is expected to become the largest grid-side energy storage project in China,' Tesla added. Such energy storage systems help to enhance stability in the electricity grid at a time when there are greater supplies of solar and wind power. Chinese media outlet Yicai reported that Tesla Shanghai, Shanghai authorities and China Kangfu International Leasing Co. held a signing ceremony Friday for the project. It added that the deal involved investments of 4 billion yuan (RM2.3 billion). The contract comes at a moment of tension between Washington and Beijing, with the two sides yet to hash out a long-term trade agreement following tariffs announced by President Donald Trump. China and the United States negotiated a 'framework' agreement in London earlier this month following two days of marathon talks. — AFP
Malay Mail
2 days ago
- Malay Mail
NIA Invites You to Countdown to "SITE 2025" Southeast Asia's Largest Innovation and Startup Expo, uniting 10 Countries and Over 100 Organizations to Drive Global Innovation Collaboration
BANGKOK, THAILAND - Media OutReach Newswire - 20 June 2025 - The National Innovation Agency (Public Organization) or NIA, under the Ministry of Higher Education, Science, Research and Innovation, has announced its readiness to host the Startup x Innovation Thailand Expo 2025 (SITE 2025) under the theme: "Global Innovation Partnership – AI & Sustainability: The Next Era of Innovation," to be held from July 4–6, 2025, at Paragon Hall, 5th Floor, Siam Paragon. This year marks a significant milestone as SITE evolves into a truly regional platform for innovation. Thailand is positioning itself as a Global Innovation Collaboration Hub, uniting the strength of 10 leading partner countries: Japan, South Korea, Hong Kong, Singapore, Czech Republic, Hungary, Finland, Switzerland, Qatar, and Chile—spanning Asia, Europe, the Middle East, and Latin America. In addition, over 100 organizations will participate, including 20 international agencies, 12+ leading private-sector companies, 7 professional associations, and 50 universities from across Krithpaka Boonfueng, Executive Director of NIA, stated:"SITE 2025 is more than an expo—it's a platform where meaningful global collaboration takes place with depth, inclusivity, and a shared purpose. With AI and sustainability at the core of the global economy, our goal is to drive real-world impact across business, investment, and innovation policy."The 2025 edition of SITE will feature more than 30 seminars, covering the full spectrum of global innovation across three major stages: Main Stage – the "heart" of the event, where international thought leaders will share visionary insights and the latest trends in AI, sustainability, and cross-border collaboration. Global Stage – in-depth policy and strategy sessions exploring opportunities, challenges, and pathways to bring innovation to the global market. Pitch Stage – a dedicated arena for Thai and international startups to pitch their ideas, promote technologies, and scale their ventures in front of global of the major highlights on the Pitch Stage will be the National Pitching Startup Thailand League 2025, alongside the 100 Startup Pitching showcase, featuring cutting-edge technologies such as AI, Deep Tech, and sustainability-driven expo will also include exhibitions from over 300 startups and leading innovation organizations, presenting real products, prototypes, and market-ready solutions. More than 30 Business Matching sessions will be held to facilitate direct connections between entrepreneurs, investors, technology owners, and partners from both Thailand and abroad. The Prime Minister Awards, a prestigious national honor, will recognize organizations, startups, and collaborations that have created significant and measurable impact both nationally and Krithpaka concluded: "NIA invites everyone with a creative spirit—innovators, investors, entrepreneurs, students, or those seeking new opportunities—to be part of SITE 2025. This is a platform to connect with global players, update on future trends, and create meaningful change together. It's also a critical opportunity to help drive Thailand forward in alignment with the 'Groom – Grant – Growth – Global' strategy." The 30+ Business Matching rounds are designed to help entrepreneurs meet with investors and technology owners in a targeted and impactful Panel at the Press Conference: "Accelerating Innovation Through Global Partnerships" This session highlighted cross-sector collaboration driving Thai innovation to the global stage. From the government perspective, Mr. Pariwat Wongsamran, Deputy Executive Director of Innovation System at NIA, said: "Innovation today knows no national borders. What NIA is doing is not just about giving opportunities to entrepreneurs—it's about laying the foundation for an ecosystem that connects Thailand to the world in a tangible way. Through partnerships in Europe and Asia, we're not only focusing on current innovations but also building infrastructure to support long-term growth so that Thai startups can scale more sustainably." Ms. Salinee Phonprapai, Deputy Director-General of the Department of International Economic Affairs, added: "Innovation is now a key engine driving Thailand's economy. The Ministry of Foreign Affairs is pushing Thai startups onto the global stage through a network of 98 embassies and consulates worldwide. We're also working with partners like NIA, BOI, EEC, depa, and the Department of Trade Negotiations to promote investment, open new markets, and facilitate trade deals. In addition, visa policies are being introduced to attract global talent to enhance Thailand's innovation ecosystem."On the private sector and innovation business side, Na Ranong, President of the Thai Venture Capital Association (TVCA), emphasized: "Global partnership is the heart of modern investment. It increases business expansion opportunities and leads to higher value exits, particularly in hot sectors like AI and Deep Tech, which are drawing attention from global investors. Today's investors are looking for startups that can truly connect to the global network." Mr. Thanawit Tonkanya, President of the Thai Startup Association (TSA), said: "Thai startups must dream big, embrace international perspectives, and develop pitch skills that appeal to investors. SITE 2025 is a real-world arena for testing, connecting, and growing global networks.""Be part of the global change! and connect with innovators, creators, and investors shaping the future at SITE 2025." The event will take place from July 4–6, 2025, at Paragon Hall, 5th Floor, Siam entry! Register at: For more updates, follow on Facebook: NIA - National Innovation Agency Thailand, and Startup ThailandHashtag: #NIA #SITE2025 #StartupThailand #InnovationThailand #DeepTech #AI #Sustainability #GlobalInnovationPartnership The issuer is solely responsible for the content of this announcement.
Malay Mail
2 days ago
- Malay Mail
CoolThink@JC × Cambodia Sharing and Exchange Programme MoU Signed to Advance Computational Thinking in Country's Schools
Ms Winnie Ying, Head of Charities (Youth Development & Poverty Alleviation; and Talent & Sector Development), The Hong Kong Jockey Club, and Mr Sok Tha, Director, Department of Digital Transformation, Ministry of Education, Youth, and Sport of Cambodia, sign an MOU for the adoption and adaptation of CoolThink@JC's curriculum and education model in Cambodian primary schools. HONG KONG SAR - Media OutReach Newswire - 20 June 2025 - The CoolThink@JC programme, initiated and funded by The Hong Kong Jockey Club Charities Trust, signed a Memorandum of Understanding (MoU) with Cambodia's Ministry of Education, Youth and Sport at Sha Tin Racecourse in Hong Kong on 18 June. Under the MoU, titled "CoolThink@JC × Cambodia Sharing and Exchange Programme", the ministry will adopt and adapt CoolThink@JC's internationally recognised curriculum and innovative education model in Cambodian primary schools. It will also foster collaboration through experience sharing between the educational systems of Hong Kong and Cambodia. The aim is to ensure that students from diverse backgrounds have equitable access to quality computational thinking education, thus nurturing future-ready talent for the digital Ministry of Education, Youth and Sport plans to introduce CoolThink@JC's curricular materials and successful experiences in 100 public schools in Cambodia, benefiting approximately 30,000 students and 500 teachers, from 2025 to 2028. By enhancing students' computational thinking and problem-solving skills, the initiative can help to bridge the digital divide and open a new chapter in the development of Cambodia's education system. As an ASEAN member state participating in the Belt and Road Initiative, Cambodia plans to share this development at international platforms such as the 2026 Global Smart Education Conference in Beijing, further amplifying the programme's exploring both local and global educational perspectives and exchanging knowledge, educators from Hong Kong and Cambodia will collaborate to create better learning environments for Asian students. Translated teaching materials will also benefit ethnic minority students in Hong Kong, further promoting integrated the MoU on behalf of Cambodia's Ministry of Education, Youth and Sport, Mr Sok Tha, Director of the ministry's Department of Digital Transformation, emphasised the significance of the CoolThink@JC programme in Cambodia's education system during the signing of the MoU. The collaboration with The Hong Kong Jockey Club Charities Trust on computational thinking education not only supports the Ministry in enhancing digital education but also aligns with the Royal Government of Cambodia's vision of producing qualified digital human resources. By integrating this program into classrooms, teachers can enhance their professional development, equipping themselves with the necessary skills to effectively incorporate technology into their teaching practices. This advancement benefits both teachers and students, improving student learning outcomes in the digital age. Through workshops and training sessions, teachers can refine their coding, problem-solving, and logical reasoning skills to better guide students. He is confident that implementing the program will enhance students' critical thinking, creativity, and innovation, leading to improved learning outcomes and better preparation for careers, especially in STEM fields and Winnie Ying, Head of Charities (Youth Development & Poverty Alleviation; and Talent & Sector Development) of The Hong Kong Jockey Club Charities Trust, said: "The CoolThink@JC programme has garnered wide international recognition, and its adoption by the Ministry of Education, Youth and Sport in Cambodia across 100 primary schools is a testament to its success and growing influence on the global stage. We are delighted to witness the establishment of the Association of Computational Thinking InnoCommunity Teachers by a group of passionate CoolThink teachers this year, facilitating experience sharing and professional exchange. Through collaboration and knowledge sharing, we aim to strengthen connections in education between Hong Kong, ASEAN and One Belt One Road regions, paving the way for high-quality, forward-looking education for the next generation. Together, we can drive innovation in education and shape a brighter future for our youth. The Club thanks the Education Bureau for its support in adopting and adapting CoolThink@JC and mainstreaming it in all publicly funded primary schools in Hong Kong."During his visit to Hong Kong, Mr Sok Tha was invited to join the first two days of the International Conference on Computational Thinking Education as a guest speaker and delivered a speech yesterday (19 June) at the conference. He remarked: "In the post- pandemic era, the demand for digital education has been growing, and Cambodia's education system is undergoing a critical period of transformation. With comprehensive support from the government and related policies, Cambodia is dedicated to building a resilient, knowledge-based society founded on innovation, science and technology. We aim to comprehensively enhance the quality of education, sports, science and technology by integrating digital tools and STEM disciplines to equip students with crucial 21st-century skills. We look forward to our collaboration with The Hong Kong Jockey Club Charities Trust and CoolThink@JC, laying an educational foundation for Cambodia's youth towards a future-ready society."CoolThink@JC created the International Conference on Computational Thinking Education (CTE) in 2016. Since 2021, it has provided support for the annual event to be held in different regions, attracting over 2,000 global education scholars, front-line teachers and IT professionals each year. This initiative effectively promotes computational thinking education and drives innovation globally. This year's conference is taking place from 18 to 20 June at The Education University of Hong Kong and Southern University of Science and Technology in Shenzhen, 2023, the CoolThink@JC programme was adopted by Hong Kong's Education Bureau as the foundation for the "Enriched Module on Coding Education for Upper Primary Level" module. The related learning and teaching materials will soon be rolled out in all publicly-funded schools in Hong Kong, supporting teachers to systematically integrate elements of innovation and technology into their curricula. This aims to cultivate students' interest and capability in information technology, while promoting and deepening STEAM and funded by The Hong Kong Jockey Club Charities Trust since 2016, the CoolThink@JC programme is co-created by The Education University of Hong Kong, Massachusetts Institute of Technology (MIT) and City University of Hong Kong, in collaboration with local educators and world-leading scholars. Since its inception, the programme has trained over 1,500 teachers and benefited more than 100,000 students. The programme is committed to mainstreaming computational thinking education into formal curricula, providing teachers with high-quality teaching materials, learning platforms and professional training. The curriculum emphasises problem-solving and logical reasoning skills, guiding students systematically through planning, analysis and debugging thought processes to effectively tackle #CoolThink@JC The issuer is solely responsible for the content of this announcement.
