大语言模型能力来源与边界
XXX
1
大语言模型基础理论突破，发表大模型相关论文80+篇
仅修改130亿参数中的1个就会使模型完全混乱
输入：FudanUniversityislocatedin
LLaMA2-13B(PPL5.877)：FudanUniversityislocatedinShanghai,China.Itislocallyknownas复旦大学.Theuniversitywasestablishedin1905.Itis
accreditedbyMinistryofEducationofthePeoplesRepublicofChina.Thereareover40,000studentsstudyinginvariouscoursesofferedbyFudan
University.ThelanguageofinstructionisChinese.
LLaMA2-13B-修改语言核心区1维扩大10倍(PPL376079936)：FudanUniversityislocatedin
ssssssssssssssssssssssssNosSsYousTheresThatsAsThisss##sssss
ThisssssThisssstssThes/ssWhatsssssssssssssssssssssssssss
sssssssssssssssssssssssssssssssssssssssssssss
sss
ssssssssssssssssssssssthssssdsvss\s{s
LLaMA2-13B-修改语言非核心区LayerNorm其他维度扩大10倍(PPL5.914)：FudanUniversityislocatedinShanghai,China,thelargestcitywiththe
mosteconomicandculturalactivitiesinChina.Withthemostadvancedinfrastructureandthebestlivingcondition,ithasbecometheinternational
educationcenterwiththelargestoverseastudents.ItconsistsofJinan,KangqiaoandFenglincampus,whichboaststhebestresourcesfromboth
educationandresearch.FudanUniversityhasbeenafamousandattractiveuniversityforinternationalstudents,especiallyinthepastonedecadefrom
2001-2010.
国际上首次提出的大语言模型语言核心区和维度依赖理论，可以有效指导大语言模型训练过程
4
1.大语言模型语言核心区与维度依赖
破坏‘Arabic/Vietnamese’区域
ArabicMMLU:AssessingMassiveMultitaskLanguageUnderstandinginArabic(Kotoetal.,arXiv2024)