ChatGPT와 후속 LLM(Large Language Model)이 등장하면서 “RLHF” 라고 불리는 “인간 피드백을 통한 강화학습”의 중요성에 대한 많은 논의가 있었다. 나는 “RL이 Supervised Learning보다 더 나은 이유가 무엇일까?”, “언어모델을 지침을 통해 학습하는것(Instruction fine-tuning)만으로 충분하지…
ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning Viet Dac Lai, Nghia Trung Ngo, Amir Pouran Ben Veyseh, Hieu Man, Franck Dernoncourt, Trung…
https://t.co/CIU4E5NLC8 LeCun은 AI 업계에서 가장 저명한 과학자 중 한 명이며 ChatGPT와 같은 AI 챗봇이 사용하는 기본 기술의 기능을 과장하는 사람들을 노골적으로 비판해 왔습니다. Barron’s: ChatGPT와 대규모 언어 모델(LLM)의 기술이 어떻게 작동하는지 설명해 주시겠습니까? LeCun: 초강력 예측 키보드라고…
SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models Vithursan Thangarasa, Abhay Gupta, William Marshall, Tianda Li, Kevin Leong, Dennis DeCoste, Sean Lie, Shreyas Saxena https:/…
Image as a Foreign Language: BEiT Pretraining for All Vision and Vision-Language Tasks Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorck, Zhiliang Peng, Qiang Liu, Kriti Aggarwal, Owais Khan Mohammed, S…
AdaMix: Mixture-of-Adapter for Parameter-efficient Tuning of Large Language Models Yaqing Wang, Subhabrata Mukherjee, Xiaodong Liu, Jing Gao, Ahmed Hassan Awadallah, Jianfeng Gao https://arxiv.org/ab…
LaMDA: Language Models for Dialog Applications Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang …
An Image is Worth 16X16 Words: Transformers for Image Recognition at Scale Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani,…
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks Patrick Lewis, Ethan Perez, Aleksandara Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau …
Integrated Eojeol Embeddings for Erroneous Sentence Classification in Korean Chatbots DongHyun Choi, IlNam Park, Myeong Cheol Shin, EungGyun Kim and Dong Ryeol Shin https://arxiv.org/abs/2004.05744 1…