구글 ML 스터디 잼 2] Introduction to Large Language Models

Define large language models(LLM)

: large, general-purpose language models can be pre-trained and then fine-tuned for specific purposes

• major features
• large 1) enormous size of training data set 2) parameter count
• general purpose : solve common probelms
• pre-trained and fine tuned : pre-train a large language model for a general purpose with a large data set and then fine tune it for specific aims with a much smaller data set

 

transformer model e.g.PaLM, LaMDA, GPT

: consists of encoder and decoder

encoder encodes the input sequence and passes it to the decoder, which learns how to decode the representations

 

Describe LLM use cases

Explain prompt tuning

prompt design is the process of creating a prompt that is tailored to the specific task that this system is being asked to perform #general_concept

prompt engineering is the process of creating a prompt that is designed to improve performance. #specialized_concept

-LLM 종류 따른 prompt tuning

 1. generic language models : a next word predictor

  - train한 데이터를 바탕으로 다음 단어에 가장 잘 어울리는 것을 토큰을 예측한다(e.g. the, a, ..)

 2. instruction tuned : trained to predict a response to the instructions given in the input

  - classify the text into neutral, negative or positive

 3. dialogue tuned : trained to have a dialog by predicting the netx response

  - expecteed to be in the context of a longer back and forth conversation, and typically works better with natural question- like phrasings

chain of thought reasoning

classification task sentiment analysis task model

Parameter-Efficient Tuning Methods(PETM)

: methods for tuning an LLM on your own custom data without duplicating the model

Describe Gen AI development tools

Generative AI Studio 

Generative AI App Builder - without any code

PaLM API

---

▼

Introduction to Large Language Models | Google Cloud Skills Boost