Evaluation for Large Language Models and Generative AI - A Deep Dive

Evaluation for Large Language Models and Generative AI - A Deep Dive
Notebooks and additional resources: https://github.com/rajshah4/LLM-Evalu...

0:00 Overview
1:30 Evaluation is broken
4:00 Reliability issues with HELM and Hugging Face Leaderboard
6:32 Evaluation before generative AI
9:33 Framework for evaluating generative AI
14:01 Reviewing 8 evaluation approaches
15:22 Exact matching approach
29:12 Similarity approach
32:37 Functional Correctness
35:50 Evaluation Benchmarks
45:12 Human Evaluation
49:03 Human Comparison/Arena
52:00 Model-Based Evaluation
1:02:22 Red Teaming
1:06:11 Operational Issues in Evaluation
1:09:10 RAG Case Study
━━━━━━━━━━━━━━━━━━━━━━━━━
★ Rajistics Social Media »
● Home Page: http://www.rajivshah.com
● LinkedIn:   / rajistics  
━━━━━━━━━━━━━━━━━━━━━━━━━