Latent Space The AI Engineer Podcast

Latent Space The AI Engineer Podcast 6: Alessio Fanelli and Swyx

Published: April 7, 2023, 2:18 a.m.

Business Name(es)

Late in Space



Key Lessons from the Episode

The most important things to take away


Benchmarks are essential for evaluating language models and driving research.

"When you ask people what are the ingredients going into a Large Language Model, they'll never talk to you about the benchmarks and it's actually a shame because they're so influential. Like that is the entirety of how we judge whether a language model is better than the other."

Benchmarks have evolved over time, becoming more challenging and diverse.

"The difficulty of benchmarks has really skyrocketed from the 1990s to today."

Over-optimizing for a single benchmark may lead to bias in model development.

"Whenever you're working on a new model, the benchmark kind of constrains what you're optimizing for."

Verifying and reproducing benchmarks can be challenging, especially with large language models.

"It's very hard to verify as well because there are certain problems with reproducing benchmarks, especially when you come to large language models."

Episode Details

Episode Gist

Customer Acquisition

Late in Space acquires customers through their ad-based podcast and by building a community around AI and technology.

Business Category



Technology and Gadgets

Business Model


Contact Alessio Fanelli and Swyx