Lennard Berger's blog

A decade of software

Sun, 19 Apr 2026 00:00:00 +0000

May 2026 marks a personal milestone: getting paid to write code for one decade. It feels surreal writing this.

I thought I’d like to take the time to share my learnings, for interested readers. I’ll will keep it short and talk about five key topics:

Software is for humans
Software is your biggest liability
The secret to get things done, is to do them
Surprising things a university degree teaches you
People trump profit every time

I hope my whimsical headings peaked your interest, so let’s dive right into it!

Building OCR pipelines with Google AI Studio

Fri, 28 Nov 2025 00:00:00 +0000

Recently I wanted to catalogue all the books I own in my home. It turns out, this isn’t really a solved problem. There’s an abundance of ISBN scanners, but no tool that simply takes a photo and tags the books in it.

Traditionally building such a system contains two steps:

Creating a segmentation model to recognise individual book titles and spines
Use the title / spine and OCR to get a title

Once these steps are done one can build a search, e.g using the Google Books API to match titles against ISBN.

Quarto for scientific presentations

Thu, 28 Aug 2025 00:00:00 +0000

If you haven’t heard about Quarto yet, it self-describes as an “open-source scientific and technical publishing system”. You’ve certainly used Quarto - or consumed media produced by Quarto, in fact this very blog is powered by Quarto.

Quarto has some simple core ideas:

use Markdown as a universal templating language
make it easy to include and its output in documents
make visualization easy (and fun)

Quarto is powerful and agnostic about the programming language and documents you want to render - it supports creating websites, documents and presentations.

Micronaut Data and pgvector

Mon, 17 Mar 2025 00:00:00 +0000

Recently vector embeddings have been all the rage. Vector embeddings are a way to project data (usually text data such as words and sentences) into a dense vector representation of a fixed size. This could be useful in natural language processing, where you want to build a retrieval augmented generation pipeline. It could be useful in object character recognition, if you want to convert your document into a searchable representation etc.

How accurate is the weather forecast?

Mon, 03 Mar 2025 00:00:00 +0000

Weather forecasts are one of these things we rely on daily and rarely actually think about. This blog post will disects how they work (at least in Germany). I was curious to see if the forecasts are acurate, and we’ll get to that as well.

Ask the weather frog

When you open the default weather app on iPhone it will use the forecasts by the DWD (Deutsche Wetterdienst) = German metrological association. The DWD runs a huge real-time weather simulation which takes into account hundreds of parameters like:

Micronaut 3x Security: Authentication with Firebase

Thu, 27 Feb 2025 00:00:00 +0000

This post was inspired by Brian Schlining’s excellent post on Micronaut authentication.

I had slightly different requirements than Brian, namely I wanted to do:

issue id tokens using Firebase’s idToken
send it to my backend to verify requests

My solution opts to directly verify the id token at request time. When hosting on GCP services, latency to Identity Platform is very low, making this approach feasible. It has a few distinct advantages:

Hosting on the Google Cloud Platform

Sun, 08 Dec 2024 00:00:00 +0000

Let’s face it, deployment is the worst part of software engineering. In the decade I’ve been writing code, deployment has been the largest driver of headaches. When Docker first debuted in 2013, many had high hopes the times of sysadmin-battles with failing hardware and complexity would finally come to an end. Well, things have improved, but many issues remain.

I recently had to deploy an application with the following requirements:

Testing linear regression empirically

Mon, 04 Nov 2024 00:00:00 +0000

Linear models are intriguing because they are both simple and powerful. This simplicity can be attributed to the fact that many natural processes actually follow simple laws.

I’ll give you an example: bigger objects are usually heavier. Both attributes are normally distributed (that is, there is an average size and mass for an object and fewer exceptions). These assumptions are what drives ordinary least squares (OLS) regression, in particular there are six key assumptions:

How much calories does an LLM burn?

Fri, 20 Sep 2024 00:00:00 +0000

Every request to GPT-4 burns 430 calories, approximately 17% of an average male diet

A clickbait quote from me for the dear reader, to tease you into the topic of AI energy consumption, something many people rarely give any thought to.

You might wonder, how did I come up with this number? Glad you ask, I’d like to do the math with you.

Let’s fuel an LLM

Many people are aware of the fact that large language models consume lots of energy. It is in the name, “large” models. We don’t have the exact tally for GPT-4, as the technical paper does not outline it. see [1] went down the rabbit hole to find out how much energy GPT-3 and Meta’s LLama family of models consumed during training.

Thinking is not hierarchical, why should AI be?

Sat, 01 Jun 2024 00:00:00 +0000

This blog post will discuss Yann LeCun’s paper see [1] and a plethora of related ideas. Below is an illustration from see [1] which summaries LeCun’s AI model:

LeCun imagines a set of hierarchical components which interact with each other. Most importantly:

the percept which can perceive the world (e.g. multi-modal input)
a world model which is built on top of the perception
a cost component which reflects how “useful” the agent is in solving a problem
a configuration component which modulates an actor to perform actions and directs the world model and perception

The whole system would be end-to-end trainable with the cost component providing the objective function for optimization.

Word embedding similarity is tricky

Mon, 06 May 2024 00:00:00 +0000

This post will be a deep dive into word embeddings. Word embeddings have made multiple appearances on this blog, such as in my talk about neural networks, or the spaCy benchmark. The reason I am captivated by word embeddings is threefold:

they are very useful, the underlying building blocks for most text-based machine learning models
word embeddings are computationally efficient (you can encode “all the language’s meaning” in a few hundred megabyte)
very few people actually understand what the consequences of embeddings are

Cosine similarity is the most popular method to date to compare embeddings (such as in document retrieval). A recent paper by see [1] investigated the usage of cosine similarity and uncovered some surprising findings:

Building a property graph database from scratch

Mon, 08 Apr 2024 00:00:00 +0000

As can be seen from the featured image above, many natural processes can be modeled as graphs. Transactions of individuals in a payment network are classic datasets for which graphs are a natural fit. Once such a dataset is loaded into a graph we can easily answer questions such as:

how many steps does an average transaction take?
who is a 2nd-level collaborator of “x”?

These questions could be answered from a relational database. However, modeling relational databases is not intuitive (for the problem), and in many cases not very performant. It is for this reason, that graph databases have steadily gained popularity, especially in the area of knowledge graphs.

Mapping spans from one document to another, an excellent interview question

Sun, 03 Mar 2024 00:00:00 +0000

Interviewing is arguably one of the worst parts of running a business. You can plan for success, and you can account for failure. When it comes to interviewing, neither applies.

To a certain degree this is certainly the very nature of the process itself. A company wants to hire the optimal candidate, whereas all candidates want to get hired. Thus, it is in the interest of the company to reduce barriers and design interviews in such a way to yield satisfactory results for everyone involved. One way to improve technical interviews is to create better interview questions.

The relationship between L2 norm and dropout in linear regression

Thu, 01 Feb 2024 00:00:00 +0000

In 2014, Dropout was introduced see [1] as a simple, yet effective way of regularizing neural networks. Dropout is conceptually simple: one randomly removes connections of neurons from the network. If a network can manage to learn the data well with a certain amount of disturbance (removed nodes), it can be said to have found a good generalisation. However, the implications of dropout are not quite as trivial. In this blog post, we will look at a simple model to gain some intuition into the effects of dropout (and how it compares to other forms of regularisation).

Playing "The game" with Monte Carlo simulations

Fri, 05 Jan 2024 00:00:00 +0000

A while ago I was introduced to a simple card game titled The game by Steffen Benndorf.

Unlike many other games, The game has a unique touch, because it is a collaborative card game.

The rules of the game are very simple. There are four piles, two of which start at 1 and ascend to 100. The other two piles start at 100 and descend to 1. Players take turns to place cards on the piles. If they manage to put down all cards, they win. Otherwise, The game wins.

NVIDIA resolves gRPC performance issues

Tue, 28 Nov 2023 00:00:00 +0000

As with any software architecture, deep learning models need a delivery platform. This is the job of NVIDIA’s Triton Inference Server. The idea is conceptionally simple:

produce your model in one of the supported model formats (such as ONNX)
package your model with a definition file, letting Triton know how to process your model
run a simple Docker container from the NGC registry with your models on an S3 or other storage bucket
profit from a standardized inference runtime and built-in metrics

Next to reliability, the main concern of any DevOps engineer will be throughput. What good does it do the production team if models have a 0% downtime, but are so slow as to be unusable?

Graph randomization is not the same as graph randomization

Sat, 25 Nov 2023 00:00:00 +0000

Recently I haven taken some interest in network science. There are many things to discuss. For this blog post I would like to share my fascination that $randomization \neq randomization$. In network science, node degree and strength [1] are important properties.

Node degree

In an undirected network the node degree refers to how many edges any given node has. As a linguist, if I wanted to model word relationships using graphs, I need to be aware that word frequency follows a Zipfian distribution. Therefore, the number of connected nodes should follow a similar distribution (if I want to model the same phenomena). Now, provided I wanted to randomise my graph to test wether a specific effect holds, I also need to take care this distribution stays intact. This is why there are two ways to randomize a network:

The missing spaCy benchmark

Thu, 28 Sep 2023 00:00:00 +0000

spaCy self-proclaims as “Industrial-Strength Natural Language Processing”.

I can wholeheartedly agree. Over many years spaCy has served me extremely well as a base component in a diverse set of projects. The top reasons to use spaCy have always been:

reliability
speed
ease of use
acceptable accuracy out-of-box

Recently, document retrieval systems have been getting a lot of attention. In the light of this, I was curious to see how spaCy would hold up to this task. Usually spaCy is the most well-documented project in the industry. As with anything, there are edge cases.

A note on compression based classifiers

Fri, 11 Aug 2023 00:00:00 +0000

Some weeks ago I stumbled across one paper in particular titled: "Low-Resource” Text Classification: A Parameter-Free Classification Method with Compressors" see [1]. This paper has been dubbed the “gzip paper”, and has been largely discussed in the NLP community because of a refreshingly simple and efficient take on text classification. In the authors words they are matching neural networks in performance:

There have been several studies in this field [2]; [3], most of them based on the intuition that the minimum cross entropy between a document and a language model of a class built by a compressor indicates the class of the document. However, previous works fall short of matching the quality of neural networks.

Modeling mindsets

Mon, 31 Jul 2023 00:00:00 +0000

Some while ago I read a fantastic book titled Modeling mindsets by Christoph Molnar. It prompted an idea to reiterate on a problem that is as old as any stastician’s playbook: student exams. Writing exams is difficult. However, a statistician may claim, modeling an exam result is fairly trivial. Bear with me for a moment, and you’ll see there is more to this simple problem than you might expect.

Exam scores

Exam scores are a popular choice in many statistican textbooks. That’s because it is largely accepted that they follow a normal distribution (hence the term student’s t-test). As they saying goes: the proof of the pudding is in the eating. To demonstrate how to model exam results, we will use a dataset of 1000 fictional exams created with Rocye Kimmon’s excellent exam scores generator. We can display the first few rows to get an idea of what data it contains.

Who wrote it first, or: why originality matters?

Thu, 15 Jun 2023 00:00:00 +0000

Ever since the introduction of language, humans have suffered from a fundamental problem: there is an exponentially growing amount of information, and only so much lifetime (of any individual), to grasp it. This has become an increasing problem with the advent of the World Wide Web. Individuals now have access to more information than any generation of humans, ever before, in history. However, access to more information doesn’t necessarily translate to better understanding.

Python dataclasses, a package full of surprises

Sun, 14 May 2023 00:00:00 +0000

Premature optimisation is the root of all evil

Virtually every software engineering lecture nowadays contains this wisdom by Sir Tony Hoarse. If you ask me, this truth has been perpetuated for good reason. With modern computing power, there are few motivations to worry about low-level code optimisation. Certainly, as it comes towards the triangle of software development, resources are inexpensive compared to engineering time, and more simple and maintainable software design is preferable over performance. But to every rule, there are exceptions. Today we will discuss one of the stumbling blocks in our software landscape: serialisation.

yGuard is now open source - obfuscation easy as pie

Wed, 08 Jan 2020 00:00:00 +0000

Disclaimer: This is not a sponsored post

I work for a company called yWorks. They make libraries for graphs that are (imho) state of the art, and also are a fun company to work at 🍺

Because these libraries are sophisticated the company is very inclined to protect their intellectual property. In the golden age of Java (which was right about the time they put this onto the market) it would have been crucial to implement some sorts of protection in order to ensure this IP. This is why yGuard was initially developed (side note: when I first touched the repository, some assets were as old as 11 years).

About me

Mon, 01 Jan 0001 00:00:00 +0000

Hi there! I am a Cognitive Scientist interested in the mechanisms underpinning (human and machine) learning.

My tinkering is primarily done in Python 🐍, which helps me prototype very quickly. My colleagues call me a “professional thing breaker”. Other than that I’m agnostic - most any problem deserves its own solution.

Work

I work on the ARC AGI challenge at Martin Butz’s lab, attempting to solve iterative planning problems in AI.