Field Data vs. Lab Data – How To Measure Web Performance?

TL;DR

• There are two ways of measuring performance – in the lab and the field. 
• Lab data is artificial and collected from a single, predefined device, location, and internet connection.
• Field data is gathered from many users, and you can filter it to focus on a specific user and their experience.
• If you want to verify the real performance experienced by users, you should use field data.
• If you’re working in a testing or debugging environment, feel free to use lab data.
• Both approaches are useful in their way, and you can easily combine them for the best results. 

So, how do you measure web performance?

You may already be familiar with Google’s tools like PageSpeed Insights, Lighthouse, or the CrUX Dashboard in Google Looker Studio (Data Studio). Or maybe you’re still looking for a solution.

To make the right choice, you should first know what you want to measure and why.

While most of these tools use similar metrics, they often rely on different data sources. And the essential distinction you need to make is between lab data and field data in web performance.

In this article, I’ll go through the differences between lab and field data to help you make the right choice.

This is because the quality of data you use to optimize your website’s performance will determine how successful you are.

Let’s dive right into it.

What is the difference between field data vs. lab data?

The concept of lab and field data wasn’t invented to measure web performance. It’s been used in various scientific tests to distinguish between the experiments conducted in a laboratory and those conducted outside of it.

Lab data

Lab data is collected in a controlled environment and uses predefined device and network specifications. This process is called synthetic monitoring.

So, when you measure your web performance with lab tools, they load your website using a predetermined device and connection speed and measure how performant it is.

When it comes to measuring web performance, lab data is useful in reproducing and debugging possible performance issues. While it doesn’t give you insight into your users’ experience, it’s viable if you don’t have access to real-user measurements.

Field data

Field data reflects the browsing experience of real users who use your website. It is affected by the connection and device they’re using while browsing. Field data often unearths the issues that are hard to capture in a lab environment.

Field data is also known as real user monitoring (RUM), real user metrics, real user measurement, or end-user experience monitoring. Google usually uses the term field data.

Why lab and field data can be different

It’s normal for lab and field data to show different values for the same metrics.  They simply measure web performance based on various sources. 

This is how Google’s Martin Splitt described the difference between lab and field data during a Google JavaScript SEO Office Hours on June 10th, 2020:

Field data is coming from real users, whereas lab data comes from a quite strong machine with probably good internet from somewhere around the world. So you might not see the same results.

source: Martin Splitt

That’s why when for lab data you may see, e.g., one simulated device and network connection, field data is based on the distribution of how all users experience your website. It means that there may be users browsing your website on different devices and both fast and slow Internet connection speed, and field data reflects all that.

In other words, with lab data based on predefined settings and field data focused on how real users interact with your website, it’s completely normal that you may see some discrepancies in the metrics.

But I understand it might be confusing, especially when lab data shows nothing to improve and the field data indicates the opposite. 

Here is our short summary of characteristics unique to field vs. lab data that you may find useful to understand the difference between them:

However, lab and field data don’t only come from different sources. You may also collect them using different tools.

So now, let’s get practical and learn how to collect web performance data.

How to collect lab data and field data

One method of collecting web performance data is to use the relevant JavaScript libraries. By setting up tracking on your own, you get more freedom to precisely choose what you’re trying to measure, and you can send the aggregated results straight to your analytics tool.

However, setting up your own tracking is time-consuming and requires you to have advanced knowledge of JavaScript.

Various third-party services offer to measure your web performance, but most of them also require you to implement code on your end, and they tend to be expensive.

So my recommended method is to use the free tools created by Google. 

Tools like Lighthouse and PageSpeed Insights run a quick lab test of your page. They’re easy to use and are convenient when you need a brief overview of your lab performance.

When it comes to field data, Google collects it from Chrome users and stores it in the Chrome User Experience Report (CrUX).

The data from the CrUX report is accessible through several tools and APIs, which you can choose depending on your specific needs.

While it’s important to note that CrUX doesn’t give you information about your users on other browsers, I still think CrUX is the ultimate source of web performance data. If you know that many people access your website on different browsers, you may implement additional tracking methods afterward, but CrUX is the best place to start.

Moreover, CrUX contains the same data that Google uses to rank websites after the Page Experience update went live in 2021. Since then, Core Web Vitals became a part of the ranking algorithm, and these three metrics (LCP, CLS, and FID) now influence your rankings.

To sum up, using Google’s tools allows you to get a mix of lab and field data that you need to get a thorough understanding of your web performance. 

Time to learn about these tools in more detail.

Field data – Chrome User Experience Report

Field data gathered by Google can be accessed via Chrome User Experience Report (also known as CrUX). The report’s data is aggregated from Google Chrome users who have opted-in to syncing their browsing history, have not set up a Sync passphrase, and have usage statistic reporting enabled.

The list of web performance metrics you can collect from CrUX is impressive:

• First Paint,
• First Contentful Paint,
• DOMContentLoaded,
• onload,
• First Input Delay (FID),
• Largest Contentful Paint (LCP),
• Cumulative Layout Shift (CLS),
• Time to First Byte,
• Notification Permissions,
• Effective Connection Type,
• Device Type,
• Country.

In December 2020, the CrUX database included over 8M origins, and it continues to grow.

The same goes for the metrics available in this report.

We hope to add more metrics and dimensions, both to provide more insight into loading and other critical factors that most affect user experience.

source: Google’s official documentation

Based on that, we can expect more metrics to be added to CrUX in the future.

Accessing CrUX

You can access the CrUX data with several tools:

• Google Search Console,
• Looker Studio (previously known as Google Data Studio Dashboard,)
• Google BigQuery,
• CrUX API,
• PageSpeed Insights API,
• Google PageSpeed Insights.

If you’re just starting, using the Search Console, Google Looker Studio, or PageSpeed Insights is the way to go – these tools are easy to use, although they only give you a sneak peek into what CrUX can offer.

By using Google BigQuery or one of the APIs, you can access more metrics and look at your performance in specific countries, on particular devices, and connection types.

Pros and cons of using Google’s field data

Lab data

Lab data is artificially collected in the test environment. It doesn’t come from real users but is simulated on a single device with a predefined location and network connection.

The examples of metrics that you can exclusively measure in the lab are Total Blocking Time and Time To Interactive.

Google’s lab data tools

The most popular lab data tools are:

• Lighthouse,
• PageSpeed Insights (contains a mix of lab and field data),
• Google Dev Tools (or developer tools in other web browsers),
• WebPageTest.

Pros and cons of using lab data

Field data or lab data – which one should you use?

If you want to know how real users are experiencing your website, field data is the right choice.

According to Google’s Martin Splitt:

Field data is probably a better indicator for how real users are experiencing your website than lab data. Because lab data is literally just someone’s server making a request to your thing. And then if that server happens to be quite beefy then you get pretty good-looking numbers, but then the real world isn’t as beefy and nice.

source: Martin Splitt

So, field data is what you should be monitoring to know if your website’s users around the world have a positive browsing experience.

Also, field data is used by Google to rank your website.

For search rankings, we use field data, as this is what your site’s users have experienced over time. This makes the data more representative for your site, taking into account where your users are located and how they access your website.

source: John Muller

However, if you are in the process of actively optimizing your site, you have no choice but to rely on lab data – it allows you to evaluate your improvements without having to wait for the real user data to come in.

That being said, there’s no need to decide that one approach is superior and more useful than the other. Your course of action should always depend on why you need the data. 

For example, you can use field data to define where your users are, how fast their internet connection is, and what device they use. With this information, you can simulate your user with lab tools when optimizing your pages and verify how much the changes you’re implementing affect the results.