Contact
Blog

Utilities of PageSpeed Insights and Screaming Frog integration

Posted at 15:39h in SEO by
1 Like

There are SEO tools and then there is THE SEO tool. Screaming Frog, the well-known and indispensable tool that every SEO expert needs to have. There are so many functionalities incorporated in one tool that sometimes when a need for analysis arises, we ask ourselves: “does the “frog” do this?”. We would be surprised how many times the answer would be “yes”.

In this post, we will focus specifically on a function that, even though it is not new (from version 12 of Screaming Frog), it is important to remember and illustrate with some example applications that are useful for day-to-day activities. We talk about the connection via API to PageSpeed that allows to get to know the Lighthouse, PageSpeed and CrUX (Chrome User Experience Report) reports. And yes, the Core Web Vitals are included, and we will be able to extract them in bulk.

How do I set up PageSpeed Insights integration?

Very easily. Open Screaming Frog, go to “Configuration”, in “API Access” select “PageSpeed Insights” and there you will have the field to include the API key that will allow you to connect to it.

 

If you need to create a password, it’s as easy as clicking on “here”. It will take you to the Google APIS page with 3 clicks:

  1. Select a project you have in your account (create a new one in case you don’t have it)

2. Once this is done, select “API Key” under “Create Credentials”:

 

3. Voilá! We already have the key to connect. Copy and paste it in the “Secret Key” field for the connection via API to PageSpeed and hit “Connect”.

4. In “Metrics” you can select both the device from which you want to track and the reports, metrics, etc. for which you are interested in extracting the information:

 

What are the benefits of this integration?

Just the thought that we can track an entire website by extracting the information from Core Web Vitals is enough to get excited about. But beware: in large websites, resource optimization and tracking time can be impacted.

Now…it is necessary to crawl aaall the website with an API connection to PageSpeed? Perhaps, if we select several examples for each type of page, we will have enough to make an evaluation: the results of PageSpeed should not vary too much between the same type of pages. So, if it is a list of URLs and not a complete crawl, we can use the option to bulk analyze these pages.

On the other hand, sometimes we just want  “data, data and more data” but what do we do with all this information? Do we really know which ones are relevant? For me, one of the most helpful ones is what we could call a “Starting Point”: I am going to make a significant change in the web (new site, change of server, change of CMS, migration, modification of web design, etc.) and I need to know the current situation at a performance level to evaluate if this modification benefits the web, and that it does not affect or harms the user experience (assuming that the loading speed is included).

How we do it? If the website has a considerable size of URLs, we select several examples among the different typologies of pages of the site in production and crawl:

1. Analysis with Screaming Frog and PageSpeed Insights before the change/migration

  • Both desktop and mobile analysis of the web in production before applying changes.

Once the URL selection has been made, in bulk mode and previously connected to the PageSpeed API, we perform two crawls to obtain the metrics we are interested in: one with the desktop version and another with the mobile version. Let’s not forget that most websites are already in the Mobile First Index (Google announced that from September 2020 it would start applying it to all websites). Even so, SEO consultants are still working to democratize the importance of mobile analysis for clients (both business and IT). In mobile the metrics will be worse than in desktop, but it is good that we cross both analysis and get the difference in both devices to have a starting point.

  • CrUX. There are still quite a few developers who rely on desktop metrics or speed control according to their own user experience on their devices to extract a rating (at least in my experience!). For this reason, it is essential to highlight that we cannot take someone else’s experience as a reference, we must base our decision making on CrUX: Chrome User Experience Report, the report of field data from real users in Chrome in the last 30 days. Here we will get a sample of real data on both devices. Be careful! This report will only be available on websites with a significant volume of visits.
  • Crossing between web in production and web in test. We assume that websites in development environments will generally have worse performance data and that with the rise to production it will improve. These analyses will never be 100% reliable, but they can help us detect key differences between what we have now and what we are trying to build. Therefore, we would recommend passing that same selection of web URLs into production, but the equivalent in test and also for both devices.

 

2. Analysis with Screaming Frog and PageSpeed Insights after the change/migration

  • Crawling the new web in production both desktop and mobile. Again, that same selection of URLs we crawl in bulk mode with the changes already applied in production, both for desktop and mobile, and cross-check with the info we have previously saved.
  • What analysis would we get?

This facilitates the comparison between important metrics such as the Core Web Vitals (LCP, CLS and FID) or the CrUX report itself to assess user experience in the set of pages analyzed. The main comparisons of interest would be:

  1. Difference between the current mobile version vs. the old one
  2. Difference between current desktop version vs. the old one
  3. Difference between current mobile-desktop vs. old mobile-desktop

All this will allow us to extract insights to know if the change has affected us positively or, if not, to locate the points of worsening that we will need to attack to improve.

Sometimes SEO consulting must focus on something that may seem obvious at first: the web changes must be made to maintain or improve the situation. If they imply a loss in any sense, we will have to analyze how it impacts on SEO/Business and thus transmit the assessment to the departments involved.  And yes, one SEO and Business. After all, we work to make the business a happy business, don’t you think?

5/5 - (1 vote)

Artículos relacionados

The following two tabs change content below.

Marta Romera

Consultora SEO y SEM en Internet República. Licenciada en Publicidad y Relaciones Públicas por la Universidad de Sevilla, sumergida en analítica web y Search Marketing. Entiendo la comunicación como la pata de un negocio, Internet como oportunidad de obligado uso, el Search Marketing como herramienta indiscutible para llegar al pódium y la analítica como pasado, presente y futuro de la estrategia.
Tags:
, ,