Collection guides

How to set up Timepath Collections: A Guide to Automating Article Extraction

Learn how to use CSS selectors to automate article extraction with Timepath. This guide provides step-by-step instructions and examples to help you find the right selectors for your needs.

Colin Lansbergen · CTO
Learn how to use CSS selectors to automate article extraction with Timepath. This guide provides step-by-step instructions and examples to help you find the right selectors for your needs.

To automatically get articles from a webpage using Timepath, you need to provide a CSS selector. This selector is like a unique address that tells our system exactly which element on the page to look at. Just like a postal address, a good selector is specific enough to find the right item but flexible enough to work if minor details change.

1. Understanding CSS Selectors

A CSS selector targets HTML elements based on their tag names, class, ID, or other attributes. Here are the most common and useful types of selectors:

  • Tag Name: This is the simplest selector, using the element's tag name. For example, a selects all anchor tags.

  • Class: This is the most common and reliable method. Use a period (.) followed by the class name. For example, .wl-tile selects all elements with the class wl-tile.

  • Attribute: Use square brackets [] to select an element based on its attributes. For example, [data-topic] selects any element with the data-topic attribute.

2. How to Find the Right Selector

The best way to find a selector is to use your browser's developer tools.

  • Inspect the Element: Right-click on the article you want and select "Inspect." This opens the developer tools and shows you the HTML code for that element.

  • Look for Unique Identifiers:

    • ID: Check for an id attribute (e.g., id="article-123"). If found, use #article-123. IDs are unique and the most reliable selectors.

    • Class: If there's no id, look for a unique class (e.g., class="wl-teaser--basic"). This is a great, specific option.

    • Data Attribute: Many modern websites use data- attributes. [data-topic] is a good example of a reliable attribute selector.

  • Test Your Selector: To confirm you have the right selector, go to the "Console" tab in the developer tools and type $$('your-selector-here'). Press Enter, and the console will show you all the elements that match your selector.

3. Example Selectors for search_result_item_url

Here are some example values you can use for search_result_item_url based on common HTML structures, including the example you provided.

Selector

What it selects

Why it works

[data-topic]

The parent <article> container.

The tool finds the <a> tag inside the parent container to get the link.

.wl-tile

The parent <article> container.

This class is a common identifier for the container holding the article link.

.wl-teaser--basic

The <a> tag itself.

This is a highly specific class that directly targets the link element.

Ready to build it?

Every guide here describes something you can do on a free account. Open the editor and follow along.

Subscribe for updates

Product news, newsroom stories and new formats — occasionally, never spam.