Haiilo is not fetching content from a URL I input in a Link post. Why?

Haiilo's content fetcher will always automatically try to fetch all the relevant content, such as title, body text, and image, from the input webpage and then populate this in Haiilo. If the content is not being fetched automatically, this can be due to, for example, the below reasons.

If none of these reasons apply to your website, please get in touch with our Service Desk for assistance.

1. The website is missing relevant metadata tags

Haiilo's content fetcher uses a Newspaper extension to fetch information from URL addresses. Most often, the content is fetched from the following tags:

  • Title: <meta property="og:title">
  • Body: <meta property="og:description">
  • Image: <meta property="og:image">

If your webpage does not include these tags, Haiilo's content fetcher can fetch either nothing or the first random tag it finds in the page source. It is recommended to contact your webpage's development team and ask them about including the correct metadata on the page.

contentfetching.png

2. The content cannot be fetched from a particular site

Haiilo's content fetcher bot will always attempt to access the post content on a website. If it is successful, some content (title, body, and/or image) will be fetched for the post. However, because website security is continuously updated, it might mean that our content fetcher bot might be able to access and fetch content for a URL one day but might fetch no content for the same URL another day.

Also, there are some cases where Haiilo's content fetcher cannot fetch content automatically due to different website structures and security restrictions, examples include Forbes.com, Bloomberg.com, and websites behind a paywall or requiring logging in to view the content.

 

Screenshot_2023-01-27_at_12.04.25.png

If your company owns a website from which Haiilo cannot access content, please ask your website security team to allowlist Haiilo's content fetcher bot. The information of the bot is User-Agent: newspaper/0.2.8.

Was this article helpful?