Publii Tips: Distinct Feeds with Pipes.digital

Unfortunately, Publii's feed functionality is still in need of major development. Different feeds based on different tags are currently not possible. The remedy is pipes.digital by a German blogger colleague of mine, onli-blogging.de, which sends feeds through a digital pipe and can filter, modify and augment them in the process.

Essentially, Publii's feed options revolve around whether a post is "featured" or not:

The pipes.digital offering, on the other hand, can filter a (complete) feed based on tags (categories), so that, for example, only German-language entries or those for a certain audience (e.g. FOSS feed) come out. But that's just the tip of the iceberg: numerous other options can be applied in the pipeline. For this there is a graphical block editor, which allows various blocks with manifold operations between source and output.
The result is then available under a link of the following format (URL/feed/FeedID):

https://www.pipes.digital/feed/AqG8bV9p

The above feed is quite simple and generates as output only posts tagged with OSBN (an aggregator for German blog posts about FOSS). Of course, in this case, every post intended for the OSBN must be tagged accordingly.

But it would also be easier to turn the tables and filter out only posts that have been tagged with, let's say NOOSBN. This is also possible with pipes. Just enter the tag name, choose item.category and check the box "block found items".

About pipes.digital

Pipes is a spiritual successor to Yahoo! Pipes, but if you did not know that site, you can think of Pipes as a visual programing editor specialized on feeds, or a visual shell, or simply as a glorified feed configurator.

Pipes gives you blocks that can fetch and create feeds, and manipulate them in various ways. Think filtering, extracting, merging and sorting. All you need to do is to connect those blocks with each other. Data just flows through such a pipe, it flows from block to block. At the end Pipes gives you a new feed, which you can give to other programs that support open web standards - such a program could be your feed reader.

As input formats Pipes supports RSS, Atom and JSON feeds, it can scrape HTML documents, and it can work with regular text files.

🗨 https://www.pipes.digital/docs

Registration and scope of services

You don't need your own account to get started, you can conveniently log in with your Google account (Portier is working under the hood).

Portier is used as the auth system. It enables logins without having to store passwords. Users log in with their email address and only have to confirm that they control that address, by either clicking on a link sent to them or, for Gmail, logging in with their Google account. That work is done by an external broker, Pipes just has to check the token sent back by that system.

🗨 https://www.pipes.digital/blog/8/The%20Architecture%20and%20Software%20behind%20Pipes

The free version offers three pipes with reduced pipe timeout. However, there are also paid plans for 5, 20 and 79 dollars - of course, each with different services behind it.
Pipes is open-source (keyword Pipes CE), so it can also be self-hosted:

https://github.com/pipes-digital/pipes

Feature requests and pull requests are welcome.

Usability

The offered selection of different function blocks can seem a bit overwhelming at first.

The remedy is the excellent documentation. There is also a video for a first impression and approach:

Under the hood

Pipes is a Ruby/Sinatra application, uses SQLite as database engine and Raphaël for the main part of the frontend, the editor. Ruby works well here, since it is a very expressive language with a healthy ecosystem around XML (oga, nokogiri) and some useful modules for RSS/ATOM normalization (feedparser, feedjira), and creation (included). Sinatra provides the webserver parts of the system and the HTML views, this part is rather small. SQLite is the data store for everything, including the pipes serialized to JSON and cache data. And finally Raphaël abstracts creating the SVG part of the javascript powered editor, painting the blocks and manipulating them.

Use cases

The combination of different blocks results in countless use cases. Some of them are described here. Basically there are no limits to your creativity. For users of Publii the tag-based filtering of feeds might be interesting until Publii offers this functionality itself. There is already a feature request on Github and a post in the forum.

Bottom line

Pipes.digital is a powerful tool for manipulating feeds.
Not least CMSs without mature feed functionality such as Publii can benefit from this.
Pipes is open-source and thus available for self-hosting. A hosted offering is available at pipes.digital.

Publii Tips: Distinct Feeds with Pipes.digital by Christian Spaan is marked with CC0 1.0