XMPP PubSub

[sjehuda]

Add support for publications that are hosted on XMPP PubSub nodes.

I have wrote a Python script which does that task.

AOX2HTML - Atom Over XMPP To HTML

• https://portal.mozz.us/gemini/woodpeckersnest.space/~schapps/projects/aox2html.gmi
• gemini://woodpeckersnest.space/~schapps/projects/aox2html.gmi

See also:

Rivista XJP

• https://schapps.woodpeckersnest.eu/rivista/
• https://video.xmpp-it.net/w/vNqcMooy3pqRAZ8Yb8grr1

[Dillo-dev] Re: XMPP PubSub Plugin

• https://lists.mailman3.com/hyperkitty/list/dillo-dev@mailman3.com/message/AZVAIYSQG24GCNAXSSLN24KOT7HNJUJU/

[snarfed]

Hmm! I'm a fan of XMPP, but I don't know a ton about it. It uses Atom as its data format, at least for the parts that granary would handle, right? granary already supports Atom input and output. Were you hoping for some other functionality?

[sjehuda]

It uses Atom as its data format, at least for the parts that granary would handle, right?

Yes.

See more information about it at

• https://blasta.woodpeckersnest.eu/help/about/xmpp/atomsub
• https://blasta.woodpeckersnest.eu/help/about/xmpp/pubsub

granary already supports Atom input and output. Were you hoping for some other functionality?

The only functionality that I expect of is to deliver XMPP PubSub node items to HTTP, as Granary does.

You can try the AOX2HTML script.

You can also try Rivista XJP, which is really a gateway to XMPP for HTTP Feed Readers.

Yet, I have managed to exted it into a publishing platform with XSLT and ECMAScript (JS), and it has no HTML, but that is for experimenting with XSLT and keeping me motivated to maintain XSLT ans CSS stylesheets to perfect appearance of feeds inside HTML browsers.

[sjehuda]

The code for XMPP has a new location https://git.xmpp-it.net/sch/xmpp-for-granary, as the project Rivista has new functionalities that might not include that code.

Of course, this is not really made for Granary; it is only the XMPP code which I want to incorporate into Granary.

---

I will exploite this comment to mention, again, that I can do it with several hours or days, provided that you would instruct me, as you are familiar with the code of Granary.

[snarfed]

@sjehuda thanks for the nudge here!

I'd love to hear more about your use case. Are you hoping to use Atom-over-XMPP in granary yourself? Or do you know someone who would? Or do you think it's just a good fit here in general?

I don't yet have a need for it myself, so if we did add it to granary, and I had to commit to maintaining it, I'd want to know that someone actively needed and used it in granary specifically, ie with converting to/from other formats, not just as an AoX client on its own.

[sjehuda]

@sjehuda thanks for the nudge here!

Thank you for accepting it.

I'd love to hear more about your use case.

Mostly, to use Granary as a gateway to XMPP with RSS readers that do
not yet support Atom Over XMPP.

Are you hoping to use Atom-over-XMPP in granary yourself?

Yes. I am.

Or do you know someone who would?

Yes. I do.

Or do you think it's just a good fit here in general?

Yes. Indeed. That would furher make more people awaare of XMPP being a
mean for publishing which would consequently increase interest and
activity.

Read the message of Vladimir.

https://lists.sr.ht/~ghost08/photon/%3C20240425100902.779d9cd7@workstation.localdomain%3E?__goaway_challenge=meta-refresh&__goaway_id=a9d5977b990984b0d6ddd2e8b7785969

He was not aware yjat XMPP can be utilized as a publishing platform.

Once seen over Granary, more people be aware of it.

I don't yet have a need for it myself, so if we did add it to
granary, and I had to commit to maintaining it, I'd want to know that
someone actively needed and used it in granary specifically, ie
converting to/from other formats, not just as an AoX client on its
own.

I intend to use it, and I would be glad to maintain it, noth that
maintenance is so necessary.

I would rather utilize Granary to follow activity and announcements
over XMPP than connecting with Movim, even though Movim and also
Libervia provide feeds.

[snarfed]

Interesting! Thanks for the explanation.

granary is a small library, with very few existing users besides me, in my own projects, so if you want to raise awareness of XMPP, adding it to granary won't be very effective marketing. Almost anything else would help more and give you better ROI. 😁

Mostly, to use Granary as a gateway to XMPP with RSS readers that do not yet support Atom Over XMPP.

OK! So if I understand right, you want to read XMPP channels in RSS feed readers? And you want to build and run a gateway/bridge service to make that possible? granary is definitely a good tool for that kind of thing, it's what I used in eg https://bluesky-atom.appspot.com/ and similar tools. Have you started building it yet? I'd love to see your plans and progress so far!

[sjehuda]

Interesting! Thanks for the explanation.

Gladly.

granary is a small library, with very few existing users besides me, in my own projects, so if you want to raise awareness of XMPP, adding it to granary won't be very effective marketing. Almost anything else would help more and give you better ROI. 😁

I could argue with you about it. I have read Granary over telecommunication networks (e.g. IRC, and XMPP) time and time again in the last couple of years.

> Mostly, to use Granary as a gateway to XMPP with RSS readers that > do not yet support Atom Over XMPP. OK! So if I understand right, you want to read XMPP channels in RSS feed readers? And you want to build and run a gateway/bridge service to make that possible? granary is definitely a good tool for that kind of thing, it's what I used in eg https://bluesky-atom.appspot.com/ and similar tools. Have you started building it yet?

Yes. Precisely.

I'd love to see your plans and progress so far!

You can watch the video from minute six. https://video.xmpp-it.net/w/vNqcMooy3pqRAZ8Yb8grr1 Please ignore the "navigation bar", which is not a part of the file; it is created with an ECMAScript in order to enable a navigation interface. * The status of the system: The system which I have already works and does that task. I only need to copy and paste the code into Granary. I would want your instructions. * Tasks that the current system already does: 1. The system requires the module Slixmpp; 2. It connects to the XMPP network as a client, with an XMPP account; 3. Upon receiving data of XMPP PEP/PubSub service (that is, only Jabber ID), it generates OPML Collection file of all Node Names; 4. Upon receiving data of XMPP PEP/PubSub service and Node Name, it generates Atom Syndication Format file; and 5. Upon receiving data of XMPP PEP/PubSub service, Node Name, and Item ID, it generates Atom Syndication Format file only of that item. * Tasks to consider: 1. We might want to set a fixed Node Name, because the Node Name for punlishing is set to "urn:xmpp:microblogging:0" (see XEP-0277 and XEP-0472). 2. Often, presence sharing is required in order to view content from PEP (a Pubsub service of Jabber IDs of type account), so Granary would have to send a request to view content.

[sjehuda]

Sorry I neglected commans. Original:

The system which I have already works and does that task. I only need to copy and paste the code into Granary.

Corrected: The system, which I have, already works and does that task. I only need to copy and paste the code into Granary.

[snarfed]

Thanks for the explanations!

I'd be happy to work with you on adding this to granary. Just so you know though, it will probably be more work than just copying and pasting. granary has a specific interface that all of its formats and protocols implement, specifically from_as1 and to_as1 functions and a Source subclass that implements a number of its abstract methods. They should also generally be a single file per protocol. And for AoX specifically, we already have an atom module, which it shoould reuse as much as possible.

Check out nostr.py, it's a good recent example that does all of that. Also see the reference docs. Feel free to try if you're still interested, I'm happy to answer any questions!

[sjehuda]

I see. I realize it.

What would be the first work for me to do?

P.S.

I did not know that Granary has capabilities other than viewing content.

  def delete(self, id):
    """Deletes a post.

[snarfed]

I'd start with the data format. See how it differs from normal Atom (if at all?), then write tests for the from_as1/to_as1 functions, then implement them.

...and if there's no difference, if it's just plain Atom, you can skip them entirely.

[sjehuda]

Mr. Barrett. Greetings.

I have recently improved the cde f downloading data from XMPP, which is significantky shorter than the previous code.

This is a CGI script of internet software Chawan (chawan.net).

This code only downloads data of "application/pubsub+xml".

Details

#!/usr/bin/env python3
# coding: utf-8
#
# Usage:
#
# printf "xmpp:pubsub.movim.eu?;node=berlin-xmpp-meetup" | python pubsub.py > berlin-xmpp-meetup.pubsub
#
# Example URIs:
#
# xmpp:pubsub.movim.eu?;node=berlin-xmpp-meetup;item=schimon-presents-blasta-a-pubsub-bookmarks-system-and-rivista-a-pubsub-dynamic-iGbveu
# xmpp:pubsub.movim.eu?;node=berlin-xmpp-meetup;item=7363a41d-1146-40b3-ac0f-8ee2559591a3#let-s-meet-up-with-flow
# xmpp:edhelas@movim.eu?;node=urn:xmpp:microblog:0;item=working-on-launching-the-movim-network-qPBzwc
# xmpp:goffi@goffi.org?;node=urn:xmpp:microblog:0;item=libervia-v0-8-la-cecilia-BdQ4
# xmpp:blog.jmp.chat?;node=urn:xmpp:microblog:0;item=september-newsletter-2024#newsletter-esim-adapter-launch

import asyncio
import configparser
import datetime
import os
import select
import sys
from urllib.parse import urlparse
import xml.etree.ElementTree as ET

class Chawan:

    def directives(cha_control, content=None, content_type=None, status_code=None) -> None:
        if status_code:
            sys.stdout.write(f"Status: {status_code}\n")
        sys.stdout.write(f"Cha-Control: {cha_control}\n")
        if content_type:
            sys.stdout.write(f"Content-Type: {content_type}\n")
        if content:
            sys.stdout.write("\n")
            sys.stdout.write(content)
            sys.stdout.write("\n")
        sys.stdout.flush()
        sys.exit()

try:
    from slixmpp import ClientXMPP
    from slixmpp.exceptions import IqError, IqTimeout
    from slixmpp.stanza.iq import Iq
    #from slixmpp.xmlstream import ET
except Exception as e:
    cha_control = f"ConnectionError InternalError {str(e)}"
    content = content_type = status_code = None
    Chawan.directives(cha_control, content, content_type, status_code)

class Xmpp:

    async def pubsub(
        client_xmpp: ClientXMPP, jid_bare: str, node_name=None, item_id=None
        ) -> dict:
        result = {}
        iq = None
        try:
            if jid_bare and node_name and item_id:
                iq = await client_xmpp.plugin["xep_0060"].get_item(
                    jid_bare, node_name, item_id, timeout=5)
            elif jid_bare and node_name:
                iq = await client_xmpp.plugin["xep_0060"].get_items(
                    jid_bare, node_name, timeout=15)
            elif jid_bare:
                iq = await client_xmpp.plugin["xep_0060"].get_nodes(
                    jid_bare, timeout=10)
        except (IqError, IqTimeout) as e:
            error_condition = e.iq["error"]["condition"]
            error_text = e.iq["error"]["text"]
        if iq:
            if "pubsub" in iq:
                #if "items" in iq["pubsub"] and "item" in iq["pubsub"]["items"]:
                if iq["pubsub"]["items"]["item"]["payload"] is not None:
                    result = {"content" : str(iq["pubsub"]),
                              "message" : "Success",
                              "error"   : False}
                else:
                    # This is not an error. It is a state of event of an empty node.
                    result = {"content" : None,
                              "message" : "PubSub node appears to be empty",
                              "error"   : True}
            elif "error" in iq:
                result = {"content" : None,
                          "message" : f"{error_condition}: {error_text}",
                          "error"   : True}
            else:
                result = {"content" : None,
                          "message" : f"Unknown error: {str(iq)}",
                          "error"   : True}
        else:
            result = {"content" : None,
                      "message" : "Unknown error: Stanza IQ is empty",
                      "error"   : True}
        return result

if __name__ == "__main__":
    directory_data = (
        os.environ.get("CHA_DATA_DIR") or
        os.path.join(os.environ.get("HOME"), ".local/share/chawan"))
    credentials = os.path.join(directory_data, "credentials.ini")
    config = configparser.ConfigParser()
    config.read(credentials)
    if ("xmpp" in config and
        "address" in config["xmpp"] and
        "password" in config["xmpp"]):
        address = config["xmpp"]["address"]
        password = config["xmpp"]["password"]
        stdin_input = sys.stdin.read()
        if stdin_input:
            uri = stdin_input
        else:
            uri_scheme = os.environ.get("MAPPED_URI_SCHEME")
            uri_path = os.environ.get("MAPPED_URI_PATH")
            uri_query = os.environ.get("MAPPED_URI_QUERY")
            uri = f"{uri_scheme}:{uri_path}?{uri_query}"
        fragmented_uri = urlparse(uri)
        if fragmented_uri.scheme == "xmpp":
            jid_bare = fragmented_uri.path
            item_id = node_name = None
            for parameter in fragmented_uri.query.split(";"):
                if parameter.startswith("node="):
                    node_name = parameter[5:]
                if parameter.startswith("item="):
                    item_id = parameter[5:]
            if jid_bare or jid_bare and node_name:
                client_xmpp = ClientXMPP(address, password)
                client_xmpp.register_plugin("xep_0060")
                #client_xmpp.register_plugin("xep_0199")
                #client_xmpp.connect()
                # ERROR:slixmpp.features.feature_mechanisms.mechanisms:No appropriate login method.
                loop = asyncio.get_event_loop()
                #rtt = loop.run_until_complete(client_xmpp.plugin["xep_0199"].ping())
                #if rtt:
                try:
                    client_xmpp.connect()
                    #result = asyncio.run(
                    #    Xmpp.pubsub(client_xmpp, jid_bare, node_name, item_id))
                    result = loop.run_until_complete(
                        Xmpp.pubsub(client_xmpp, jid_bare, node_name, item_id))
                    client_xmpp.disconnect(
                        ignore_send_queue=False,
                        reason="Chawan: Receiving PubSub data was completed.",
                        wait=2.0)
                    # TODO if no items in result node (an empty node)
                    if result["error"]:
                        message = result["message"]
                        cha_control = f"ConnectionError ConnectionRefused {message}."
                        content = content_type = status_code = None
                    else:
                        # These directives would restructure the XML document to
                        # assign XML prefix at each element.
                        """
                        xml = ET.fromstring(result["content"])
                        namespaces = {
                            "atom"   : "http://www.w3.org/2005/Atom",
                            "avatar" : "urn:xmpp:avatar:metadata",
                            "pubsub" : "http://jabber.org/protocol/pubsub",
                            "xhtml"  : "http://www.w3.org/1999/xhtml"}
                        for namespace in namespaces:
                            ET.register_namespace(
                                namespace, namespaces[namespace])
                        ET.indent(xml, space="  ", level=0)
                        content = ET.tostring(xml, encoding="unicode")
                        """
                        # This directive will not set an XML prefix to elements.
                        content = result["content"]
                        cha_control = "ControlDone"
                        content_type = "application/pubsub+xml"
                        status_code = "200"
                except Exception as e:
                    message = f"Check whether credentials or XMPP URI are valid; {str(e)}"
                    cha_control = f"ConnectionError ConnectionRefused {message}."
                    content = content_type = status_code = None
            else:
                message = f"Jabber ID or Node Name appear to be missing: {uri}"
                cha_control = f"ConnectionError InvalidURL {message}."
                content = content_type = status_code = None
        else:
            message = f"Invalid XMPP PubSub Query URI: {uri}"
            cha_control = f"ConnectionError InvalidURL {message}."
            content = content_type = status_code = None
    else:
        message = f"XMPP credentials are missing {credentials}"
        cha_control = f"ConnectionError InternalError {message}."
        content = content_type = status_code = None
    Chawan.directives(cha_control, content, content_type, status_code)

I created The Focus to handle Atomsub documents of "application/pubsub+xml".

#1066