AJagucki.log

Martin Bishop recently asked me why he couldn't see my User Tune from the tooltip of my entry in the MUC room occupant list. Naturally, we were both using Psi where PEP info is often displayed in the tooltips of the Contact List entries. I explained to him that some MUC rooms can be configured as anonymous where the JID of the occupants are not exposed. Without the JID of course, you would be hard pressed to figure out a way to determine whose PEP nodes to retrieve, not to mention determine subscriptions and node access models. However, after answering his question I had an interesting idea for an Openfire plugin...

Remember during the stone age when people used IRC? Neither do I, but stay with me. There were many client side scripts that let you type a command and have your currently playing song sent to the channel. In fact, when I used Kopete there was a plugin that did the very same thing, even over Jabber. So, the idea was to provide that same functionality, except server-side instead of client-side and utilizing PEP.

Thus, last night my friends and I came up with an Openfire plugin called PEP in MUC.

Sending one's tune to the room:

Currently you can also send your mood to the room in the same manner. There are some caveats of course. This only works for local users to the server, and by its nature the node access models are not being respected. But since the user themselves are initiating the 'explicit pubsub notification' this latter caveat does not seem like an issue to me.

Grab the plugin for your Openfire server and ... revert back to the stone age?

With Google Summer of Code 2007 coming to an end, project repositories at code.google.com are now setup for each organization and are becoming populated with code samples from participating students. You can download my code sample here, from the XSF project page.

My previous post announcing this project's completion already contained a wrap up blurb at the end. I'll quote it here for dramatic effect.

[...] I want to let it be known that I had an absolute blast coding this summer. Meeting Gaston and being able to collaborate with him in person at the Jive offices was a real highlight of it all. I am looking forward to further collaboration with Gaston as I maintain the PEP implementation in the future. I am also happy to have met everyone in XSF, including Peter Saint-Andre and Kevin Smith. You guys are awesome.

My efforts will not stop here. With new versions of XEP-0163 and XEP-0060 on the way, I have already talked to Gaston about helping to implement these new versions in the future.

Until then, cheers!

Openfire now has a complete implementation of Personal Eventing via Pubsub (XEP-0163). According to Gaston, my work on PEP will be included in the next Openfire release (3.4.0 I believe). For more information on the implementation and pointers to its source code, visit the project's section on my main site.

Below you will find the deliverables that I indicated I would provide in the Google Summer of Code proposal at the beginning of the summer:

• System test plan document
• Architectural and detailed design document
• Implementation disseminated through a SCMS
• Documentation
• Report on final testing and evaluation will be available on code.google.com in the GSoC dashboard.

With the meat of this post out of the way, I want to let it be known that I had an absolute blast coding this summer. Meeting Gaston and being able to collaborate with him in person at the Jive offices was a real highlight of it all. I am looking forward to further collaboration with Gaston as I maintain the PEP implementation in the future. I am also happy to have met everyone in XSF, including Peter Saint-Andre and Kevin Smith. You guys are awesome.

My efforts will not stop here. With new versions of XEP-0163 and XEP-0060 on the way, I have already talked to Gaston about helping to implement these new versions in the future.

Until then, cheers!

I am pleased to announce that the Google Summer of Code project to implement PEP in Openfire is very nearly complete. It is indeed usable at the moment and further below in this post you will find a public test server running the latest revision available for you to play with.

A whole lot of progress was made in the last two weeks, including implementations (along with c2s and s2s verifications) for the following sections:

• Account Owner Publishing
• Contact Notification Filtering
• Generating Notifications
• Sending the Last Published Item
• Recommended Defaults

The only section left to implement and verify is 14.1: 'Cancelling Subscriptions.' The PEP project page on my main site has a table with the current status of the project. That page might also contain links to a beta release with pre-compiled JAR packages available to download in the near future.

Until that time comes, you are welcome to join my friends and I on the public test server by connecting to jabber.esoteriq.net. If you are looking for a Jabber client to test PEP with, I recommend using either the latest version of Psi or Coccinella from their respective versioning repositories. Keep in mind that this is a development test server. Expect restarts, database resets, and even temporary downtime when the server gets updated to the latest revision. Join us in the group chat at pepdev@conference.jabber.esoteriq.net and say hello . Direct bug reports (and/or praises ) to one of the addresses listed in the contact section of my weblog.

Also, thanks to Ben Slote for generously hosting the public test server.

Work on implementing XEP-0163 in Openfire is nearly complete. Its current state is very usable now that PEP service auto-subscribe on presence subscriptions has been included in a revision I committed earlier in the week.

Contact Subscriptions

A subscription to a PEP service's root collection node (and thus all leaf PEP nodes) is generated and processed by the server during two different situations. The first instance can occur if user Foo has a PEP service in memory on the server and user Bar decides to subscribe to Foo's presence. Upon Bar's presence subscription being authorized, a subscription to Foo's root PEP node is automatically created by the server. The second instance occurs when both of our example users already have established presence subscriptions to each other, but neither have a PEP service created yet (since PEP services are virtual and are actually created dynamically in Openfire as the need arises for efficiency purposes). Once either user decides to publish some personal eventing data and a PEP service is created on the server, anyone who already has a presence subscription to the owner of the new PEP service automatically gets a subscription created for the owner's root PEP node as well.

With the Contact Subscription section of XEP-0163 now implemented, testing with PEP supporting clients like Coccinella and Psi is much more viable (before this point, raw packets needed to be created and sent manually for PEP service/node subscriptions). Below are screenshots taken during my latest tests, including tests for publishing User Geolocation and User Activity data to Openfire using Coccinella. There is also a shot of my friend buchan (aka Jared Hendry) publishing his User Tune data taken from iTunes using Psi.

Next week I will be implementing the last couple sections of the XEP, starting with Contact Notification Filtering.

Client Tests

Publishing geolocation using Coccinella:

Viewing geolocation in Coccinella:

Viewing geolocation in Psi:

Publishing user activity using Coccinella:

Viewing user activity in Coccinella:

Viewing buchan's tune in Psi:

Over the last few days I finished implementing Contact Service Discovery. Doing so was not as straightforward as I thought it might be. I once again needed to extend Openfire's API with the introduction of a new class I called UserItemsProvider. I then needed to modify IQDiscoItemsHandler so that it would accumulate items from all implementers of UserItemsProvider anytime a disco#items query was sent to a user's JID. The default discoverable items for a user include every resource they can be addressed with (if the disco#items query came from someone with a bidirectional subscription to the addressee's presence). These default items were hardcoded into IQDiscoItemsHandler, so I refactored it some to make use of the new UserItemsProvider interface. Ultimately, this allowed me to make IQPEPHandler an implementor of UserItemsProvider as well, enabling me to provide PEP nodes as items while respecting the access models in place for each node.

With all of this done, I went ahead and created the same Shakespearean dummy users as those in the examples for the Contact Service Discovery section of the XEP. Everything worked out as planned, with the access models being respected for each disco#items query. Check here for a complete log of the XML console during my tests.

However, this was only after I discovered and corrected what I believe to be typos in XEP-0163 in which access models are set within fields surrounding <value> elements with superfluous <option> elements. After checking the mailing list I discovered that others had encountered this as well. As this thread explains, everything works as expected when the seemingly needless <option> elements are removed in the example node creation packets. As Peter Saint-Andre has told me before, there is a newer version of XEP-0163 being finalized which does not suffer these same typos. I will bring this little issue up during our next GSoC meeting in jdev@conference.jabber.org just to be sure I and those from the mailing list correctly assumed the example packets to contain typos. Since Openfire's node access model configuration code works as expected without the <option> elements, I agree with those from the mailing list that it was probably a typo.

With another section from XEP-0163 implemented, my goal for next week is to attack the next section: Contact Subscription. Currently, PEP nodes can be subscribed to with explicit pubsub subscription packets, but I will need to implement the auto-subscribe with presence functionality as well.

When Contact Subscription and Contact Notification Filtering are finished, I will be very close to done with the complete implementation of XEP-0163. Any extra time left in the summer I will likely spend implementing the new versions of XEP-0163 and XEP-0060. Why would I stop when I am having this much fun?

Last week's refactoring of the PubSub framework underwent a more thorough testing and verification for compatibility with PEP. With the changes working as planned, much progress was made towards completion of the implementation of the actual protocol specification (as opposed to implementation of the architectural design of PEP in Openfire).

At the beginning of this week, the PEP implementation was at a stage where I could make use of Psi's preliminary PEP support to test with Openfire. When a user publishes his/her Mood in Psi, and the node they are publishing to does not already exist, their PEP service must automatically create the node. After I setup this bit of logic in my IQPEPHandler, I was able to give Psi's mood support a try. Below are screenshots of the trial along with Psi's XML console output.

Publishing mood with PEP:

Viewing the node contents in the contact tooltip:

XML Console Output:

<iq type="set" id="aabda" >
<pubsub xmlns="http://jabber.org/protocol/pubsub">
<publish node="http://jabber.org/protocol/mood" >
<item id="current" >
<mood xmlns="http://jabber.org/protocol/mood">
<flirtatious/>
<text>Denise is near...</text>
</mood>
</item>
</publish>
<configure>
<x>
<field type="hidden" var="FORM_TYPE" >
<value>http://jabber.org/protocol/pubsub#node_config</value>
</field>
<field var="pubsub#access_model" >
<value>presence</value>
</field>
</x>
</configure>
</pubsub>
</iq>


<iq type="result" id="aabda" to="ajagucki@127.0.0.1/Psi" />


<message from="ajagucki@127.0.0.1" to="ajagucki@127.0.0.1"
         id="http://jabber.org/protocol/mood__ajagucki@127.0.0.1__g1tc1" >
<event xmlns="http://jabber.org/protocol/pubsub#event">
<items node="http://jabber.org/protocol/mood" >
<item id="P1tcu6gwk9RQChT" >
<mood xmlns="http://jabber.org/protocol/mood">
<flirtatious/>
<text>Denise is near...</text>
</mood>
</item>
</items>
</event>
<headers xmlns="http://jabber.org/protocol/shim">
<header name="pubsub#subid">yShj3lEQl1F8kjoI1ag7XYP41pbyNaysqNAOVn76</header>
</headers>
</message>

Below is a table that gives an overview of which sections in the specification have been implemented and verified for correctness. One should keep in mind that some of the work that appears to be incomplete may already be implemented in Openfire's PubSub framework, requiring only minor tweaks to be considered implemented for PEP.

If you wish to follow the progress even more closely, you can checkout the latest revision from my PEP branch in Openfire's svn repository (thanks Gaston ):

svn co http://svn.igniterealtime.org/svn/repos/openfire/branches/pep

Refactoring PubSubEngine

Between now and my last post I have spent most of my time improving upon the (not so efficient) design for PEP that was in place. As it was previously, each user's PEPService had its own PubSubEngine to handle the actual underlying PubSub related work involved with PEP. Consider the memory overhead incurred if there are 100,000 users each with their own PubSubEngine that is performing the same exact PubSub functions. Clearly this is a waste of memory. However, the PubSubEngine was designed in a manner so that it was deeply intertwined with a single PubSubService instance which was one of its member variables.

In one of my previous meetings with Gaston we looked at the feasibility of abstracting out the PubSubService from the PubSubEngine in an effort to allow every user's PEPService to work with a single PubSubEngine from within the IQPEPHandler. Unfortunately, it was not a very trivial change to make. There were a number of class variables in PubSubEngine whose constructors required a PubSubService. A lot of code was modified or moved elsewhere from PubSubEngine. More than I expected in fact. Of course, the PubSubService interface along with its implementors (PubSubModule) needed to be modified as well to accommodate the changes made in this major refactor.

As arduous as it was, I have completed this refactoring and Openfire happily compiles and runs. It would have been nice to have unit tests available for the PubSub framework in place already, but that is not the case. If there are any bugs introduced from the refactoring I will take care of them as they are encountered with my own tests as I continue implementation of XEP-0163. Although my task may be more difficult as a result, the savings in memory overhead on large deployments will be well worth the effort. Gaston said this refactoring would be the hardest part of the project, so it is a relief to have completed it. The implementation from here on out should be fairly trivial.

Goals

My next few goals are as follows:

• Make PEPService implement the Cacheable and Externalizable interfaces for even more efficiency.
• Create PEPServices dynamically when a user attempts to utilize PEP. This ensures we only allocate PEP resources for those users interested (instead of preallocating resources for every user).
• Make adding 'identity' elements to the server's disco#info result dynamic in the same manner that is done for features.
• Proceed with implementation.

In my previous post I mentioned several Java classes in Openfire's codebase that would be useful to digest before I started my design phase for the PEP implementation. In this update I will go into more detail about these and several other classes as I describe the current design and initial implementation that I have been working on throughout the week.

Before I do so, here is a more concise version of this post for you 'tl;dr' folks out there: Initial design phase is complete and implementation has started on the PEP skeleton classes that are in place. Things are proceeding as planned and there are no major hiccups.

Now if that does not cut it for you, read on.

The design of PEP in Openfire is primarily concerned with the flow of IQ packets through the server. Openfire uses its IQRouter class to route particular packets off to IQHandlers depending on the namespace of the handler's IQHandlerInfo. I have written an IQHandler called IQPEPHandler which handles IQ packets with PubSub related namespaces. With the ability to intercept PubSub related packets I can determine if the packet is meant for Personal Eventing and act accordingly.

The IQPEPHandler class also has a map of users to PEPServices. PEPService implements the PubSubService interface and allows the server to interact with each user's PEP nodes using the PubSubEngine.

Here is a bit of eyecandy showing Psi's PEP related menu items enabled and a PEP node being created from the XML console:

With this skeleton in place I will be continuing implementation at a nice pace. Gaston will be creating my own svn branch soon and once that happens I will import my code into that repository and continue my work in public.

Yesterday I took a trip downtown to the Jive Software offices here in Portland to meet my mentor, Gaston. After we met up and grabbed some tea from their break room, we made our way to his work area and started discussing exactly how PEP could fit into Openfire's codebase. I took note of several key classes that I may need to either modify or utilize in some manner. I will list them below for those curious:

• IQRouter
• IQDiscoInfoHandler
• IQDiscoItemsHandler
• PubSubModule
• PubSubEngine

When I begin my design and implementation phases I will go into detail about these classes and explain why they are note worthy.

I had a good time with Gaston at the Jive HQ and I left motivated. We both agreed to make our meeting a weekly recurrence.

Today I spent some time writing up a system test document. Here is an excerpt from its introductory paragraph:

The purpose of this document is to provide an overview of the system tests to be performed when implementing the Personal Eventing Protocol (PEP) in the Openfire Jabber server. These tests are designed to ensure all of the use case examples outlined in XEP-0163 (an XMPP extension protocol document) are accounted for. As of the time this document has been typed, there are very few (if not only one) Jabber client(s) that support PEP. Psi is one such Jabber client that has included early PEP support in its version 0.11 release candidate. This version of Psi is able to discover Jabber servers with PEP support and includes PEP functionality such as publishing Mood, Tune, and Avatar nodes. Thus, Psi will be the client used for testing where possible. In addition we will be observing the raw XML packets flowing from Psi's XML console and ensuring correct behavior with respect to XEP-0163.

After writing up a list of system tests I had a top-down view of the project and it gave me ideas about things that will need to be addressed during my design phase. Beginning the design is on my agenda for tomorrow.

Also, for those curious about Google's surprise for GSoC students this year, here it is. Signed by the author with "Happy Hacking." I thought it was a fitting gift.