Nepomuk – What Comes Next

After a very generous start to my fundraiser (thank you so much for your support) it is time I get into more detail about what you are actually supporting. Originally I wanted to do that by updating nepomuk.kde.org. I will still do that but it will take a little more time than anticipated. Thus, I will simply start with another blog post.

Well then, apart from cleaning out the bug database at bugs.kde.org (this will be a hard one), continuing to support app developers with Nepomuk integration, maintaining the whole Nepomuk stack, Soprano, the Shared-desktop-ontologies, and some smaller Nepomuk-based applications there are some very specific tasks I want to work on in the near future (In this case the near-future roughly spans the next half year).

Semantic Saving and Loading of Documents

Pretty much forever we have managed documents in a very nerdy manner: the way they are stored on the local file system. We navigate physical folders, create complex hierarchies, get lost in them, recreate parts of them, never find our files again, and still keep on doing it.

The vision I have is that we do not think about folders at all any more since for me they are a restriction of the 3-dimensional world that has no place in a computer. A document on the real world can only be archived in a single folder. On the computer there is no such restriction. We can do whatever we want. Thus, the idea is to organize documents closer to the way our brain organizes information: based on context and familiar topics and relations.

This vision, however, is not feasible in the near future. There is simply too much legacy data and too many applications relying on the classical folder structure. Thus, the idea would be a hybrid approach which combines classical local folders with advanced semantic relations and meta-data. This is a development which I already started with fantastic input from the community.

The next steps include finishing the prototype and creating its counterpart, the file open dialog. This will be a very tough one for which I will ask your support again since that works out so great with the save dialog.

Excerpts

A typical use case is bookmarking pages or copying specific parts of a document into some collage of snippets. However, as always we loose the relation to the source. This is were Nepomuk will shine: instead of copying the part of the document we simply define the excerpt (the portion the user is interested in. This can be a section which is marked, it can be a specific position in the document ranging up to its end, or it can be part of an image.) as a resource in Nepomuk which we can annotate like any other resource. This means that we relate it to topics, people, projects, files, other snippets, web pages, comment on it, and so on – all the while we keep the relation to the original document

This allows for nice things like automatic collages (think of selecting all snippets which mention a certain topic or relate to a certain project and were created before some date and merging them all into one view), simpler quoting of things you read before (since the relation to the original document is in tact you have easy access to the details required for the quote – very interesting for academic workers), and a simple listing of all interesting quotes from documents by some person you like (an example query).

Sharing Nepomuk Data – Step 1

Whenever we create information we want to share it with others. Vishesh Handa already started a very ambitious project to support several types of data sharing through a plugin system. What I want to do first is much less but nonetheless interesting: sharing bits of Nepomuk data manually.

This means that you define the information you want to share and then simply export it into a file which you can then send to someone else. They in turn can import this information into their own Nepomuk system. For starters there will be tracking of origin of the data or anything like keeping two ratings at the same time. That is for later.

This is a very simple first step to sharing which should be fairly easy to implement, the GUI being the only actually hard part. The Data Management Service already takes care of export and import for us.

Once this works adding the same to EMail sending or Telepathy communications ill be very simple. In fact the Telepathy-KDE guys (namely Daniele E. Domenichell aka Dr. Danz) have been interesting in that for a long time. (I wish I were with you guys at Cambridge now!)

To this end I will probably finally get to work on Ginkgo, the generic Nepomuk resource management tool developed by Mandriva’s Stephane Lauriere.

For App Developers: Resource Watcher

For the longest time the only way of getting notified of changes in the Nepomuk database were the very generic Soprano signals Model::statementsAdded and Model::statementsRemoved. Checking for specific changes meant to check each statement which was added or removed or doing a pull each time one of those signals was emitted. An ugly and not very fast solution.

With the introduction of the Data Management Service this will finally change. We already have a draft API for the Nepomuk::ResourceWatcher which allows to opt in for change notifications of different kinds: changes on specific resources, new resource of specific types, changes to specific properties.

The initial API is there and partially integrated with the Data Management Service already. However, I would like to add some more nice features like only watching for non-indexed data or excluding changes done by a specific application (useful for an app which does changes itself and does not want to bother with that data). Also integration into the DMS needs to be finished as not all features exposed in the API are supported yet.

The technical aspect: KDE frameworks

With KDE 5.0 kdelibs and kde-runtime will be split into smaller parts to make it simpler for application developers to depend our powerful technologies. This also means a split for Nepomuk. I already started this split but a lot more work needs to be done to make Nepomuk an independent part in the KDE frameworks family.

Part of this also involves getting rid of deprecated legacy API and improving API where we were previously restricted by binary-compatibility issues.

So this is it for now. Reading over it again I get the feeling that it might be too much already – especially since I am fairly certain that new things will pop up all over the place. Nonetheless I will try to stay the course for once. ;)

Thanks again for your support.

Semantic Saving – Next Next Try

After my last post on Semantic Saving I got a lot of responses, most of them very positive and productive – thanks a lot for that. Today I tried to incorporate most of what you suggested into a new mockup. The new mockup contains of two versions: The default small one only shows a file name, a document type (I still think this is very important), and the folder selection (I fear we still need that although I hope that need will go away in the future):

A “More” button allows to expand the dialog into the bigger version. In there I did not change that much: The title has been replaced with a name field since the title of a file is essentially its name, there is no need to confuse users with that distinction. The comment has been moved below the annotations. I agree that it is less important, especially when saving files that contain text.

Maybe we can continue the discussion on this revised mockup before I start implementing something.

GSoC Wrap-up Part 2

Last time I presented the work Adam Kidder did on Nepomuk virtual folders in the GSoC. Today the story continues with the work by Alessandro Sivieri, my second GSoC student.

Whenever we handle files on the computer we need to bother with folder structures and file names. We need to come up with good naming schemes which allow us to find our files. We need to decide several times a day in which folder a file should go – should it go into folder A or B or should I create a subfolder? In the end there is always a little bit of chaos, even with the most structured minds. Alessandro tried a different approach in his project: save and load documents based on meta data and annotations rather than file and folder names.

This is not an easy task but I dare say that he succeeded. Alessandro created two new dialogs for saving and loading documents (we do not talk about files anymore – way too technical). The saving dialog allows to create arbitrary annotations for the document using the Nepomuk annotation plugin system which also brings in Scribo text analysis features. The loading dialog on the other hand uses a fancy filter system to narrow down the list of documents to open.

Saving Documents

We start by looking at the document saving dialog. Our example is KWord from which we want to save a fancy little text document. (No, it is not a test document, I really wrote this, this is real data, I assure you! … Yeah, OK, I admit it, just random words…) Hitting the save button opens up the new smart save dialog as can be seen in the screenshot below.

Smart Saving of a KWord document

The first thing we notice is that there is no filename and no folder selection. Name and folder are selected by Nepomuk. However, we get to give the document a name (it makes things much easier for us later on) and a description (in a future version applications will be able to prefill these fields with some meaningful defaults). But the interesting part is the meta data. The dialog suggests certain possible annotations which we can approve). Below the recently used annotations we have the possiblity to add any annotation we want through the existing Nepomuk annotation system. Last but not least we can give the document a type. This type does not identify the document on a mime-type level but much more real-life oriented. The idea is that users either define their own types based on pimo:Document or use ontologies that provide them. Typical examples include invoices or letters or project descriptions. This way documents are saved on a much higher abstraction level than with the classical file chooser: instead of a text file we save an invoice.

Once we specified the meta data we want to apply to the new document and hit the save button the smart save dialog generates a folder and file name and saves the document. We do not need to care about the location.

(Hint: there are certainly situations in which we want to use the classic file chooser. That is why the smart save dialog allows to switch over to the old ways by the simple click of a button.)

Loading Documents

But if documents are saved in some random folder which we do not know, how do we find them again? Well, that is the real beauty of the new approach. The idea is that you tell the open dialog what you want to open by specifying some details that you remember.

Let us have a look at the smart open dialog as it opens from within Okular.

We see two main views: on the left hand side we see a list of filters and on the right hand side we see a long list of files/documents. This might look overwhelming in the beginning but wait until we specify the first detail about the document we want to open: we tell the dialog that the document has mime type image/png (Yes, in the future this will look less technical) and the file view changes only showing png images.

These are still way too many to search for the one we need, so we give more detail. We remember that we accessed the document sometime this week:

Again the list of files is changed and now after only choosing two filters we are down to seven documents to choose from. Although this would be enough we do one better just to show that the filter system obviously also includes manual annotations such as tags:

And after activating the tag filter we are down to a single document. Nice, isn’t it?

A Few Technical Details

There are a few technical aspects worth mentioning about Alessandro’s work.

First of all: he makes direct use of Adam’s work on the virtual folders. The file list on the right is a simple KDirModel listing a nepomuksearch:/?sparql=… query. I find this very nice as my two students shared knowledge and discussed their work to find good solutions to their problems.

The second thing I find important is the creation of the filter list. The list of filters is created dynamically based on the existing annotations of the files in the current selection. In essence the idea is to only show filters that would actually change the list of available files (as you can see in the last screenshot this does not work 100% yet but we are close).

The GUI is obviously a prototype and we hope that you will give feedback and ideas to improve its usability. As Adam, Alessandro will continue working on KDE and Nepomuk and the smart file dialog will evolve until KDE 4.4.

Try It

To test the smart file dialog you need three things:

1. My kdelibs patch which makes the KFileDialog pluggable. This is actually a very simple one as the file dialog already loads the backend from a separate lib. While you are on it, please review the patch so it can get into KDE 4.4.
2. The Nepomuk-KDE playground module which also contains the smart save dialog. I recommend installing the whole module as the smart save dialog makes use of pretty much every Nepomuk lib available.
3. Tell KFileDialog to load the smartfilemodule instead of the default by adding “file module=smartfilemodule” into the “KFileDialog Settings” group of kdeglobals.

Obviously nepomuk needs to be enabled for it to work. Have fun.