Overview

A subscription feed is a unified format for dynamic data to be made available to the routerconsole.

A user can subscribe to multiple feeds in the routerconsole. Feeds are stored in separate subtrees, and their data is combined/presented appropriately by the routerconsole. All subscribed feeds are considered equal; the only thing that distinguishes the official feed from other feeds is that it is provided by default in router updates.

In addition to the subscribed feeds, there is a special manual feed which stores local changes made by the user. This feed stores data which, pre-feeds, was stored in the router.config file.

Structure of a feed

Feeds are implemented using the Java Preferences API (with a custom backend), which provides a tree structure for storing and retrieving data.

A Preferences subtree can be imported from (or exported to) an XML file, so this is the format in which feeds are presented to the routerconsole. The default implementation for importing Preferences XML has been extended to provide security and verifiability.

The structure of the XML file is important; a feed builder will be provided which can convert between XML and JSON (which can represent the same structure but is much easier to edit by hand).

Feed security

As described in the overview, feeds are a way to make data available to the routerconsole. This data is controlled by a remote party, and while it is the user's decision to subscribe to a feed, there could be potential attack vectors where e.g. a subscribed feed is hijacked by another party. Because allowing arbitrary untrusted data to be imported and used locally is generally a Bad Idea (TM), several restrictions are placed upon the feed contents.

Feeds are restricted to specifying a single Preferences node (and its subtree), and the node must be the child of a defined path. The currently-defined paths are:

  • routerconsole/feeds

(Other Preferences paths may be defined as importable at later stages to expand functionality.)

Additionally, the feed must be signed: the node specified by the feed must have a child node named signature which contains the following signature information for the entire feed subtree:

  • The name of the signer (this is optional but is used for display purposes).
  • The public key of the signer.
  • The digest method to use for converting the subtree into a signable string (currently supports SHA256).
  • The signature method to use (currently supports DSA-SHA1).
  • A timestamp.
  • The value of the signature.

The signature value is obtained from signed data consisting of the other five signature properties concatenated together followed by the digest of the subtree that is to be signed (i.e. signedDate = signerName + signerPubKey + digestMethod + signatureMethod + timestamp + getSubtreeDigest(digestMethod)).

The digest is a recursively-generated, unique String representation of the subtree. The keys at each node are sorted alphabetically, as are the sub-nodes (to ensure a repeatable digest). Note that any signature nodes in the subtree are ignored for the purpose of hash calculation (so no signature nodes are themselves signed by signatures of parent nodes).

Updating a feed

Feeds can have multiple update urls to provide fallback locations for obtaining the feed (as indicated by the presence of the routerconsole/feeds/feedname/feeduris path in the feed layout). Multiple update types are supported by the update mechanism, but only HTTP is currently implemented. In the future, Seedless might potentially be leveraged as a feed url distribution medium.

The update process for a feed via HTTP is as follows:

  • A FeedChecker task is fired for the feed, either by a timed task (to be implemented) or the user.
  • The FeedChecker task gets the HEAD of the feed from the first successful URI, and notifies that a new version is available (with the version number being the lastUpdated time).
  • If the notified version is newer than the current version, the ConsoleUpdateManager fires off a FeedFetcher task for the feed.
  • The FeedFetcher task GETs the feed from the first successful URI.
  • SignedPreferences.importPreferences() is called, which does the following:
    • The feed is imported into a temporary tree.
    • The feed is checked and verified:
      • The feed only specifies a single subtree under a single allowed path.
      • The feed name does not match any restricted/special feed names (currently only manual).
      • A signature exists for the feed subtree.
      • The signature matches the feed subtree hash.
    • If the checks and verifications all pass:
      • If an existing data subtree exists for the feed name, it is deleted.
      • The new data subtree is imported into the main tree.
  • If the feed is successfully imported (i.e. no InvalidPreferencesFormatException is thrown), the FeedFetcher task checks the feed for any updates contained within it, such as signed router updates (none are implemented currently).

Feed layout

Below is the current reference layout for feed data. The majority of the layout is in alpha, and is still being designed.

The currently-implemented paths are:

  • routerconsole/feeds/feedname/apps

These paths are in beta, and are unlikely to have any significant changes.

foo - a fixed node name (specifically referenced in the codebase)
bar - a variable node name (representing multiple possible subtrees)
baz - a preference of the encompassing node

routerconsole/feeds

  • feedname
    Name of the feed - MUST be unique across ALL feeds (this will be enforced on the "Add feed" page)

    • title
      Title of the feed (displayed in the routerconsole)

    • description
      Description of the feed (displayed in the routerconsole)

    • lastupdated
      When the feed was last updated

    • ___$$$___signatureValue
      Signature of the subtree based at the parent of this node.

    • ___$$$___signatureInfo
      Signature information for the subtree based at the parent of this node.

      • signerName
      • signerPubKey
      • digestMethod
        The digest method to use (currently supports SHA256).
      • signatureMethod
        The signature method to use (currently supports DSA-SHA1).
      • timestamp
        This is appended to the signed data prior to signing.
    • trustedKeys
      The trusted signers for the parent of this node.

      • signer
        The name field is not used, but must be unique to each specified signer

        • signerName
        • signerPubKey
    • feeduris
      The urls to search for this feed

      • uritype
        The type of update method the child URIs are for.

        • urlname
          The name field is not used, but must be unique to this feed

          • uri
            Feed uri to pull from

          • priority
            Feed urls to search are ordered by this value

    • apps
      Apps to populate /home

      • categories
        The subtree defines all categories used by this feed (except for the manual feed, as outlined below).

        • favorites
          Categories for the "Eepsites of Interest" section

          • catname
            • title
            • description
            • icon
        • services
          Categories for the "Local Services" section

          • catname
            • title
            • description
            • icon
      • favorites
        Apps to populate the "Eepsites of Interest" section

        • appname
          • title
          • category
            The category the App is in. For pulled feeds, the category must exist in the corresponding categories subtree above. For the manual feed, the category can exist in any feed's categories subtree.

          • description

          • url
          • icon
            If icon contains a remote url, and the image doesn't exist locally, it will be pulled.
      • services
        Apps to populate the "Local Services" section

        • appname
          • title
          • category
          • description
          • url
          • icon
      • search
        Apps to populate the "Search" section

        • appname
          • title
          • url
    • news
      News for this feed

      • entryname
        The name field is not used, but must be unique to this feed

        • title
          Title to display for the entry (will be escaped)

        • date
          News entries are sorted by the date field

        • content
          This field will be escaped. Maybe allow basic formatting?

    • updateurls
      Update urls distributed by this feed

      • signed
        Signed update urls. A feed may only reference a single signed update. Child urls are a list of possible update locations.

        • should the signature be passed in here? Or the signing key?

          • urlname
            The name field is not used, but must be unique to this feed

            • url
              Update url
      • unsigned
        Unsigned update urls. These will manifest on /configupdate in a dropdown box. A user may only select one unsigned update url at a time across all feeds. Alternatively the user can manually enter an unsigned update url.

        • urlname
          The name field is not used, but must be unique to this feed

          • title
            Should these have a title field? So the update title is displayed instead of the update url?

          • url
            Update url

Persistent feed information

The routerconsole/feeds path stores unaltered feed data, which is overwritten each time the feed is updated. Information about feeds that must be persistent is stored in the routerconsole/feedinfo path.

The current layout of the feed subtrees below the routerconsole/feedinfo path is as follows:

  • feedname
    • title
    • lastfetched
    • feedurls
      This is a backup of the feed's feedurls subtree
    • hiddenapps
      • favorites
        • appname
      • services
        • appname
      • search
        • appname

Example feed

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE preferences SYSTEM 'http://java.sun.com/dtd/preferences.dtd'>
<preferences EXTERNAL_XML_VERSION="1.0">
  <root type="user">
    <map />
    <node name="routerconsole">
      <map />
      <node name="feeds">
        <map />
        <node name="official">
          <map>
            <entry key="title" value="I2P Official Console Feed" />
            <entry key="lastupdated" value="2012-10-12" />
            <entry key="signature" value="1234567890asdfghjkl" />
          </map>
          <node name="feedurls">
            <map />
            <node name="mainsite">
              <map>
                <entry key="url" value="http://www.i2p2.i2p.xyz/consolefeed.xml" />
                <entry key="priority" value="10">
              </map>
            </node>
            <node name="clearnetmainsite">
              <map>
                <entry key="url" value="https://www.i2p2.de/consolefeed.xml" />
                <entry key="priority" value="20">
              </map>
            </node>
          </node>
          <node name="apps">
            <map />
            <node name="favorites">
              <map />
              <node name="i2p">
                <map>
                  <entry key="title" value="I2P">
                  <entry key="description" value="Official project website">
                  <entry key="url" value="http://www.i2p2.i2p.xyz/">
                  <entry key="icon" value="/themes/console/images/i2p_32x32.png">
                </map>
              </node>
              <node name="torrents">
                <map>
                  <entry key="title" value="Torrents">
                  <entry key="description" value="Torrent websites">
                  <entry key="icon" value="/themes/console/images/magnet.png">
                  <entry key="category" value="true">
                </map>
                <node name="postman">
                  <map>
                    <entry key="title" value="Postman">
                    <entry key="description" value="Postman's tracker">
                    <entry key="url" value="http://tracker2.postman.i2p.xyz/">
                    <entry key="icon" value="http://tracker2.postman.i2p.xyz/logo.png">
                  </map>
                </node>
              </node>
            </node>
            <node name="services">
              <map />
              <node name="configupdate">
                <map>
                  <entry key="title" value="Update">
                  <entry key="description" value="Manage your router updates">
                  <entry key="url" value="/configupdate">
                  <entry key="icon" value="/themes/console/images/update.png">
                </map>
              </node>
            </node>
            <node name="search">
              <map />
              <node name="epsilon">
                <map>
                  <entry key="title" value="Epsilon search">
                  <entry key="url" value="http://epsilon.i2p.xyz/search?query=%s">
                </map>
              </node>
            </node>
          </node>
          <node name="news">
            <map />
            <node name="0-9-2_released">
              <map>
                <entry key="title" value="0.9.2 released">
                <entry key="date" value="2012-09-21">
                <entry key="content" value="0.9.2 includes extensive low-level changes to improve the performance and efficiency of the router. We have updated our UPnP library, to hopefully make UPnP work for more people. I2PSnark now has DHT support, but it is not yet enabled by default, as we plan to do more testing during the upcoming 0.9.3 development cycle. As usual, there's also lots of bug fixes in this release, so updating is recommended.">
              </map>
            </node>
          </node>
          <node name="updateurls">
            <map />
            <node name="signed">
              <map />
              <node name="mainsite">
                <map>
                  <entry key="url" value="http://www.i2p2.i2p.xyz/_static/i2pupdate.su2">
                </map>
              </node>
              <node name="kytv">
                <map>
                  <entry key="url" value="http://update.killyourtv.i2p.xyz/i2pupdate.su2">
                </map>
              </node>
              <node name="echelon">
                <map>
                  <entry key="url" value="http://echelon.i2p.xyz/i2p/i2pupdate.su2">
                </map>
              </node>
              <node name="postman">
                <map>
                  <entry key="url" value="http://update.postman.i2p.xyz/i2pupdate.su2">
                </map>
              </node>
            </node>
            <node name="unsigned">
              <map />
              <node name="kytv">
                <map>
                  <entry key="title" value="KillYourTV's dev updates (pack200)">
                  <entry key="url" value="http://update.killyourtv.i2p.xyz/mtn/200/i2pupdate.zip">
                </map>
              </node>
            </node>
          </node>
        </node>
      </node>
    </node>
  </root>
</preferences>