Monitoring Jio storage design

Monitoring  storage for jio is based on OPML storage. OPML Storage is a collection of rss feed.

Each OPML represent a hosting subscription in SlapOS, and rss feed in opml, which represent an instance, contains the result of instances promisses. The goal of this storage is to group all OPML and rss storage of hosting subscriptions into a single Jio storage.

Storage specification

 The json specification of the storage is:

{
  "type": replicatedopml,
  "local_sub_storage": {
     // specification of the local sub storage where data should be stored (ex: indexeddb storage).
  } 
}

Storage post/remove method

The storage will allow to put or remove OPML description. An OPML decription entry describe the opml storage which should be created to sync opml file, it has the attribute portal_type set to "opml".

An OPML object in the storage should like this:

opml = {
  "url": "http://example.com/opml.xml",
  "login": "username",
  "basic_login": "credential_btoa_hash",
  "portal_type": "opml",
  "active": true
}

active say if this opml should be synced or not.

storage.put(opml.url, opml) // will add a new opml specification
storage.remove(opml.url) // will remove added opml to storage.

Storage repair (sync)

Repair method is used to sync the storage, it fetch all opml then all sub rss and store them into the local_sub_storage. Here is the description of repair method:

1- Get the list of storage to sync
storage_list = storage.allDocs({include_docs: true, query: "(portal_type:'opml') AND (active: true)"})
2- Sync OPML storage
for storage_desc in storage_list
  opml_storage = JIO.createStorage(storage_desc)
  opml_item_list = opml_storage.allDocs({...})

  // save opml_item into storage, opml_item has the portal_type "opml-outline"

  for opml_item in opml_item_list
    storage.put(opml_item)

    rss_spec = {
      url: opml_item.xmlUrl,
      type: rss
    }
    rss_storage = JIO.createStorage(rss_spec)
    rss_item_list = rss_storage.allDocs({...})

    // put each rss item with portal_type "promise"
    for rss_item in rss_item_list
      rss_storage.put(rss_item)

Description of an rss_item into the storage:

{
  "portal_type": "promise",
  "title": "promise title",
  "parent_id": "ID of OPML outline of this rss"
  "status": "OK",
  "active": true,
  "description": "....",
  ... // all others rss item fields
}

Storage allDocs

Items will be fetched in the local_storage by theirs portal_type, to get all promises item the request should be: storage.allDocs({query: 'portal_type: "promise"'}).

Get the list of instances:  storage.allDocs({query: 'portal_type: "opml-outline"'})

Get the list of promises of instance: storage.allDocs({query: ('portal_type: "promise") AND (parent_id: "XXXXX"'}).