P2Pcollab

protocols for local-first peer-to-peer collaboration

About

P2Pcollab is a collaborative effort towards creating decentralized, asynchronous protocols for local-first peer-to-peer collaboration.

We develop decentralized internet protocols and tools that allow us ownership and control over our data and digital identities, and enable us to collaborate with our peers and facilitate content publishing, access and discovery, based on open protocols and open source software, without centralized censorship or opaque algorithmic bias.

We aim to shift the paradigm from centralized services with user lock-in to open, decentralized protocols with data ownership, pushing data to edge networks where we can locally access, search, discover, and collaborate, even offline.

We realize this through the research and development of local-first network protocols and data models, and their implementation as composable and reusable libraries and applications.

Design principles

We design networks & systems that empower & respect users, and ensure sustainability of hardware, software, and human resources.

The following principles guide us to achieve this.

Local first: network & data locality without dependency on always-on global networks & services.
Identity & data ownership: Full control over user data and identities
Privacy: Respect user privacy and minimize the amount of data and metadata exposed about users
End-to-end security: Only the intended recipients should be able to read data stored and transmitted in the network.
Self-*: Self-organization, self-optimization, self-repair of networks and systems.
Resiliency: Resilient networks & systems that can recover from network partitions and system failures.
Minimalism: Minimize software dependencies and hardware resources to reduce complexity
Composability: Design systems with composable and reusable components.

Design overview

Network architecture

The network architecture follows a two-tier peer-to-peer model that consists of global core network and local edge networks. Within each network the system is further organized as independent suboverlays for group communication and a common overlay for peer discovery and clustering to find suboverlay members.

Topic-based publish/subscribe

Topic-based publish-subscribe is a messaging pattern where publishers publish content to topics, which subscribers can subscribe to and receive published messages. This messaging pattern is the basis of various publishing, group communication and database replication schemes, and serves as a basic building block for our system.

The current internet stack is based on a centralized, host-based, one-to-one communication pattern, while most of our communications involve interaction within groups. IP multicast has been proposed as a group communication primitive, but it is not generally available to end-users due to security and scalability limitations.

We propose a relay-free, asynchronous peer-to-peer publish-subscribe protocol suite that provides a scalable, public key-addressed group communication primitive, that keeps user location & subscriptions private by employing brokers that store-and-forward encrypted messages for users, providing an authenticated causal multicast scheme for group communication that serves as a transport layer for secure group communication and data syncronization protocols.

It enables direct communication of mobile nodes on edge networks and also offers asynchronous communication with remote nodes through the internet via store-and-forward brokers that form a core P2P network and keep encrypted messages for mobile nodes while they are offline.

A broker acts on behalf of a user in the P2P core network. It collects subscriptions and facilitates publishing, search, and discovery of content. A user may use multiple brokers for redundancy or for different identities. Brokers can be run by users, communities, or commercial providers.

The protocol suite consists of peer sampling, subscription-based clustering, and reliable, authenticated causal multicast protocols.

Read more about the protocol design for further details.

Mergeable Data Repositories

Conflict-free replicated data types (CRDT) enable asynchronous, conflict-free collaboration on shared data repositories, and make eventual consistency possible among a set of replicas.

Authenticated CRDT operations disseminated over pub/sub to subscribers form a tamper-proof log, which is stored in mergeable data repositories and replicated to subscribers with access control on the allowed operations.

This enables decentralized collaboration on data repositories without relying on a centralized server for coordination, and allows resource-constrained mobile and IoT devices on edge networks to participate in the network.

Authorization and access control

Authorization is based on public-key cryptography, where the repository owner can grant access rights to members based on their public key. Each operation is signed and encrypted by its author and disseminated to all replicas subscribed to the repository. Before a replica can merge an operation, it needs to verify that its causal dependencies are merged already and that the author is allowed to perform the operation according to the CRDT access control rules defined by the repository owners.

Immutable objects

Next to mutable objects, data repositories also need to store immutable objects using a content-addressed object store that stores encrypted chunked objects in the repository. These objects are referenced from the mutable store.

Decentralized Content Recommendation, Search & Discovery

Gossip-based collaborative filtering protocols enable recommendation, search, and discovery of relevant pieces of content (e.g. news items, blog posts, etc) and data repositories with content to subscribe to, based on users' obfuscated interest profiles.

This allows discovery of relevant information in both implicit networks formed around common interests and in explicit networks based on group membership.

Community & Contact

Funding

The project is supported by NLnet and the NGI0 Discovery fund of the EU's Next Generation Internet initiative.