Difference between revisions of "Kredits:IPFS"

From Kosmos Wiki
Jump to navigation Jump to search
 
(8 intermediate revisions by the same user not shown)
Line 3: Line 3:
 
It is content-addressable, meaning a filename is just a hash of its content. Thereby, identical content is never duplicated, and the authenticity of the content at a certain name or address can easily be verified by hashing it again after downloading it from the p2p network.
 
It is content-addressable, meaning a filename is just a hash of its content. Thereby, identical content is never duplicated, and the authenticity of the content at a certain name or address can easily be verified by hashing it again after downloading it from the p2p network.
  
We use IPFS to store Kredits data, like for example contributor profiles and contribution details. Check out these two documents for example:
+
We use IPFS to store Kredits data, like contributor profiles and contribution details. Check out these two documents for example:
  
 
* Contributor profile data: [https://ipfs.kosmos.org/ipfs/QmdjFBqJ6Cq9z1Z1mqNPB9LuScaCwUj7tCRwgKu4fEubwp QmdjFBqJ6Cq9z1Z1mqNPB9LuScaCwUj7tCRwgKu4fEubwp]
 
* Contributor profile data: [https://ipfs.kosmos.org/ipfs/QmdjFBqJ6Cq9z1Z1mqNPB9LuScaCwUj7tCRwgKu4fEubwp QmdjFBqJ6Cq9z1Z1mqNPB9LuScaCwUj7tCRwgKu4fEubwp]
Line 11: Line 11:
  
 
== Kosmos IPFS setup ==
 
== Kosmos IPFS setup ==
 +
 +
We run a few IPFS nodes, which are linked together to effectively form a cluster, in which kredits data can be accessed very quickly. This is how we mirror our data in a fast and reliable way:
 +
 +
* Due to all kredits data being public by design, instead of setting up a private cluster, our IPFS nodes simply run [https://gitea.kosmos.org/kosmos/kredits-ipfs-pinner/ a program] which listens to new IPFS hashes being published by the smart contracts on the RSK blockchain. Whenever they see a new document being published, they automatically pin it locally (meaning it is downloaded and marked to be saved indefinitely on that node).
 +
* Our smaller nodes only connect to a few other p2p nodes in order to save memory and CPU resources. However, they are connected to at least one other Kosmos node at all times.
 +
* Our largest node, which is also running the public gateway at [https://ipfs.kosmos.org/ipfs/ ipfs.kosmos.org], is connected to a few hundred other IPFS nodes, so that anyone else can access kredits data reasonably fast.
 +
* Some developers and users also run the [https://gitea.kosmos.org/kosmos/kredits-ipfs-pinner/ Kredits IPFS Pinner] on their own machines. They may run it once every now and then to simply back up the data, or they may also run it on an always-on server of theirs, in order to mirror all kredits data live.
 +
 +
== Peering with Kosmos IPFS nodes ==
 +
 +
As a user of Kredits, or someone who wants to support our project by mirroring the data on your node, we recommend that you configure your node to peer with one of our nodes. This means your node will automatically connect to ours, and it will try to keep that p2p connection alive from then on.
 +
 +
=== go-ipfs ===
 +
 +
In the official reference client, [https://github.com/ipfs/go-ipfs go-ipfs], Peering is supported since version 0.6.0. The relevant configuration section is <code>[https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#peeringpeers Peering.Peers]</code>.
 +
 +
==== List of current peers====
 +
 +
<nowiki>[
 +
  {
 +
    "ID": "QmRw21aC3TroRVdZhrE2Qh4W6PBA67kbE8p3fNfsVcfW8D",
 +
    "Addrs": ["/dns4/draco.kosmos.org/tcp/4001"]
 +
  }
 +
]</nowiki>
 +
 +
==== CLI command for applying the peer config ====
 +
 +
<nowiki>ipfs config --json Peering.Peers '[{ "ID": "QmbqZCZ2RzVr4r1UEdFak6ra76kHxtGmfV9r3e1Ev6Tj5D", "Addrs": ["/dns4/barnard.kosmos.org/tcp/4001"] }]'</nowiki>

Latest revision as of 08:17, 10 September 2023

IPFS is a public, decentralized, p2p file system.

It is content-addressable, meaning a filename is just a hash of its content. Thereby, identical content is never duplicated, and the authenticity of the content at a certain name or address can easily be verified by hashing it again after downloading it from the p2p network.

We use IPFS to store Kredits data, like contributor profiles and contribution details. Check out these two documents for example:

(Or you could fetch the same ones from a different node: contributor example, contribution example)

Kosmos IPFS setup

We run a few IPFS nodes, which are linked together to effectively form a cluster, in which kredits data can be accessed very quickly. This is how we mirror our data in a fast and reliable way:

  • Due to all kredits data being public by design, instead of setting up a private cluster, our IPFS nodes simply run a program which listens to new IPFS hashes being published by the smart contracts on the RSK blockchain. Whenever they see a new document being published, they automatically pin it locally (meaning it is downloaded and marked to be saved indefinitely on that node).
  • Our smaller nodes only connect to a few other p2p nodes in order to save memory and CPU resources. However, they are connected to at least one other Kosmos node at all times.
  • Our largest node, which is also running the public gateway at ipfs.kosmos.org, is connected to a few hundred other IPFS nodes, so that anyone else can access kredits data reasonably fast.
  • Some developers and users also run the Kredits IPFS Pinner on their own machines. They may run it once every now and then to simply back up the data, or they may also run it on an always-on server of theirs, in order to mirror all kredits data live.

Peering with Kosmos IPFS nodes

As a user of Kredits, or someone who wants to support our project by mirroring the data on your node, we recommend that you configure your node to peer with one of our nodes. This means your node will automatically connect to ours, and it will try to keep that p2p connection alive from then on.

go-ipfs

In the official reference client, go-ipfs, Peering is supported since version 0.6.0. The relevant configuration section is Peering.Peers.

List of current peers

[
  {
    "ID": "QmRw21aC3TroRVdZhrE2Qh4W6PBA67kbE8p3fNfsVcfW8D",
    "Addrs": ["/dns4/draco.kosmos.org/tcp/4001"]
  }
]

CLI command for applying the peer config

ipfs config --json Peering.Peers '[{ "ID": "QmbqZCZ2RzVr4r1UEdFak6ra76kHxtGmfV9r3e1Ev6Tj5D", "Addrs": ["/dns4/barnard.kosmos.org/tcp/4001"] }]'