|Rate (per day)||2.75||237.19|
Generated with BBoe's Subreddit Stats
|Rate (per day)||82.22||1245.55|
Generated with BBoe's Subreddit Stats
For initial block sync, [Bitcoin] sort of works [like BitTorrent] already.~ jtoomim
You download a different block from each peer. That's fine.
However, a mechanism does not currently exist for downloading a portion of each [new] block from a different peer.
That's what I want to add.
[bitcoin-dev] BlockTorrent: Torrent-style new-block propagation on Merkle treeshttps://np.reddit.com/Bitcoin/comments/484nbx/bitcoindev_blocktorrent_torrentstyle_newblock/
As I understand it, the current block propagation algorithm is this:where:
(If I'm missing anything, please let me know.)
- A node mines a block.
- It notifies its peers that it has a new block with an inv. Typical nodes have 8 peers.
- The peers respond that they have not seen it, and request the block with getdata [hash].
- The node sends out the block in parallel to all 8 peers simultaneously. If the node's upstream bandwidth is limiting, then all peers will receive most of the block before any peer receives all of the block. The block is sent out as the small header followed by a list of transactions.
- Once a peer completes the download, it verifies the block, then enters step 2.
The main problem with this algorithm is that it requires a peer to have the full block before it does any uploading to other peers in the p2p mesh. This slows down block propagation to:
O( p • log_p(n) )
A Bittorrent-inspired algorithm might be something like this:
It's like the Napster era of file-sharing. We can do much better than this.
- n is the number of peers in the mesh,
- p is the number of peers transmitted to simultaneously.
Bittorrent can be an example for us.
Bittorrent splits the file to be shared into a bunch of chunks, and hashes each chunk.
Downloaders (leeches) grab the list of hashes, then start requesting their peers for the chunks out-of-order.
As each leech completes a chunk and verifies it against the hash, it begins to share those chunks with other leeches.
Total propagation time for large files can be approximately equal to the transmission time for an FTP upload.
Sometimes it's significantly slower, but often it's actually faster due to less bottlenecking on a single connection and better resistance to packet/connection loss.
(This could be relevant for crossing the Chinese border, since the Great Firewall tends to produce random packet loss, especially on encrypted connections.)
Bitcoin uses a data structure for transactions with hashes built-in. We can use that in lieu of Bittorrent's file chunks.
Features and benefits
- (Optional steps to build a Merkle cache; described later)
- A seed node mines a block.
- It notifies its peers that it has a new block with an extended version of inv.
- The leech peers request the block header.
- The seed sends the block header. The leech code path splits into two.
- (a) The leeches verify the block header, including the PoW. If the header is valid,
- (a) They notify their peers that they have a header for an unverified new block with an extended version of inv, looping back to 2. above. If it is invalid, they abort thread (b).
- (b) The leeches request the Nth row (from the root) of the transaction Merkle tree, where N might typically be between 2 and 10. That corresponds to about 1/4th to 1/1024th of the transactions. The leeches also request a bitfield indicating which of the Merkle nodes the seed has leaves for. The seed supplies this (0xFFFF...).
- (b) The leeches calculate all parent node hashes in the Merkle tree, and verify that the root hash is as described in the header.
- The leeches search their Merkle hash cache to see if they have the leaves (transaction hashes and/or transactions) for that node already.
- The leeches send a bitfield request to the node indicating which Merkle nodes they want the leaves for.
- The seed responds by sending leaves (either txn hashes or full transactions, depending on benchmark results) to the leeches in whatever order it decides is optimal for the network.
- The leeches verify that the leaves hash into the ancestor node hashes that they already have.
- The leeches begin sharing leaves with each other.
- If the leaves are txn hashes, they check their cache for the actual transactions. If they are missing it, they request the txns with a getdata, or all of the txns they're missing (as a list) with a few batch getdatas.
The main feature of this algorithm is that a leech will begin to upload chunks of data as soon as it gets them and confirms both PoW and hash/data integrity instead of waiting for a fully copy with full verification.Inefficient cases, and mitigations
This algorithm is more complicated than the existing algorithm, and won't always be better in performance.
Because more round trip messages are required for negotiating the Merkle tree transfers, it will perform worse in situations where the bandwidth to ping latency ratio is high relative to the blocksize.
Specifically, the minimum per-hop latency will likely be higher.
This might be mitigated by reducing the number of round-trip messages needed to set up the BlockTorrent by using larger and more complex inv-like and getdata-like messages that preemptively send some data (e.g. block headers).
This would trade off latency for bandwidth overhead from larger duplicated inv messages.
Depending on implementation quality, the latency for the smallest block size might be the same between algorithms, or it might be 300% higher for the torrent algorithm.
For small blocks (perhaps < 100 kB), the BlockTorrent algorithm will likely be slightly slower.
Why is [BTCC] limiting their block size to under 750 all of a sudden?https://np.reddit.com/Bitcoin/comments/486o1u/why_is_bttc_limiting_their_block_size_to_unde
Future work: possible further optimizations
For large blocks (e.g. 8 MB over 20 Mbps), I expect the BlockTorrent algo will likely be around an order of magnitude faster in the worst case (adversarial) scenarios, in which none of the block's transactions are in the caches.
One of the big benefits of the BlockTorrent algorithm is that it provides several obvious and straightforward points for bandwidth saving and optimization by caching transactions and reconstructing the transaction order.
A cooperating miner [could] pre-announce Merkle subtrees with some of the transactions they are planning on including in the final block.Leveraging other features from BitTorrent
Other miners who see those subtrees [could] compare the transactions in those subtrees to the transaction sets they are mining with, and can rearrange their block prototypes to use the same subtrees as much as possible.
In the case of public pools supporting the getblocktemplate protocol, it might be possible to build Merkle subtree caches without the pool's help by having one or more nodes just scraping their getblocktemplate results.
Even if some transactions are inserted or deleted, it [might] be possible to guess a lot of the tree based on the previous ordering.
Once a block header and the first few rows of the Merkle tree [had] been published, they [would] propagate through the whole network, at which time full nodes might even be able to guess parts of the tree by searching through their txn and Merkle node/subtree caches.
That might be fun to think about, but probably not effective due to O( n2 ) or worse scaling with transaction count.
Might be able to make it work if the whole network cooperates on it, but there are probably more important things to do.
There are also a few other features of Bittorrent that would be useful here, like:
DDoS started again. Have a nice day, guys :)https://np.reddit.com/Bitcoin_Classic/comments/47zglz/ddos_started_again_have_a_nice_day_guys/d0gj13y
(It might be good if we could get Bram Cohen to help with the implementation.)Using the existing BitTorrent algorithm as-is - versus tailoring a new algorithm optimized for Bitcoin
Another possible option would be to just treat the block as a file and literally Bittorrent it.Related work: IBLT (Invertible Bloom Lookup Tables)
But I think that there should be enough benefits to integrating it with the existing bitcoin p2p connections and also with using bitcoind's transaction caches and Merkle tree caches to make a native implementation worthwhile.
Also, BitTorrent itself was designed to optimize more for bandwidth than for latency, so we will have slightly different goals and tradeoffs during implementation.
Concerns, possible attacks, mitigations, related work
One of the concerns that I initially had about this idea was that it would involve nodes forwarding unverified block data to other nodes.
At first, I thought this might be useful for a rogue miner or node who wanted to quickly waste the whole network's bandwidth.
However, in order to perform this attack, the rogue needs to construct a valid header with a valid PoW, but use a set of transactions that renders the block as a whole invalid in a manner that is difficult to detect without full verification.
However, it will be difficult to design such an attack so that the damage in bandwidth used has a greater value than the 240 exahashes (and 25.1 BTC opportunity cost) associated with creating a valid header.
As I understand it, the O(1) IBLT approach requires that blocks follow strict rules (yet to be fully defined) about the transaction ordering.Remark
If these are not followed, then it turns into sending a list of txn hashes, and separately ensuring that all of the txns in the new block are already in the recipient's mempool.
When mempools are very dissimilar, the IBLT approach performance degrades heavily and performance becomes worse than simply sending the raw block.
This could occur if a node just joined the network, during chain reorgs, or due to malicious selfish miners.
Also, if the mempool has a lot more transactions than are included in the block, the false positive rate for detecting whether a transaction already exists in another node's mempool might get high for otherwise reasonable bucket counts/sizes.
With the BlockTorrent approach, the focus is on transmitting the list of hashes in a manner that propagates as quickly as possible while still allowing methods for reducing the total bandwidth needed.
The BlockTorrent algorithm does not really address how the actual transaction data will be obtained because, once the leech has the list of txn hashes, the standard Bitcoin p2p protocol can supply them in a parallelized and decentralized manner.
Fee revenue for miners is still extremely small compared to the 25 BTC block subsidy. The average amount of fees per block has been around 0.5 BTC for the last 20 blocks, or about 2% of total revenue.~ jtoomim
If block sizes increase, then miners can include a greater number of transactions, each of which has a smaller fee. Whether this results in greater or lesser total fees is purely speculative right now, and depends on a lot of different variables (like demand and the actual block size).
I personally think that miners' best bet for achieving long-term survival is increasing the block size to about 10 MB over 4 years while keeping fee/kB about the same as it is now. I think that it will be easier to increase the volume of transactions 10x than it will be to increase the cost per transaction 10x.
One of the issues with increasing the fee per tx is that doing so would make Bitcoin less attractive as a currency to investors, speculators, and users, which in turn would likely result in a decrease in the exchange rate. This would reduce the real value of the 25 BTC bitcoin block reward, which will have a much larger effect on miner revenue than direct fees, at least for the next 4.5 years. Thus, the idea of keeping block sizes small in order to maximize revenue through fees is likely to backfire.
I'm working on a project called Bitcoin Classic to bring democracy and Satoshi's original vision back to Bitcoin development.~ jtoomim
Jonathan Toomim says the markets are dumb Jonathan Toomim who is the Bitcoin Cash developer talks about the fact whether the Bitcoin Cash halving has been priced in or not. He says that only an idiot would predict the price movements of cryptocurrency and says that the markets are dumb: Mining firm operator Jonathan Toomim went so far as to say he is excited for Zcash because he believes it holds the potential to undermine bitcoin’s value. “Bitcoin had four main value Posted in r/Bitcoin by u/bit_novosti • 20 points and 177 comments Jonathan Toomim - Bitcoin Miner 157 points · 12 days ago. Because it's a threat. level 2. 82 points · 12 days ago. coins but as there can be millions of cryptocurrencies you better hold one that is also usable and thereby creates value that lets it retain its network effect. level 2. Bitcoin miner and developer Jonathan Toomim, FinalHash CEO Marshall Long and former Bitcoin Foundation board member Olivier Janssens launched another Bitcoin implementation this week. Named Bitcoin Classic , the Bitcoin Core fork intends to increase the block-size limit to 2 megabytes later this year.
[index]          
Jonathan Tan 4,177 views. 19:49. Most Realistic Bitcoin Price Prediction for June 2020 Best Bitcoin Price Prediction 2020 Charts 👀 - Duration: 10:15. Altcoin Daily 46,355 views. The Bitcoin Cash difficulty adjustment algorithm (DAA) was activated in a rush on Nov 13, 2017 to fix some extreme mining profitability and hashrate swings that occurred starting August 1st. While ... Watch the BCH Network Stress Test by Jonathan Toomim: https: ... BITCOIN CASH ETP Lists on Leading Swiss Stock Exchange ... Transaction volume is defined by the total amount of value transacted ... BITCOIN TODAY: In this video, I'll go through the Bitcoin news today & I'll make a Bitcoin price analysis. The BTC news & analysis can be inspiration for your own Bitcoin trading or investing, but ... bitcoin value evolution chart bitcoin historical value bitcoin history value bitcoin historical volatility index ... bitcoin evolution jonathan ross bitcoin evolution is it real