Tuesday, January 17, 2023

Confederacy of Dunces: Ignatius Reilly modeled after Samuel Johnson?

I wonder if Ignatius Reilly was modeled more on Samuel Johnson than Don Quixote or Thomas Aquinas. Johnson lived in the 1700's where Ignatius may have felt more at home. Johnson was hopelessly married to London. He was slovenly, pious, and grotesque (identified and unidentified physical maladies, suspected of having Tourette syndrome). His father died when he was young & he married a woman twice his age. He was famous for his literary intellect, wit, brutal honesty, and devotion to religion. The work that made him famous was an English dictionary that was the premier dictionary for 170 years until the Oxford English Dictionary. Here are a few entries where he let his wit & humour influence the work:


Pension.  'An allowance ... generally understood to

     mean pay given to a state hireling for treason to his

     country.' 

Oats.  'A grain which in England is generally

     given to horses, but in Scotland supports the people.'

Excise.  'A hateful tax levied upon commodities, and

     adjudged not by the common judges of property, but wretches

     hired by those to whom excise is paid.'


Relevant excerpts from Boswell's biography on Johnson:


After dinner, our conversation first turned upon Pope. Johnson said, his characters of men were admirably drawn, those of women not so well. He repeated to us, in his forcible melodious manner, the concluding lines of the Dunciad. While he was talking loudly in praise of those lines, one of the company ventured to say, "Too fine for such a poem:—a poem on what?" JOHNSON, (with a disdainful look,) "Why, on dunces. It was worth while being a dunce then. Ah, Sir, hadst thou lived in those days! It is not worth while being a dunce now, when there are no wits."

....

To the utter astonishment of all the passengers ... he defended the Inquisition, and maintained, that 'false doctrine should be checked on its first appearance; that the civil power should unite with the church in punishing those who dared to attack the established religion'

...

'Some people have a foolish way of not minding, or pretending not to mind, what they eat. For my part, I mind my belly very studiously, and very carefully; for I look upon it, that he who does not mind his belly will hardly mind anything else.' ... Yet I have heard him, upon other occasions, talk with great contempt of people who were anxious to gratify their palates ...  nor would he, unless when in very high company, say one word, or even pay the least attention to what was said by others, till he had satisfied his appetite, which was so fierce, and indulged with such intenseness, that while in the act of eating, the veins of his forehead swelled, and generally a strong perspiration was visible. To those whose sensations were delicate, this could not but be disgusting; 

...

He was so much displeased with the performances of a nobleman's French cook, that he exclaimed with vehemence, "I'd throw such a rascal into the river," and he then proceeded to alarm a lady at whose house he was to sup, by the following manifesto of his skill: "I, Madam, who live at a variety of good tables, am a much better judge of cookery, than any person who has a very tolerable cook..."' When invited to dine, even with an intimate friend, he was not pleased if something better than a plain dinner was not prepared for him. I have heard him say on such an occasion, 'This was a good dinner enough, to be sure; but it was not a dinner to ASK a man to.'  On the other hand, he was wont to express, with great glee, his satisfaction when he had been entertained quite to his mind.

...

About this time he was afflicted with a very severe return of the hypochondriack disorder, which was ever lurking about him. He was so ill, as, notwithstanding his remarkable love of company, to be entirely averse to society, the most fatal symptom of that malady. Dr. Adams told me, that as an old friend he was admitted to visit him, and that he found him in a deplorable state, sighing, groaning, talking to himself, and restlessly walking from room to room. He then used this emphatical expression of the misery which he felt: 'I would consent to have a limb amputated to recover my spirits.'

Talking to himself was, indeed, one of his singularities ever since I knew him. I was certain that he was frequently uttering pious ejaculations; for fragments of the Lord's Prayer have been distinctly overheard. 

...

His sincere regard for Francis Barber, his faithful negro servant, made him so desirous of his further improvement, that he now placed him at a school at Bishop Stortford, in Hertfordshire. 

...

at last silenced her by saying, 'My dear Lady, talk no more of this. Nonsense can be defended but by nonsense.'

...

Concerning americans in 1769:

'Sir, they are a race of convicts, and ought to be thankful for any thing we allow them short of hanging.'

...

On the superiority of Irishmen over Scottsmen, of whom he always spoke very ill of in front of his good friend and biographer Boswell, who was Scottish. 

"You are almost the only instance of a Scotchman that I have known, who did not at every other sentence bring in some other Scotchman [of literary importance]. ... The Irish are not in a conspiracy to cheat the world by false representations of the merits of their countrymen. No, Sir; the Irish are a FAIR PEOPLE;—they never speak well of one another. "

...

He thought portrait-painting an improper employment for a woman. 'Publick practice of any art, and staring in men's faces, is very indelicate in a female.' ... "Would not a gentleman be disgraced by having his wife singing publickly for hire? "



Saturday, March 9, 2019

Reverse Nakamoto Consensus

Abstract

POW consensus is problematic due to 51% attacks. Even BTC is potentially vulnerable as it shifts from rewards to fees.[1][2]  Much higher security can be obtained by replacing hashrate with stake-rate in a way that adheres to Nakamoto consensus, preventing POS problems. POW security comes from a high ratio of dedicated to non-dedicated hashrate. Dedication is achieved by capital investment in equipment that isn't useful for other purposes, including other coins. Electrical and depreciation costs reduce security by reducing the amount of speculative capital spent on equipment.[2][3][4] Capital in stake is potentially as dedicated as capital in equipment and there is 50x more capital in stake available for securing the chain.[5] But stake does not produce a hashrate in proportion to the capital. Hashrate proves the capital in equipment was occupied during the vote, preventing the capital from voting twice. A Verifiable Delay Function (VDF) can be used to provide a time denominator to get stake-rate, preventing the need for stakers (as in traditional POS) to have time locks that require registration and centralization. Another difference between equipment and stake capital is that stake capital is on the chain prior to the leader elections (aka lotteries) whereas equipment capital proves its existence during each election. This difference results in POS systems having to deal with grinding attacks that can taint randomness in leader elections. The solution is to reverse Nakamoto consensus by beginning with a nonce that all stake-miners must use and ending with block creation. POW must still be used to create and distribute coins (stake) because the only known proof of unique identity that's necessary in  distributed consensus is proof of cost. The amount of proven cost is proportional to the amount of "identity weight" necessary in decentralized elections and lotteries. Two difficulty algorithms must be used: one for the above reverse Nakamoto POS-VDF that provides consensus on transactions and one for the POW that creates and distributes coin. Interestingly, the POW for coin creation can utilize a rental market and waste electricity, both of which decrease security when POW is used for consensus. Isolating coin creation with self-hashing txns allows solo POW mining, making pools obsolete. Isolating consensus with reverse POS-VDF can copy BTC's deflating emission rate, but it can also enable a truly stable-value coin (without reference to a fiat). Stakers are not paid for securing the consensus because running a node is the only cost and because paying for consensus results in centralization (such as pools, private ASIC manufacturing, and node centralization in POS). "Freedom (aka decentralization) is not free" and "paying voters to vote subverts the vote."

This is a simplified and updated version of my previous VDF-POS-POW article that gives greater detail in some areas. See also POW Consensus as Registered Voters with Digital Signatures for how Nakamoto consensus can be viewed within the framework of classical Byzantine consensus.

Time-occupied equipment in POW & POS-VDF prevents double-voting.

POS is clumsy because it requires registration, time-locks, and placing the stake at risk in order to keep stake from "double-voting" in block-leader elections. This is not a problem in POW because the equipment is automatically time-occupied in elections in a way that makes it costly to double-vote. A VDF may help POS by keeping the stake time-occupied in the same way.

Reversing Nakamoto consensus is needed because stake is already on the chain.

When the VDF is used in an obvious, straightforward way (as in Chia), the random number output by the VDF that elects the leader can be hacked because its seed can be varied without cost (a grinding attack). All POS has this problem that must be dealt with. But randomness without a taintable seed comes natural to POW.  This is because the POS stake (or Chia's space) is on the chain prior to elections whereas POW equipment value is off-chain, proving its existence during each election. The solution is to reverse Nakamoto consensus, beginning with a nonce the staking "miner" can't choose, and ending with block creation.

Capital in POW & POS substitutes for identity to prevent Sybil attacks.

Backing up, the capital required to establish stake (POS) or equipment (POW) substitutes for establishing unique identity when participating in block-leader elections. We need proven unique identities (or substitute it with proof of capital) to prevent Sybil attacks. But proof of capital alone does not prevent that "identity" from voting twice over time in a future or past election. As discussed above, POW handles this automatically, but POS needs a VDF. The stake in this POS-VDF does not need to be put explicitly at risk (as in other POS systems) for the same reason POW equipment is already "at risk" of losing value if there is enough collusion to conduct a 51% attack. But POS-VDF requires orders of magnitude more collusion in terms of capital required (compared to POW) to conduct an attack if the majority of coin holder participate in block-leader elections.

Electrical costs reduce POW consensus security

In the above I only mentioned the capital cost of POW equipment, so I need to address a common misconception that electrical costs are part of POW security. It is well-established in research that electrical and depreciation costs reduce POW security to the extent they reduce capital that could have been spent on mining equipment that will not be able to find alternate sources of revenue. Proof of security in POW comes from a high ratio of dedicated to un-dedicated hashrate, and only mining equipment that can't find alternative sources of revenue can be assumed to be dedicated. If proof of electrical costs were the only source of security, a 51% attack that costs half the reward+fees would be enough to gain 100% of the reward+fees and enable additional profits from double spending. But expensive, electrically-efficient equipment like ASICs that will recoup their costs over many blocks is very expensive to compete with. BTW I've argued with Bob McElrath over this view.
    POW for Generating Coin with Self-hashing TXNs

    This eliminates the need for pools. Coins in this scheme are created by POW using a self-hashing txn that replaces coinbase txns. Block headers have a difficulty/block (aka a maxTarget/target) for consensus and a difficulty/coin (aka a maxTarget/coin/target) for coin creation. A self-hashing txn has a destination address, a block height that is close to the block in which it will be included, the quantity of coin being mined, and a nonce. The miner hashes the txn while changing the nonce until the hash is < Target*coin /maxTarget/requestedCoin. Validators confirm the txn meets this requirement before accepting a block that includes it. The POW for coin creation can and maybe should be the "opposite" of the ideal POW for consensus. It can and maybe should focus on proving electricity (or depreciation) was wasted instead of proving the most capital in non-depreciating equipment. The goal is to minimize and equalize the barriers to entry, not giving an advantage to proprietary equipment. But a problem is that some locations have free electricity. Whatever the case, hopefully a rental market will equalize the barrier to entry.

    Stable without a peg to any fiat

    The above coin generation can be based on a standard exponential decrease like BTC. A part of BTC fluctuations is the result of sudden halvings (it jumps after halvings), so a continuous equation can be used:

    h = height
    T = target solvetime
    HalfLife = HL = 4*365*24*3600; // 4-yr half life in seconds
    TotalCoins = TC = 21E6;
    // Replace T/HL with half life in blocks if you want.
    RewardPerBlock = T*TC*ln(2)/HL*0.5^(h*T/HL)

    However, the protocol could allow the staking POS-VDF winners of blocks to increase or decrease the difficulty per coin in the block after theirs by 0.01%. If the stakers (coin holders) want a stable-valued coin, a stable-valued coin will result. If they attempt to make the value of their holdings increase by making difficulty per coin too hard too quickly, they can't expect new adopters who will have a stable-valued clone to choose from. If they make difficulty too easy, the value of their coin holdings' will drop from inflation and again late adopters will not like the coin. Their best option seems to be to simply seek a store of value, allowing all future newcomers the same cost of entry, automatically adjusting for Moore's law.

    A Simple DAG can be used to get fast confirmations

    Reverse Nakamoto Consensus
    • Normal POW sequence is 
      1. Get random seed from previous block (it's hash)
      2. Create block, out of many options
      3. Suffer a delay
      4.  Determine nonce & length of delay. 
      5. Create random seed for next block (it's hash)
    • vPOW sequence is 
      1. Get random seed from previous block (it's VDF-output) 
      2. Determine nonce ("Seed") & length of delay
      3. Suffer the delay
      4. Create block, out of many options
      5. Create random seed for next block (it's VDF-output)
    VDF definition (Verified Delay Function)
    • Stakers calculate: setup(security parameter, TimeDelay) = pp 
    • 5 GHz servers calculate time-delay function: eval(pp, seed) = a unique y & proof π 
    • Validators: verify(pp, y, seed, π) = True or False
    • I can the y value "VDF-output".
      Reversing Nakamoto Consensus:
      1. Staker hashes a concatenation of an existing UTXO (that is at least 500 blocks in the past) & the previous block's VDF-out to get a Seed. He normalizes a hash of the Seed for a 0 to 1 value to get a RandValue. 
      2. ST = solvetime = 2*RandValue*TargetSolvetime*Difficulty/(UTXO qty)
        Note: this is better than a Poisson by having a flat probability distribution. This allows faster (but smaller) blocks because it prevents an excess of collisions in solutions for fast solve times that the Poisson causes. 
      3. Stakers with fast STs calculate VDF's pp=setup(security parameter,ST) & send pp & Seed to a fast VDF server to cause a delay in calculating eval(pp,Seed) which returns y = VDF-output & π proof
      4. If staker is first, he creates the block. The header includes the UTXO, staker's signature for proof the UTXO is his (requires spending UTXO to a new UTXO?), previous block hash, the seed, VDF-output & π, and merkle root. If he is allowed to select a timestamp instead of simply adding the solvetime to the previous block (which will cause block time to get far away from real time which should not be a problem), a grinding attack becomes possible which I address below.. He hashes the header once to get the block hash.
      5. Notice the private key of the UTXO connects the block to the proof of "work" (sum of difficulties for the POS-VDF), but the block is not part of the VDF randomization seed. This is a practical way of seeing why reverse Nakamoto consensus works and is better than other POS..
      6. Validators confirm validity of all the header's elements. 
      7. If a miner sees two or more different blocks with the same VDF output and correctly signed by the winner, he only works on the first valid block he sees. See "Multiple block attack" below.
      Notes:
      • Precision errors in VDF are corrected by difficulty because stakers assign the timestamps.
      • Exploiting the VDF This is probably the biggest security risk. VDFs are new and if an attacker finds a way to speed it up 100x or 1000x, a fork to fix it will leave the chain prior to the fork vulnerable.
      • Randomness is preserved. Current and previous stakers can't control solvetime or VDF-output. Randomness comes from staker (instead of miner) population and previous VDF-output, so it can't be tainted (subject to grinding).
      • Chain work rule remains sum of difficulties. CAP & Byzantine problems are solved as in POW. The chain with highest sum of difficulties wins because it proves the largest number of stakers were present (fewest network partitions) and double-voting (rewriting history) is not possible w/o going through equal time-cost multiplied by stake.
      • Stake-key re-use of old keys that the attacker has bought from old stakers to do a medium to long range attack (since the staker selling his keys had better have spent his UTXOs) is not possible because the attacker has to still wait on the VDF cycles. 
      Multiple UTXO grinding attack is stopped by 500 block rule.
      An attacker can privately mine duplicates of a block and create many different UTXO's for the same input coin (stake), which can enable him to affect the seed to randomness infuture blocks, increasing his chances of having a fast solvetime. This is alleviated by not allowing recent UTXOs. This is a little like a time-lock, which I claimed could be eliminated by reversing Nakamoto consensus. It is needed to prevent a grinding attack which I said was not possible. The theoretical foundations I gave are not in error because to do the reverse Nakamoto consensus perfectly, the seed for the VDF would have to go all the way back to the POW-based self-hashing transaction that created the coin as opposed to the UTXOs. But that is not practical because it means most generated coins can never be transferred if we want secure consensus. Notice the typical POS staker registration that would cause a large communication overhead and/or centralization is not needed for this "time lock".

      Attacker trying multiple VDF-outputs
      Nodes can compare who has the fastest VDF time before the VDF has to be performed so they might be able to weed out alternatives before they propagate very far. But if an attacker can see many VDF-outputs for slightly longer solvetimes he can try all of them to see which ones will allow him to have a faster solvetime, so that the sum of his solvetime and the previous solvetime that was slightly faster than the other one other miners are building on will be faster than the "main" chain. A 20% miner considering 5 different VDF-output can significantly increase his odds of finding blocks. Chia's solution is for everyone is to consider all the fast alternatives but I have studied this aspect yet.  

      Timestamps are subject to grinding

      By selecting a timestamp between > 10 different values, miners can increase their chances of winning the block after the next block by > 10x (timestamp affects next difficulty which affects the seed after that block).  It's possible to require the timestamp to be set to the previous timestamp plus the VDF calculated solvetime even if VDF solutions are faster or slower than expected. This can be done in a scheme where stakers who win blocks adjust a factor in the block header (by say 0.001%) that keeps the VDF solvetimes close to real time. This is different from difficulty's affect on solvetimes which will respond a lot faster. This works as long as it does not affect the solvetime for the next say 100 blocks, which I'll discuss below. This scheme is necessary if the VDF-POS-POW scheme is to be used with a DAG.  The timestamp grinding solution below requires step-like difficulty changes instead of "continuous" which DAGs need to rank txns to prevent double spending.

      If miners must be allowed to set the timestamp, let's consider the consequences.  Choosing a timestamp does not affect a miner's chances for the 2nd generation (the next block) more than anyone else because it does not change the seed to that block, but it does affect the seed to the block after that (the 3rd). So a miner can run many parallel VDFs for all the timestamps he can assign, and for each of those VDF-outputs he does it again, and then again for the 3 generation. This allows him to choose optimum solvetimes for 3rd and 4th generation (he can see the solvetime for his 4th before having to suffer it).  100 different timestamps could possibly affect the output of a fast-responding difficulty algorithm, giving the attacker 100x higher stake. Doing this for 3 generations would require 100^3 = 1 M parallel VDFs which might be feasible. By this scheme he can't affect the 1st or 2nd generations, but he multiplies the power of his stake by 100 in 3rd generation. If a largish staker gets a lucky fast solvetime in the 1st generation, he can be sure he will also get 3rd and 4th blocks by this method, giving him a 50% chance to get all 4 blocks if he has 25% stake-rate (this is a 2x effect because he does not control 1st or 2nd generation solvetimes). Chia may have a worse situation where the 2nd generation can be affected by an assigned timestamp.

      To address this, Chia is using a 28 hour delay in difficulty in a 4.5 day averaging window. Delaying a response in difficulty like this, as a large percentage of the averaging window, causes catastrophic oscillations unless it's the largest coin for a POW. This occurred in nearly all Cryptonote/Monero clones which had only about a 1.5 hour delay with a 1 day averaging window.  This is not a delay like BTC's two weeks. BTC applies the calculated difficulty immediately after the 2-week calculation, so it's not the kind of delay that causes oscillations.   I informed Chia in Twitter about this. This warning applies if their "farmers" have a profit motive like POW, or if they're not the largest coin for their proof of space, but they are, so they should be safe, but it seems like a better option should be tried. My scheme here has no rewards, so stakers are not motivated to come and go so I could use Chia's method, but I would like to avoid it.

      If miner's can assign timestamps, and Chia's method is not used, BTC's algorithm (without the Zeitgeist and timespanLimit attack holes) can be used and adjusted every 200 blocks. But the difficulty should only be allowed to increment in tranches like +/- 5%, 10%, 15% to reduce the amount of grinding.  Otherwise a big staker could get several blocks at the transition. If difficulty could be set to 100 different options, then a 25% staker could have a 70% chance of getting perhaps 10 blocks. This is done by not choosing the fastest solution, but by choosing a solution that would result in the next 10 blocks being solved quick on average (requiring 100 concurrent VDFs at the beginning and having to re-write the public chain because it will take some time).

      Another way to restrict the grinding is to make the timestamp range that nodes allow (FTL and MTP) as tight as possible.

      Multiple block creation attack stopped
      The other "attack" is a direct result of reverse Nakamoto consensus. The winning block is formed AFTER the winner is determined. In other words, the winner can immediately create hundreds or thousands of valid blocks after winning, and submit them to different parts of the network. The solution is for all miners to simply accept and work on only the first valid block they see for that VDF output. The "mutated" blocks will all have the same VDF output, so miners seeing these block will get different solvetimes by selecting a different block. The chain work rule will naturally sort out any problems. Rejecting all similar valid blocks that have the same VDF output does not work because a duplicate could be delayed and nodes would have to reject the current chain back to that point.

      [1] See equations 7 and 8.
      https://s3.amazonaws.com/tld-documents.llnassets.com/0013000/13402/bis%20report.pdf

      [2] See page 13
      https://faculty.chicagobooth.edu/eric.budish/research/Economic-Limits-Bitcoin-Blockchain.pdf
      Thoughtful PhD reviewers consider it simple and correct:
      https://medium.com/orbs-network/the-economic-limits-of-the-blockchain-847f79b452db
      https://www.aier.org/article/sound-money-project/economic-limits-bitcoin
      https://promarket.org/founder-blockchain-discusses-new-research-inherent-limitations-bitcoin/
      https://twitter.com/jenniferdoleac/status/1011052217980391424

      [3] Even Nick Szabo tweeted about the importance of capital in equipment, and keeping a high ratio of dedicated equipment.
      https://twitter.com/NickSzabo4/status/1092891074555633664

      [4] Alt coins intuitively are aware of this definition of security: they have frequently reduced total hashrate in an attempt to increase the dedicated to non-dedicated hashrate. They try to find a unique POW and try to avoid anything that increases non-dedicated hashrate like ASICs, Nicehash, and botnets. They seem to always regret merged mining.

      [5] It was estimated in the above paper that BTC mining equipment cost up to $2 B in June 2018 when the market cap had dropped to $130 B.


      Saturday, March 2, 2019

      Fixing POW problems with a POW-VDF-POS-DAG scheme

      Introduction

      This is a coin design that combines a lot of ideas to prevent common POW problems and enable it function as a stable value currency (without any reference to fiat) instead of just an asset. It isolates the functionality of POS, VDF, POW, and DAG methods to optimize their individual potential.
      1. Block creation: Hash-rate is replaced with stake-rate to simulate POW mining equipment for higher-security Nakamoto consensus & efficiency. VDFs are used to give a time-denominator for stake-rate. A potentially important discovery is that reversing Nakamoto consensus seems to prevent PoS problems.These 4 tweets summarize it.
      2. Coin creation: Solo mining for coin creation and distribution is by regular POW equipment in self-hashing transactions. This prevents pool centralization. It does not influence block-creation consensus. 
      3. Dynamic coin parameters: Consensus vote winners (block creators) are able to slowly adjust coin parameters (coin emission rate, block size, block time, and fees) up or down by 0.02% per block in block headers, reducing developer power. 
      4. Stable value: I argue that coin holders will initially vote for coin inflation, but settle on a stable-value (without reference to a fiat like other "stable" coins), in accordance with a store of value asset evolving into a stable-valued currency. This replaces arbitrary coin emission schedules with an economic intelligence (a feedback mechanism).
      5. Fast Finality: See the DAG article .("finality" only applies if stake is not concentrated >40%)
      6. High transaction rate:  See the DAG article.(If CPUs can validate 50 TB chains)
      Problems not addressed:
      • Lack of BTC privacy and anonymity
      • POW waste is still used to generate & distribute coin
      Security assumptions:
      • Honest stakers > attacking stakers.
      • VDF works and is not hacked
      • Old stake keys are not sold en masse for an attack.
      • Reverse Nakamoto consensus can overcome a "multiple block" problem.
      • Coins for staking are in wallets on active nodes instead of cold storage. This could be changed.
      How this is not like regular PoS
      • Like POW, there is only 1 message per vote, the block header.
      • There's no stake registration or masternodes
      • There's no time lock on stakes
      • There's no stake-at-risk
      • There's no need for grinding attack protection [correction: difficulty needs to be done carefully to prevent grinding of timestamps ]
      • There's no need for a random beacon
      • There's no subjectivity
      • There's no fake-stake attack
      • There no "rich get richer" problem with stake

      Contents

      • Part 1: Theory & Discussion
        • Overview: how stake value differs from equipment value
        • Importance of time (not waste) in Nakamoto consensus
        • Time in distributed consensus networks
        • The physics of generating randomness from POW & VDF equipment 
        • Future Work: Converting to a DAG
        • Why POW coins lower hashrate to increase security
        • POW security comes from risk of equipment value loss (not waste)
        • Hashrate is ideally low-entropy/second production, not joules/second waste
        • Replacing hash-rate with stake-rate by assigning stakes a time denominator
        • Chia's VDF as the time denominator
        • VDF definition
        • Why stake-rate POW requires reversing Nakamoto consensus
        • Splitting coin creation from consensus prevents pool centralization
        • Do not pay voters (stakers) to vote (stake). Freedom isn't free.
        • Getting stable value by letting stakers be the Fed
      • Part 2: Details of reverse Nakamoto Consensus
        • Overview
        • Details
        • Notes
      • Part 3: Coin Specifics
        • Solo-mining with self-mining txns
        • Stake-controlled parameters in block headers
          • hashes/coin required to get coin
          • Block Size & block time
          • Fees
        • Block Header
        • Self-staking txns

      Part 1: Theory & Discussion

      Overview: How stake capital differs from equipment capital

      Papers have shown (and I'll demonstrate it below) that electrical costs do not affect POW security and even decrease it to the extent it reduces up-front equipment investment. The source of security comes from expected equipment value loss plus social reputation losses (if a miner attacks) being greater than the gains from an attack.  But why can't we replace "equipment value" in POW with "coin value" in PoS? Why does PoS (but not POW) require a lot of complexity due grinding attacks, the need for a random beacon, an increase in centralization, and/or require stakers to register participation? Why are most coins lowering hashrate in order to get higher security? Why are coins trying to avoid NiceHash & ASICs which should make mining & its marketplace more optimal?

      POW equipment is an engine that produces hashes/second but stakes don't. It wastes energy doing so only because mining equipment is not ideally efficient.. The difference between equipment value (a stake) and coin stake is that equipment stake is occupied during the hashing of a block. It can't double-vote at zero cost like coin stake. The equipment engine being "real" is deeply connected to proving the uniqueness of the bits in transactions (blocking double-spends).  Equipment is value that is occupied over time while stake is just value. Value in both POW and PoS is used to solve cyberspace's identity problem, preventing Sybil attacks on consensus votes. Stake has value and uniqueness, but that's not enough to cast a unique vote. Time in the denominator of hashrate is what proves the equipment value did not double-vote without double cost during or after the vote. This article shows how to create a simple stake-rate that can replace hash-rate, avoiding PoS problems and complexity to get better security and less waste than POW.

      POW is great for creating and distributing coins (value), which I'll retain. The equipment may or may not waste electricity. The cost of buying and operating equipment is a relatively consistent amount of waste of value in the real world that puts value on the chain. But waste (and I'll argue even paying voters to vote in consensus) reduces the security of coming to an honest and accurate consensus. So I'll use waste to create value on the chain, but use efficient consensus to transfer it.

      Importance of time (not waste) in Nakamoto consensus

      Here is my background article to show how Nakamoto consensus fits within classical Byzantine fault tolerance, and how any classical consensus mechanism that proves a chain of votes had the route of fewest partitions must similarly use an eventual and probabilistic scheme. 

      Nakamoto consensus ("POW") is amazing in that it uses only 1 message per block (the solved block itself) from a single miner to to show he won the election process, and to announce the difficulty for the next election. A sequence of these shows any newly-joining node which chain has had the fewest network partitions (the largest vote participation).  In discussing distributed networks that need to reach consensus agreement to a fact, I say "vote" or "voter" instead of hash, stake, or node (of a distributed system of voting nodes not blockchain nodes) because it immediately and more precisely conveys the goal of and commonality between these other terms.  Traditionally (in a distributed network of nodes trying to reach consensus), voters need to register, prove their participation in each vote, and communicate back and forth to declare they agree on the consensus. This is an enormous amount of communication. In contrast, everyone in Nakamoto consensus immediately agrees to a block by seeing if the hash solved the puzzle, without registering (it's permissionless). POW replaces the communication complexity with local computing. As I discuss below, POW does not need to waste energy any more than the traditional method. Both require time.  For a given amount of latency, more communication among more nodes requires more time. In POW terminology, a time cost enables a randomization in the election of a block leader with smallish likelihood of colliding with a concurrent winner. Instead of a communication overhead to prove unique identity and presence of nodes (in a traditional distributed network) during that time period, miners prove a hash cost. The hash cost is each hash's claim to unique identity. POW does not require the existence of electricity & depreciation costs. It works optimally on up-front equipment-only costs. This is the same as up-front stake costs except POW has a time cost where the equipment must prove its existence during the vote. Stake in POS can have a cost and the initial stake itself can be created by POW. This proves unique identity, but it has no time cost that proves the identity voted only once during each vote. This inability results in complexity in PoS schemes. They have to bend over backwards to inject a source of randomness to elect a block leader, but a properly functioning election process generates its own randomness.

      Time in distributed network consensus 

      In the next two paragraphs, a single hash (or the smallest unit of a stake) serves as a single "voting node" that's participating in reaching consensus. We can't identify individual "miners" or "stakers", so we treat individual hashes or stakes as "individuals" in the consensus vote.

      Update: 
      CAP theorem & tradeoff triangle combined to show bandwidth is the limitation:
      (C+P+PL) * N * LL  = O
      C & P are bytes per round per voting node to prove Consistency (Correctness) exists on the largest Partition of voting nodes.
      PL = payloand = txs / round / voting node.
      LL = Low Latency = fast finality = Availability = Scalability = rounds/second
      O = Overhead = bytes/sec required
      Scalability trilemma's "security" means C & P are proven. N = number of nodes = number of independent voters with equal voting weight = decentralized. With non-equal weight it has lower decentralization but allows faster LL. 
      In POW, the C and P byte requirements are incredibly low at 1 header per round and needing only a few headers, and N (distinct miners) can be small. Mining is not necessarily decentralized but it does not need to be: the security is achieved by the profit motive of miners not wanting to lose more value in their equipment than they can gain by colluding. POW is able to move the C&P bandwidth requirements off the network to computation.This frees up O for transmitting more txs.

      Older text: 

      In distributed consensus, the CAP theorem says we can't have Consistent data that is immediately Available to every node, and have network Partition tolerance all at the same time.  If each variable can vary 0 to 1 from worst to best, you can think of this theorem as C*A*P = 0.5 so that one of them has to be 0.5 to enable the other two to be 1. This is not a literal mathematical fact but my way of thinking about how the variables are constrained.  So we have to have a time cost (lower A) to get more Consistent data and more Partition tolerance.

      The tradeoff triangle says we can't have fast finality (the C*A in CAP), Low Overhead (fewer bytes/second/transaction), and a large number of Nodes (voters) at the same time. To reveal if this is just a restatement of the CAP theorem I use (C*A)*1/(O/N) = 0.5 where P = 1/(O/N) = the inverse of Overhead per Node. This seems to be a claim that partition tolerance is easier with many voters (Nodes) and less communication (Overhead).  If we require a given level of Consistency, we can use a higher time cost (1/A) to lower fast finality (C*A) in order to get higher attack tolerance (higher P = N/O). Time cost enables POW to maximize this ratio. This might be mathematized another way: C/T = O/N where T=1/A = time. Faster finality equals more communication per node. More completely:

      Another limitation, again related to the CAP theorem, is the scalability trilemma which means we can't have scalability, security, and decentralization. It's a restatement of the tradeoff triangle where scalability means number of possible transactions which means lower communication overhead per transaction when given a limit on network bandwidth (which includes the time cost to validate).  Decentralization simply put means many nodes and security means the finality in "fast finality" can be trusted.

      By having a time-range in which a winner can be found, it's uncommon for two more valid winners to announce themselves at about the same time. The size of the blocks and network latency place a lower limit on the time range a vote in POW needs to occur (to reduce the frequency of "simultaneous" winners).

      Time isn't everything: In the above CAP and tradeoff triangle discussion, there's an assumption that Sybil attacks are not being performed.  To some extent the nodes are assumed to be independent actors (or at least not colluding to attack the consensus results).  The value cost of equipment stake or coin stake addresses the Sybil/identity/uniqueness problem. Time cost prevents the same identity (based on value) from double-voting based on time as opposed to faking identity to double vote.

      Physics of randomness from POW & VDF equipment 

      POS without VDF faces a deep problem of where to get a random number for electing the winner in a consensus vote. Generating the random number is the reason POW equipment and VDF equipment (with POS) is needed.  The number of options (states) needed to elect a winner is limited by physics to equipment efficiency*energy*time. POW has an efficiency loss at converting energy to hashes while VDF has an efficiency loss at ensuring a computation took a number of clock cycles, so it can be a lot more efficient in establishing consensus through random selection of winner. The max number of states (at the quantum level) in the perfect POW or VDF equipment is energy*time/4/h, which is extremely small.

      Decentralized consensus requires equipment that can change state. VDF clicks with time and POW generates a hash. VDF assumes there is a max clock rate which is a source of weakness in the conversion my argument depends on, but it may have a fundamental limit such as 10 GHz possibly radiating too much energy from the line traces in the ICs.  Each hash is a unique identity that isolates the winner in both space (the location of the equipment) and time (which vote it won). The lowest theoretical energy cost per block to elect a winner is joules = 4*h*N/T where N = number of candidates in the election and T = how long the election takes. Presence of attackers does not increase this cost because there is no way to hash faster except to spend more upfront costs to have more equipment that can start with a different nonce. This is a cost which function like a provable unique identity. The election's equipment cost is zero per election if there is a large number of elections for which the equipment is used. So protection against attacks is like a "potential" energy cost (stored, not wasted as heat except for the initial expense and consequent heat) to participate in elections, but the election cost is the "kinetic" (heat, aka randomization) cost of elections whose lower limit is 4*h*N/T. In other words, capital costs are the source of BTC's security, not rolling costs such as wasted electricity. This has been repeatedly shown in research such as this article, but popular twitter pundits do not seem aware of it.

      In both POW and this POS-VDF-POW scheme, POW proves a unique identity. Using POW for identity in consensus allows temporary identities to vote by "off-chain" POW equipment. POS-VDF-POW uses POW to create a stake for the POS which becomes an on-chain identity. POW consensus voters are more like mercenaries than citizens. In a mature and ideal marketplace, which may likely be the case as we shift from rewards to fees, POW mining should be rentable (like a mercenary) which makes 51% attacks easy and profitable on their own in addition to enabling double spending.attacks.

      By using stake derived from the cumulative hashes (aka cumulative difficulty) in the coin's past (as opposed to current POW hashrate (difficulty) for say 6 blocks) as the proof of citizenship in each election, POS-VDF can require much more equipment "energy" (equipment plus electrical expenses) to establish unique identity for voting than POW. This is why it should be a lot easier to attack with 51% hashrate in POW than with 51% total hashes (stake) in POS-VDF, if all holders of the coin ("citizens") are voting (staking). Some day POW should be be rentable.

      BTW, POS without VDF requires time-locking the stakes to function like a VDF. It's inferior because it does not block long-range attacks.

      Converting to a DAG

      See https://github.com/zawy12/difficulty-algorithms/issues/40

      Why coins lower hashrate to increase security

      ASICs, botnets, NiceHash, and Merge Mining are examples of increased hashrate (HR) that usually decrease security.  This is because security is based on:

      (X equipment HR)/(non-X equipment HR) > 50%

      Where X can be described as staked, non-colluding, or dedicated equipment. Coins do things to lower the non-X portion which decreases the total HR but increases security.  To increase the X (to be sure it is "staked"), it might be morally or culturally motivated to not attack, but it's better if the X is at risk of loss in order to prevent 51% of the miners from naturally colluding (which can evolve without communication) to harm the other 49% and users (see next section).

      It's difficult if not impossible to keep the non-X equipment portion low and to keep the X portion from colluding. All coins who do not depend on a central control system can be attacked (for example, the Chinese government could force its miners to harm BTC). If stake-based consensus were possible without its usual drawbacks, it could be much more reliable in keeping the ratio above 50%.

      POW security comes from risk of equipment loss (not waste)

      The "rolling costs" of equipment depreciation and electricity costs are not in the previous equation. They even decrease security by decreasing the reward+fees that miners can spend on equipment. This up-front cost keeps them dedicated, assuming they do not have a more profitable coin to turn to. If equipment depreciation could be minimized (Moore's law bypassed), even more value could be spent on up-front equipment costs.  If large equipment owners attack the coin, and the coin is the largest for that POW, then their equipment value decreases (if the coin loses value due to the attack). This is why Chinese miners do not collude to do 51% attacks on BTC. All this can be summarized in a second equation that determines POW security.

      (current+future equip value lost in attacking) > (attack profits)

      If you do not agree with the above, here are appeals to authority (aka references):  A popular research paper last year and Nick Szabo has said pretty much the same thing.

      Hashes/second is low-entropy/second production, not joules/second

      It's clear you can spend more on equipment to spend less on electricity. Hashes do not theoretically require any energy, except for Bremermann's incredibly low physical limit (Energy per bit change = h/4/second => 1.5 watts to switch 100 quadrillion perfectly efficient transistors at 10 quandrillion Hz). This shows a measurable amount of energy is not a fundamental to equipment operation, but only in manufacturing it, which is a value. Likewise in this new consensus mechanism, a "stake-rate" to replace hash-rate will not require energy, but it does require energy in its creation. Hashing looks for low-entropy solutions, so POW equipment is an engine that produces lower-entropy/second instead of joules/second. Hashing requires  energy only because our equipment is not yet close to the ideal. Stake-rate in this system will also produce lower-entropy/second and it is already at the ideal. Wasted energy puts value on the chain. Low-entropy production produces consensus. BTW, lower entropy increases net available work (energy) in systems at a given temperature, so it may have a value that can be viewed as energy, but I think that's a red herring.

      Replacing hash-rate with stake-rate by assigning stakes a time denominator

      I've shown the perfect POW equipment does not waste energy other than in its creation, so that the hashes/time it produces can be equated to equipment value. I've shown stake value can't replace equipment value in solving the identity problem for voting. But we can look at the hash/time output of equipment and replace hash value with stake value and give the stake a time denominator by forcing it to be occupied (in time) during a vote to prevent double voting. To force stakes to be occupied, we need the VDF function in the next section. We can't simply use a time-lock or tighten the FTL and MTP limits on timestamps. There is a random number seed (and it's modification during each block provided by the indeterminacy of the wide voting population) that needs to pass "through the stake-time marriage" during the vote (during the block-leader selection process).

      Chia's use of VDF is Not Like This

      Chia was started by BitTorrent creator Bram Cohen and has at least $3.3 M in funding.  They are developing and paying $100,000 in competition for optimizing their Verified Delay Functions (VDF).  A VDF requires many sequential steps that can go only as fast as the computer's clock. They are supposed to be the ultimate in non-parallelizable functions, except they also provide a way for validators to prove the "miner" (farmer) expended the time. I was directed to them after tweeting about the importance of time if we want to use stakes, and the VDF was exactly what I was looking for even though I did not know what I was looking for. All I knew is that I needed a time denominator to create stake-rate as opposed to hashrate. I was not even thinking about needing a way to validate a time delay and a random output as proof the delay was suffered. It helped congeal my thoughts and made this article possible, solving POW's 51% attack problems and greatly simplifying POS. But I am doubtful Chia's proof of space is as good as using POS.

        Why stake-rate POW requires reversing Nakamoto consensus

        The only working method of using VDF-delayed stake I could find required reversing the consensus process by starting with a nonce and ending with block creation.  If it's not done this way, grinding attacks and/or contamination of the randomness occurs. From a theoretical standpoint, it had to be done backwards because POW takes time to prove equipment value. Time comes before the proof of value. But in stake-rate POW, the stake's proof of value must come before the time-delay calculation because we have to use the stake's quantity to determine how much time it should be delayed compared to other stakes of different quantity. Surprisingly, this conflict can be resolved by running the consensus backwards. In other words, we can view the consensus as going backwards in time, so that the time delay still comes "before" the proof of value, as in POW. But we still get to use the stake quantity to properly adjust the time delay.

        BTW, doing block creation at the end allows the staker to create many different blocks quickly, but it does not allow a grinding attack because the block hash is not our seed for randomness. We use the output of the VDF as the seed for the next block. As in POW, our random adjustment to the previous seed comes from the population of value-weighted voters (hashers or stakers).

        Splitting coin creation & consensus removes pool centralization

        POW's proof of waste is still used in this scheme to put value on the chain. Splitting them has a tremendous benefit. It enables solo mining via self-mining txns that prevent the need for pools. See Coin Specifics section.

        Do not pay voters to vote.  Freedom isn't free.

        Paying stakers to stake could cause an unwanted concentration of wealth, but there are other reasons to believe they shouldn't be paid, and shouldn't need to be. Cryptography enables security without cost. We don't have to pay for a vault. Similarly, I hope to show  decentralized consensus without cost is possible, other than operating a node.  If coin holders want to keep their value, they better run a node. "Freedom isn't free." Voters should vote. They are more honest if we don't pay them to vote, but we need to try to make it inexpensive to vote.  The ethos in the community should be "run your own node, or the coin's going to be attacked."  Schemes that do not need us to run nodes to cast our votes are schemes that sacrifice freedom (that is, they sacrifice decentralization).

        Full-time staking might be motivated with 2% interest per year to off-set an across-the-board 2% demurrage, but I will not try to pursue that route. It might help prevent large stakers from concentrating wealth.  It would act like an efficient lottery (your odds are even) for small coin holders (a few lucky small holders get a big payout while many unlucky ones lose the 2%).

        Getting stable value by letting stakers be the Fed

        Each block winner can increase or decrease the difficulty that is used for the self-mining txns (see below). This will indirectly determine coin emission rate. Ostensibly coin holders will set difficulty low to begin with to get coin cheaply and increase it later to limit coin inflation. So hopefully there will be a shift from "store of value" to "currency". I believe they will eventually decide on stable value. The emission rate should be more intelligent by letting coin holders dynamically determine it instead of arbitrarily setting it at coin creating.

        Moore's law prevents us from using a constant difficulty to keep stable value. Difficulty increases precisely with price over the short term when coin emission rate is fixed.  But due to the unpredictability of Moore's law and other advances, price would decrease if difficulty is constant for a long time due to inflation. We need an oracle to adjust for Moore's law and software changes. We do not want the centralization of off-chain oracles. Coin holders (stakers) may serve as an ideal decentralized oracle to make the adjustment on the chain as they are forming blocks.

        A cryptocurrency has no inherent long-term value without stable value. A long-term holder (staker) realizing this will vote for stable value. If the majority of stakers try to increase the difficulty too much to get better returns, they will eventually lose value from insufficient users wanting to use it as a currency.

        The coin's white paper and devs may need to only recommend that block winners set the difficulty to track constant value so that it always costs miners 1 dollar in terms of year 2020 to get 1 coin (stable value in terms of 2020 dollars). There may be a desire by purchasers and users for it to track the dollar even as the dollar slowly devalues so that a wider audience immediately has a reference for its value. But there's no mechanism for determining that desire and holders (stakers) would not be willing to respond to it anyway.

        A stable-valued currency occurs when coin quantity remains proportional to GDP/velocity.  For example, assuming constant velocity, if $1 B is spent a week and total coin quantity is $100 B, and if $2 B/week is spent a year later due to a larger GDP, then coin quantity should become $200 B or it will be valued 2x more, which is harmful to it as a currency because that would invalidate contracts (prices and wages) and law that are expressed in terms of the coin. I would like to incorporate this knowledge as part of the difficulty adjustment, assisting (or counteracting) staker decisions. If the txn volume per week per total coin qty is increasing,  I don't have a metric to determine if the increase is from a real GDP increase or if velocity is increasing, so I don't know if coin emission rate should be higher or lower. If I knew, how is the target GDP/velocity chosen? How much should coin emission rate increase to avoid too much lag on the one hand and oscillations on the other?.  The fees on txns should discourage a manipulation of the metric with do-nothing txns. Transactions would be unconscious fee-weighted votes on a  difficulty adjustment. Stakers could revolt by omitting txns, but that would destroy the value of their stakes.

        The minimum unit should be 0.001 coin because 1/10th of a penny is dust and the txns/sec limitation in this technology is severe.  This is the value of a Satoshi at 1 BTC = $100,000.

        Part 2: reverse Nakamoto Consensus

        This enables stake-rate to replace hash-rate. A Verifiable Delay Function (VDF) is used to get the time denominator for stake-rate. Since equipment value (hash-rate) requires time to prove its value during the vote, time comes before the proof of value. But stake value is used to adjust the time delay in stake-rate, so stake value comes before time. This prevents the value from demonstrating to the chain that it did not vote twice.This is remedied by doing the consensus backwards. This prevents problems normally seen in PoS

        This rest of section has been superseded by this article.


          Part 3: Coin Specifics

            POW to generate & distribute coins

            POW waste is still needed to fairly distribute coin. We can use self-hashing txns to enable solo mining (aka self-mining txns).  People could use their own mining equipment or rent NiceHash.  The self-hashing txn consists of a destination addr, a difficulty that determines how many coins will be obtained based on the  difficulty_2 (hashes/coin, see below) that is in the block header, a block number that is 20 blocks into future, and a nonce. The amount of coin he gets is determined by the a difficulty_2 setting in the block header in where his txn is included. Miner hashes the txn until the nonce solves the difficulty that's in the txn.  After he releases it, hopefully a staker will include it in the next 20 blocks. The staker must confirm the nonce is a solution and that the txn was not previously submitted in the past 20 blocks. This allows stakers time to include it while preventing the need to check for the same txn in the distant past. If the stable value idea is not used, a regular difficulty algorithm is used to determine emission rate just like regular POW. Either way, pools are obsolete. Txn fees will prevent too many mining txns, so there is no minimum. Keccak256 may be a good choice of POW. The ideal POW is one which "everyone" has equal access to (not giving an undue advantage to botnets or undisclosed specialized ASICs. A POW with a good ASICs on the open market  would be good.

            Coin holders adjust coin parameters

            A block creator can change the numerically-valued parameters up or down 0.02% from previous block. For example, if a fee is 0.1%, then a single block can increase it to 0.10002%. If 60% of block-winners (stakers) want fees to be 2x higher and 20% stakers vote against them, it will take N = 8665 blocks (30 days with 300 second blocks) to make the change. The equation is:  N = log(2)/log(1.0002)/(0.6-0.2). This is the same as if 40% are voting for the increase while 60% are silent.

            Block headers will have fields under the control of stakers for mining difficulty, block size, block time, fees per byte, and fees per coin in txns.

            Block Header

            • Same as BTC, except no nonce.
            • Consensus-related:
              • a reference to Stake txn (its "address")
              • VDF-value "y" & proof π
              • Signature of block header with stake key
            • Block winner controlled coin parameters
              • Difficulty2 target for self-mining (nBits type)
              • Max block size in 100 kB (3 Bytes)
              • Target solvetime (3 Bytes)    3 bytes needed for voting accuracy
              • txn fee per byte (3 bytes)
              • txn % fee per coin (3 bytes)
            • Total coin quantity mined (7 bytes)
            • Foundation wallet address for fees (34 bytes) ? Modifiable by who?
            • Code URL (probably Github)  ?  Modifiable by who?

            Initial settings in block headers 

            Consensus vote will slowly modify these in every block.
            • difficulty = $0.05/coin 
            • block time = 300
            • max block size = 4 MB
            • % fees = 0
            • fees / byte = 0.001
            • dev or foundation address

              Self-staking transactions?

              Not believed to be necessary, but self-staking txns could validate double-spends and allow two parallel blocks to merge to form a much higher chain work block. Every transaction would supply 1x stake-at-risk. Stake is lost if txn is spent twice. This has faster finality at a cost of only being able to send 1/2 the remaining coin in a txn

              Thursday, February 28, 2019

              Consistent value in various contexts is the source of money's properties

              An ideal money has the same value in all relevant contexts or "dimensions".  The numerous properties ascribed to money are just referring those contexts.

              The purpose of a thing is more fundamental to defining it than its properties. For example, ask a person where the chair is in a picture of a forest and he'll know it's the log or stump, but an A.I. won't be able to find legs or a back. Consider the purposes of money authors have mentioned:
              • Medium of exchange
              • Unit of account
              • Store of value
              Less frequently mentioned:
              • Deferred payment (a unit of debit or credit)
              • Legal tender (e.g. a unit of account in contracts)
              Value is inherent to all of these, and stability in value is obviously also important. If you add "consistent" or "stable" before them it makes sense and sounds idealistic or even redundant.

              Here are 15 properties I was able to find, taking the liberty of adding the word "value":
              • Stable value in time
              • Stable value in different locations
              • Divisible value
              • Fungible value (aka "Uniform")
              • Portable value
              • Durable value
              • Acceptable value (aka "Convenient")
              • Trustworthy value ( aka "Confidence")
              • Liquid value (this is vague and encompasses most of the others)
              "Consistent value in every way" seems to be an accurate summary. I found two properties which are kind of oblique or re-enforce the others..
              • Limited in Supply  (re-enforces stable value and trustworthy value)
              • Long history of acceptable value (re-enforces trustworthy / confidence in value)
              There is another property:
              • Has value in itself
              This might be a circular reference, or it breaks money out of a different circular reference "money has value because we agree it has value". This property is saying it should have value because we can use it for something besides exchange. It refers to something like copper, silver, food or vodka (a unit of exchange when the USSR was falling apart). Coins have had this property off and on. For maybe 2 or 3 decades, the copper in a penny was worth about a penny. Then there are silver and gold coins.  So the trades in these types of money are also barter.

              Barter, energy, and cryptocurrencies
              Continuing on about this final property: it always has taken a lot of energy to get silver and gold. Similarly, POW cryptocoins waste energy to "prove their worth". But the worth in metals is also like stored energy (literally, metals can be burned to get a lot of energy out, but being able to use them saves energy). Especially silver: it's biggest use right now is in solar cells. Buying silver is akin to buying potential energy.  The "inherent" value in a barter-type money is the amount of economic "energy" (possibly literally) it can produce or save, but all the other properties only demand that the "value" is the amount of energy it can control through mutual agreement.  If you could bottle up electrical energy in different quantities that could be easily extracted by anyone and could transfer it over the internet, that would probably be the perfect money.

              Importance of stable value to contracts
              Contracts (including wages and prices) are just an agreement between economic players. In order for an economic system to be intelligent, it seems a constant value is as important as keeping the definition of a kg of wheat constant.

              Currency quantity should track GDP
              If the "real" GDP of the currency being used increases, then the amount of currency in circulation must increase in order to maintain stable value. This is if the GDP is increasing from the economy getting more efficient, or if production increases, or if the currency is being demanded by previously "external" economic actors like the rest of the world increasingly using your currency. GDP increases from simply printing more currency (inflation) has to be subtracted from the "real" GDP.  If the real GDP is trying to grow and the currency is not increased with it, it slows the growth rate by strangling trade. Increasing the amount of currency ahead of time can help the GDP to grow, but if too much currency is produced, inefficient decisions are made with the excess currency, leading to a future reduction in GDP.  For example asset prices can artificially rise while inflation is kept low so it can seem like everything is fine, but this leads to a boom-bust cycle in assets.

              The "real" GDP can be viewed as a net energy that is acquired and used over time. It is used to sustain (maintain) and increase itself (the economy). But the net energy is not necessarily physical joules (or how efficiently they are used, hence "net"). We may place higher value on things that can't be measured with physical energy. For example, we may print more money to increase the apparent GDP (since the money quantity is higher) that actually reduces "real" (joule-based) GDP. An example of this is wanting an even distribution of joule-based wealth more than total joule-based wealth. In other words "efficient" use of the joules may not be a physical conversion efficiency. But I will assume "real" GDP refers to net work energy in joules.

              To keep constant value the quantity of the currency needs to be in proportion to the amount of power (energy per time) the infrastructure can produce, provided the currency's velocity (turnover rate) is constant. So the quantity of money divided by the time it takes the money to "turnover" (its 1/velocity) should remain proportional to the productive power of the infrastructure, which indicates the currency is in units of joules. That is, (money qty)*(velocity) = (net work energy in joules) / (time). But since constant value depends on (money qty)*(velocity) it does not strictly connect money to constant value as in coins with inherent value. The solution is to make money proportional to the infrastructure that creates the GDP. That infrastructure is an engine that has a net work output per time.  It took energy to create the infrastructure, so it's like a potential energy. So money can retain units of joules like the infrastructure and yet be directly connected to a joules/time.

              The amount of currency in circulation should "lead" that power. For example, if a new discovery is going to increase efficiency and needs a large capital investment, an amount of currency needs to be created immediately in proportion to the expected benefits of the discovery and loaned to those who will profit from the discovery.  If the discovery increases real GDP as expected and thereby the loaned (created) money is repaid, the issuing authority (like a government) can spend it without inflation. If the venture fails and it's not repaid, there is inflation. Doing it this way pulls marginally unemployed infrastructure into action and/or causing slight temporary inflation that "steals" relative power from other sectors to get the discovery up and going quickly. Intellectual property, culture, and resource depletion affect the efficiency of the infrastructure's production and the efficiency of its use, so knowing the changes in the "power" for the purpose of increasing or decreasing the currency to keep constant value is not easy. We can make an initial error in estimating the true watts of production for the purpose of determining the amount of coin to issue, but it's OK is we are consistent in that error consistent (initial accuracy can be bad, but long term precision should be good). We only need to know that the amount of coin is staying proportional to the power of production, provided the velocity has not changed.  "Net work energy" is clearly defined in physics but we may not want to turn the net work output of our GDP infrastructure into fun heat energy. Evolution indicates we "want" to create more infrastructure that will capture more energy in the future to build more sustainable infrastructure, more quickly. A currency-issuing authority that guides its market in that direction the best is the one who will have the dominant currency. We might want more fun heat energy, but in the end the infrastructure that seeks to expand itself will dominate, pushing for a currency issuing authority that assists it in controlling assets (including people) to this end, eliminating liabilities (including people) along the way. China's rise and strict control of trade and currency is not an accident. USSR's fall in 1989 was a wake up call that economics is important, causing them to intelligently guide macroeconomics. The square caused the government to fear its people which is the opposite of the U.S. government which acts with ignorant impunity as a result of the wealth that resulted from winning the currency war. We've printed an excess for free foreign labor as fast as the increasing world GDP could absorb it, greatly slowing inflation, but reducing our own infrastructure.

              A lot of currency is created as banks follow rules set out by governments to create it out of thin air using the asset and the credit-worthy borrower's promise to repay as assets in the banks books that offset the thin-air money.

              Economics as an A.I.
              Economic systems economize limited resources with competing (evolving) agents. Part of programming interacting A.I. agents is to create a currency that gives access to CPU time and memory space (I'll assume CPU time is primary concern). The quantity of the currency turnover per time must be proportional to CPU calculations per time. Each calculation requires energy and expansion of the A.I. system would mean gaining access to (creating or stealing) more CPUs (infrastructure). So a perfect parallel can be made between a specific type of A.I. and economics.

              Slow inflation may be practical, violating constant value
              How to increase and decrease the quantity of currency to assist the survival and expansion of the infrastructure is not obvious. It may be necessary to violate constant value. For example, there's a long history of erasing past debts as a way remove the "1%" from having too much power (see Michael Hudson's "The Lost Tradition of Biblical Debt Cancellations").  A 2% annual inflation puts pressure on large holders of the currency to invest the capital in the economy directly or via loans, or lose their value if they don't. 

              Monday, February 18, 2019

              The Problem with Avalanche (BCH & Ava)

              [

              update #3.  Here's my rant in a comment to their Sept 26, 2019 dev meeting

              Avalanche is not a consensus mechanism for two related reasons: it does not quantify the voting population or detect network partitions. Not having Sybil or eclipse protection is not as big of a problem.   It proves consensus only among its peers without knowing what the wider network thinks, even if it has Sybil & eclipse protection.  It does not meet the "agreement" requirement mentioned in Wikipedia to be called a consensus mechanism. See Leslie Lamport's requirements for consensus and Coda Hale's "You Can’t Sacrifice Partition Tolerance" as an example of a researcher getting exasperated with people calling algorithms like Avalanche a consensus mechanism.  Nakamoto consensus was Earth-shattering in its ability to get consensus in a distributed permissionless setting with Sybil, Eclipse, and partition resolution (not just detection via slow solvetimes). VDF-POS is the only alternative (POS alone requires more excessive bandwidth as centralization & permission are increased). If you find something better like centralized staking for post-consensus, then you do not need Nakamoto consensus because you're unconsciously doing POS where Avalanche gets fast "consensus" at the cost of ruining partition detection which means you must let a real consensus mechanism override it.  I bugged deadalnix & Emin about this 9 months ago and their position is "partitions are rare". That's true, but if you get Sybil protection, you can still have an eclipse problem and more importantly it means it must not be the final say in consensus. Even if you let POW override it, when you get close to something working you'll realize a simpler semi-centralized technique using classical consensus will work better because Avalanche is only useful for quickly resolving the opinion of a large set of voters. If you have a large set, your Sybil protection is going to require a lot of communication. A Sybil solution may also contain  partition & eclipse protection, but keep in mind this conjecture: you can't carry Sybil etc protection over to the speed of Avalanche in a way that maintains the combination of protection, speed, and a level of decentralization that exceed POS + classical methods. Maybe there is a reason the Avalanche researchers want to be anonymous. They are clearly well-published, so why hide when publishing this?  If you use a smaller set of voters, I think you'll find a better solution such as semi-centralized mempools using classical consensus to prevent double spends. So merchants would trust the mempools for small-valued txns but realize they are not a guarantee like the actual blocks. If all nodes agreeing on individual txns are used, then Avalanche can be used, but it's only a suggestion for merchants and miners. To avoid full-blown POS that makes the consensus part of POW pointless, you would just not worry about Sybil protection, so POW would retain the right to over-rule the centralized mempool or node-based Avalanche.  The potential for preventing 51% attacks can only be achieved if you are basically subtly switching to a POS coin, not using the POW as consensus.  It makes no sense to keep Nakamoto consensus if you're going to overrule it with pre- and post- consensus.  You can just use POW in self-hashing txs to generate and distribute coin and just throw Nakamoto consensus out the window. VDF-POS as I've described is the only other option.

              If you want fast consensus that maintains Nakamoto consensus, use a DAG.  See the issue in my github for how to do a DAG.


              ]

              [
               Update #2. I recently learned BCH may allow past 100 block winners to be a committee to participate in Avalanche to confirm txs. This is to provide Sybil protection which was my main complaint below.  However, Avalanche's main benefit in speed with little communication by sampling only your peers, hoping they are connected to a much larger network. With only 100 blocks (and maybe only 20 actual distinct mines or pools in the committee), there seems to be little to no advantage over classical consensus methods which will have the advantage of proving consensus instead of hoping for it. Avalanche is not a consensus mechanism because it does not prove agreement between all non-faulty nodes (see Wikipedia). It can't know if a majority consensus has been reached because it does not quantify membership participation. It does not know if the network is split (as in a DoS or eclipse attack that can be combined with a double spend) with each side giving different results. All it knows is if your immediate peers agree. See this tweet thread for more of my more recent comments on Avalanche and 0-conf
              https://twitter.com/zawy3/status/1174006755925417986

              ]



              [ update #1:  I believe BCH and Ava are using Avalanche advantageously in this way: if the recipient is confident there is not a 51% attack or network partition in progress, then he can be a sure double spend will not be allowed.  But it invites 33% Sybil attacks on nodes (either locally or globally) to trick nodes into pre-approving txns that POW will have to overturn. The difference between a DAG and Avalanche is that a DAG measures network hashrate integrity by having lots of blocks solved quickly. Neither avoids POW's 51% problem.  ]

              The problem with Avalanche is that it assumes a high level of node participation (the "membership" must not change too much, section 3.7).  So there's no protection against network partitions. It assumes the network remains largely intact and does not say what happens when the minority side comes to a different conclusion. There's no mechanism to tell which is the larger side. The authors said they would address this in a later paper, which is harder to do than Avalanche itself. It achieves Consistency and Availability but assumes there is no network Partition. Ava and BCH said partitions are not part of the real world, but if they were not a big issue,  Nakamoto (POW) consensus did not need inventing.

              POW's magic is in selecting chain history with the least sum of partitions (via highest cumulative work) with only one voting (hashrate) member (miner) per election (block) needing to communicate that he won, and everyone immediately agreeing without even communicating an acknowledgment. The next vote begins without any other communication. The size of the voting membership (hashrate) is also determined from those single winning announcements, which set a variable for the next election to get an accurate average block time.  It's an amazing achievement. An enormous amount of communication overhead is avoided by making the voters work. No membership list is needed because POW does not prove there was no partition. It only proves the chain had the route of least partitions, assuming a 51% attack has not or will not occur.

              If there is a network partition with Avalanche that coincides with conflicting spends on each side of the partition, the network is permanently forked. There's no mechanism to tell nodes which fork is correct unless it defaults back to POW. But if it defaults back to POW, the hidden chain's double-spends will overwrite the Avalanche-approved txns.  BCH said their implementation will allow miners to include txns that Avalanche has not voted on (and not required to include Avalanche-approved txns...both as a way to claim POS is not superseding POW).  This means there is no protection against a double spend because the attacker only needs to get one block on the public chain to include txns that did not receive Avalanche approval, paving the way for double-spends on the hidden chain.

              I've tried to come up with ways to "repair" Avalanche with membership metrics that will enable it to detect network partitions. Fast finality, if not all basic POS, requires proof of sufficient network integrity. If centralization is to be avoided, the nodes must independently conclude the necessary percentage of voting members are known to be participating. This is not trivial. I assume Casper and Dfinity are solving this problem in complicated (suspect) ways.  I'm attempting my own design in a future post.





              Sunday, September 9, 2018

              Zipf's law: causes & derivable from Pareto

              I specified the relationship between Pareto and Zipf distributions in wikipedia:


              Wikipedia's excuse for Zipf's prevalence:


              Preferential Attachment
              A simple, common, and possibly accurate view is that there is a preferential attachment ("the rich get richer") going on, e.g. people are more strongly attracted to large cities because there are more opportunities. A common math example for preferential attachment is the Yule process.

              Yule process
              This is one of the best contenders for explaining Zipf's law and other Pareto (power law) distributions. In biology, the number of species in a genus (or members of a specie?) seems to follow a Yule process that results in a Pareto (power law) in its long tail. This process says a new specie is more likely to form in proportion to the number of species in that genera. It's a simple "the big get bigger in proportion to their size".  It assumes that no species die out. If that condition were included, its tail probably dies off quicker, which is seen in realistic data. Overall, this process gives a hump, or at least a dip at the front end, that is commonly seen in real-world data in their log-log plots. Zipf's law is rho = 0.

              Deviation of Front and Tail from straight-line log-log plots
              A simple power law such as Pareto (the continuous form of Zipf's) is a straight line on a log-log plot.  But most real data forms a hump, possibly more than the Yule process.  So the front and tail ends dip down from the straight line. The preferential attachment idea is amenable to this: after cities get beyond a certain size, there are drawbacks. If it's not beyond a certain size, there's no benefit to from the limited ability to cooperate.  If a word is used too often, it's not conveying information.  If  a word is used too rarely, no one understands it.  So a double Pareto has been suggested, to cover the front and back tails, but it gives too much of a sharp hump in the middle, so it seems a triple Pareto (power law) would be better. But in many cases, a primary Pareto with a secondary Pareto for the head or tail adjustment might be close enough to the best possible.

              BTW,  Zipf-Madelbrot is simply a slightly more general form of Zipf's law, throwing in another constant that might be used to create a more general Pareto distribution (continuous form) by scaling it in a way that results in a CDF =1 at infinity.

              This chapter  by a physicist is excellent.

              This paper says Zipf law works because it is half way between order and disorder.  It normalized entropy per character is 1/2, half the maximum possible. (At N=100 it's 0.37 and at N=1,000 it's 0.44, using  normalized entropy = H(CDF)/log(N) in a spreadsheet).


              Other possible explanations for Zipf's law
              Zipf's law usually refers to systems that have an exponent of s = 1. The reason for this simple 1/rank is considered a mystery for a long time. A lot of theories have been proposed (two possibilities are mentioned above, but I don't know if they were s = 1).

              An IEEE article (with more comments here) claims many experts say Metcalf's law should be that the "value" of the network is N*log(N) instead of N^2 and that this gives rise to Zipf's law. N^2 assumes the value of every additional connection is the same with every node connected to every other node. Log(N) allows loss in efficiency of the connections as the number of nodes increases. They point out the harmonic sum 1+1/2+1/3+ ... 1/N =~ ln(N) + 0.577. This is not surprising since the integral of 1/x is ln(x).So the network gives ln(N)+0.577 value to each of the N nodes, so the total network value is N*[ln(N)+0.577]. But the node math can't be converted (at least directly) to words and city populations because the nodes have an equal number of connections and the same distributions.  Maybe ideas (for words) or occupations (for city populations) could be treated like nodes. Maybe cities or words could be treated as different networks.



              Tuesday, September 4, 2018

              Analaogy between Gravity/Momentum and Magnetism/Electrostatics

              There is a bewildering array of different ways to find mathematical analogies between mechanics and electromagnetism. Feynman discusses analogues to electrostatics, stating at least that part of the analogies is possibly merely the result of everything needing to be calculated in terms of space, and since vector math is the best way to deal with it, the equations will naturally come out similar. (The vector math of curl, divergence, and gradient should show up a lot.) For a simple example, the 1/R^2 rule shows up a lot. It is the strength of something at a distance R from a point source of some quantity.  The source is emitting "rays" of some sort that spread out as you get further from the source. If the quantity we're interested is proportional to the number of rays per surface area of an imaginary sphere of radius R, then we have a 1/R^2 rule.  It shows up in gravity, electrostatics, sound intensity, and the probability P per second of getting hit by a bullet from a madman (point source) firing N bullets per second in completely random 3D directions. 

              Wikipedia has a bunch of analogies between mechanical and electrical systems but the most natural one seems to be the first one mentioned, the Impedance_analogy. It's presented as 4 different substitutions, but it looked like there should be a simple source of the relationships.  I noticed simply replacing charge movement (current)  with "meters movement" (velocity) would derive the relationships.  Replacing charge with meters is a bizarre idea which could explain why respectable sources do not seem to mention this connection.

              Two basic electricity equations are:

              V = L di/dt
              i = 1/J dV/dt

              V = volts
              L = inductance
              i = q/second
              q = charge
              1/J = C = capacitance

              The charge => meters substitution gives valid analogous relationships:

              F = M dv/dt
              v = 1/K dF/dt

              F = force
              M = mass
              v = x/second
              x = meters
              K = Hook's law

              These equations extend to valid energy equations:
              E = 1/2 J q^2  (capacitive energy)
              E = 1/2 K x^2  (spring energy)
              E = 1/2 L i^2  (inductive energy)
              E = 1/2 M v^2 (kinetic energy)

              Separating a capacitor's plates by distance x does not result in the spring equation. The analogy works because as a charge q is moved the distance across a capacitor's dielectric, there's not only a V force it's resisting, but it's presence once it's there increases the voltage for future charges trying to move against it. 

              The most interesting possibility is a deep parallel between L and M because L is just the result of how charge flows. It creates a self-interacting magnetic field and magnetism (in a non-quantum world) can be derived from q and relativity, i.e. L is not a thing in and of itself in the way we normally think of mass being something "real". L is just the result of q being forced to flow in a self-interacting way  We know the kinetic energy is also the result of relativity increasing the mass as velocity increases.  Is mass just as fictitious as L? It's interesting that inductors are in the same shape as a spring. Is mass in some sense have a capacitor shape? We know in relativity the distance shortens in the direction of travel. This leads to the idea than areas of like charge are like capacitor plates being pushed together and this is the source of how extra energy is being stored when you accelerate a mass. Inertia would just be the force needed to push like charges closer together, although the plate view is not supposed to be the correct view because as I mentioned capacitor energy is not the result of plates pushed together. 

              The above are the integrals over either a charge or distance. In other words, these are the derivatives of the above:

              V = J*q
              F = K*x
              magnetic flux = L*i
              momentum = M*v

              J = 1/C is the difficulty with which q can move onto the capacitor, and K is the difficulty with which "distance can move onto a spring".  Similarly Mass and inductance are difficulty to increasing v and i.

              i is to magnetic flux what v is to momentum. We think of pushing charges through inductors, so maybe we should think of moving meters through mass instead of moving a mass through meters.   

              Two more equations to point out:

              E = F*x
              E = V*q

              Electrical: Moving charges are confined in a spatial arrangement that we call an inductor. If we try to accelerate them, we encounter a push-back. If we overcome the push-back so that they move faster, it will cause the inductor to have a higher internal energy. (The inductor can be thought of as a superconducting  toroidal type.

              Mechanical: "Moving meters" are confined (via charges?) in a an arrangement  we call mass. If we try to accelerate them, we encounter a push-back from their "self-inductance". If we overcome the push-back so that they move faster, it will cause the mass to have a higher internal energy.

              To go deeper, I would like to insert v in place of i in Maxwell's equations because Einstein said Maxwell's equations are a great filter for weeding out false theoretical ideas because all relativistic ideas are subject to them. But a straight substitution into Maxwell's equations would seem to have a problem. Maxwell's equations are all about spatial relationships. Throwing in an extra spatial dimension to replace charge seems drastic. Do I exchange an x and q instead of replace the q?

              Notice in the all the charge equations above, meters were not present except implied in say J or L.

              Will it require somehow replacing the 3D spatial system of the equations with some kind of 3D charge system?  To be clear, this means there would be some sort of 3D "charge-space".  I was once told spatial dimensions are the result of quantum spin and I know spin is at the charge level, so I did a quick Google search for "spin charge" which showed electrons consist of 3 quasi-particles. Are the 3 quasiparticles related to charge in a way that is analogous to how the 3 spatial dimensions' are related to mass? 

              Maxwell's equations can be derived from the idea of point charge emitting rays of force in 3D space, and them using relativity to generate the magnetic pair of equations. Magnetism = a relativistic effect of charges (Feynman and Schwartz cover this, but I didn't learn it from school, but came across it in a 1930's Encyclopedia Britannica before looking for it elsewhere). Magnetism is perpendicular to the movement of charge in space.