Friday, August 25, 2017

Environmental impact of POW algorithms

hush slack comment

The current equihash is pro-GPU (ASIC-resistant). Making the algo more CPU or smart phone friendly will use more electricity compared to hardware expense because so many of those hardware units are sitting idle and miner competition means more money will be spent on electricity than new hardware. Maybe in the beginning there could be an evnvironmental benefit, but over the long term this is not to say a CPU/smartphone algo will have less of an environmental impact. Also, any algo that could do this is too new for my comfort. Equihash had a small hope for helping CPUs compete with GPUs, but it did not work out. The environmental impact is exclusively determined by how important society thinks POW coins are. The more society pays for POW security, the more miner competition there will be. Miner competition will involve paying for both hardware and electricity. If the algo shifts costs to hardware, the money is ultimately being spent on engineers or shareholders lucky enough to own good intellectual property (like NVIDIA). Those engineers and shareholders are just going to spend the money on things that will ultimate require more electricity. So there's no way to reduce expense on the total amount of electricity or other resources are use: economics demands POW because it is the only proven security. Competition means hardware and electricity expense (waste) will equalize with society's demands for the POW security. A caveat is that some jurisdictions allow free electricity which means those miners have more money available to spend on hardware to compete with each other, wasting electricity. So a shift to hardware can be good for that reason. Also a shift to CPU/smartphone hardware could be good for the coin because it means better even distribution of mining which invites more users and provides 51% protection. But HUSH devs can't fix these problems (shift to hardware or shift away from POW) because they are the most difficult to fix that the best minds in the world have not yet solved. But they can be smart enough to not try any of the newer ideas because it will reduce the security of the coin.

Thursday, August 24, 2017

BitcoinCash difficulty algorithm problems

BitcoinCash allows a drop in difficulty down to 1/4 if the last 5 blocks took > 12 hours. But the rise in D appears to take 2016 blocks like bitcoin. They did this so that D could drop quickly after the fork. But this asymmetry (long time to adjust up, but short time to adjust down) is causing an unexpected feedback. Deciding to mine or not depends on a threshold of profit. Let's say price and Diff are stable, but then for some reason price falls below the profit threshold. Some jump ship but the average Diff of the past 2016 block may still require a higher difficulty setting. This will occur when the price drop is at the end of 2016 block cycle. This can occur in BTC, but hey can tolerate price swings better because they have been forced t be more conservative: the big BTC miners have no place else to go that will absorb their hashrate without making diff jump sky high. But with BCH, they can just jump back to BTC which is able to absorb them. If the wait for difficulty adjustment were symmetrical as ni bitcoin, they would have to wait 2016 blocks before difficulty could adjust again. That would be a 4x longer wait for each block if 75% miners jump ship. So it would need a huge drop in diff at the end of the 2016 cycle. So they all pile back on and the difficulty ends up being way too low for 2016 blocks, so the blocks start coming every minute instead of 10 minutes. All these coins are going to be a pressure downward on the price, so when the next Diff kicks in high again due to the new interest, it's going to be combined with a low price. They really will jump ship again for 2016 blocks. The selling pressure is also gone, preparing it for another huge swing back the other way due to low difficulty and higher price. So there's a bad cycle. So maybe the lack of symmetry may not be the problem, but just the existence of Bitcoin as a safe haven for the swings. But the asymmetry appears to amplify it, ruining the average solvetime: in BCH they do not have to wait 2016 blocks for the difficulty to drop. It can happen in maybe less than 10 blocks. So the fear is that miners are going to continue mining them 10x faster, with only short what times in between, permanently, until they hard fork.

Sunday, August 20, 2017

Gandalf's difficulty algorithm


The following looks simple, but the link above shows how much work I've done in advanced techniques
like a variable averaging window, hash-attack detection using statistics, least-squares slope prediction,
significant-slope-change detection, mixtures of avg and median, recent-blocks-more-heavily-weighted, and
"discrete D/T averaging" as a hashrate measurement. I was able to get all of them only as good as simple
averaging, and none of them better than simply going to a smaller N for averaging.

# Gandalf's difficulty algorithm v1.  "Hash attacks shall not pass!"
# This is Zawy v1b with low N=8 (and X=5)
# Timestamp manipulation protection is not needed due to limits on next_D.  
# Negative timestamps must be allowed.
# 0.92 keeps avg solvetime on track but makes median come out 8% low. I can't find a fix.
# Increase N to 30 if you want correct avg & median, but expect 10% of blocks in small 
# coins to be stolen.   Median mixed in w/ avg will not help. 
# Do not use sum(D's) * T / [max(timestamp)-min(timestamp)] method.
#  5% block solvetimes are > 5x target solvetime under constant HEAVY attack.
#  1.3%  block solvetimes > 5x target solvetime when hashrate is constant. 
#  3% of blocks will be < 0.5x or > 2x the correct difficulty on accident.
#  When N=15 these numbers are 5%,, 1%, and 0.5%.   
https://github.com/seredat/karbowanec/commit/231db5270acb2e673a641a1800be910ce345668a

next_D = avg(last 8 Ds) * T / avg(last 8 solvetimes] *0.92;
next_D = 1.6*previous_D  if next_D/previous_D > 1.6;
next_D = 0.63*previous_D  if next_D/previous_D < 0.63;

The only potential problem I see is that the D will swing below 0.5 the correct D a lot, attracting an attack.
But they can only get ~4 blocks at low difficulty. A question I need an answer to is this: how short of an
attack can an attacker tolerate before it is not worth switching to the coin? I can go to N=5 which will
result in a lot of D < 0.33 correct D, which is really attractive, but is worth 3 blocks for them to switch coins?

Friday, August 18, 2017

Bitcoin Cash & SegWit2x causing BTC-chain splits to have constant value, increasing advantage of the 1% hodlers

the merkle says it's insatiable demand from South Korea. People are going to want SegWit2x fork even more due to the success of BCH and that will drive BTC itself even higher. It seems like BTC is headed towards the ideal constant-value currency simply by forking. But totally unexpectedly, old holders will profit even more due to the forks. BTC hodlers are becoming exactly what they were fighting against: banks and governments (esp U.S.) profiting from inflation of a coin. The forks make it feel easier for new entrants to come on board and the "1%" that joined early continue to profit massively. But when others profit so massively, it comes out of the 99% one way or the other. BTC is not zero sum, but the eventual total off-chain real world assets it will control are limited to how fast world GDP can grow. The "1%" BTC hodlers will gain a disproportionate share of those assets, just as the U.S. enjoys "slave labor" in Asia simply from having the world's default currency due to a "might is right" military (which itself was largely a geographic advantage of WW2). We are able to print new dollars just as fast as world GDP can expand without causing inflation. We print 2% or 3% even faster, causing persistent mild inflation, which prevents the 1% from forever keeping their wealth unless they reinvest it in good ideas that get more than the 2% or 3%. Likewise BTC forks have a might is right advantage in both devs and mining by being well planned and 1st.

what I mean by "BTC" inflation is that it is chain-splitting. BCH and impending SegWit2x means there will be 3x more "BTC". I can't see how it's any different from alts, except that there is a huge marketing advantage to new entrants (aka the 'unwary') to calling them "bitcoin" (flavors?) but also existing BTC hodlers have no need to complain or fight against them like they do with alts. This last point is my perspective that a lot of other people hold, even if it's not correct. The effect is the same: the perception of new entrants and existing hodlers is more important than most facts when it comes to a coin's value. And our ignorance (if new entrants and hodlers are "wrong" about the usefulness of a new coin) that results in higher value can in turn spur development back towards a more useful coin. The 3 different forks could evolve back into equivalent code simply because people "wrongly" believed in the initial bad ideas. The macroscopic invisible hand driving all this is the desire for constant value. The visible part of the hand is simply new entrants not wanting to pay high prices and hodlers liking the extra coins from nothing.

Friday, August 11, 2017

Strong Drink mix for Parkinson's

I have been putting together a really strong drink. I guess it's about $15 a day, with most of the cost being in the powder extracts, $2 to $3 per day each, straight from China in bulk.

12 oz pomegranate juice from Hispanic store (not the expensive POM)

added sweet concentrates:

=================

black cherry concentrate 12 g

black molasses 12 g (sugar cane juice after most of the white sugar is removed)

Jallab 12 g (Arabic grape skin extract plus others)

powder 10:1 extracts:

==================

blueberry extract 12 g (my eyesight sharpened enough to not need my barely-needed glasses in 4 days)

strawberry extract 12 g

apple peel extract 12 g

tangerine peel extract 12 g

Citrus flavonoids with animal studies in PD, bought from china in bulk.

These doses are 1/4 the human-equivalent doses because the studies are "shock" studies on the animals by which I mean they are very short term to see how the chemicals work in response to PD-like toxin challenges.

===================================

nobiletin 500 mg

naringin 300 mg

tangeretin 100 mg

Other stuff in pills with strong animal and epidemiological evidence for PD and ability to absorb and cross blood brain barrier (pills not in the drink):

====================

black tea extract

green tea extract

grape seed extract

fisetin

(inosine to be added)

The American producer of patented fisetin is not clear that it is pure fisetin and the brand is hiding details about what it is, so I'll spend 1/3 as much to get pure fisetin from China and then sell the excess on ebay. Inosine in bulk is also 1/3 the cost from china.

Canola mayonnaise, the bomb!

Broccoli, Sardines, home-made very yeasty beer, olive oil

1 hour exercise, then drink it to absorb the sugar.

Tuesday, August 8, 2017

Potential value of bitcoin, empires, and taxes

M3 is roughly "all cash". For US dollars, it's about $30 trillion. The rest of the world I'll estimate at $25 trillion because the Euro is about $12 T in USD. I expect bitcoin and alts to roughly follow this ratio, so BTC would be compared to dollars. so the max would be 30T/21M = $1.5M per BTC. As this happens, dollars all over the world will come flooding home making them worthless, so it's more important for Americans to switch early than in other countries, to maintain current lifestyle. The ability to print dollars and the growing world economy accepting them has been the biggest boon any country has ever seen. We were basically allowed to print them as fast as the world's economy grew. We spent half the surplus on a military which pushed and supported the use of the dollar, enabling stability and exchangeability in the same way MicroSoft "helped" software. Bitcoin is the Linux of money. Wealth will be more evenly distributed as the dollar monopoly in currency ceases.

Not just the U.S. but all governments will lose power to control if they lose control of the currency that their citizens demand. Countries enforce a currency by demanding taxes and legal disputes be settled in their dollars. Empires use currency to enslave other countries. So countries can start their own cryptocoin, enabling them to enforce law more directly and automatically extract taxes by being privy to every transaction. This would relegate BTC to replacing only gold for private holders, which is $7T, $350,000/BTC, but potentially a lot more since a lot of countries want to be more fair in international exchange instead of being stuck with the dollar. Gold is harder to move so there's a great desire to switch to BTC. Also, buying stuff directly from other countries instead of Amazon will need BTC, which is a "black market" as far as the U.S. government will be concerned. It takes away their power as the dollars come home. They can make it illegal to import things from other countries without dollars. The U.S. desperately needs dollars to stay out of the country. When foreigners like the Chinese government start giving us wads of dollars to get BTC, that is NOT the time to switch back to dollars. That is the end of the U.S. as the world power. That is when great powers fall: they spend too much on military to support a coin or gold, lose their skills at production due to enslaving people in the distant lands (via the coin supported by the military), then find themselves powerless as their coin collapses. In the case of Spain getting gold, they just spent it all, then the armada fell and Britain's superior skill at ship building took over. There is also the possibility that BTC will be a basis for establishing ownership of assets and enforcing smart contracts, again putting it well over the $1M/BTC range. I do not expect it to go over $500,000 in 20 years. If it reaches $100,000 it will be a primary way of buying $1M beach houses as old money finds itself increasingly poor and BTC millionaires start looking for something to do with their gains.

50x = $150k/BTC does not need BTC to be lucky. It only needs to be the best idea. When BTC reaches $150k it will be because it is starting to be used as an international standard for trade. It will be because dollars are coming home which will make them lose all their value. The U.S. government will then have to decide to cancel all social security and most government expenses (pollution, law, roads, retirements) and foreign debts, or print money (hyperinflation). That will not stop the inflation because there will be 3x more dollars inside the U.S. from foreigner not wanting them. If it takes 15 years, that will be 7% inflation plus our current 3%. 10% inflation is far from hyperinflation, but still a disaster. Actually the disaster was letting there be a "balance of payments" surplus the past 50 years which means more money going out (via free trade, military, and dept) than what was coming in. This results in erosion of the country's ability to support itself. Free trade is a disaster if it makes the balance of payments worse. The U.S. (like China) got out from under enslavement of a foreign currency by enacting trade tariffs. China's devaluation of currency is in effect a trade tariff on the external world's imports which forces its people to work harder and develop more skill. I believe the U.S. is sophisticated enough not to have hyperinflation. When it reaches $150k, it is NOT the time the sell, but a time to keep holding, unless you see a better option. But I think a capped-quantity coin is not a good solution and not the solution the rest of the world will want due to late-comers being at a disadvantage. But unless a new coin lets smartphones determine their own time via the stars or random or 3rd party consensus trust, and combine it with a local trust network to decentralize the coin (protecting it from big miners), BTC may be the best option. This is because all alts subject to 51% can be destroyed via simple forward-stamping timestamps, and if BTC miners are hodlers, they will soon find it more profitable to destroy alts than to mine, forcing more money into a few coins. They may even use their BTC value gains to buy more equipment to retain power by destroying alts instead of mining BTC. The miners may turn into BTC's military. This is what happens to all empires: they win by might is right until all the slave countries figure out a way to get out from under the coin that controls them. The coin is backed by a military. Coins are how governments exert control. Some argue BTC has no government, that devs are not really in control. However that may be, anyone who holds BTC will be the new lords, enslaving the late comers, backed by our military, the miners. At least this is our best-case scenario in our search for personal profit.

Thursday, August 3, 2017

Shaking marbles in a jar as the simplest exposition of evolution, A.I., and economics


I am not using "entropy" in this article in a vague sense. I am talking about the entropy measured by physics equations.(see note if you're interested in the physics)

Executive Summary of the Physics
Physical evolution in its simplest form is shaking a jar of randomly-packed marbles in a gravitational field. In short, complex (but not completely random) energy injected into a "closed thermodynamic system" results in entropy being released to the universe as black body radiation (more low-energy photons go out than high-energy ones came in...all energy transfer is a sending of photons). Since entropy is a conserved quantity, the entropy inside the container is reduced. A reduction in entropy exhibits itself as a higher degree of order by becoming more densely packed, harder, and repeating patterns. The 2nd law is not "entropy always increases" (see good physics texts such the famous Feynman lectures on thermodynamics that mention this). "Always increases" is an engineering idealization for isolated systems. There is no isolated system in the universe thanks to black body radiation. Gravitational systems are emitting entropy "so that the Universe can expand" (or alternatively "as a result of the Universe expanding"), leaving a lower-entropy state behind. (more detail: The entropy of the Universe is constant on a co-moving volume basis and reducing on a constant-volume basis. This last statement is not speculation but I do not have a good reference. It is not required for the following, but provides the deepest explanation for the origin of life that I have.)

The physics of shaking marbles in a jar
When you randomly and slowly place marbles in a jar they will pack with about 56% fill ratio, leaving 44% space. If you shaking them afterwards, starting first with hard shakes and then softer shakes, they will pack with > 63%. The harder shakes allow for the bottom layer to form first. The softer shakes allow the higher levels to settle without upsetting the lower levels. The highest theoretical packing is 75%. The shaking can't be completely random and consistent because it will continually un-pack. Non-random shaking can be thought of as a "periodic" or "semi-periodic" force (or energy injection). For packing differently sized and shaped small objects there is a more complicated way that does the same basic thing: add heat while lowering pressure, then raise pressure as the heat drops, and then repeat, but lowering the temperature and pressure each time (see wiki's "Simulated annealing"). The heat checks options while the pressure that follows secures and compacts the solutions. The pressure is a force like gravity. The heat is the shaking.

Chemical bonds
I'm not going to hardly mention the chemical bonds that are crucial to life other than to say the reduction in entropy in the products of life can be measured via molar aka specific entropy. There exists in chemistry potential energy gradients due to charges just as in gravity. They can react to form durable bonds that releases entropy and heat to the universe, leaving behind more durable arrangement with lower entropy. Getting into specific chemical bonds can and should be done, but for this post it would be like getting into the motivations of an employee in a company when I only want to discuss company-wide profit seeking that enables the employees to exist. Similarly, when getting into employees motiviations, we do not address the chemical reactions in his brain, nor the genes and environment that led to those reactions. But this outline is trying to show how the "environment" (the Sun, Moon, and Earth via thermodynamics and cosmology) resulted in those genes based on physics as opposed to some nebulous beliefs about "evolution".

Correlation between the jar and life on Earth
The Sun, moon, and Earth's rotation are the initial source of non-random energy coming into the biosphere. The "non-random" (low entropy) placement of the moon's mass away from the Earth has been crucial for life's development (Isaac Asimov once discusses this). Its effect decreases with time as the moon get further away each year, in keeping with the jar analogy but I do not want to stress the "decreasing" part because it is not as important as simply not being random. The effect of the moon is apparent to NASA's life-hunters: the most promising places seem to involve an external gravitational force periodically affecting a celestial body (usually moons close to a planet). The Sun is more important via photosynthesis these days, but I have investigated the extent to which life initially capitalized on ("extracted") the low entropy created by the moon's initial placement. I calculated in a previous post that today the Sun is providing 150x more energy than the yearly loss in Earth's rotational energy due to the moon, but at the beginning of life, the moon was >3x closer with >9x more gravitational effect, and the Earth was turning a lot faster. My calculation (from available data) indicated the moon provided about 20% of what the Sun was providing. This is a "mass moving" quality of energy rather than the "heating" quality provided by the Sun. The combination may have been crucial: periodic heating was a "periodic shaking" at the molecular level while the "mass moving" shaking provided a larger-scale directional force to where the resulting molecules would go. I considered the parallel this has with law guiding individual free market transactions, but have rejected it. Law will be the jar within Earth's jar.

As the moon gets further away, the entropy of the Earth-moon system is increasing due to a volume increase. The Earth's rotation rate is slowing, giving some energy to the moon in order for it to get further away, but a much larger percent is used to churn the air, seas, and mantle via the moon's gravity.To what extent were an excess of ocean vents present (where the oldest known life fossils have been found) due to the moon churning the mantle? The second oldest place fossilized life has been found is in bays with ocean tidal zones, which is more obviously assisted by the moon. To what extent would our massive economic machine not be possible if the moon had not churned the mantle enough to make more ore concentrations possible via volcanic activity? That non-random placement of mass is again a physically-measurable lower state of entropy that resulted from a lowing of the Earth's rotation via the moon.

I have cast it as an low-entropy injection from the Earth-moon system's volume increasing, but most of the lower entropy is coming from balancing the black body radiation (entropy) being emitted as a result of churning/heating caused by the Sun and slowing of the Earth's rotation.

The Sun affected molecules with periodic (daily) heating to enable them to form stronger bonds, a key ingredient for life (the success of "genes" is a combination of ability to spread quickly and last a long time via the really strong bonds of DNA crystals). I will not dive into chemistry because I think that is missing the forest for the trees. It is important to keep in mind genes have no force of their own. They are just enzymes. Future genes model on past genes. Crucial to efficient packing is that the next layer of marbles are guided by the lower levels, obviously copying their arrangement. In what sense is this not like genes? Obviously the marbles do not evolve diversity, but I'm only allowing one type of object into the jar to try to see the essence of physical evolution.

I should mention the Sun and rotation combine to cause a bulk periodic movement of mass like the moon, not just heating. A side effect of this causes charge potential energy gradients that result in lightening strikes which been theorized to be crucial for life. Those strikes are like sudden large "shakes of the jar" at the molecular and atomic layers.

I discussed this in more detail earlier this year, but the above section is more efficient and has new material to show more parallels.

Correlation to artificial intelligence
A large part of A.I. in finding the solution to complex problems is starting with random values in neural nets, Bayesian probabilities, and genetic algorithms. The marbles (in our brain) are the neural nodes, Bayesian nodes, or genes (well, not genes exactly). You typically start with random values and give the program computational energy (shaking) for the nodes or genes to request CPU time and memory (energy) while they are under a system-wide A.I. algorithmic constraint (the jar) and a gravity provided by the training data to find an efficient (low entropy) solution. The solution will show many patterns in the node weights, probabilities, or genes which is the marbles' repeatinng arrangement. The patterns may be eliminated for condensing the node weights, probabilities, and genes into a smaller set of nodes that is more random on a per weight/probability/gene basis. This is like taking marbles out of the jar which is less entropy by a factor of roughly equal to N2/N1 (this is exact if it's a specific molar entropy) and it allows a smaller jar which is also less entropy by a factor described at the note at the bottom. The final result is not a repeated pattern but it is the same(?) or less total entropy. It is more entropy on a per weight/probability/gene basis in an absolute sense, but the entropy per weight/probability/gene "per this system" is the same or less. Nodes in neural and Bayesian nets might be identical in an algorithmic sense (like the jar constraint) and change only their weighting factors (placement). In genetic algorithms, the genes would repeat and/or reduce in number. So the algorithm representing the jar is the set of rules of gene interaction rather than a description of the genes. The genes vary while the marbles do not. There is only 1 type of marble. To compress the marble packing to reduce the number of marbles in the jar, but to completely describe the pattern, only a few marbles are needed. A.I. seeks to find repeating patterns of marbles and normally compress into a single pattern. Fewer marbles are better if they are smart.

Correlation to economics
The A.I. must have some sort of direct, implied, or unnoticed limited-quantity currency that corresponds to the amount of CPU time (FLOPS) and memory that is available, if there is a limit on the amount of CPU power (FLOPS) and memory space that the algorithm can use. The currency is the energy being transmitted from the jar (the algorithm constraints) to each marble (the weight/probability change or gene replication). The CPU time and memory are kinetic and potential energies. The hardware itself is a potential energy. The currency quantity corresponds to computing quantity. If the hardware increases then the currency can be expanded by the algorithm (the government) to keep constant value so that nothing else in the algorithm needs to change. This is like increasing the level of shaking: the currency is the amount of energy being transferred from marble to marble from the jar. The energy comes from the governing jar or A.I via the permission granted by possession of the currency and eventually goes back to it (taxes). In my jar there is no currency I can point to except the energy itself. Although the energy coming in and size of the jar may not change, the entropy of the system gets smaller as it gets more efficient. The lower entropy means better command, control, and strength that is typically used to increase the incoming energy, the "size of the jar", and the number of "marbles". If the allowable nodes or genes are increased, then the value of the currency per node or gene decreases by that same proportion if the hardware has not improved because they will have to compete to get the same computer time and memory. So the currency quantity needs to decrease if the currency should represent the same amount of FLOPs and byte space.

In other words, for contracts to remain valid, the currency quantity should change in proportion to the energy per person that is under the system's control.

Physics note: All other things being equal, the entropy increase of the moon getting further away is S=S2-S1 where S1=a*[ln(b*V1) + c] where V is the volume of the Earth-moon system and a, b, and c are constants. There is rotational energy decrease and gravitational energy increase, but these are internal energies that do not change the kinetic energy of the system (that could have affected the entropy) because the Earth's temperature is about a constant. But those lost energies do emit a lot of entropy away from the system as waste heat. As another example important to the following: harder materials have lower entropy due to the atoms having fewer places per volume that they can occupy. Specifically, for a single harmonic oscillator (in a solid) S=k*ln(kT/hf + 1) where f is frequency of the oscillations which is higher for stronger bonds.

Tuesday, August 1, 2017

difficulty algorithms

===============

# Zawy v1b difficulty algorithm (nice and simple)
# Based on next_diff=average(prev N diff) * TargetInterval / average(prev N solvetimes)
# Thanks to Karbowanec and Sumokoin for supporting, refining, testing, discussing, and using.
# Dinastycoin may be 3rd coin to use it, seeking protection that Cryptonote algo was not providing.
# Original impetus and discussion was at Zcash's modification of Digishield v3. The median method
# Zcash uses should be less accurate and should not be needed for timestamp error protection.
# Wider allowable limit for difficulty per block change provides more protection for small coins.
# Miners should be encouraged to keep accurate timestamps to help negate the effect of attacks.
# Large timestamp limits allows quick return after hash attack. Needed to prevent timestamp manipulation.
# (1+0.693/N) keeps the avg solve time at TargetInterval.
# Low N has better response to short attacks, but wider variation in solvetimes. 
# Sudden large 5x on-off hashrate changes with N=11 sometimes has 30x delays verses 
# 20x delays with N=17. But N=11 may lose only 20 bks in 5 attacks verse 30 w/ N=17.
# For more info: 
# https://github.com/seredat/karbowanec/commit/231db5270acb2e673a641a1800be910ce345668a
#
# D = difficulty, T = TargetInterval, TS = timestamp, TSL = timestamp limit

N=17;  # can possibly range from N=4 to N>30.  N=17 seems to be a good idea.
TSL=10 if N>11 else TSL = 0.8*N; # stops miner w/ 50% from lowering  D>25% w/ forward TS's.
current_TS=previous_TS + TSL*T if current_TS > previous_TS + TSL*T;
current_TS=previous_TS - (TSL-1)*T if current_TS < previous_TS - (TSL-1)*T;
next_D = sum(last N Ds) * T / [max(last N TSs) - min(last N TSs] / (1+1/N);
next_D = previous_D*1.2 if next_D < 0; 
next_D = 2*previous_D  if next_D/previous_D > 2;
next_D = 0.5*previous_D  if next_D/previous_D < 0.5;
==============

I had thought the following was best, but if I use v1b with N=8 and replace 1/(1+0.693/N) with
*0.89 then they are about the same.

===========
#  Zawy v1c difficulty algorithm  
# For more info: 
# https://github.com/seredat/karbowanec/commit/231db5270acb2e673a641a1800be910ce345668a
# D = difficulty, T = TargetInterval, TS = timestamp, TSL = timestamp limit
# N should not be changed
N=16;
M=int(N/2);
O=int(N/4);
k = 0.85; # adjustment to keep avg solvetime correct when hash rate is constant.
TSL=9; # should not be changed
current_TS=previous_TS + TSL*T if current_TS > previous_TS + TSL*T;
current_TS=previous_TS - (TSL-1)*T if current_TS < previous_TS - (TSL-1)*T;
temp1=  k * sum(last N Ds) * T / [max(last N TSs) - min(last N TSs];
temp2 = k * sum(last M Ds) * T / [max(last M TSs) - min(last M TSs];
temp3 = k * sum(last O Ds) * T / [max(last O TSs) - min(last O TSs];
next_D = 1/3*(temp1+temp2+temp3);

==================


# Measures relative effectiveness of difficulty algorithm under attack scenarios.
# The algorithm with the lowest "attack" number wins.
# Getting blocks at low diff attracts attacks, so they are weighted same as long wait times.
for (  i = all block numbers )  { 
      for ( j=0 to 9 )  {  total_ST += ST[i+j]  }
      if  (10-total_ST/T) / SQRT(total_ST/T) > 2.5  sum_diff_was_too_low += total_ST/T/10 ;     
      if  (10-total_ST/T) / SQRT(total_ST/T) < -2.5  sum_diff_was_too_high += total_ST/T/10 ;     
}

=================
# zawy v3c  difficulty algorithm, 2nd best
# same as v3b with the following changes
# replace N=17 with
N=16 
sumNs = N/2*(N+1)

# totalHashrate = totalHashrate + D[i]/ST[i] * 4 * (1-e^(-ST[i]/T)) * e^(-ST[i]/T))
# is now

totalHashrate = totalHashrate + D[i]/ST[i] * 4 * (1-e^(-ST[i]/T)) * e^(-ST[i]/T))*(N+1-i)/sumNs
================
# Zawy v3b 
# ST=Solvetime, T=TargetInterval
# next_D = T * avg(N linearly mapped D/ST) 
# UNLESS past N/2 ST's are > 2.5 std devs too long (after an attack) 
# THEN use v1b for N/2 blocks:  next_D = T * avg(N/2 of D) / avg(N/2 of ST) / (1+0.69/(N/2))
N=16
M=int(N/2)

&correct_negative_solvetimes

# check to see if last M blocks were dropping significantly fast.
Mexpected = Sum(last  M ST's) / T
std_dev = (M - Mexpected) / SQRT(Mexpected)
if (std_dev > 2.1 OR wait > 0 ) then
    next_D = sum(last M Ds) * T / [max(last M timestamps) - min(last M timestamps] / (1+0.69/M)
    if (wait == 0) then (wait = M) else (wait=wait-1)
else 
   totalHashrate=0
   for i=1 to N
        totalHashrate = totalHashrate + D[i]/ST[i] * 4 * (1-e^(-ST[i]/T)) * e^(-ST[i]/T))
   next i
   next_D = T * 0.64 * totalHashrate / N  
end if
# finished

sub correct_negative_solvetimes {
# Eliminate negative ST before the mapping loop above. 
# N = number of block going in reverse.  N=1 is previous block.
# The following ST changes are permanent.

# Avg past 2 ST's if this ST is < 0.
if ST[1] < 0 then 
    if  ST[1] + ST[2] > 0 then 
        ST[1] = ( ST[1] + ST[2] ) / 2
        ST[2] = ST[1]
    else
         next_D=previous_D
         exit / return completely....get out of next_D routine
    end if
end if
if ST[2] < 0    #   ST[1]+ST[2] in previous block was neg. Previous ST[1] is now ST[2]
    if ST[1] + ST[2]  + ST[3]  > 0      
       ST[1] = ( ST[1] + ST[2]  + ST[3]  ) / 3
       ST[2] = ST[1] 
       ST[3] = ST[1] 
   else        # there were 2 or 3 negative timestamps in a row.
       # We don't know if ST[2] and ST[3] were early or if ST[4] and/or ST[5] were late.  
       # Best and safest thing is to make them median ST
        ST[1] =median ( ST[1] to ST[N] ) 
        ST[2] = ST[1]
        ST[3] = ST[1] 
    end if
end if
}


=========

#!/usr/bin/perl

# Zawy v1 algo for detecting hash attacks when diff algo is not responding fast enough (July 30, 2017)
# Let me know if there is an error:  zawy@yahoo.com
# Only detects when the diff algo is not responding fast enough.
# Prints a graphic to command line and writes block, solvetime, and difficulty to diff3.dat 
# For difficulty algorithm ideas (zawy v1c recommended)
# see https://github.com/seredat/karbowanec/commit/231db5270acb2e673a641a1800be910ce345668a

$instructions="
Detects hash attacks.
This diff3.pl file needs editing for the coin-cli location and the block interval.
Output goes to screen and diff3.txt
Command line format:
perl diff3.pl  
Default is past 500 blocks\n";

unless ($#ARGV == 1 or $#ARGV == -1) {   print $instructions; }

$coin_name='hush';
$dir="./$coin_name/src"; # Use "." if cli program is in same dir as this script. No ending "/".
$n=500; # number of blocks in past to look if start and end block not passed
$m=10; # Number of blocks to check for statistical aberration. Recommend m=10 to 15. 
# m < attack length is good. 
$ti=150; # the coin's targetinterval in seconds.  150 for 2.5 minute blocks.
$sd=3; # std devs of statistical significance.  Recommend sd=3.  See next paragraph.

# sd < 3 will result in too many false positives. sd = 4 will get only 1/4 of smart short 3x attacks
# The number of hits on accident are the 2*(one_tail_probability)*W/15*n/m
# where W=block in your diff algo averaging window.  sd=3 is 0.0015 for one_tail.  
# Expect to have a false appearance of attacks more often if W is larger or m is smaller.
# Short < W/2 attacks that are 2x network hashrate are detected 1/2 time and more often are just noise.
# But if m < 1.5 attack length, then sd=2.5 detects 80% of 2x attacks with minor false positives.
# Short 3x attacks are 80% detected with little error from noise.

if ($#ARGV == 1) { $n=$ARGV[1] - $ARGV[0]; }

@a=`$dir/hush-cli getinfo`;
foreach $a  (@a) { 
    if ($a=~s/.+blocks\"[^\d]+(\d+).+/$1/) {$height=$a;}
}
if ($#ARGV == 1) {$height=$ARGV[1]; }

for ($i=$height-$n-1;$i<$height;$i++) { 
   
   # print $i;
   $hash=`$dir/hush-cli getblockhash $i`;
   # print $hash;
   $info=`$dir/hush-cli getblock $hash`;
   $diff=$info;
   $diff=~s/.+"difficulty"\D+(\d+).+/$1/sg;
   $time=$info;
   $time=~s/.+"time"\D+(\d+).+/$1/sg;
   if ($i == $height-$n-1 ) { $prev_time=$time;next;  }
   # print "t: $time d: $diff\n";
   unless ($i==$height) {
      $t{$i} = $time-$prev_time;
      $neg{$i} = "negative" if $t{$i} <0;
      $d{$i} = int($diff);
   }
   $prev_time=$time;
 }
foreach $blk (sort keys %d) {
   $avg_diff= $avg_diff + $d{$blk}/$n; 
   $avg_t=$avg_t+$t{$blk}/$n;
   $j++;
   $sum_t=$sum_t+$t{$blk}; # add solvetimes
   $std_dev{$blk}="---";
   unless ($j < $m) {
      # take out oldest solvetime
      $sum_t=$sum_t-$t{$blk-$m};
      # do statistical calculation 
      $blks_expected=$sum_t/$ti;
      $est_diff_inc_needed{$blk} = $m/$blks_expected; 
      $std_dev=($m-$blks_expected)/($blks_expected)**0.5;
      if ($std_dev > $sd) {
         $std_dev{$blk}=int($std_dev*10)/10;  # store measured std dev if > sd
      }
      else  { $std_dev{$blk}="---"; }
   } 
}

open (F, ">diff3.txt") or die $!;
print F "height solvetime difficulty std_dev\n"; 

print "format: height, std dev if attack detected, scaled difficulty, solve time (amt diff needs increasing) difficulty\n";
foreach $blk (sort keys %d) {
    print "$blk $std_dev{$blk} " . 'O' x int($d{$blk}/$avg_diff*50) . ", $t{$blk} (" . int($est_diff_inc_needed{$blk}*100)/100 . ") $d{$blk} $neg{$blk}\n";
    print F "$blk $t{$blk} $d{$blk} $std_dev{$blk}\n";
}
close F;
print "avg difficulty: " . int($avg_diff) ."\n";
print "avg solvetime: " . int($avg_t) ."\n";
print $instructions;