DATA MINING WITH SUPERALGORES

Old pots produce the best soup.

Science has always inspired indicator design. From physics to signal processing, many indicators use concepts from mechanical engineering, electronics, and probability. In Superalgos' Data Mining section, we've explored using thermodynamics and information theory to construct indicators and using statistical and probabilistic techniques like reduced normal law to take advantage of low probability events.

An asset's price is like a mechanical object revolving around its moving average. Using this approach, we could design an indicator using the oscillator's Total Energy. An oscillator's energy is finite and constant. Since we don't expect the price to follow the harmonic oscillator, this energy should deviate from the perfect situation, and the maximum of divergence may provide us valuable information on the price's moving average.

Definition of the Harmonic Oscillator in Few Words

Sinusoidal function describes a harmonic oscillator. The time-constant energy equation for a harmonic oscillator is:

With

Time saves energy.

In a mechanical harmonic oscillator, total energy equals kinetic energy plus potential energy. The formula for energy is the same for every kind of harmonic oscillator; only the terms of total energy must be adapted to fit the relevant units. Each oscillator has a velocity component (kinetic energy) and a position to equilibrium component (potential energy).

The Price Oscillator and the Energy Formula

Considering the harmonic oscillator definition, we must specify kinetic and potential components for our price oscillator. We define oscillator velocity as the rate of change and equilibrium position as the price's distance from its moving average.

Price kinetic energy:

It's like:

With

and

L is the number of periods for the rate of change calculation and P for the close price EMA calculation.

Total price oscillator energy =

Given that an asset's price can theoretically vary at a limitless speed and be endlessly far from its moving average, we don't expect this formula's outcome to be constrained. We'll normalize it using Z-Score for convenience of usage and readability, which also allows probabilistic interpretation.

Over 20 periods, we'll calculate E's moving average and standard deviation.

We calculated Z on BTC/USDT with L = 10 and P = 21 using Knime Analytics.

The graph is detrended. We added two horizontal lines at +/- 1.6 to construct a 94.5% probability zone based on reduced normal law tables. Price cycles to its moving average oscillate clearly. Red and green arrows illustrate where the oscillator crosses the top and lower limits, corresponding to the maximum/minimum price oscillation. Since the results seem noisy, we may apply a non-lagging low-pass or multipole filter like Butterworth or Laguerre filters and employ dynamic bands at a multiple of Z's standard deviation instead of fixed levels.

Kinetic Detrender Implementation in Superalgos

The Superalgos Kinetic detrender features fixed upper and lower levels and dynamic volatility bands.

The code is pretty basic and does not require a huge amount of code lines.

It starts with the standard definitions of the candle pointer and the constant declaration :

let candle = record.current
let len = 10
let P = 21
let T = 20
let up = 1.6
let low = 1.6

Upper and lower dynamic volatility band constants are up and low.

We proceed to the initialization of the previous value for EMA :

if (variable.prevEMA === undefined) {
    variable.prevEMA = candle.close
}

And the calculation of EMA with a function (it is worth noticing the function is declared at the end of the code snippet in Superalgos) :

variable.ema = calculateEMA(P, candle.close, variable.prevEMA)
//EMA calculation
function calculateEMA(periods, price, previousEMA) {
    let k = 2 / (periods + 1)
    return price * k + previousEMA * (1 - k)
}

The rate of change is calculated by first storing the right amount of close price values and proceeding to the calculation by dividing the current close price by the first member of the close price array:

variable.allClose.push(candle.close)
if (variable.allClose.length > len) {
    variable.allClose.splice(0, 1)
}
if (variable.allClose.length === len) {
    variable.roc = candle.close / variable.allClose[0]
} else {
    variable.roc = 1
}

Finally, we get energy with a single line:

variable.E = 1 / 2 * len * variable.roc + 1 / 2 * P * candle.close / variable.ema

The Z calculation reuses code from Z-Normalization-based indicators:

variable.allE.push(variable.E)
if (variable.allE.length > T) {
    variable.allE.splice(0, 1)
}
variable.sum = 0
variable.SQ = 0
if (variable.allE.length === T) {
    for (var i = 0; i < T; i++) {
        variable.sum += variable.allE[i]
    }
    variable.MA = variable.sum / T
for (var i = 0; i < T; i++) {
        variable.SQ += Math.pow(variable.allE[i] - variable.MA, 2)
    }
    variable.sigma = Math.sqrt(variable.SQ / T)
variable.Z = (variable.E - variable.MA) / variable.sigma
} else {
    variable.Z = 0
}
variable.allZ.push(variable.Z)
if (variable.allZ.length > T) {
    variable.allZ.splice(0, 1)
}
variable.sum = 0
variable.SQ = 0
if (variable.allZ.length === T) {
    for (var i = 0; i < T; i++) {
        variable.sum += variable.allZ[i]
    }
    variable.MAZ = variable.sum / T
for (var i = 0; i < T; i++) {
        variable.SQ += Math.pow(variable.allZ[i] - variable.MAZ, 2)
    }
    variable.sigZ = Math.sqrt(variable.SQ / T)
} else {
    variable.MAZ = variable.Z
    variable.sigZ = variable.MAZ * 0.02
}
variable.upper = variable.MAZ + up * variable.sigZ
variable.lower = variable.MAZ - low * variable.sigZ

We also update the EMA value.

variable.prevEMA = variable.EMA

Conclusion

We showed how to build a detrended oscillator using simple harmonic oscillator theory. Kinetic detrender's main line oscillates between 2 fixed levels framing 95% of the values and 2 dynamic levels, leading to auto-adaptive mean reversion zones.

Superalgos' Normalized Momentum data mine has the Kinetic detrender indication.

All the material here can be reused and integrated freely by linking to this article and Superalgos.

This post is informative and not financial advice. Seek expert counsel before trading. Risk using this material.

Mainstream economics haven't noticed.

What come to mind when I say the word "economics"?

Probably GDP, unemployment, and inflation.

If you've ever watched the news or listened to an economist, they'll use data like these to defend a political goal.

The issue is that these statistics are total bunk.

I'm being provocative, but I mean it:

• The economy is not measured by GDP.

• How many people are unemployed is not counted in the unemployment rate.

• Inflation is not measured by the CPI.

All orthodox economists' major economic statistics are either wrong or falsified.

Government institutions create all these stats. The administration wants to reassure citizens the economy is doing well.

GDP does not reflect economic expansion.

GDP measures a country's economic size and growth. It’s calculated by the BEA, a government agency.

The US has the world's largest (self-reported) GDP, growing 2-3% annually.

If GDP rises, the economy is healthy, say economists.

Why is the GDP flawed?

GDP measures a country's yearly spending.

The government may adjust this to make the economy look good.

GDP = C + G + I + NX

C = Consumer Spending

G = Government Spending

I = Investments (Equipment, inventories, housing, etc.)

NX = Exports minus Imports

GDP is a country's annual spending.

The government can print money to boost GDP. The government has a motive to increase and manage GDP.

Because government expenditure is part of GDP, printing money and spending it on anything will raise GDP.

They've done this. Since 1950, US government spending has grown 8% annually, faster than GDP.

In 2022, government spending accounted for 44% of GDP. It's the highest since WWII. In 1790-1910, it was 3% of GDP.

Who cares?

The economy isn't only spending. Focus on citizens' purchasing power or quality of life.

Since GDP just measures spending, the government can print money to boost GDP.

Even if Americans are poorer than last year, economists can say GDP is up and everything is fine.

How many people are unemployed is not counted in the unemployment rate.

The unemployment rate measures a country's labor market. If unemployment is high, people aren't doing well economically.

The BLS estimates the (self-reported) unemployment rate as 3-4%.

Why is the unemployment rate so high?

The US government surveys 100k persons to measure unemployment. They extrapolate this data for the country.

They come into 3 categories:

• Employed

People with jobs are employed … duh.

• Unemployed

People who are “jobless, looking for a job, and available for work” are unemployed

• Not in the labor force

The “labor force” is the employed + the unemployed.

The unemployment rate is the percentage of unemployed workers.

Problem is unemployed definition. You must actively seek work to be considered unemployed.

You're no longer unemployed if you haven't interviewed in 4 weeks.

This shit makes no goddamn sense.

Why does this matter?

You can't interview if there are no positions available. You're no longer unemployed after 4 weeks.

In 1994, the BLS redefined "unemployed" to exclude discouraged workers.

If you haven't interviewed in 4 weeks, you're no longer counted in the unemployment rate.

If unemployment were measured by total unemployed, it would be 25%.

Because the government wants to keep the unemployment rate low, they modify the definition.

If every US resident was unemployed and had no job interviews, economists would declare 0% unemployment. Excellent!

Inflation is not measured by the CPI.

The BLS measures CPI. This month was the highest since 1981.

CPI measures the cost of a basket of products across time. Food, energy, shelter, and clothes are included.

A 9.1% CPI means the basket of items is 9.1% more expensive.

What is the CPI problem?

Here's a more detailed explanation of CPI's flaws.

In summary, CPI is manipulated to be understated.

Housing costs are understated to manipulate CPI. Housing accounts for 33% of the CPI because it's the biggest expense for most people.

This signifies it's the biggest CPI weight.

Rather than using actual house prices, the Bureau of Labor Statistics essentially makes shit up. You can read more about the process here.

Surprise! It’s bullshit

The BLS stated Shelter's price rose 5.5% this month.

House prices are up 11-21%. (Source 1, Source 2, Source 3)

Rents are up 14-26%. (Source 1, Source 2)

Why is this important?

If CPI included housing prices, it would be 12-15 percent this month, not 9.1 percent.

9% inflation is nuts. Your money's value halves every 7 years at 9% inflation.

Worse is 15% inflation. Your money halves every 4 years at 15% inflation.

If everyone realized they needed to double their wage every 4-5 years to stay wealthy, there would be riots.

Inflation drains our money's value so the government can keep printing it.

The Solution

Most individuals know the existing system doesn't work, but can't explain why.

People work hard yet lag behind. The government lies about the economy's data.

In reality:

• GDP has been down since 2008

• 25% of Americans are unemployed

• Inflation is actually 15%

People might join together to vote out kleptocratic politicians if they knew the reality.

Having reliable economic data is the first step.

People can't understand the situation without sufficient information. Instead of immigrants or billionaires, people would blame liar politicians.

Here’s the vision:

A decentralized, transparent, and global dashboard that tracks economic data like GDP, unemployment, and inflation for every country on Earth.

Government incentives influence economic statistics.

ShadowStats has already started this effort, but the calculations must be transparent, decentralized, and global to be effective.

If interested, email me at trevorstark02@gmail.com.

Here are some links to further your research:

1. MIT Billion Prices Project

2. 1729 Decentralized Inflation Dashboard Project

3. Balaji Srinivasan on “Fiat Information VS. Crypto Information”

I've spent over 50 hours researching one-person firms, which interest me. I've found countless one-person enterprises that made millions on the founder's determination and perseverance.

Throughout my investigation, I found three of the most outstanding one-person enterprises. These enterprises show that people who work hard and dedicate themselves to their ideas may succeed.

Eric Barone (@ConcernedApe) created Stardew Valley in 2011 to better his job prospects. Eric loved making the game, in which players inherit a farm, grow crops, raise livestock, make friends with the villagers, and form a family.

Eric handled complete game production, including 3D graphics, animations, and music, to maintain creative control. He stopped job hunting and worked 8-15 hours a day on the game.

Eric developed a Stardew Valley website and subreddit to engage with gamers and get feedback. Eric's devoted community helped him meet Steam's minimum vote requirement for single creators.

Stardew Valley sold 1 million copies in two months after Eric launched it for $15 in 2016. The game has sold 20 million copies and made $300 million.

The game's inexpensive price, outsourcing of PR, marketing, and publication, and loyal player base helped it succeed. Eric has turned down million-dollar proposals from Sony and Nintendo to sell the game and instead updates and improves it. Haunted Chocolatier is Eric's new game.

Is farming not profitable? Ask Stardew Valley creator Eric Barone.

Gary Brewer established BuiltWith to assist users find website technologies and services. BuiltWith boasts 3000 paying customers and $14 million in yearly revenue, making it a significant resource for businesses wishing to generate leads, do customer analytics, obtain business insight, compare websites, or search websites by keyword.

BuiltWith has one full-time employee, Gary, and one or two part-time contractors that help with the blog. Gary handles sales, customer service, and other company functions alone.

BuiltWith acquired popularity through blog promotions and a top Digg ranking. About Us, a domain directory, connected to BuiltWith on every domain page, boosting it. Gary introduced $295–$995 monthly subscriptions to search technology, keywords, and potential consumers in response to customer demand.

Gary uses numerous methods to manage a firm without staff. He spends one to two hours every day answering user queries, most of which are handled quickly by linking to BuiltWiths knowledge store. Gary creates step-by-step essays or videos for complex problems. Gary can focus on providing new features based on customer comments and requests since he makes it easy to unsubscribe.

BuiltWith is entirely automated and successful due to its unique approach and useful offerings. It works for Google, Meta, Amazon, and Twitter.

Digital Inspiration develops Google Documents, Sheets, and Slides plugins. Digital Inspiration, founded by Amit Agarwal, receives 5 million monthly visits and earns $10 million. 40 million individuals have downloaded Digital Inspirations plugins.

Amit started Digital Inspiration by advertising his blog at tech events and getting Indian filter blogs and other newspapers to promote his articles. Amit built plugins and promoted them on the blog once the blog acquired popularity, using ideas from comments, friends, and Reddit. Digital Inspiration has over 20 free and premium plugins.

Mail Merge, Notifications for Google Forms, YouTube Uploader, and Document Studio are some of Digital Inspiration's most popular plugins. Mail Merge allows users to send personalized emails in bulk and track email opens and clicks.

Since Amits manages Digital Inspiration alone, his success is astounding. Amit developed a successful company via hard work and creativity, despite platform dependence. His tale inspires entrepreneurs.

The holy grail of plain text task management is finally within reach

Plain text task management? Are you serious?? Dedicated task managers exist for a reason, you know. Sheesh.

—Oh, I know. Believe me, I know! But hear me out.

I've managed projects and tasks in plain text for more than four years. Since reorganizing my to-do list, plain text task management is within reach.

Data completely yours? One billion percent. Beef it up with coding? Be my guest.

Enter: The List

The answer? A list. That’s it!

Write down tasks. Obsidian, Notenik, Drafts, or iA Writer are good plain text note-taking apps.

List too long? Of course, it is! A large list tells you what to do. Feel the itch and friction. Then fix it.

• But I want to be able to distinguish between work and personal life! List two things.

• However, I need to know what should be completed first. Put those items at the top.

• However, some things keep coming up, and I need to be reminded of them! Put those in your calendar and make an alarm for them.

• But since individual X hasn't completed task Y, I can't proceed with this. Create a Waiting section on your list by dividing it.

• But I must know what I'm supposed to be doing right now! Read your list(s). Check your calendar. Think critically.

Before I begin a new one, I remind myself that "Listory Never Repeats."

There’s no such thing as too many lists if all are needed. There is such a thing as too many lists if you make them before they’re needed. Before they complain that their previous room was small or too crowded or needed a new light.

A list that feels too long has a voice; it’s telling you what to do next.

I use one Master List. It's a control panel that tells me what to focus on short-term. If something doesn't need semi-immediate attention, it goes on my Backlog list.

Todd Lewandowski's DWTS (Done, Waiting, Top 3, Soon) performance deserves praise. His DWTS to-do list structure has transformed my plain-text task management. I didn't realize it was upside down.

This is my take on it:

D = Done

Move finished items here. If they pile up, clear them out every week or month. I have a Done Archive folder.

W = Waiting

Things seething in the background, awaiting action. Stir them occasionally so they don't burn.

T = Top 3

Three priorities. Personal comes first, then work. There will always be a top 3 (no more than 5) in every category. Projects, not chores, usually.

S = Soon

This part is action-oriented. It's for anything you can accomplish to finish one of the Top 3. This collection includes thoughts and project lists. The sole requirement is that they should be short-term goals.

Some of you have probably concluded this isn't for you. Please read Todd's piece before throwing out the baby. Often. You shouldn't miss a newborn.

As much as Dancing With The Stars helps me recall this method, I may try switching their order. TSWD; Drilling Tunnel Seismic? Serenity After Task?

Master List Showcase

My Master List lives alone in its own file, but sometimes appears in other places.  It's included in my Weekly List template. Here's a (soon-to-be-updated) demo vault of my Obsidian planning setup to download for free.

Here's the code behind my weekly screenshot:

## [[Master List - 2022|✓]]  TO DO

![[Master List - 2022]]

FYI, I use the Minimal Theme in Obsidian, with a few tweaks.

You may note I'm utilizing a checkmark as a link. For me, that's easier than locating the proper spot to click on the embed.

Blue headings for Done and Waiting are links. Done links to the Done Archive page and Waiting to a general waiting page.

Read my full article here.