They depend heavily on open source..

Discord users send billions of messages daily. Users wish to search these messages. How do we index these to search by message keywords?

Let’s find out.

1. Discord utilizes Elasticsearch. Elasticsearch is a free, open search engine for textual, numerical, geographical, structured, and unstructured data. Apache Lucene powers Elasticsearch.
   

2. How does elastic search store data? It stores it as numerous key-value pairs in JSON documents.
   

3. How does elastic search index? Elastic search's index is inverted. An inverted index lists every unique word in every page and where it appears.

4. Elasticsearch indexes documents and generates an inverted index to make data searchable in near real-time. The index API adds or updates JSON documents in a given index.

1. Let's examine how discord uses Elastic Search. Elasticsearch prefers bulk indexing. Discord couldn't index real-time messages. You can't search posted messages. You want outdated messages.

6. Let's check what bulk indexing requires.
1. A temporary queue for incoming communications.
2. Indexer workers that index messages into elastic search.

1. Discord's queue is Celery. The queue is open-source. Elastic search won't run on a single server. It's clustered. Where should a message go? Where?

8. A shard allocator decides where to put the message. Nevertheless. Shattered? A shard combines elastic search and index on. So, these two form a shard which is used as a unit by discord. The elastic search itself has some shards. But this is different, so don’t get confused.

1. Now, the final part is service discovery — to discover the elastic search clusters and the hosts within that cluster. This, they do with the help of etcd another open source tool.
   

A great thing to notice here is that discord relies heavily on open source systems and their base implementations which is very different from a lot of other products.

TOP APPS

Who survives spring cleaning?

Relax. Notion is off-limits. This topic is popular.

(I wrote about it 2 years ago, before everyone else did.) So).

These apps are probably new to you. I hope you find a new phone app after reading this.

Outdooractive

ViewRanger is Google Maps for outdoor enthusiasts.

This app has been so important to me as a freedom-loving long-distance walker and hiker.

This app shows nearby trails and right-of-ways on top of an Open Street Map.

Helpful detail and data. Any route's distance,

You can download and follow tons of routes planned by app users.

This has helped me find new routes and places a fellow explorer has tried.

Free with non-intrusive ads. Years passed before I subscribed. Pro costs £2.23/month.

This app is for outdoor lovers.

Google Files

New phones come with bloatware. These rushed apps are frustrating.

We must replace these apps. 2017 was Google's year.

Files is a file manager. It's quick, innovative, and clean. They've given people what they want.

It's easy to organize files, clear space, and clear cache.

I recommend Gallery by Google as a gallery app alternative. It's quick and easy.

Trainline

App for trains, buses, and coaches.

I've used this app for years. It did the basics well when I first used it.

Since then, it's improved. It's constantly adding features to make traveling easier and less stressful.

Split-ticketing helps me save hundreds a year on train fares. This app is only available in the UK and Europe.

This service doesn't link to a third-party site. Their app handles everything.

Not all train and coach companies use this app. All the big names are there, though.

Here's more on the app.

Battlefield: Mobile

Play Store has 478,000 games. Few can turn my phone into a console.

Call of Duty Mobile and Asphalt 8/9 are examples.

Asphalt's loot boxes and ads make it unplayable. Call of Duty opens with a few ads. Close them to play without hassle.

This game uses all your phone's features to provide a high-quality, seamless experience. If my internet connection is good, I never experience lag or glitches.

The gameplay is energizing and intense, just like on consoles. Sometimes I'm too involved. I've thrown my phone in anger. I'm totally absorbed.

Customizability is my favorite. Since phones have limited screen space, we should only have the buttons we need, placed conveniently.

Size, opacity, and position are modifiable. Adjust audio, graphics, and textures. It's customizable.

This game has been on my phone for three years. It began well and has gotten better. When I think the creators can't do more, they do.

If you play, read my tips for winning a Battle Royale.

Lightroom

As a photographer, I believe your best camera is on you. The phone.

2017 was a big year for this app. I've tried many photo-editing apps since then. This always wins.

The app is dull. I've never seen better photo editing on a phone.

Adjusting settings and sliders doesn't damage or compress photos. It's detailed.

This is important for phone photos, which are lower quality than professional ones.

Some tools are behind a £4.49/month paywall. Adobe must charge a subscription fee instead of selling licenses. (I'm still bitter about Creative Cloud's price)

Snapseed is my pick. Lightroom is where I do basic editing before moving to Snapseed. Snapseed review:

These apps are great. They cover basic and complex editing needs while traveling.

Final Reflections

I hope you downloaded one of these. Share your favorite apps. These apps are scarce.

I was entering tens of millions of records per hour when I first published Slonik PostgreSQL client for Node.js. The data being entered was usually flat, making it straightforward to use INSERT INTO ... SELECT * FROM unnset() pattern. I advocated the unnest approach for inserting rows in groups (that was part I).

However, today I’ve found a better way: jsonb_to_recordset.

jsonb_to_recordset expands the top-level JSON array of objects to a set of rows having the composite type defined by an AS clause.

jsonb_to_recordset allows us to query and insert records from arbitrary JSON, like unnest. Since we're giving JSON to PostgreSQL instead of unnest, the final format is more expressive and powerful.

SELECT *
FROM json_to_recordset('[{"name":"John","tags":["foo","bar"]},{"name":"Jane","tags":["baz"]}]')
AS t1(name text, tags text[]);
 name |   tags
------+-----------
 John | {foo,bar}
 Jane | {baz}
(2 rows)

Let’s demonstrate how you would use it to insert data.

Inserting data using json_to_recordset

Say you need to insert a list of people with attributes into the database.

const persons = [
  {
    name: 'John',
    tags: ['foo', 'bar']
  },
  {
    name: 'Jane',
    tags: ['baz']
  }
];

You may be tempted to traverse through the array and insert each record separately, e.g.

for (const person of persons) {
  await pool.query(sql`
    INSERT INTO person (name, tags)
    VALUES (
      ${person.name},
      ${sql.array(person.tags, 'text[]')}
    )
  `);
}

It's easier to read and grasp when working with a few records. If you're like me and troubleshoot a 2M+ insert query per day, batching inserts may be beneficial.

What prompted the search for better alternatives.

Inserting using unnest pattern might look like this:

await pool.query(sql`
  INSERT INTO public.person (name, tags)
  SELECT t1.name, t1.tags::text[]
  FROM unnest(
    ${sql.array(['John', 'Jane'], 'text')},
    ${sql.array(['{foo,bar}', '{baz}'], 'text')}
  ) AS t1.(name, tags);
`);

You must convert arrays into PostgreSQL array strings and provide them as text arguments, which is unsightly. Iterating the array to create slices for each column is likewise unattractive.

However, with jsonb_to_recordset, we can:

await pool.query(sql`
  INSERT INTO person (name, tags)
  SELECT *
  FROM jsonb_to_recordset(${sql.jsonb(persons)}) AS t(name text, tags text[])
`);

In contrast to the unnest approach, using jsonb_to_recordset we can easily insert complex nested data structures, and we can pass the original JSON document to the query without needing to manipulate it.

In terms of performance they are also exactly the same. As such, my current recommendation is to prefer jsonb_to_recordset whenever inserting lots of rows or nested data structures.

WRITE FOR THE WEB3

Why web3’s messaging is lost and how crypto winter is growing growth seeds

People interested in crypto, blockchain, and web3 typically read Bitcoin and Ethereum's white papers. It's a good idea. Documents produced for developers and academia aren't always the ideal resource for beginners.

Given the surge of extremely technical material and the number of fly-by-nights, rug pulls, and other scams, it's little wonder mainstream audiences regard the blockchain sector as an expensive sideshow act.

What's the solution?

Web3 needs more than just builders.

After joining TikTok, I followed Amy Suto of SutoScience. Amy switched from TV scriptwriting to IT copywriting years ago. She concentrates on web3 now. Decentralized autonomous organizations (DAOs) are seeking skilled copywriters for web3.

Amy has found that web3's basics are easy to grasp; you don't need technical knowledge. There's a paradigm shift in knowing the basics; be persistent and patient.

Apple is positioning itself as a data privacy advocate, leveraging web3's zero-trust ethos on data ownership.

Finn Lobsien, who writes about web3 copywriting for the Mirror and Twitter, agrees: acronyms and abstractions won't do.

Web3 preached to the choir. Curious newcomers have only found whitepapers and scams when trying to learn why the community loves it. No wonder people resist education and buy-in.

Due to the gender gap in crypto (Crypto Bro is not just a stereotype), it attracts people singing to the choir or trying to cash in on the next big thing.

Last year, the industry was booming, so writing wasn't necessary. Now that the bear market has returned (for everyone, but especially web3), holding readers' attention is a valuable skill.

White papers and the Web3

Why does web3 rely so much on non-growth content?

Businesses must polish and improve their messaging moving into the 2022 recession. The 2021 tech boom provided such a sense of affluence and (unsustainable) growth that no one needed great marketing material. The market found them.

This was especially true for web3 and the first-time crypto believers. Obviously. If they knew which was good.

White papers help. White papers are highly technical texts that walk a reader through a product's details. How Does a White Paper Help Your Business and That White Paper Guy discuss them.

They're meant for knowledgeable readers. Investors and the technical (academic/developer) community read web3 white papers. White papers are used when a product is extremely technical or difficult to assist an informed reader to a conclusion. Web3 uses them most often for ICOs (initial coin offerings).

White papers for web3 education help newcomers learn about the web3 industry's components. It's like sending a first-grader to the Annotated Oxford English Dictionary to learn to read. It's a reference, not a learning tool, for words.

Newcomers can use platforms that teach the basics. These included Coinbase's Crypto Basics tutorials or Cryptochicks Academy, founded by the mother of Ethereum's inventor to get more women utilizing and working in crypto.

Discord and Web3 communities

Discord communities are web3's opposite. Discord communities involve personal communications and group involvement.

Online audience growth begins with community building. User personas prefer 1000 dedicated admirers over 1 million lukewarm followers, and the language is much more easygoing. Discord groups are renowned for phishing scams, compromised wallets, and incorrect information, especially since the crypto crisis.

White papers and Discord increase industry insularity. White papers are complicated, and Discord has a high risk threshold.

Web3 and writing ads

Copywriting is emotional, but white papers are logical. It uses the brain's quick-decision centers. It's meant to make the reader invest immediately.

Not bad. People think sales are sleazy, but they can spot the poor things.

Ethical copywriting helps you reach the correct audience. People who gain a following on Medium are likely to have copywriting training and a readership (or three) in mind when they publish. Tim Denning and Sinem Günel know how to identify a target audience and make them want to learn more.

In a fast-moving market, copywriting is less about long-form content like sales pages or blogs, but many organizations do. Instead, the copy is concise, individualized, and high-value. Tweets, email marketing, and IM apps (Discord, Telegram, Slack to a lesser extent) keep engagement high.

What does web3's messaging lack? As DAOs add stricter copyrighting, narrative and connecting tales seem to be missing.

Web3 is passionate about constructing the next internet. Now, they can connect their passion to a specific audience so newcomers understand why.

My growth-killing mistake

When you initially start developing your audience, you need guidance.

What does my audience like? What do they not like? How can I grow more?

When I started writing two years ago, I inquired daily. Taking cues from your audience to develop more valuable content is a good concept, but it's simple to let them destroy your growth.

A small audience doesn't represent the full picture.

When I had fewer than 100 YouTube subscribers, I tried several video styles and topics. I looked to my audience for what to preserve and what to change.

If my views, click-through rate, or average view % dropped, that topic or style was awful. Avoiding that style helped me grow.

Vlogs, talking head videos on writing, and long-form tutorials didn't fare well.

Since I was small, I've limited the types of films I make. I have decided to make my own videos.

Surprisingly, the videos I avoided making meet or exceed my views, CTR, and audience retention.

A limited audience can't tell you what your tribe wants. Therefore, limiting your innovation will prohibit you from reaching the right audience. Finding them may take longer.

Large Creators Experience The Same Issue

In the last two years, I've heard Vanessa Lau and Cathrin Manning say they felt pigeonholed into generating videos they didn't want to do.

Why does this happen over and over again?

Once you have a popular piece of content, your audience will grow. So when you publish inconsistent material, fewer of your new audience will view it. You interpret the drop in views as a sign that your audience doesn't want the content, so you stop making it.

Repeat this procedure a few times, and you'll create stuff you're not passionate about because you're frightened to publish it.

How to Manage Your Creativity and Audience Development

I'm not recommending you generate random content.

Instead of feeling trapped by your audience, you can cultivate a diverse audience.

Create quality material on a range of topics and styles as you improve. Be creative until you get 100 followers. Look for comments on how to improve your article.

If you observe trends in the types of content that expand your audience, focus 50-75% of your material on those trends. Allow yourself to develop 25% non-performing material.

This method can help you expand your audience faster with your primary trends and like all your stuff. Slowly, people will find 25% of your material, which will boost its performance.

How to Expand Your Audience Without Having More Limited Content

Follow these techniques to build your audience without feeling confined.

• Don't think that you need restrict yourself to what your limited audience prefers.

• Don't let the poor performance of your desired material demotivate you.

• You shouldn't restrict the type of content you publish or the themes you cover when you have less than 100 followers.

• When your audience expands, save 25% of your content for your personal interests, regardless of how well it does.

Trigger warning — Andrew Tate is running a genius marketing campaign

Andrew Tate is a 2022 internet celebrity.

Kickboxing world champion became rich playboy with controversial views on gender roles.

Andrew's get-rich-quick scheme isn't new. His social media popularity is impressive.

He’s currently running one of the most genius marketing campaigns in history.

He pulls society's pendulum away from diversity and inclusion and toward diversion and exclusion. He's unstoppable.

Here’s everything you need to know about Andrew Tate. And how he’s playing chess while the world plays checkers.

Cobra Tate is the name he goes by.

American-born, English-raised entrepreneur Andrew Tate lives in Romania.

Romania? Says Andrew,

“I prefer a country in which corruption is available to everyone.”

Andrew was a professional kickboxer with the ring moniker Cobra before starting Hustlers University.

Before that, he liked chess and worshipped his father.

Emory Andrew Tate III is named after his grandmaster chess player father.

Emory was the first black-American chess champion. He was military, martial arts-trained, and multilingual. A superhuman.

He lived in his car to make ends meet.

Andrew and Tristan relocated to England with their mother when their parents split.

It was there that Andrew began his climb toward becoming one of the internet’s greatest villains.

Andrew fell in love with kickboxing.

Andrew spent his 20s as a professional kickboxer and reality TV star, featuring on Big Brother UK and The Ultimate Traveller.

These 3 incidents, along with a chip on his shoulder, foreshadowed Andrews' social media breakthrough.

• Chess

• Combat sports

• Reality television

A dangerous trio.

Andrew started making money online after quitting kickboxing in 2017 due to an eye issue.

Andrew didn't suddenly become popular.

Andrew's web work started going viral in 2022.

Due to his contentious views on patriarchy and gender norms, he's labeled the King of Toxic Masculinity. His most contentious views (trigger warning):

• “Women are intrinsically lazy.”

• “Female promiscuity is disgusting.”

• “Women shouldn’t drive cars or fly planes.”

• “A lot of the world’s problems would be solved if women had their body count tattooed on their foreheads.”

Andrew's two main beliefs are:

1. “These are my personal opinions based on my experiences.”

2. “I believe men are better at some things and women are better at some things. We are not equal.”

Andrew intentionally offends.

Andrew's thoughts began circulating online in 2022.

In July 2022, he was one of the most Googled humans, surpassing:

• Joe Biden

• Donald Trump

• Kim Kardashian

Andrews' rise is a mystery since no one can censure or suppress him. This is largely because Andrew nor his team post his clips.

But more on that later.

Andrew's path to wealth.

Andrew Tate is a self-made millionaire. His morality is uncertain.

Andrew and Tristan needed money soon after retiring from kickboxing.

“I owed some money to some dangerous people. I had $70K and needed $100K to stay alive.”

Andrews lost $20K on roulette at a local casino.

Andrew had one week to make $50,000, so he started planning. Andrew locked himself in a chamber like Thomas Edison to solve an energy dilemma.

He listed his assets.

• Physical strength (but couldn’t fight)

• a BMW (worth around $20K)

• Intelligence (but no outlet)

A lightbulb.

He had an epiphany after viewing a webcam ad. He sought aid from women, ironically. His 5 international girlfriends are assets.

Then, a lightbulb.

Andrew and Tristan messaged and flew 7 women to a posh restaurant. Selling desperation masked as opportunity, Andrew pitched his master plan:

A webcam business — with a 50/50 revenue split.

5 women left.

2 stayed.

Andrew Tate, a broke kickboxer, became Top G, Cobra Tate.

The business model was simple — yet sad.

Andrew's girlfriends moved in with him and spoke online for 15+ hours a day. Andrew handled ads and equipment as the women posed.

Andrew eventually took over their keyboards, believing he knew what men wanted more than women.

Andrew detailed on the Full Send Podcast how he emotionally manipulated men for millions. They sold houses, automobiles, and life savings to fuel their companionship addiction.

When asked if he felt bad, Andrew said,

“F*ck no.“

Andrew and Tristan wiped off debts, hired workers, and diversified.

Tristan supervised OnlyFans models.

Andrew bought Romanian casinos and MMA league RXF (Real Xtreme Fighting).

Pandemic struck suddenly.

Andrew couldn't run his 2 businesses without a plan. Another easy moneymaker.

He banked on Hustlers University.

The actual cause of Andrew's ubiquity.

On a Your Mom’s House episode Andrew's 4 main revenue sources:

1. Hustler’s University

2. Owning casinos in Romania

3. Owning 10% of the Romanian MMA league “RXF”

4. “The War Room” — a society of rich and powerful men

When the pandemic hit, 3/4 became inoperable.

So he expanded Hustlers University.

But what is Hustler’s University?

Andrew says Hustlers University teaches 18 wealth-building tactics online. Examples:

• Real estate

• Copywriting

• Amazon FBA

• Dropshipping

• Flipping Cryptos

How to swiftly become wealthy.

Lessons are imprecise, rudimentary, and macro-focused, say reviews. Invest wisely, etc. Everything is free online.

You pay for community. One unique income stream.

The only money-making mechanism that keeps the course from being a scam.

The truth is, many of Andrew’s students are actually making money. Maybe not from the free YouTube knowledge Andrew and his professors teach in the course, but through Hustler’s University’s affiliate program.

Affiliates earn 10% commission for each new student = $5.

Students can earn $10 for each new referral in the first two months.

Andrew earns $50 per membership per month.

This affiliate program isn’t anything special — in fact, it’s on the lower end of affiliate payouts. Normally, it wouldn’t be very lucrative.

But it has one secret weapon— Andrew and his viral opinions.

Andrew is viral. Andrew went on a media tour in January 2022 after appearing on Your Mom's House.

• The Full Send Podcast

• The Fresh & Fit Podcast

• “BFFs” by Barstool Sports

And many, many more…

He chatted with Twitch streamers. Hustlers University wanted more controversy (and clips).

Here’s the strategy behind Hustler’s University that has (allegedly) earned students upwards of $10K per month:

1. Make a social media profile with Andrew Tates' name and photo.

2. Post any of the online videos of Andrews that have gone viral.

3. Include a referral link in your bio.

Effectively simple.

Andrew's controversy attracts additional students. More student clips circulate as more join. Andrew's students earn more and promote the product as he goes viral.

A brilliant plan that's functioning.

At the beginning of his media tour, Hustler’s University had 5,000 students. 6 months in, and he now has over 100,000.

One income stream generates $5 million every month.

Andrew's approach is not new.

But it is different.

In the early 2010s, Tai Lopez dominated the internet.

His viral video showed his house.

“Here in my garage. Just bought this new Lamborghini.”

Tais' marketing focused on intellect, not strength, power, and wealth to attract women.

How reading quicker leads to financial freedom in 67 steps.

Years later, it was revealed that Tai Lopez rented the mansion and Lamborghini as a marketing ploy to build social proof. Meanwhile, he was living in his friend’s trailer.

Faked success is an old tactic.

Andrew is doing something similar. But with one major distinction.

Andrew outsources his virality — making him nearly impossible to cancel.

In 2022, authorities searched Andrews' estate over human trafficking suspicions. Investigation continues despite withdrawn charges.

Andrew's divisive nature would normally get him fired. Andrew's enterprises and celebrity don't rely on social media.

He doesn't promote or pay for ads. Instead, he encourages his students and anyone wishing to get rich quick to advertise his work.

Because everything goes through his affiliate program. Old saying:

“All publicity is good publicity.”

Final thoughts: it’s ok to feel triggered.

Tate is divisive.

His emotionally charged words are human nature. Andrews created the controversy.

It's non-personal.

His opinions are those of one person. Not world nor generational opinion.

Briefly:

• It's easy to understand why Andrews' face is ubiquitous. Money.

• The world wide web is a chessboard. Misdirection is part of it.

• It’s not personal, it’s business.

• Controversy sells

Sometimes understanding the ‘why’, can help you deal with the ‘what.’