Integrity
Write
Loading...
rekt

rekt

3 years ago

LCX is the latest CEX to have suffered a private key exploit.

The attack began around 10:30 PM +UTC on January 8th.

Peckshield spotted it first, then an official announcement came shortly after.

We’ve said it before; if established companies holding millions of dollars of users’ funds can’t manage their own hot wallet security, what purpose do they serve?

The Unique Selling Proposition (USP) of centralised finance grows smaller by the day.

The official incident report states that 7.94M USD were stolen in total, and that deposits and withdrawals to the platform have been paused.

LCX hot wallet: 0x4631018f63d5e31680fb53c11c9e1b11f1503e6f

Hacker’s wallet: 0x165402279f2c081c54b00f0e08812f3fd4560a05

Stolen funds:

  • 162.68 ETH (502,671 USD)
  • 3,437,783.23 USDC (3,437,783 USD)
  • 761,236.94 EURe (864,840 USD)
  • 101,249.71 SAND Token (485,995 USD)
  • 1,847.65 LINK (48,557 USD)
  • 17,251,192.30 LCX Token (2,466,558 USD)
  • 669.00 QNT (115,609 USD)
  • 4,819.74 ENJ (10,890 USD)
  • 4.76 MKR (9,885 USD)

**~$1M worth of $LCX remains in the address, along with 611k EURe which has been frozen by Monerium.

The rest, a total of 1891 ETH (~$6M) was sent to Tornado Cash.**

Why can’t they keep private keys private?

Is it really that difficult for a traditional corporate structure to maintain good practice?

CeFi hacks leave us with little to say - we can only go on what the team chooses to tell us.

Next time, they can write this article themselves.

See below for a template.

More on Web3 & Crypto

Yusuf Ibrahim

Yusuf Ibrahim

4 years ago

How to sell 10,000 NFTs on OpenSea for FREE (Puppeteer/NodeJS)

So you've finished your NFT collection and are ready to sell it. Except you can't figure out how to mint them! Not sure about smart contracts or want to avoid rising gas prices. You've tried and failed with apps like Mini mouse macro, and you're not familiar with Selenium/Python. Worry no more, NodeJS and Puppeteer have arrived!

Learn how to automatically post and sell all 1000 of my AI-generated word NFTs (Nakahana) on OpenSea for FREE!

My NFT project — Nakahana |

NOTE: Only NFTs on the Polygon blockchain can be sold for free; Ethereum requires an initiation charge. NFTs can still be bought with (wrapped) ETH.

If you want to go right into the code, here's the GitHub link: https://github.com/Yusu-f/nftuploader

Let's start with the knowledge and tools you'll need.

What you should know

You must be able to write and run simple NodeJS programs. You must also know how to utilize a Metamask wallet.

Tools needed

  • NodeJS. You'll need NodeJs to run the script and NPM to install the dependencies.
  • Puppeteer – Use Puppeteer to automate your browser and go to sleep while your computer works.
  • Metamask – Create a crypto wallet and sign transactions using Metamask (free). You may learn how to utilize Metamask here.
  • Chrome – Puppeteer supports Chrome.

Let's get started now!

Starting Out

Clone Github Repo to your local machine. Make sure that NodeJS, Chrome, and Metamask are all installed and working. Navigate to the project folder and execute npm install. This installs all requirements.

Replace the “extension path” variable with the Metamask chrome extension path. Read this tutorial to find the path.

Substitute an array containing your NFT names and metadata for the “arr” variable and the “collection_name” variable with your collection’s name.

Run the script.

After that, run node nftuploader.js.

Open a new chrome instance (not chromium) and Metamask in it. Import your Opensea wallet using your Secret Recovery Phrase or create a new one and link it. The script will be unable to continue after this but don’t worry, it’s all part of the plan.

Next steps

Open your terminal again and copy the route that starts with “ws”, e.g. “ws:/localhost:53634/devtools/browser/c07cb303-c84d-430d-af06-dd599cf2a94f”. Replace the path in the connect function of the nftuploader.js script.

const browser = await puppeteer.connect({ browserWSEndpoint: "ws://localhost:58533/devtools/browser/d09307b4-7a75-40f6-8dff-07a71bfff9b3", defaultViewport: null });

Rerun node nftuploader.js. A second tab should open in THE SAME chrome instance, navigating to your Opensea collection. Your NFTs should now start uploading one after the other! If any errors occur, the NFTs and errors are logged in an errors.log file.

Error Handling

The errors.log file should show the name of the NFTs and the error type. The script has been changed to allow you to simply check if an NFT has already been posted. Simply set the “searchBeforeUpload” setting to true.

We're done!

If you liked it, you can buy one of my NFTs! If you have any concerns or would need a feature added, please let me know.

Thank you to everyone who has read and liked. I never expected it to be so popular.

Coinbase

Coinbase

4 years ago

10 Predictions for Web3 and the Cryptoeconomy for 2022

By Surojit Chatterjee, Chief Product Officer

2021 proved to be a breakout year for crypto with BTC price gaining almost 70% yoy, Defi hitting $150B in value locked, and NFTs emerging as a new category. Here’s my view through the crystal ball into 2022 and what it holds for our industry:

1. Eth scalability will improve, but newer L1 chains will see substantial growth — As we welcome the next hundred million users to crypto and Web3, scalability challenges for Eth are likely to grow. I am optimistic about improvements in Eth scalability with the emergence of Eth2 and many L2 rollups. Traction of Solana, Avalanche and other L1 chains shows that we’ll live in a multi-chain world in the future. We’re also going to see newer L1 chains emerge that focus on specific use cases such as gaming or social media.

2. There will be significant usability improvements in L1-L2 bridges — As more L1 networks gain traction and L2s become bigger, our industry will desperately seek improvements in speed and usability of cross-L1 and L1-L2 bridges. We’re likely to see interesting developments in usability of bridges in the coming year.

3. Zero knowledge proof technology will get increased traction — 2021 saw protocols like ZkSync and Starknet beginning to get traction. As L1 chains get clogged with increased usage, ZK-rollup technology will attract both investor and user attention. We’ll see new privacy-centric use cases emerge, including privacy-safe applications, and gaming models that have privacy built into the core. This may also bring in more regulator attention to crypto as KYC/AML could be a real challenge in privacy centric networks.

4. Regulated Defi and emergence of on-chain KYC attestation — Many Defi protocols will embrace regulation and will create separate KYC user pools. Decentralized identity and on-chain KYC attestation services will play key roles in connecting users’ real identity with Defi wallet endpoints. We’ll see more acceptance of ENS type addresses, and new systems from cross chain name resolution will emerge.

5. Institutions will play a much bigger role in Defi participation — Institutions are increasingly interested in participating in Defi. For starters, institutions are attracted to higher than average interest-based returns compared to traditional financial products. Also, cost reduction in providing financial services using Defi opens up interesting opportunities for institutions. However, they are still hesitant to participate in Defi. Institutions want to confirm that they are only transacting with known counterparties that have completed a KYC process. Growth of regulated Defi and on-chain KYC attestation will help institutions gain confidence in Defi.

6. Defi insurance will emerge — As Defi proliferates, it also becomes the target of security hacks. According to London-based firm Elliptic, total value lost by Defi exploits in 2021 totaled over $10B. To protect users from hacks, viable insurance protocols guaranteeing users’ funds against security breaches will emerge in 2022.

7. NFT Based Communities will give material competition to Web 2.0 social networks — NFTs will continue to expand in how they are perceived. We’ll see creator tokens or fan tokens take more of a first class seat. NFTs will become the next evolution of users’ digital identity and passport to the metaverse. Users will come together in small and diverse communities based on types of NFTs they own. User created metaverses will be the future of social networks and will start threatening the advertising driven centralized versions of social networks of today.

8. Brands will start actively participating in the metaverse and NFTs — Many brands are realizing that NFTs are great vehicles for brand marketing and establishing brand loyalty. Coca-Cola, Campbell’s, Dolce & Gabbana and Charmin released NFT collectibles in 2021. Adidas recently launched a new metaverse project with Bored Ape Yacht Club. We’re likely to see more interesting brand marketing initiatives using NFTs. NFTs and the metaverse will become the new Instagram for brands. And just like on Instagram, many brands may start as NFT native. We’ll also see many more celebrities jumping in the bandwagon and using NFTs to enhance their personal brand.

9. Web2 companies will wake up and will try to get into Web3 — We’re already seeing this with Facebook trying to recast itself as a Web3 company. We’re likely to see other big Web2 companies dipping their toes into Web3 and metaverse in 2022. However, many of them are likely to create centralized and closed network versions of the metaverse.

10. Time for DAO 2.0 — We’ll see DAOs become more mature and mainstream. More people will join DAOs, prompting a change in definition of employment — never receiving a formal offer letter, accepting tokens instead of or along with fixed salaries, and working in multiple DAO projects at the same time. DAOs will also confront new challenges in terms of figuring out how to do M&A, run payroll and benefits, and coordinate activities in larger and larger organizations. We’ll see a plethora of tools emerge to help DAOs execute with efficiency. Many DAOs will also figure out how to interact with traditional Web2 companies. We’re likely to see regulators taking more interest in DAOs and make an attempt to educate themselves on how DAOs work.

Thanks to our customers and the ecosystem for an incredible 2021. Looking forward to another year of building the foundations for Web3. Wagmi.

JEFF JOHN ROBERTS

3 years ago

What just happened in cryptocurrency? A plain-English Q&A about Binance's FTX takedown.

Crypto people have witnessed things. They've seen big hacks, mind-boggling swindles, and amazing successes. They've never seen a day like Tuesday, when the world's largest crypto exchange murdered its closest competition.

Here's a primer on Binance and FTX's lunacy and why it matters if you're new to crypto.

What happened?

CZ, a shrewd Chinese-Canadian billionaire, runs Binance. FTX, a newcomer, has challenged Binance in recent years. SBF (Sam Bankman-Fried)—a young American with wild hair—founded FTX (initials are a thing in crypto).

Last weekend, CZ complained about SBF's lobbying and then exploited Binance's market power to attack his competition.

How did CZ do that?

CZ invested in SBF's new cryptocurrency exchange when they were friends. CZ sold his investment in FTX for FTT when he no longer wanted it. FTX clients utilize those tokens to get trade discounts, although they are less liquid than Bitcoin.

SBF made a mistake by providing CZ just too many FTT tokens, giving him control over FTX. It's like Pepsi handing Coca-Cola a lot of stock it could sell at any time. CZ got upset with SBF and flooded the market with FTT tokens.

SBF owns a trading fund with many FTT tokens, therefore this was catastrophic. SBF sought to defend FTT's worth by selling other assets to buy up the FTT tokens flooding the market, but it didn't succeed, and as FTT's value plummeted, his liabilities exceeded his assets. By Tuesday, his companies were insolvent, so he sold them to his competition.

Crazy. How could CZ do that?

CZ likely did this to crush a rising competition. It was also personal. In recent months, regulators have been tough toward the crypto business, and Binance and FTX have been trying to stay on their good side. CZ believed SBF was poisoning U.S. authorities by saying CZ was linked to China, so CZ took retribution.

“We supported previously, but we won't pretend to make love after divorce. We're neutral. But we won't assist people that push against other industry players behind their backs," CZ stated in a tragic tweet on Sunday. He crushed his rival's company two days later.

So does Binance now own FTX?

No. Not yet. CZ has only stated that Binance signed a "letter of intent" to acquire FTX. CZ and SBF say Binance will protect FTX consumers' funds.

Who’s to blame?

You could blame CZ for using his control over FTX to destroy it. SBF is also being criticized for not disclosing the full overlap between FTX and his trading company, which controlled plenty of FTT. If he had been upfront, someone might have warned FTX about this vulnerability earlier, preventing this mess.

Others have alleged that SBF utilized customer monies to patch flaws in his enterprises' balance accounts. That happened to multiple crypto startups that collapsed this spring, which is unfortunate. These are allegations, not proof.

Why does this matter? Isn't this common in crypto?

Crypto is notorious for shady executives and pranks. FTX is the second-largest crypto business, and SBF was largely considered as the industry's golden boy who would help it get on authorities' good side. Thus far.

Does this affect cryptocurrency prices?

Short-term, it's bad. Prices fell on suspicions that FTX was in peril, then rallied when Binance rescued it, only to fall again later on Tuesday.

These occurrences have hurt FTT and SBF's Solana token. It appears like a huge token selloff is affecting the rest of the market. Bitcoin fell 10% and Ethereum 15%, which is bad but not catastrophic for the two largest coins by market cap.

You might also like

Alexandra Walker-Jones

Alexandra Walker-Jones

3 years ago

These are the 15 foods you should eat daily and why.

Research on preventing disease, extending life, and caring for your body from the inside out

Photo by Isra E on Unsplash

Grapefruit and pomegranates aren't on the list, so ignore that. Mostly, I enjoyed the visual, but those fruits are healthful, too.

15 (or 17 if you consider the photo) different foods a day sounds like a lot. If you're not used to it  — it is.

These lists don't aim for perfection. Instead, use this article and the science below to eat more of these foods. If you can eat 5 foods one day and 5 the next, you're doing well. This list should be customized to your requirements and preferences.

“Every time you eat or drink, you are either feeding disease or fighting it” -Heather Morgan.

The 15 Foods That You Should Consume Daily and Why:

1. Dark/Red Berries

(blueberries, blackberries, acai, goji, cherries, strawberries, raspberries)

The 2010 Global Burden of Disease Study is the greatest definitive analysis of death and disease risk factors in history. They found the primary cause of both death, disability, and disease inside the United States was diet.

Not eating enough fruit, and specifically berries, was one of the best predictors of disease (1).

What's special about berries? It's their color! Berries have the most antioxidants of any fruit, second only to spices. The American Cancer Society found that those who ate the most berries were less likely to die of cardiovascular disease.

2. Beans

Soybeans, black beans, kidney beans, lentils, split peas, chickpeas.

Beans are one of the most important predictors of survival in older people, according to global research (2).

For every 20 grams (2 tablespoons) of beans consumed daily, the risk of death is reduced by 8%.

Soybeans and soy foods are high in phytoestrogen, which reduces breast and prostate cancer risks. Phytoestrogen blocks the receptors' access to true estrogen, mitigating the effects of weight gain, dairy (high in estrogen), and hormonal fluctuations (3).

3. Nuts

(almonds, walnuts, pecans, pistachios, Brazil nuts, cashews, hazelnuts, macadamia nuts)

Eating a handful of nuts every day reduces the risk of chronic diseases like heart disease and diabetes. Nuts also reduce oxidation, blood sugar, and LDL (bad) cholesterol, improving arterial function (4).

Despite their high-fat content, studies have linked daily nut consumption to a slimmer waistline and a lower risk of obesity (5).

4. Flaxseed

(milled flaxseed)

2013 research found that ground flaxseed had one of the strongest anti-hypertensive effects of any food. A few tablespoons (added to a smoothie or baked goods) lowered blood pressure and stroke risk 23 times more than daily aerobic exercise (6).

Flax shouldn't replace exercise, but its nutritional punch is worth adding to your diet.

5. Other seeds

(chia seeds, hemp seeds, pumpkin seeds, sesame seeds, fennel seeds)

Seeds are high in fiber and omega-3 fats and can be added to most dishes without being noticed.

When eaten with or after a meal, chia seeds moderate blood sugar and reduce inflammatory chemicals in the blood (7). Overall, a great daily addition.

6. Dates

Dates are one of the world's highest sugar foods, with 80% sugar by weight. Pure cake frosting is 60%, maple syrup is 66%, and cotton-candy jelly beans are 70%.

Despite their high sugar content, dates have a low glycemic index, meaning they don't affect blood sugar levels dramatically. They also improve triglyceride and antioxidant stress levels (8).

Dates are a great source of energy and contain high levels of dietary fiber and polyphenols, making 3-10 dates a great way to fight disease, support gut health with prebiotics, and satisfy a sweet tooth (9).

7. Cruciferous Veggies

(broccoli, Brussel sprouts, horseradish, kale, cauliflower, cabbage, boy choy, arugula, radishes, turnip greens)

Cruciferous vegetables contain an active ingredient that makes them disease-fighting powerhouses. Sulforaphane protects our brain, eyesight, against free radicals and environmental hazards, and treats and prevents cancer (10).

Unless you eat raw cruciferous vegetables daily, you won't get enough sulforaphane (and thus, its protective nutritional benefits). Cooking destroys the enzyme needed to create this super-compound.

If you chop broccoli, cauliflower, or turnip greens and let them sit for 45 minutes before cooking them, the enzyme will have had enough time to work its sulforaphane magic, allowing the vegetables to retain the same nutritional value as if eaten raw. Crazy, right? For more on this, see What Chopping Your Vegetables Has to Do with Fighting Cancer.

8. Whole grains

(barley, brown rice, quinoa, oats, millet, popcorn, whole-wheat pasta, wild rice)

Whole-grains are one of the healthiest ways to consume your daily carbs and help maintain healthy gut flora.

This happens when fibre is broken down in the colon and starts a chain reaction, releasing beneficial substances into the bloodstream and reducing the risk of Type 2 Diabetes and inflammation (11).

9. Spices

(turmeric, cumin, cinnamon, ginger, saffron, cloves, cardamom, chili powder, nutmeg, coriander)

7% of a person's cells will have DNA damage. This damage is caused by tiny breaks in our DNA caused by factors like free-radical exposure.

Free radicals cause mutations that damage lipids, proteins, and DNA, increasing the risk of disease and cancer. Free radicals are unavoidable because they result from cellular metabolism, but they can be avoided by consuming anti-oxidant and detoxifying foods.

Including spices and herbs like rosemary or ginger in our diet may cut DNA damage by 25%. Yes, this damage can be improved through diet. Turmeric worked better at a lower dose (just a pinch, daily). For maximum free-radical fighting (and anti-inflammatory) effectiveness, use 1.5 tablespoons of similar spices (12).

10. Leafy greens

(spinach, collard greens, lettuce, other salad greens, swiss chard)

Studies show that people who eat more leafy greens perform better on cognitive tests and slow brain aging by a year or two (13).

As we age, blood flow to the brain drops due to a decrease in nitric oxide, which prevents blood vessels from dilatation. Daily consumption of nitrate-rich vegetables like spinach and swiss chard may prevent dementia and Alzheimer's.

11. Fermented foods

(sauerkraut, tempeh, kombucha, plant-based kefir)

Miso, kimchi, and sauerkraut contain probiotics that support gut microbiome.

Probiotics balance the good and bad bacteria in our bodies and offer other benefits. Fermenting fruits and vegetables increases their antioxidant and vitamin content, preventing disease in multiple ways (14).

12. Sea vegetables

(seaweed, nori, dulse flakes)

A population study found that eating one sheet of nori seaweed per day may cut breast cancer risk by more than half (15).

Seaweed and sea vegetables may help moderate estrogen levels in the metabolism, reducing cancer and disease risk.

Sea vegetables make up 30% of the world's edible plants and contain unique phytonutrients. A teaspoon of these super sea-foods on your dinner will help fight disease from the inside out.

13. Water

I'm less concerned about whether you consider water food than whether you drink enough. If this list were ranked by what single item led to the best health outcomes, water would be first.

Research shows that people who drink 5 or more glasses of water per day have a 50% lower risk of dying from heart disease than those who drink 2 or less (16).

Drinking enough water boosts energy, improves skin, mental health, and digestion, and reduces the risk of various health issues, including obesity.

14. Tea

All tea consumption is linked to a lower risk of stroke, heart disease, and early death, with green tea leading for antioxidant content and immediate health benefits.

Green tea leaves may also be able to interfere with each stage of cancer formation, from the growth of the first mutated cell to the spread and progression of cancer in the body. Green tea is a quick and easy way to support your long-term and short-term health (17).

15. Supplemental B12 vitamin

B12, or cobalamin, is a vitamin responsible for cell metabolism. Not getting enough B12 can have serious consequences.

Historically, eating vegetables from untreated soil helped humans maintain their vitamin B12 levels. Due to modern sanitization, our farming soil lacks B12.

B12 is often cited as a problem only for vegetarians and vegans (as animals we eat are given B12 supplements before slaughter), but recent studies have found that plant-based eaters have lower B12 deficiency rates than any other diet (18).


Article Sources:

  1. The Global Burden of Disease Study 2010 (GBD 2010)

2. I. Darmadi-Blackberry, M. Wahlqvist, A. Kouris-Blazos, et al. Legumes: the most important dietary predictor of survival in older people of different ethnicities. Asia Pac J Clin Nutr. 2004;13(2):217–20.

3. Guha N, Kwan ML, Quesenberry CP Jr, Weltzien EK, Castillo AL, Caan BJ. Soy isoflavones and risk of cancer recurrence in a cohort of breast cancer survivors: the Life After Cancer Epidemiology study. Breast Cancer Res Treat. 2009 Nov;118(2):395–405.

4. Y. Bao, J. Han, F. B. Hu, E. L. Giovannucci, M. J. Stampfer, W. C. Willett, C. S. Fuchs. Association of nut consumption with total and cause-specific mortality. N. Engl. J. Med. 2013 369(21):2001–2011.

5. V. Vadivel, C. N. Kunyanga, H. K. Biesalski. Health benefits of nut consumption with special reference to body weight control. Nutrition 2012 28(11–12):1089–1097.

6. D Rodriguez-Leyva, W Weighell, A L Edel,R LaVallee, E Dibrov,R Pinneker, T G Maddaford, B Ramjiawan, M Aliani, R Guzman R, G N Pierce. Potent antihypertensive action of dietary flaxseed in hypertensive patients. Hypertension. 2013 Dec;62(6):1081–9. doi: 10.1161/HYPERTENSIONAHA.113.02094.

7. Vuksan V, Jenkins AL, Dias AG, Lee AS, Jovanovski E, Rogovik AL, Hanna A. Reduction in postprandial glucose excursion and prolongation of satiety: possible explanation of the long-term effects of whole grain Salba (Salvia Hispanica L.). Eur J Clin Nutr. 2010 Apr;64(4):436–8. doi: 10.1038/ejcn.2009.159. Epub 2010 Jan 20. PMID: 20087375.

8. W. Rock, M. Rosenblat, H. Borochov-Neori, N. Volkova, S. Judeinstein, M. Elias, and M. Aviram. Effects of date (Phoenix dactylifera L., Medjool or Hallawi Variety) consumption by healthy subjects on serum glucose and lipid levels and on serum oxidative status: a pilot study. J. Agric. Food. Chem., 57(17):8010{8017, 2009.

9. Eid N, Enani S, Walton G, et al. The impact of date palm fruits and their component polyphenols, on gut microbial ecology, bacterial metabolites and colon cancer cell proliferation. J Nutr Sci. 2014;3:e46.

10. Li Y, Zhang T, Korkaya H, Liu S, Lee HF, Newman B, Yu Y, Clouthier SG, Schwartz SJ, Wicha MS, Sun D. Sulforaphane, a Dietary Component of Broccoli/Broccoli Sprouts, Inhibits Breast Cancer Stem Cells. Clin Cancer Res. 2010 May 1;16(9):2580–90.

11. Lappi J, Kolehmainen M, Mykkänen H, Poutanen K. Do large intestinal events explain the protective effects of whole grain foods against type 2 diabetes? Crit Rev Food Sci Nutr. 2013;53(6):631–40.

12. S. S. Percival, J. P. V. Heuvel, C. J. Nieves, C. Montero, A. J. Migliaccio, J. Meadors. Bioavailability of Herbs and Spices in Humans as Determined by ex vivo Inflammatory Suppression and DNA Strand Breaks. J Am Coll Nutr. 2012 31(4):288–294.

13. Nurk E, Refsum H, Drevon CA, et al. Cognitive performance among the elderly in relation to the intake of plant foods. The Hordaland Health Study. Br J Nutr. 2010;104(8):1190–201.

14. Melini, F.; Melini, V.; Luziatelli, F.; Ficca, A.G.; Ruzzi, M. Health-Promoting Components in Fermented Foods: An Up-to-Date Systematic Review. Nutrients2019, 11, 1189.

15. H. Funahashi, T. Imai, T. Mase, M. Sekiya, K. Yokoi, H. Hayashi, A. Shibata, T. Hayashi, M. Nishikawa, N. Suda, Y. Hibi, Y. Mizuno, K. Tsukamura, A. Hayakawa, S. Tanuma. Seaweed prevents breast cancer? Jpn. J. Cancer Res. 2001 92(5):483–487.

16. Chan J, Knutsen SF, Blix GG, Lee JW, Fraser GE. Water, other fluids, and fatal coronary heart disease: the Adventist Health Study. Am J Epidemiol. 2002 May 1;155(9):827–33. doi: 10.1093/aje/155.9.827. PMID: 11978586.

17. Fujiki H, Imai K, Nakachi K, Shimizu M, Moriwaki H, Suganuma M. Challenging the effectiveness of green tea in primary and tertiary cancer prevention. J Cancer Res Clin Oncol. 2012 Aug;138(8):1259–70.

18. Damayanti, D., Jaceldo-Siegl, K., Beeson, W. L., Fraser, G., Oda, K., & Haddad, E. H. (2018). Foods and Supplements Associated with Vitamin B12Biomarkers among Vegetarian and Non-Vegetarian Participants of the Adventist Health Study-2 (AHS-2) Calibration Study. Nutrients, 10(6), 722. doi:10.3390/nu10060722

Gajus Kuizinas

Gajus Kuizinas

3 years ago

How a few lines of code were able to eliminate a few million queries from the database

I was entering tens of millions of records per hour when I first published Slonik PostgreSQL client for Node.js. The data being entered was usually flat, making it straightforward to use INSERT INTO ... SELECT * FROM unnset() pattern. I advocated the unnest approach for inserting rows in groups (that was part I).

Bulk inserting nested data into the database

However, today I’ve found a better way: jsonb_to_recordset.

jsonb_to_recordset expands the top-level JSON array of objects to a set of rows having the composite type defined by an AS clause.

jsonb_to_recordset allows us to query and insert records from arbitrary JSON, like unnest. Since we're giving JSON to PostgreSQL instead of unnest, the final format is more expressive and powerful.

SELECT *
FROM json_to_recordset('[{"name":"John","tags":["foo","bar"]},{"name":"Jane","tags":["baz"]}]')
AS t1(name text, tags text[]);
 name |   tags
------+-----------
 John | {foo,bar}
 Jane | {baz}
(2 rows)

Let’s demonstrate how you would use it to insert data.

Inserting data using json_to_recordset

Say you need to insert a list of people with attributes into the database.

const persons = [
  {
    name: 'John',
    tags: ['foo', 'bar']
  },
  {
    name: 'Jane',
    tags: ['baz']
  }
];

You may be tempted to traverse through the array and insert each record separately, e.g.

for (const person of persons) {
  await pool.query(sql`
    INSERT INTO person (name, tags)
    VALUES (
      ${person.name},
      ${sql.array(person.tags, 'text[]')}
    )
  `);
}

It's easier to read and grasp when working with a few records. If you're like me and troubleshoot a 2M+ insert query per day, batching inserts may be beneficial.

What prompted the search for better alternatives.

Inserting using unnest pattern might look like this:

await pool.query(sql`
  INSERT INTO public.person (name, tags)
  SELECT t1.name, t1.tags::text[]
  FROM unnest(
    ${sql.array(['John', 'Jane'], 'text')},
    ${sql.array(['{foo,bar}', '{baz}'], 'text')}
  ) AS t1.(name, tags);
`);

You must convert arrays into PostgreSQL array strings and provide them as text arguments, which is unsightly. Iterating the array to create slices for each column is likewise unattractive.

However, with jsonb_to_recordset, we can:

await pool.query(sql`
  INSERT INTO person (name, tags)
  SELECT *
  FROM jsonb_to_recordset(${sql.jsonb(persons)}) AS t(name text, tags text[])
`);

In contrast to the unnest approach, using jsonb_to_recordset we can easily insert complex nested data structures, and we can pass the original JSON document to the query without needing to manipulate it.

In terms of performance they are also exactly the same. As such, my current recommendation is to prefer jsonb_to_recordset whenever inserting lots of rows or nested data structures.

James White

James White

3 years ago

I read three of Elon Musk's suggested books (And His Taste Is Incredible)

A reading list for successful people

Daniel Oberhaus via Flickr

Elon Musk reads and talks. So, one learns. Many brilliant individuals & amazing literature.

This article recommends 3 Elon Musk novels. All of them helped me succeed. Hope they'll help you.

Douglas Adams's The Hitchhiker's Guide to the Galaxy

Page Count: 193
Rating on Goodreads: 4.23

Arthur Dent is pulled off Earth by a buddy seconds before it's razed for a cosmic motorway. The trio hitchhikes through space and gets into problems.

I initially read Hitchhiker's as a child. To evade my mum, I'd read with a flashlight under the covers. She'd scold at me for not sleeping on school nights when she found out. Oops.

The Hitchhiker's Guide to the Galaxy is lighthearted science fiction.

Goodreads

My favorite book quotes are:

  • “Space is big. You won’t believe how vastly, hugely, mind-bogglingly big it is. I mean, you may think it’s a long way down the road to the chemist’s, but that’s just peanuts to space.”

  • “Far out in the uncharted backwaters of the unfashionable end of the western spiral arm of the Galaxy lies a small unregarded yellow sun. Orbiting this at a distance of roughly ninety-two million miles is an utterly insignificant little blue-green planet whose ape-descended life forms are so amazingly primitive that they still think digital watches are a pretty neat idea.”

  • “On planet Earth, man had always assumed that he was more intelligent than dolphins because he had achieved so much — the wheel, New York, wars, and so on — whilst all the dolphins had ever done was muck about in the water having a good time. But conversely, the dolphins had always believed that they were far more intelligent than man — for precisely the same reasons.”

the Sun Tzu book The Art Of War

Page Count: 273
Rating on Goodreads: 3.97

It's a classic. You may apply The Art of War's ideas to (nearly) every facet of life. Ex:

  • Pick your fights.

  • Keep in mind that timing is crucial.

  • Create a backup plan in case something goes wrong.

  • Obstacles provide us a chance to adapt and change.

This book was my first. Since then, I'm a more strategic entrepreneur. Excellent book. And read it ASAP!

Goodreads

My favorite book quotes are:

  • “Victorious warriors win first and then go to war, while defeated warriors go to war first and then seek to win.”

  • “Engage people with what they expect; it is what they are able to discern and confirms their projections. It settles them into predictable patterns of response, occupying their minds while you wait for the extraordinary moment — that which they cannot anticipate.”

  • “If you know the enemy and know yourself, you need not fear the result of a hundred battles. If you know yourself but not the enemy, for every victory gained, you will also suffer a defeat. If you know neither the enemy nor yourself, you will succumb in every battle.”

Peter Thiel's book Zero to One

Page Count: 195
Rating on Goodreads: 4.18

Peter argues the best money-making strategies are typically unproven. Entrepreneurship should never have a defined path to success. Whoever says differently is lying.

Zero to One explores technology and society. Peter is a philosophy major and law school graduate, which informs the work.

Peters' ideas, depth, and intellect stood out in Zero to One. It's a top business book.

Goodreads

My favorite book quotes are:

  • “The most valuable businesses of coming decades will be built by entrepreneurs who seek to empower people rather than try to make them obsolete.”

  • “The next Bill Gates will not build an operating system. The next Larry Page or Sergey Brin won’t make a search engine. And the next Mark Zuckerberg won’t create a social network. If you are copying these guys, you aren’t learning from them.”

  • “If your goal is to never make a mistake in your life, you shouldn’t look for secrets. The prospect of being lonely but right — dedicating your life to something that no one else believes in — is already hard. The prospect of being lonely and wrong can be unbearable.”