My main witness node crashed for no reason when I was asleep around 1.AM midnight.

Thanks for @fancybrothers who notified me on the discord channel. When I got the message, it has been 5 hours and I ‘ve missed 310 blocks.

Sh*t happens!

Thus, I decide to write a tool that monitors the witness node, in case of failure, it will detect and switch to your backup node.

Project: https://github.com/DoctorLai/SteemWitnessAutoSwitch

It is easy to use: you first need to configure:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

{
    "account": "Your Steem Witness Account",
    "key": "Your Active Key",
    "signing_keys": [
        "Witness Signing Key 1",
        "Witness Signing Key 2",
        "STM1111111111111111111111111111111114T1Anm"
    ],
    "url": "https://steemyy.com",
    "fee": "3.000 STEEM",
    "interval": 60,
    "period": 360,
    "threshold": 4  
}

Make the last siging key disabled one so it will disable your node in case all your witness nodes are down. I have taken out the code to send a email for notification since it is quite customised to my settings but you can easily add it, the easily way to send a email would be to launch the mail utility.

The default setting is to switch if there are 4 missed blocks in the last 6 minutes. You can adjust if you are outside TOP 20. The interval is the time interval to check if there are new misses.

Last but not least, I would recommend running this using screen or pm2

Every little helps! I hope this helps!

Steem On!~

If you like my work, please consider voting for me, thanks!
https://steemit.com/~witnesses type in justyy and click VOTE

Alternatively, you could proxy to me if you are too lazy to vote!

Also: you can vote me at the tool I made: https://steemyy.com/witness-voting/?witness=justyy

Visit me at: https://steemyy.com

This page is synchronized from the post: ‘Open Source: Auto Switch Your Witness Node In the Event of Failure’

Let’s introduce a concept i.e. witness-voting factor that represent the average witness voting each other in the TOP 20. The maximum value is 20, and the minimal is 0.

If the witness-voting factor is 20, it means that all top 20 witnesses are voting each other. This is not healthy as the gap (of votes) between TOP 20 and the rest will be always increasing. Also, since witnesses are voting each other, it is kinda controlled by a group of people who share the same interests - this is centralisation admit it or not.

SteemJs code to compute the witness-voting factor

The idea is to retrieve the list of the votes of the TOP 20, and group them, count the frequency and sort them. Compute the average. The witness-voting factor for Top 20 on STEEM blockchain is 4.5, compared to 12.65 on HIVE.

Run the following in SteemJs Editor

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89

function getTotalWitnesses() {
    return new Promise((resolve, reject) => {
      steem.api.getWitnessCount(function(err, result) {
          if (!err) {
              resolve(result);
          } else {
              reject(err);
          }
      });
    });
}

function getAllWitnessAccounts(total) {
    return new Promise((resolve, reject) => {
          steem.api.getWitnesses([...Array(total).keys()], function(err, result) {
            if (!err) {
                resolve(result);
            } else {
                reject(err);
           }
        });
    });
}   

function getAccounts(accounts) {
    return new Promise((resolve, reject) => {
          steem.api.getAccounts(accounts, function(err, result) {
            if (!err) {
                resolve(result);
            } else {
                reject(err);
           }
        });
    });
}   


(async function () {
    const totalWitnesses = await getTotalWitnesses();
    let data = await getAllWitnessAccounts(totalWitnesses);
    data = data.filter(x => {
        // reduce the noise
        return (x.votes > 0) && (x.running_version === "0.23.1");
    });
    // sort by votes
    data.sort((a, b) => {
        return b.votes - a.votes;
    });
    // only count TOP 20 witnesses
    data = data.slice(0, 20);
    let top = [];
    data.map((x) => {
        top.push(x.owner);
    });
    const accountData = await getAccounts(top);
    let votes = {};
    data.map((x) => {
        const account = accountData.filter(v => {
            return v.name === x.owner
        });
        // count the vote frequency for each witness
        for (let w of account[0].witness_votes) {
            if (typeof votes[w] === "undefined") {
                votes[w] = 1;
            } else {
                votes[w] ++;
            }
        }
    });    
    
    var items = Object.keys(votes).map(function(key) {
      return [key, votes[key]];
    });
    
    items.sort(function(first, second) {
      return second[1] - first[1];
    });
    
    let x = 0, cnt = 0;
    for (let w of items) {
        // TOP 20 witnesses only
        if (top.includes(w[0])) {
            log(w[0] + ", " + w[1]);
            x += w[1];
            cnt ++;
        }
    }
    log(x/cnt);
})();

Here is the detail list:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

future.witness, 7
justyy, 7
steem-agora, 7
steemchiller, 7
dev.supporters, 6
dlike, 6
maiyude, 6
steem-supporter, 6
symbionts, 6
steem-dragon, 4
scissor.sisters, 4
cryptoking777, 3
hivei0, 3
hoasen, 3
juddsmith079, 3
matreshka, 3
protoss20, 3
hinomaru-jp, 2
inwi, 2
rnt1, 2

Every little helps! I hope this helps!

Steem On!~

Reposted to Blog

If you like my work, please consider voting for me, thanks!
https://steemit.com/~witnesses type in justyy and click VOTE

Alternatively, you could proxy to me if you are too lazy to vote!

Also: you can vote me at the tool I made: https://steemyy.com/witness-voting/?witness=justyy

Visit me at: https://steemyy.com

This page is synchronized from the post: ‘Are Top 20 Witnesses Voting Each Other? Introducing Witness-voting Factor’

How many witnesses are running on 23.1 and how many of them are active? It turns out answering this question does not require preprocess blocks on steem blockchain e.g. SteemSQL. Rather, we can get the answer by pure SteemJS.

Get All Witnesses

First, we can use the steemp.api.getWitnessCount to return the list of all registered witnesses - which is a lot more than we thought, currently more than 1400 witnesses but of course many of them have been disabled or never produced a block.

1
2
3
4
5
6
7
8
9
10
11

function getTotalWitnesses() {
    return new Promise((resolve, reject) => {
      steem.api.getWitnessCount(function(err, result) {
          if (!err) {
              resolve(result);
          } else {
              reject(err);
          }
      });
    });
}

Get All Witnesses Accounts

Then, we can use steem.api.getWitnesses to retreive the witnesses information for multiple accounts at the same time.

1
2
3
4
5
6
7
8
9
10
11
12

function getAllWitnessAccounts(total) {
    return new Promise((resolve, reject) => {
          steem.api.getWitnesses([...Array(total).keys()], function(err, result) {
            if (!err) {
                resolve(result);
            } else {
                reject(err);
           }
        });
    });
}   
`

Filtering

Then, we can chain those two functions to filter out those witnesses that are running 23.1 and are active.

1
2
3
4
5
6
7

(async function () {
    const totalWitnesses = await getTotalWitnesses();
    let data = await getAllWitnessAccounts(totalWitnesses);
    log(data.filter(x => {
        return x.running_version === "0.23.1" && (x.signing_key !== "STM1111111111111111111111111111111114T1Anm");
    }).length);
})();

The answer is 46 witnesses are running on 23.1 and all of them are active.

If we point the RPC node to HIVE chain, we get 107 witnesses running on 23.0.

Run the code using SteemJs

Slightly changing the query, we know:
There are 24 Witnesses running at 0.23.0 but they are disabled.
And there are 9 witnesses running at 0.23.0 and they are ‘active’ which may be those HIVE witnesses who didn’t take offline they witnesses.

BTW, i have added the dSteem into the SteemJs tool: https://steemyy.com/steemjs/

Every little helps! I hope this helps!

Steem On!~

Reposted to Blog

If you like my work, please consider voting for me, thanks!
https://steemit.com/~witnesses type in justyy and click VOTE

Alternatively, you could proxy to me if you are too lazy to vote!

Also: you can vote me at the tool I made: https://steemyy.com/witness-voting/?witness=justyy

Visit me at: https://steemyy.com

This page is synchronized from the post: ‘SteemJs: How Many Witnesses are Running on 23.1?’

Hello,
I decide to write a simple steem block explorer. And viewing the transactions in a block is the first step.

https://steemyy.com/block.php
Chinese: https://steemyy.com/steemit-tools/block.php

Known Issues

1. Latest blocks are showing 0 transactions, please refresh in a few minutes (I am looking into solving this issue, any help would be appreciated). The Last Irreversible blocks are fine.
2. customizing known operations - I will gradually improve it.
3. showing account’s history page
4. search field, filtering

Every little helps! I hope this helps!

Steem On!~

If you like my work, please consider voting for me, thanks!
https://steemit.com/~witnesses type in justyy and click VOTE

Alternatively, you could proxy to me if you are too lazy to vote!

Also: you can vote me at the tool I made: https://steemyy.com/witness-voting/?witness=justyy

Visit me at: https://steemyy.com

This page is synchronized from the post: ‘New Tool: Introducing a Simple Steem Block Explorer’

Blockchain is a fancy word, and not many people understand it. If you want to explain this to your GF/partner, what would be the most simple way?

Right, blocks are chained in one way, as time goes on, in particular for STEEM blockchain, every 3 seconds, we append a new block to the end of the current chain. So the chain goes longer and longer each day.

We need a way to tell if the current block can be attached (we can’t just produce a random block and force appending it). For each block, we will have a unique block ID, and we will have a previous field that match the previous Block ID. The first block on steem blockchain has previous set to 0000000000000000000000000000000000000000.

Run this in SteemJS

1
2
3
4

const blockNum = 1;
steem.api.getBlock(blockNum, function(err, result) {
  console.log(err, result);
});

You will see the block_id and previous value of the first block.

1

{"previous":"0000000000000000000000000000000000000000","timestamp":"2016-03-24T16:05:00","witness":"initminer","transaction_merkle_root":"0000000000000000000000000000000000000000","extensions":[],"witness_signature":"204f8ad56a8f5cf722a02b035a61b500aa59b9519b2c33c77a80c0a714680a5a5a7a340d909d19996613c5e4ae92146b9add8a7a663eef37d837ef881477313043","transactions":[],"block_id":"0000000109833ce528d5bbfb3f6225b39ee10086","signing_key":"STM8GC13uCZbP44HzMLV6zPZGwVQ8Nt4Kji8PapsPiNq1BK153XTX","transaction_ids":[]}

And the second block gives:

1

{"previous":"0000000109833ce528d5bbfb3f6225b39ee10086","timestamp":"2016-03-24T16:05:36","witness":"initminer","transaction_merkle_root":"0000000000000000000000000000000000000000","extensions":[],"witness_signature":"1f3e85ab301a600f391f11e859240f090a9404f8ebf0bf98df58eb17f455156e2d16e1dcfc621acb3a7acbedc86b6d2560fdd87ce5709e80fa333a2bbb92966df3","transactions":[],"block_id":"00000002ed04e3c3def0238f693931ee7eebbdf1","signing_key":"STM8GC13uCZbP44HzMLV6zPZGwVQ8Nt4Kji8PapsPiNq1BK153XTX","transaction_ids":[]}

As you can see, the previous value is the same as the block_id for the Block 1. This goes on and on. Let’s print the first 10 blocks’ previous and block_id:

Run the Code in SteemJS

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

function getBlock(blockNum) {
    return new Promise((resolve, reject) => {
        steem.api.getBlock(blockNum, function(err, result) {
            if (err) reject(err);
            resolve({
                block: blockNum,
                previous: result.previous,
                block_id: result.block_id,
            });
        });    
    });
}

(async function() {
    for (let i = 1; i <= 10; ++ i) {
        log(await getBlock(i));    
    }
})();

And you will see the rule applies. In fact, this is the fundamental rule of the blockchain - the rule to chain the blocks.

1
2
3
4
5
6
7
8
9
10

{"block":1,"previous":"0000000000000000000000000000000000000000","block_id":"0000000109833ce528d5bbfb3f6225b39ee10086"}
{"block":2,"previous":"0000000109833ce528d5bbfb3f6225b39ee10086","block_id":"00000002ed04e3c3def0238f693931ee7eebbdf1"}
{"block":3,"previous":"00000002ed04e3c3def0238f693931ee7eebbdf1","block_id":"000000035b094a812646289c622dba0ba67d1ffe"}
{"block":4,"previous":"000000035b094a812646289c622dba0ba67d1ffe","block_id":"00000004f9de0cfeb08c9d7d9d1fe536d902dc4a"}
{"block":5,"previous":"00000004f9de0cfeb08c9d7d9d1fe536d902dc4a","block_id":"00000005014b5562a1133070d8bee536de615329"}
{"block":6,"previous":"00000005014b5562a1133070d8bee536de615329","block_id":"00000006e323e35687e160b8aec86f1e56d4c902"}
{"block":7,"previous":"00000006e323e35687e160b8aec86f1e56d4c902","block_id":"000000079ff02a2dea6c4d9a27f752233d4a66b4"}
{"block":8,"previous":"000000079ff02a2dea6c4d9a27f752233d4a66b4","block_id":"000000084f957cc170a27c8330293a3343f82c23"}
{"block":9,"previous":"000000084f957cc170a27c8330293a3343f82c23","block_id":"00000009f35198cfd8a866868538bed3482d61a4"}
{"block":10,"previous":"00000009f35198cfd8a866868538bed3482d61a4","block_id":"0000000aae44a2f4d57170dab16fb1619f9e1d0e"}

Steem Blockchain is a public database. Every 3 seconds, one witness helps to package operations (comment, vote, transfer etc) into a block, seal it with the signing key, and finally push to the chain for a small reward.

As the blockchain gets bigger and bigger, it takes enormous efforts to alter the chain, as you will need to modify every prior blocks.

Every little helps! I hope this helps!

Steem On!~

Reposted to Blog

If you like my work, please consider voting for me, thanks!
https://steemit.com/~witnesses type in justyy and click VOTE

Alternatively, you could proxy to me if you are too lazy to vote!

Also: you can vote me at the tool I made: https://steemyy.com/witness-voting/?witness=justyy

Visit me at: https://steemyy.com

This page is synchronized from the post: ‘Illustrating the Blockchain via SteemJs - Blocks are Chained’

Let’s say, I have an application that is written in NodeJs, that process the latest blocks of the steem blockchain and write data into a database. I have three ways of running it on my server.

Using Screen

As you probably know, when you are disconnected from a SSH session, the programs that are launched in the session will be terminated. There is any easy way to fix this. You can start a session by using command screen -S name before you run any long-running program e.g. system updates or a program that needs to run forever.

In case the SSH is disconnected, the programs will be still running. And you can use command screen -r name to re-connect to the session. You can screen -ls to list the current sessions, and screen -d name to detach a session.

The advantage of using this method is:

1. you can easily monitor the application (by seeing its output)
2. you can easily terminate or restart the application e.g. Ctrl + C
3. the application runs continuously, meaning that you can keep up to the latest blocks and provide a less than 3 second sync time.

Some disadvantages:

1. If the application exists abnormaly, it is not restarted automatically - thus you would need to add the error-handling in your code.
2. Long-running code may tend to have more problems: e.g. memory-leaks, crash at some point.

Putting it in Crontab

Another way is to run your program in crontab. However, the finest granularity is every minute i.e. you can specify your application to run every minute.

Advantages are:

1. You probably don’t need to do error-handling. In case it fails, it fails. You can just wait for next minute (restart automatically).
2. Not to worry about memory leaks. It is less likely to go wrong due to memory issue.

Disadvantages:

1. Lose the ability to sync real-time with the blocks
2. Not easy to monitor the output (you can redirect the errors to files though)

Using a Process Manager

You can combine the both advantages using a Process Manager. For example, if it is a NodeJs application, you can use pm2 utility.

Advantages:

1. automatically restart your application in case it crashes or some conditions are met e.g. memory constraints, age.
2. easy to monitor the output of each application e.g. you can pm2 show app
3. easy to terminate pm2 delete app or restart pm2 restart app
4. the application runs continuously
5. error-handling is optional - but considered a good practise not to let pm2 restart your app.

Every little helps! I hope this helps!

Steem On!~

Reposted to Blog

If you like my work, please consider voting for me, thanks!
https://steemit.com/~witnesses type in justyy and click VOTE

Alternatively, you could proxy to me if you are too lazy to vote!

Also: you can vote me at the tool I made: https://steemyy.com/witness-voting/?witness=justyy

Visit me at: https://steemyy.com

This page is synchronized from the post: ‘Three ways of Running a continuous NodeJS Application on Your Server’