Just see following test.

Environment

  • OS: 4.15.0-47-generic #50-Ubuntu SMP.
  • CPU: Intel(R) Xeon(R) CPU E5-2697 v3 @ 2.60GHz
  • NodeJs: 10.15.3

Normal

function f() {}

async function main() {
    let tm = -Date.now();
    for (let i = 0; i < 100000; i++) {
        f();
    }
    tm += Date.now();
    console.log(tm);
}
main()
$ node test.js
3

With operations

function f() {
    a = [];
    for (let i = 0; i < 30; i++) { a.push(i); }
}

async function main() {
    let tm = -Date.now();
    for (let i = 0; i < 100000; i++) {
        f();
    }
    tm += Date.now();
    console.log(tm);
}
main()

$ node test.js
27

With await

function f() {}

async function main() {
    let tm = -Date.now();
    for (let i = 0; i < 100000; i++) {
        await f();
    }
    tm += Date.now();
    console.log(tm);
}
main()
$ node test.js
19

With async/await

async function f() {}

async function main() {
    let tm = -Date.now();
    for (let i = 0; i < 100000; i++) {
        await f();
    }
    tm += Date.now();
    console.log(tm);
}
main()
$ node test.js
26

Introduction

3-way-merge(henceforth 3WM) is popularly used at SCM(Source code Configuration Management) tool. But, there is no standard way of implementation. And most of them is for text(source code). So, there are already lots of study regarding Text-3WM. But, for me, it's very difficult to find study regarding Json-3WM. Someone may think Json-3WM is subset of Text-3WM. And I think it can be one of possible algorithm of Json-3WM. But, is it enough? This article is for this topic.

Assumption

Array vs. Object

At Json, Array is considered as Object. At Javascript, typeof [] gives object. For example, [1, 2] can be represented as {0:1, 1:2}. In this article, Array is handled as one of primitive value, not an object - just like JSON-Merge-Patch(RFC 7386).

Merge

Way of merging two Json object is clearly defined at JSON-Merge-Patch(RFC 7386). Therefore this article is only focusing on 3-way-mergee.

3-Way Merge: Clear

In most common cases, it looks clear. For example,

Mergable

<base>
{
    "a": "b",
    "c": {
        "d": "e",
        "f": "g"
    }
}

<our>
{
    "a": "b",
    "c": {
        "d": "e",
        "f": "g",
        "h": "i"  # <=
    }
}


<their>
{
    "a": "b",
    "c": {
        "d": "e",
        "f": "z"  # <=
    }
}

<merged>
{
    "a": "b",
    "c": {
        "d": "e",
        "f": "z",  # <=
        "h": "i"   # <=
    }
}

Conflict

<base>
{
    "a": "b",
    "c": {
        "d": "e",
        "f": "g"
    }
}

<our>
{
    "a": "b",
    "c": {
        "d": "e",
        "f": "z"  # <=
    }
}


<their>
{
    "a": "b",
    "c": {
        "d": "e",
        "f": "y"  # <=
    }
}

3-Way Merge: Not clear

This is main topic of this discussion. Consider following example

<base>
{
    "a": "b",
    "c": {
        "d": "e",
        "f": "g"
    }
}

<our>
{
    "a": "b",
    "c": {
        "d": "e",
        "f": "z"  # <=
    }
}

And

<their>
{
    "a": "b"
              # <=
}

It seems conflict. Then, which property is conflicted? /c or /c/f?

Case: /c/f (Incorrect)

In this interpretation, merged result with Accept-Our option (resolve strategy for conflicted property) should be

<merged: accept-our>
{
    "a": "b",
    "c": {
        "f": "z"  # <=
    }
}

What is issue of above? In this case, there is no difference from below version of <their> object.

<their>
{
    "a": "b",
    "c": {}  # <=
}

In case of above version, everything looks clear. Definitely, /c/f is property conflicted. Then, Is it natural that two different changes gives same merged result? I think No. Therefore this is not correct interpretation of given example.

Case: /c

In this case, interpretation of changes are

  • <their>: Deleting /c.
  • <our>: Changing /c/f and /c(sub-property of /c is changed).
    So, /c is conflicted and merged result with Accept-Our option (resolve strategy for conflicted property) should be
    <merged: accept-our>(same with <our>)
    {
      "a": "b",
      "c": {
          "d": "e",
          "f": "z"
      }
    }
    Whole sub-object /c is resolved with /c of <our> object because of Accept-Our merge strategy.

Summary

This article describes private opinion related with 3-way-merge algorithm of Json object. There is NO correct answer. This is just one of suggestion.

Extremely basic concept!. But, sometimes importance of this concept is ignored becase of several reasons

  • Easy-to-use with libraries and other functions.
  • Easy-to-use becase of language characteristics.
  • To save typing efforts.
  • ...

But, to increase readability of code, it's very important keeping fundamental concepts and following use-cases of each data structure.

  • Array: Order is matter. And duplication is allowed.
  • Set: Order is NOT matter. And duplication is NOT allowed.

Fabric has great document for it.
See https://hyperledger-fabric.readthedocs.io/en/release-1.2/build_network.html#

But, once you try you own network from scratch, you will face lots of unexpected obstacles.
So, in addition to this guide, I would like to mention some more information. If you are not familiar with docker and docker-compose, you may confuse what is exactly happending inside of byfn(Build Your First Network). Therefore, before staring building your network, please understand what is docker and docker-compose.

One important thing regarding docker-compose is, by default docker-compose uses user-defined bridge network mode. And, instantiation CC means running docker-container. Problem is, container for CC also should join to network in where peer is joined.

Environment variable CORE_VM_DOCKER_HOSTCONFIG_NETWORKMODE is for it!

You should NOT forget this!
Without this, container cannot connect to peer nodes!

In case of promise is NOT fullfilled forever, mocha doens't give any information about this.
See test below.

    describe('test', function () {
        it('test', async () => {
            await new Promise((r, j) => {});
            console.log('hello');
        });
    })

This gives only following result in terminal

    test
      test
Waiting for the debugger to disconnect...
Killed

No more informations!

Environment: Hyperledger Fabric v1.2

Fabric uses 'npm install --production' to build chaincode docker image.
(https://github.com/hyperledger/fabric/blob/release-1.2/core/chaincode/platforms/node/platform.go#L188)

And run CC by using 'npm start -- --peer.address'
(https://github.com/hyperledger/fabric/blob/release-1.2/core/chaincode/container_runtime.go#L147)

So, only following files are needed to be released

  • javascript files
  • package.json
  • package-lock.json

In case of NodeJs, javascript files should be deployed to peer nodes. So, usually uglifying and optimization is required.
And webpack is most popular tool for this requirements.
And in case that all source codes are bundled to one file - bundle.js - then, releasing only three files are enough!

  • bundle.js
  • package.json
  • package-lock.json

And 'npm start' may look like 'node bundle.js'.

        {
            "type": "node",
            "request": "launch",
            "name": "Mocha Current File",
            "program": "${workspaceFolder}/node_modules/.bin/_mocha",   ⇐ “_mocha” NOT “mocha”
            "runtimeArgs": [
                "--nolazy",
                "--max_old_space_size=16384",
                "--preserve-symlinks"    <=== Important. Related with way of resolving module by node.
            ],
            "args": [
                "-r",
                "ts-node/register",
                "--no-timeouts",
                "--colors",
                "src/index.spec.ts"
            ],
            "outFiles": [
                "${workspaceFolder}/build"  <=== This is very important NOT "${workspaceFolder}/build/**/*.js"
            ],
            "console": "integratedTerminal",
            "sourceMaps": true,
            "skipFiles": [
                "node_modules/**/*.js",
                "<node_internals>/**/*.js"
            ],
            "internalConsoleOptions": "neverOpen"
        }


<IMPORTANT NOTE>

if “program” is “ts” file; then
    outFiles: ["${workspaceFolder}/build/**/*.js"]
else if “program” is other executable using ‘ts-node/register’; then
    outFiles SHOULD be removed.
fi

*** In case of Mocha, if ‘outFiles’ are described, breakpoint cannot be reached!

Hyperledger Fabric v1.2


./users/Admin@example.com/tls/client.key   (*E)
./users/Admin@example.com/tls/ca.crt   (*b)
./users/Admin@example.com/tls/client.crt   (*E)


# AdminMsp {

./users/Admin@example.com/msp/cacerts/ca.example.com-cert.pem   (*a)
./users/Admin@example.com/msp/tlscacerts/tlsca.example.com-cert.pem   (*b)
./users/Admin@example.com/msp/admincerts/Admin@example.com-cert.pem   (*c) (Auth. by (*a))
./users/Admin@example.com/msp/keystore/a331923a189a86dca0832b2841e077a3701cfe0063d6792e88f593a243c3338b_sk   (*C)
./users/Admin@example.com/msp/signcerts/Admin@example.com-cert.pem   (*c)(*C) (Auth. by (*a))

# } // AdminMsp



./orderers/orderer.example.com/tls/server.crt   (*F)
./orderers/orderer.example.com/tls/server.key   (*F)
./orderers/orderer.example.com/tls/ca.crt   (*b)


# OrdererMsp {

./orderers/orderer.example.com/msp/cacerts/ca.example.com-cert.pem   (*a)
./orderers/orderer.example.com/msp/tlscacerts/tlsca.example.com-cert.pem   (*b)
./orderers/orderer.example.com/msp/admincerts/Admin@example.com-cert.pem   (*c) (Auth. by (*a))
./orderers/orderer.example.com/msp/keystore/caec6e5979df95564f57b0a50174cc5004977cbdf3f667d1b9f1e1d5128ff490_sk   (*D)
./orderers/orderer.example.com/msp/signcerts/orderer.example.com-cert.pem   (*D) (Auth. by (*a))

# } // OrdererMsp


./ca/ca.example.com-cert.pem   (*a)(*A)
./ca/74652114cc0ad0b0b9c3365af181f9dc3e240fbe8373ec30f3cd10de4bde6221_sk   (*A)

./tlsca/tlsca.example.com-cert.pem   (*b)(*B)
./tlsca/534fde3483b518f3887590a10553f6b274d23692eee5bf5976f9be4943c3926d_sk   (*B)

./msp/cacerts/ca.example.com-cert.pem   (*a)
./msp/tlscacerts/tlsca.example.com-cert.pem   (*b)
./msp/admincerts/Admin@example.com-cert.pem   (*c)


-------------------------------------------------------

NOTE:
- (*a) (*b) ... are same files.
- (*A) (*B) ... matching crypto (pub/priv matching)


To run operations requiring ‘admin’ policy, ‘AdminMsp’ is required.
That is, cli-node or peer-node having ‘AdminMsp’ can run admin operations.
(CORE_PEER_MSPCONFIGPATH should be path to AdminMsp)

Note: This review is written while closing very small projects with Hyperledger Fabric(henceforth HLF).


You can easily find articles regarding pros and cons of HLF network architecture - ex. consensus algorithm, permission and so one - comparing with others - ex. Bitcoin, ethereum. So, in this articile, I would like to discuss things in a point of developing Chaincode.

Updating same state at multiple transactions

I think most important and biggest difference in terms of developing Chaincode between HLF and other popular networks, is


In case that several transactions try to update same global state, only one transaction is allowed in a block!


This comes from architectural design of HLF - Orderer(I think this is a kind of debt for hight TPS). Even if time interval of creating block, is very short comaring with other networks, this is very serious constraints. For example, in case of money trading system, only one transfer-transaction is allowed in a block for same account. Because of this characteristics - high TPS with contraints described above, HLF is very good for assets having it's own ID like house, products and so one. But, it's not good for assets requiring couting and calculation like money.

To overcome this constraints, HLF suggests sample codes at their <fabric-sample>/high-troughput. But I think this is not enough. For example, to know current balance, transaction have to read all accumulated variables. And to transfer money, one variable is added to global state. That is, still only one transaction is allowed in a block because read-set to calculate current balance is updated(added)!

Other Misc.

I think followings are not HLF specific issues. But to me, these are also annoying. And I can't find any good way to reduce these pain points at SDKs or libraries for Chaincode.

  • Read/Write operation on global state is very expensive. And reading variable that is updated in same transaction gives it's original value(not updated value)
  • ECDSA is algorithm for digital signature, not for en/decryption.
  • Testing Chaincode requires HLF networks. So, test and debugging cycle takes longer time than expected.

I hope that HLF team provides good solutions for them. And until then, I hope my sample template - hlfcc-node-starter - is helpful to other developers using typscript as their Chaincode-language.

+ Recent posts