Living on the (Bleeding) Edge
Software development in the blockchain space is fast paced, constantly evolving, and constantly innovating. There is a paradox in software between standardization and innovation: standardization brings consistency, documentation, knowledge transfer, and easier adoption, while innovation forges ahead with new ideas, asking what could be possible and making it real, exploring new concepts, and pushing the limits.
Often times standardization struggles to keep up with innovation; we see this most often in the browser wars — balancing cross-browser compatibility and being the first to enable cool new features for rich applications.
My own struggle with balancing standardization and innovation in the EOS space comes in the form of token design and smart contract engineering. BlockOne created the first EOS token in the form of eosio.token which essentially set the de-facto standard for what an EOS based token would look and act like.
Block explorers most frequently use this standard to track balances for the myriad of tokens that now exist, 4,047 according to eosflare.io, and any deviation can make balances difficult to track. To provide some understanding on this token standard I’ll provide a brief explanation.
How EOS Tokens Work
Token balances are essentially an internal database stored within the smart contract that hosts the token. This balance is stored within a table names accounts, scoped to the owner of the balance (essentially meaning you must know the account name in advance to find its balance) with a single “column” aptly named `balance` that holds the token quantity.
At any given time one may query a smart contracts accounts table for an account name and determine the actual live balance. However, with over a million EOS accounts and 4000 tokens existence it is unreasonable for a block explorer to be making all these queries to the blockchain all the time.
This is where actions come into play. EOS Tokens have two primary actions that create or modify token balances: issue and transfer. These actions have a from and to parameter and a quantity to indicate how many tokens are being created or moved around. These actions also have notifiers that essentially “tell” an account they have received or given away some of their balance.
The majority of block explorers use these actions as markers to internally track however many tokens an account may have. They apply a “delta” to the token balance, add or subtract, from the collection of these actions occurring to calculate the actual balance. Because these actions only occur “when something happens” the block explorer does not need to constantly query the blockchain for every token balance, which improves how quickly a block explorer can tell you the information you want to know.
Innovation Breaks Everything
The majority of block explorers use the methods described above, so when any detail changes outside this standard they no longer “appear to work” for the token that decides that standard isn’t good enough to achieve what they want to achieve.
Some of the first instances where I began running up against this problem was when I created claimable tokens, when I created staking functionality for KARMA, and when I created auto-staked reward drops for Parsl.
Claimable tokens was not so bad: I added a simple true or false claimed column to the accounts table to track whether a token was claimed. Block explorers do not have a provision to display additional detail about token tables, so this was only view-able in specialized portals.
When it came time to recover unclaimed tokens this used a new action recover which moved the tokens away from the non-participating user and back to the issuing account. Most block explorers do not listen to this non-standard action, and so they would not accurately reflect that these tokens have been removed from an account.
For staking functionality on KARMA, the powerup, powerdown, and refund actions do not use transfers, they modify the `account` table directly to save on the CPU cost a user would have to pay to perform these actions. Once again, block explorers may not listen and balances might “appear” incorrect.
Lastly for the Parsl reward drop, rather than using a transfer action a custom rewarddrop action was used instead. This action means that a users new tokens are automatically staked, and once again most block explorers might not reflect these changes.
It’s important to note that the actual balance within the smart contracts internal database was always accurate. Reading this database directly would always have the correct result, however, block explorers may not show this on a casual glance.
Could It Have Been Better?
There are workarounds that may have been employed to support block explorers and the methods they use to track balances. For example, a complex series of inline transfer actions may be used every time one of these other custom actions are made. However, this dramatically increases CPU cost and processing overhead as discussed by Dan Larimer himself in his Medium article Developing Efficient Contracts
It essentially becomes a balance of two evils… make actions more costly for users so block explorers can appear accurate, or sacrifice the block explorers so the dApp is as efficient and affordable as possible for the user.
And Now For Something Completely Different1
Most recently I’ve had the privilege of developing the innovative Air-HODL smart contract for LiquidApps. This innovative airdrop model uses a block by block vesting schedule that enables participants to both actively use vested tokens for DAPP services without penalty, protects the market by providing constraints on how many tokens a recipient can actual sell, and rewards “HODLers” by automatically redistributing forfeited unvested tokens.
Creating a robust smart contract that enabled all these features was a challenge and I was privileged to work alongside Tal Muskal and have the support of Michael Gucci of AirdropsDAC to create something so innovative for the EOS community.
In this specific case, the accounts table of the contract was modified to have these columns: balance, staked, allocation and claimed. Because of the unique vesting nature of these tokens, even after they are issued to participants in some arbitrary quantity (let’s say 1000.0000 DAPPHDL for example’s sake) their balance would be 0 and allocation would be 1000.
The issue action was modified to not include a notifier to the recipient. On the block explorer side this meant that the issue action was essentially invisible for the sake of tracking token balances, however because the actual balance would be zero, even if the notifier was included block explorers would still show an invalid balance, because the tokens have not vested yet.
Note: An advantage of not including the notifier was that dApps that use notifiers to reject tokens (for example, casinos) were still able to receive their allocation of Air-HODL tokens. In traditional airdrops, these accounts reject the tokens and have no way of receiving them. This caveat was discovered during the many airdrops successfully deployed by AirdropsDAC.
Once more the method of using inline “dummy” transfer actions may have been employed, however this would have introduced tremendous complexity and CPU overhead which is not desired.
While block explorers may have difficulty or challenges in supporting such innovation, the community can look forward to a number of new DSP Portals being created by DSP’s (DAPP Service Providers) that will make managing and tracking AirHODL balances simple and efficient.
What Can Block Explorers Do?
We all turn to our trusty favourite block explorers to manage and track our EOS Accounts and the many tokens and interactions we enjoy on the EOS Mainnet. As mentioned above all these innovations still use traditional internal databases within the smart contract to accurately track token balances, so while block explorers may appear inaccurate, you can feel rest-assured that your token balances are in fact safe.
Some of the new paradigms that may be explored is reading the tables directly, tracking RAM and Table deltas specific to these token contracts (something that Dfuse does very well), creating custom tracking plugins for tokens (something that eosflare.io does very well), and new upcoming EOSIO features such as WasmQL and the variety of new and efficient history plugins that are being created such as Chronicle.
Never Stop Innovating
There can be challenges, caveats, and barriers to adoption with any innovation, but that never means stop. Overcoming these challenges, and seeing the community grow and take advantage of pushing the limits is extremely rewarding.
We see greater challenges overcome all the time throughout history: petrol stations for the first automobiles, power distribution to our homes, fast internet in Australia (this may never be solved…), charging stations for electric automobiles, renewable power generation, and exploring the stars. As developers we are fortunate that we can overcome the challenges of innovation without needing monumental global infrastructure. Don’t let anything stop you.
