Python Demux for EOS - py-demux-eos
py-demux-eos is an implementation of the Demux library for Python
What is Demux?
Demux is a library and design pattern introduced by block.one in July 2018. It allows programmers to reduce the cost of querying data in EOSIO blockchain applications. EOSIO smart contracts can store application data either in RAM or in the chain history. RAM is expensive and the chain state would be slow if the end user has to poll every block to look for events. The demux library polls the block chain, parses the actions and passes them onto another program which can write them into a local database and can quickly serve queries. A good example of this would be if we wrote a DEX (decentralised exchange), we would need to store the order book in a table in RAM but when a user wishes to complete a sale they would obviously want to get the best deal. If we do that in RAM the user who posts an order would pay with RAM and CPU time to update the indexes in the table so that the buyer can quickly find the best deal. With Demux those indexes can be computed off chain and the transaction is quicker and cheaper for both parties. We all want faster and cheaper don't we? That's why we have chosen the EOSIO platfom.
Why write in Python?
Even though Python is one of the worlds most widely used programming languages the main reason is because because Python is often considered a more expressive language, that is writing a program in Python requires less decisions than writing the same program in Javascript. This can be seen in the library we have produced which is approximately one third the size of its inspiration from block.one and we are just a team of three developers doing some block chain and web consulting work with EOSphere.
What are these start_block and commit_block functions you added and why are they useful?
The original Demux also contains an architectural flaw. If a program crashes or otherwise unexpectedly stops part way through processing a block you could end up with only some of the actions in a block recorded in your off chain database. The only way to get back to a known good state might be to reprocess the entire blockchain - millions of blocks. These situations aren't actually very uncommon nor always accidental. In the past we have used deployment strategies that don't guarantee that servers are cleanly shut down before an update in all cases. The core idea is that the programmer using the py-demux-eos library will start a database transaction in the start_block callback function and commit it in the commit_block callback function. This would then guarantee the consistency in the target database. i.e that this would be an atomic operation.
How can I use this library in my program?
The instructions are in the readme.md
To install py-demux-eos on your computer.
pip install …
In your Python import the Demux class
from demuxeos import Demux
Instantiate the Demux object, there are several optional parameters
d = Demux('https://node2.eosphere.io', start_block, commit_block, rollback)
Register action functions
d.register(action)
Tell Demux to start processing blocks
d.process_blocks(1)
Demux will call your callback function as it encounters new blocks and actions.
Demux is a class so we can connect to multiple chains and process them in the same app (our friends EOSphere are working on both EOS-Mainnet, WORBLI and soon to be TELOS). Doing so would require using multi threading though.
How do I get the code?
You can find it here: https://github.com/blockrepublictech/py-demux-eos
There is also a demonstration app here: https://github.com/blockrepublictech/py-demux-eos-runner
What's next?
We are intending to build a module that will integrate this with the Django web framework and store blockchain data in a relational database for easier processing for web applications.
How can I help?
Our work is supported by the EOSphere Australia. If you vote for us `eosphereiobp` we will be able to create more stuff for you and the ecosystem.
Maybe we can help you with building blockchain and web applications, you can contact us through EOSphere or at hello@blockrepublic.tech
Mark Lockett is a Developer on the EOSphere Team his work is supported by EOSphere
EOSphere are a passionate group of EOS enthusiasts and data centre professionals currently Standby Block Producers providing services for the EOS Mainnet.
Based in Australia, EOSphere are focussed on supporting the EOS community and serving the entire EOSphere through provision of infrastructure, governance and community benefit initiatives.
Congratulations @eosphere! You received a personal award!
Click here to view your Board